Model compression methods, compressed model decompression methods, devices, electronic equipment

Abstract

This application provides a model compression method, a compressed model decompression method, an apparatus, and an electronic device. The model compression method includes: acquiring a high-precision model; the high-precision model refers to a model whose parameters are expressed using floating-point numbers; converting the model parameters into frequency domain parameters; and quantizing and compressing the frequency domain parameters to obtain a compressed model with low-precision frequency domain parameters. The model compression method provided in this application can achieve effective model compression without reducing the model's prediction accuracy, obtaining a high compression ratio, improving computational speed, and optimizing both computational speed and storage space of the compressed model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a model compression method, a compressed model decompression method, an apparatus, an electronic device, and a computer-readable storage medium. Background Art

[0002] Existing neural network models often have high storage and computational requirements, which limits their application on resource-constrained devices. Therefore, compressing neural network models is a feasible solution.

[0003] Currently, linear quantization is commonly used to compress models. However, because this method reduces the number of parameters in the original model to a smaller scale during the quantization process, it significantly impacts the model's accuracy, leading to decreased prediction accuracy. Furthermore, this method struggles to balance compression ratio and accuracy, thus failing to effectively compress the model. Summary of the Invention

[0004] To achieve a high compression ratio without reducing the accuracy of model predictions, this application provides a model compression method, a compressed model decompression method, an apparatus, an electronic device, and a computer-readable storage medium.

[0005] In a first aspect, this application provides a model compression method, comprising: obtaining a high-precision model; wherein the high-precision model refers to a model whose parameters are expressed using floating-point numbers; converting the parameters of the model into parameters expressed in the frequency domain; and quantizing and compressing the parameters expressed in the frequency domain to obtain a compressed model of parameters expressed in the low-precision frequency domain.

[0006] Since the parameters expressed in the time domain have a much larger data volume than those expressed in the frequency domain, converting the parameters expressed in the time domain to those expressed in the frequency domain can reduce the computational cost and storage space required for model quantization compression, thereby achieving better compression results. Furthermore, compared with existing technologies, the model compression method of this application does not lose the number of model parameters, so the compressed model does not reduce accuracy. Moreover, the precision of the model parameters is effectively reduced through parameter quantization compression, thus reducing the model's storage space requirements.

[0007] Optionally, converting the parameters of the model into parameters expressed in the frequency domain includes: using a Fourier transform algorithm to convert the parameters of the model into parameters expressed in the frequency domain.

[0008] Because the Fourier transform is a linear transform, it can decompose complex signals into several simple sine and cosine functions. Furthermore, there are fast algorithms for the Fourier transform, which have a periodic structure and can employ divide-and-conquer or recursive methods to reduce computational complexity. Therefore, using the Fourier transform algorithm can simply, quickly, and efficiently convert parameters expressed in the time domain into parameters expressed in the frequency domain.

[0009] Optionally, the quantization compression of the parameters expressed in the frequency domain includes: quantizing and compressing the parameters expressed in the frequency domain using an asymmetric quantization method.

[0010] Since asymmetric quantization can better adapt to the characteristics of data distribution, provide different quantization strategies for different data distributions, and provide higher quantization accuracy, using asymmetric quantization to quantize and compress the parameters expressed in the frequency domain can yield quantization results with higher accuracy and more uniform distribution.

[0011] Optionally, before quantizing and compressing the parameters of the frequency domain representation to obtain the parameters of the low-precision frequency domain representation, the method further includes: compressing the parameters of the frequency domain representation to reduce the sparsity of the parameter matrix of the frequency domain representation.

[0012] Since the matrix contains useless elements, these elements consume storage space and computational resources, and amplify errors in the calculation process and rounding errors. Therefore, compressing the parameters in the frequency domain representation and removing useless elements can effectively reduce the sparsity of the parameter matrix in the frequency domain representation, save storage space and computational resources, and improve computational efficiency. Optionally, the compression of the parameters in the frequency domain representation includes: compressing the parameters in the frequency domain representation using a gradient compression algorithm.

[0013] Gradient compression is simple to implement, has fast compression and decompression speeds, and is suitable for real-time data processing. Furthermore, gradient compression can achieve a high compression ratio. Therefore, using gradient compression algorithms to compress parameters expressed in the frequency domain can more quickly yield models with higher compression ratios.

[0014] Optionally, the step of compressing the frequency domain-represented parameters using a gradient compression algorithm includes: calculating the absolute values ​​of the frequency domain-represented parameters; sorting the absolute values ​​to obtain the indices of the sorted absolute values; dividing the sorted absolute values ​​into a preset number of numerical intervals; mapping the absolute values ​​to their corresponding numerical intervals; mapping the indices to the indices of the numerical intervals corresponding to their respective absolute values; compressing the mapped indices to a unit numerical interval to obtain gradient-compressed values; and reordering the gradient-compressed values ​​according to the order before the absolute values ​​were sorted.

[0015] By dividing the sorted absolute values ​​into a preset number of numerical intervals; mapping the absolute values ​​to the corresponding numerical intervals; and mapping the indices to the indices of the numerical intervals corresponding to their respective absolute values, the parameters expressed in the frequency domain can be more discretized. Since the number of parameters in the model is huge, directly using the absolute values ​​of the parameters expressed in the frequency domain for gradient compression results in high computational complexity and is not easy to process in parallel. Therefore, using the mapped indices for gradient compression can reduce computational and storage complexity.

[0016] Secondly, this application provides a method for decompressing a compressed model, comprising: converting the parameters of the compressed model in a low-precision frequency domain expression into parameters in a low-precision time domain expression to obtain a decompressed model; wherein the compressed model is generated according to the model compression method described in the first aspect, and the conversion method between the parameters in the low-precision frequency domain expression and the parameters in the low-precision time domain expression is the inverse transformation of the conversion method between the parameters in the first aspect and the parameters in the frequency domain expression.

[0017] By converting the parameters of the compressed model in the low-precision frequency domain to those in the low-precision time domain, a decompressed low-precision model can be obtained.

[0018] Optionally, the compression model uses Fourier transform to convert the parameters expressed in the time domain into parameters expressed in the frequency domain. The conversion of the low-precision frequency domain parameters of the compression model into low-precision time domain parameters includes: using an inverse Fourier transform algorithm to convert the low-precision frequency domain parameters of the compression model into the low-precision time domain parameters.

[0019] By employing the inverse Fourier transform algorithm, the compressed model can be decompressed quickly and efficiently.

[0020] Thirdly, this application provides a model compression apparatus, comprising: an acquisition module for acquiring a high-precision model; wherein the high-precision model refers to a model whose parameters are expressed using floating-point numbers; a first parameter conversion module for converting the parameters of the model into parameters expressed in the frequency domain; and a model compression module for quantizing and compressing the parameters expressed in the frequency domain to obtain a compressed model of parameters expressed in the low-precision frequency domain.

[0021] Fourthly, this application provides a compressed model decompression apparatus, comprising: a second parameter conversion module, used to convert the parameters of the compressed model in a low-precision frequency domain expression into parameters in a low-precision time domain expression, to obtain a decompressed model; the compressed model is generated according to the model compression method described in the first aspect, and the conversion method between the parameters in the low-precision frequency domain expression and the parameters in the low-precision time domain expression is the inverse transformation of the conversion method between the parameters and the parameters in the frequency domain expression in the first aspect.

[0022] Fifthly, this application provides an electronic device, including: a processor and a memory; the processor is configured to execute a program stored in the memory to implement the method as described in any one of the first aspects.

[0023] In a sixth aspect, this application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the method as described in any of the first aspects. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic flowchart of a model compression method provided in Embodiment 1 of this application;

[0026] Figure 2 A schematic flowchart illustrating another model compression method provided in Embodiment 1 of this application;

[0027] Figure 3 This is a schematic diagram of the structure of a model compression device provided in Embodiment 3 of this application;

[0028] Figure 4 This is a schematic diagram of the structure of a compression model connected to a compression device provided in Embodiment 4 of this application;

[0029] Figure 5 This is a schematic diagram of the structure of the electronic device provided in Embodiment 5 of this application. Detailed Implementation

[0030] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0031] The method provided in this application can be applied to resource-constrained devices. For example, model inference can be performed on embedded devices. Since embedded devices have very limited storage and computing resources, the model compression method provided in this application can achieve a better compression ratio without reducing the model's prediction accuracy, thereby reducing the model's storage space footprint and improving the model's computational efficiency.

[0032] To better understand the model compression method provided in this application, the relevant technologies are introduced below.

[0033] Related model compression methods generally employ linear quantization, which maps the original data to a finite set of quantization levels. Typically, equidistant or non-equidistant quantization methods are used for data compression.

[0034] However, linear quantization methods have a significant impact on model accuracy and struggle to balance compression ratio and accuracy. During model quantization compression, the number of parameters in the original model is reduced to a smaller scale, which often affects the model's accuracy, making it unsuitable, especially in scenarios requiring high precision. Furthermore, while this method can reduce model file size to some extent, practical applications often require balancing model size and accuracy, resulting in a mutually restrictive effect and making it difficult to simultaneously achieve high compression ratios and high accuracy.

[0035] In view of this, this application provides a model compression method, a compressed model decompression method, an apparatus, an electronic device, and a computer-readable storage medium. The model compression method reduces the computational load and storage space occupied by model quantization compression by converting the parameters expressed in the time domain of the model into parameters expressed in the frequency domain. By quantizing and compressing the parameters expressed in the frequency domain, the high-precision frequency domain parameters are converted into low-precision frequency domain parameters. During model inference, the low-precision frequency domain parameters are restored back to the low-precision time domain parameters. While compressing the number of model parameters, the accuracy of model prediction can be maintained, and a higher model compression ratio can be achieved.

[0036] The following describes in detail, with specific embodiments, the model compression method, compressed model decompression method, apparatus, electronic device, and computer-readable storage medium provided in this application.

[0037] Example 1

[0038] To achieve better model compression ratio and higher model computation efficiency without reducing model accuracy, embodiments of this application provide a model compression method, such as... Figure 1 As shown, the method includes:

[0039] S101: Obtain the high-precision model. A high-precision model refers to a model whose parameters are expressed using floating-point numbers.

[0040] Here, parameters refer to the model's weights and biases. There is no restriction on the number of bits in the floating-point numbers; for example, they can be FP64, FP32, or FP16.

[0041] Specifically, obtain a pre-trained high-precision model.

[0042] Optionally, the trained high-precision model has an accuracy of FP32.

[0043] A high-precision model refers to a neural network model, and the type of neural network model is not specifically limited. For example, it can be a fully connected neural network, a convolutional neural network, or a recurrent neural network.

[0044] S102: Convert the model's parameters into parameters expressed in the frequency domain.

[0045] Specifically, because time-domain data contains continuous data related to time, while frequency-domain data decomposes time-domain data into discrete frequency components with assignments and phases, frequency-domain data has a smaller data volume compared to time-domain data. Therefore, converting parameters from the time domain to the frequency domain can significantly reduce the computational cost during model quantization compression and improve the computational speed of model compression. The method for transforming parameters from the time domain to the frequency domain is not specifically limited; examples include wavelet transform, Laplace transform, and Fourier transform.

[0046] In one feasible implementation, step S102 converts the model's parameters into frequency domain parameters, including:

[0047] The Fourier transform algorithm is used to convert the parameters of the model into parameters expressed in the frequency domain.

[0048] The Fourier transform is a linear transform that decomposes complex signals into several simple sinusoidal pre-functions. The formula for the Fourier transform can be expressed as:

[0049]

[0050] Where n is the number of sequences, x n It is a time-domain signal sequence, where j is the imaginary unit, k is the frequency domain sampling point index, N is the number of sampling points, and X... k It is the frequency domain representation at a certain frequency, and its value range is a complex number.

[0051] The following example, using weight parameters, illustrates the parameter transformation method:

[0052] Suppose a weight tensor x = [x0, x1, x2], where x0, x1, and x2 are the sampling points of x at times t0, t1, and t2 in the time domain sequence, x0 = 3, x1 = 1, and x2 = 4, i.e., x = [3, 1, 4], and N = 3. Substituting these values ​​into the Fourier transform formula, we get:

[0053]

[0054] X1 and X2 are solved using Euler's formula.

[0055] When the weight tensor is a multidimensional tensor, it is set to a one-dimensional tensor for easier computation.

[0056] For example, suppose the dimension tensor of weights is 1*3*3, with the following values:

[0057] Weights=[[[-1.1807, 0.4312, 0.2082],

[0058] [0.2906, 0.4785, -1.1250],

[0059] [0.0762, -0.3177, 0.0225]].

[0060] The weight is compressed into a one-dimensional tensor, as shown below:

[0061] [-1.1807, 0.4312, 0.2082, 0.2906, 0.4785, -1.1250, 0.0762, -0.3177, 0.0225]. Substituting the one-dimensional tensor values ​​into the Fourier transform formula using the method described above, the following frequency domain data is obtained:

[0062] [-1.1162+0.0j, -0.42804883-1.51472284j, 1.67766721+0.63402646j, -0.66275-1.28717356j, -1.98658396+1.5917 2176j, -1.98658396-0.59172176j, -0.66275+1.28717356j, -1.67766721-0.63402646j, -0.42804883+1.51472284j].

[0063] In one feasible implementation, the Fourier transform algorithm is used to convert the parameters of the model into parameters expressed in the frequency domain, including:

[0064] The Fast Fourier Transform algorithm is used to convert the parameters of the model into parameters expressed in the frequency domain.

[0065] The Fast Fourier Transform (FFT) is a fast algorithm for calculating the Discrete Fourier Transform (DFT), with a time complexity of O(NlogN). The FFT is an efficient algorithm for calculating the Fourier Transform. Because the Fourier Transform has high computational complexity, the FFT significantly reduces computational costs.

[0066] The FFT algorithm reduces computation time by dividing the signal into smaller sub-signals and reusing the results when calculating the Fourier transform. The FFT algorithm can be implemented using recursive and non-recursive methods. Using the recursive method, the signal is divided into two parts, and the results are combined after calculating the Fourier transforms of the sub-signals. Using the non-recursive method, the signal is rearranged, the Fourier transforms are calculated with different step sizes, and the results are combined afterward.

[0067] S103: Quantize and compress the parameters expressed in the frequency domain to obtain a compressed model of the parameters expressed in the frequency domain with low precision.

[0068] Quantization refers to the process of converting model parameters from floating-point to fixed-point representation. The aim is to reduce the model's precision, decrease its storage footprint, and improve its inference speed. Fixed-point numbers are a way for computers to represent numbers. In fixed-point numbers, the position of the decimal point is fixed, and the digits before and after the decimal point are represented in binary. These digits are then combined to store the number in the computer. This representation method is called fixed-point representation, and numbers represented in this way are called fixed-point numbers.

[0069] Specifically, since the parameters obtained after Fourier transform are still expressed as floating-point numbers in the frequency domain, quantization compression is needed to convert the high-precision frequency domain parameters into low-precision fixed-point parameters in order to reduce the model's storage space requirements.

[0070] There are many methods for quantizing models, and no specific limitations are made here. For example, there are linear quantization methods and nonlinear quantization methods. Linear quantization methods include symmetric quantization and asymmetric quantization. Nonlinear quantization methods include pairwise quantization, exponential quantization, and Laplace quantization.

[0071] In one feasible implementation, step S103 quantizes and compresses the parameters expressed in the frequency domain, including:

[0072] Asymmetric quantization is used to quantize and compress the parameters expressed in the frequency domain.

[0073] Specifically, the asymmetric quantization method first obtains the maximum and minimum values ​​of the high-precision frequency domain representation of the input parameters as boundary values ​​of the input parameters, uses [QMIN, QMAX] as the numerical range of the mapping, calculates the scaling factor and offset based on the boundary values ​​of the input parameters and the maximum and minimum values ​​of the numerical range, and then performs fixed-point conversion on the frequency domain representation of the parameters.

[0074] The formula for asymmetric quantization can be expressed as:

[0075]

[0076] Where max(x) is the maximum value of the input frequency domain parameter, min(x) is the minimum value of the input frequency domain parameter, QMAX is the maximum value of the mapped numerical range, QMIN is the minimum value of the mapped numerical range, Z is the offset, and the round() function rounds up. uint8 is the converted 8-bit fixed-point number.

[0077] For example, x = [-0.070477, -0.77491695, 0.63402296], where max(x) = 0.63402296 and min(x) = -0.77491695. In int8 type, QMAX = 255 and QMIN = 0. Substituting these values ​​into the above formula yields:

[0078]

[0079]

[0080]

[0081] Optionally, in order to evaluate the quantization effect, the parameters expressed in the low-precision frequency domain after quantization can be dequantized, the dequantization result can be compared with the parameters expressed in the original high-precision frequency domain, the error can be calculated, and error analysis can be performed.

[0082] In one feasible implementation, to avoid the sparsity of the parameter matrix expressed in the frequency domain, before step S103, which involves quantizing and compressing the parameters in the frequency domain to obtain a compressed model of the parameters in the low-precision frequency domain expression, the following method is further included:

[0083] The parameters expressed in the frequency domain are compressed to reduce the sparsity of the parameter matrix.

[0084] Specifically, there are many compression methods to reduce matrix sparsity, which will not be specifically limited here. Examples include proportional compression, linear predictive compression, wavelet compression, and gradient compression. Proportional compression refers to reducing the data in a proportional manner, with the scaling factor being a constant, to reduce the data's storage space. Linear predictive compression is a data compression method based on a linear predictive model; common examples include LineFit and LPC. Wavelet compression utilizes wavelet transform to decompose data into coefficients at multiple scales and different frequencies. Compression can be achieved by filtering out some coefficients, while maintaining the signal's multi-resolution characteristics and a certain frequency-resolved characteristic.

[0085] In one feasible implementation, the parameters of the frequency domain representation are compressed to reduce the sparsity of the parameter matrix of the frequency domain representation, including:

[0086] Gradient compression algorithm is used to compress the parameters expressed in the frequency domain.

[0087] like Figure 2 As shown, in one feasible implementation, a gradient compression algorithm is used to compress the parameters expressed in the frequency domain, including:

[0088] S1041. Calculate the absolute value of the parameters expressed in the frequency domain.

[0089] Specifically, since the absolute value of the parameters expressed in the frequency domain can reflect the response intensity of the time domain signal at a specific frequency, taking the absolute value of the parameters expressed in the frequency domain can be used for feature extraction.

[0090] Since the parameter expressed in the frequency domain is a complex number, for a complex number z = x + jy, where x and y represent the real and imaginary parts respectively, the absolute value of the complex number can be calculated using the following formula:

[0091]

[0092] In the example above, the absolute values ​​of the nine frequency domain parameters are:

[0093] [1.1162, 1.57404291, 1.79347618, 1.4477753, 2.54560287, 2.54560287, 1.4477753, 1.79347618, 1.57404291].

[0094] S1042. Sort the absolute values ​​and obtain the index of the sorted absolute values.

[0095] The index records the subscript of the frequency domain representation parameter in the array before sorting.

[0096] For example, the absolute values ​​of the parameters expressed in the frequency domain after Fourier transform are [7, 6, 8], and the subscripts of the corresponding absolute values ​​in the array are [0, 1, 2], so their indices are [0, 1, 2]. The absolute values ​​after ascending order are [6, 7, 8], and the corresponding indices are [1, 0, 2].

[0097] S1043. Divide the sorted absolute values ​​into a preset number of numerical intervals.

[0098] The preset number is a positive integer greater than the number of elements in the absolute value array of the parameters expressed in the frequency domain.

[0099] Specifically, after the parameters expressed in the frequency domain are sorted in ascending order of absolute value, they have a maximum value and a minimum value. In order to make the absolute values ​​of the parameters expressed in the frequency domain more discrete, the maximum value and the minimum value are used as the boundary values ​​of the discrete interval, and the discrete interval is divided into a preset number of numerical intervals.

[0100] For example, with a preset number of 10, the absolute values ​​of the parameters expressed in the frequency domain [6, 7, 8] are divided into 10 numerical intervals, with each pair of adjacent values ​​forming a numerical interval, as shown below:

[0101] [6, 6.2, 6.4, 6.6, 6.8, 7, 7.2, 7.4, 7.6, 7.8, 8].

[0102] S1044. Map the absolute values ​​to their corresponding numerical intervals. Map the indices to the indices of the numerical intervals corresponding to their respective absolute values.

[0103] Specifically, to make the absolute values ​​of the parameters expressed in the frequency domain more discrete, each absolute value needs to be mapped to its corresponding numerical interval within the discrete interval. Since the model has a huge number of parameters, directly using the absolute values ​​of the parameters expressed in the frequency domain for gradient compression results in high computational complexity and is not easily parallelized. Therefore, gradient compression is generally performed using the indices of each absolute value within the discrete interval. Thus, it is also necessary to map the index of the absolute value to the index of the corresponding numerical interval within the discrete interval.

[0104] For example, the absolute values ​​of the parameters expressed in the frequency domain [6, 7, 8] are mapped to the elements with values ​​of 6, 7, and 8 in the discrete interval [6, 6.2, 6.4, 6.6, 6.8, 7, 7.2, 7.4, 7.6, 7.8, 8], and the indices of the absolute values ​​of the parameters expressed in the frequency domain [0, 1, 2] are mapped to the indices of the corresponding elements 6, 7, and 8 in the discrete interval [0, 5, 10].

[0105] S1045. Compress the mapped index to a unit value range to obtain the gradient compressed value.

[0106] For example, if the unit value interval is [-1, 1], the gradient compression formula is:

[0107]

[0108] Where, index is the index of the absolute value of the parameter expressed in the frequency domain, mapped to the index of the corresponding numerical interval in the discrete interval, and num... levels It is the number of numerical intervals in the discrete interval, where n is the number of absolute values ​​of the parameters expressed in the frequency domain, and grad i i is the absolute value of the parameter expressed in the i-th frequency domain. Where 0 ≤ i <n。

[0109] Substituting the mapped index values ​​[0, 5, 10] into the above formula, we get:

[0110]

[0111] S1046. Reorder the gradient compression values ​​according to the order before sorting by absolute value.

[0112] Specifically, in the example above, the mapping is [6, 7, 8] to [-0.77491695, -0.070447, 0.63402296]. Therefore, it is necessary to restore the mapping to [7, 6, 8]. The mapping after restoration is [-0.070447, -0.77491695, 0.63402296].

[0113] Since the parameters expressed in the time domain involve a much larger amount of data compared to those expressed in the frequency domain, converting the time-domain parameters to the frequency-domain parameters can reduce the computational load and storage space required for model quantization compression, thereby achieving better compression results. Furthermore, compared to existing technologies, the method provided in this application restores the low-precision parameters expressed in the frequency domain back to the low-precision parameters expressed in the time domain during model inference without losing the number of model parameters. Therefore, the compressed neural network model does not reduce accuracy, and the precision of the model parameters is effectively reduced through parameter quantization compression. Thus, the computational efficiency of model inference is improved, and the storage space occupied by the model is reduced.

[0114] Example 2

[0115] In order to perform inference and prediction using the model, the compressed model needs to be decompressed. This application provides a method for decompressing a compressed model, including:

[0116] The parameters of the compressed model in the low-precision frequency domain are converted into parameters in the low-precision time domain to obtain the decompressed model. The compressed model is generated according to the model compression method in any of the embodiments in Example 1. The conversion method between the parameters in the low-precision frequency domain and the parameters in the low-precision time domain is the inverse transformation of the parameter conversion method between the parameters and the frequency domain in Example 1.

[0117] In one feasible implementation, the compressed model uses Fourier transform to convert the parameters expressed in the time domain to parameters expressed in the frequency domain, and converts the low-precision frequency domain parameters of the compressed model to low-precision time domain parameters, resulting in a decompressed model, including:

[0118] The inverse Fourier transform algorithm is used to convert the parameters of the compressed model from the low-precision frequency domain expression to the low-precision time domain expression.

[0119] The formula for the inverse Fourier transform is:

[0120]

[0121] Where N is the signal length, and its value is a non-negative integer: a iParameters in the frequency domain representation, taking complex values, where i is the subscript of the parameter in the frequency domain representation; 0 ≤ i < N; k is the subscript of the parameter in the time domain, 0 ≤ k < N; j is the imaginary unit; e is the base of the natural constant; x k are the parameters in the converted time domain representation.

[0122] After the inverse Fourier transform, the model is decompressed into parameters in a low-precision time domain representation, which can be used for inference.

[0123] To compensate for the quantization deviation of the model accuracy, the difference between the parameters in the original time domain representation and the parameters in the low-precision time domain representation after the inverse transform can be calculated, and the difference is normalized. The normalized difference is added to the parameters in the low-precision time domain representation after the inverse transform to update the parameters in the time domain representation.

[0124] Embodiment III

[0125] The embodiment of the present application provides a model compression device 300, as Figure 3 shown. The model compression device 300 includes: an acquisition module 301, a first parameter conversion module 302, and a model compression module 303.

[0126] The acquisition module 301 is used to acquire a high-precision model. The high-precision model means that the parameters of the model are expressed in floating-point numbers.

[0127] The first parameter conversion module 302 is used to convert the parameters of the model into parameters in the frequency domain representation.

[0128] The model compression module 303 is used to perform quantization compression on the parameters in the frequency domain representation to obtain a compressed model of the parameters in the low-precision frequency domain representation.

[0129] Optionally, the first parameter conversion module 302 is specifically used to convert the parameters of the model into parameters in the frequency domain representation by using the Fourier transform algorithm.

[0130] Optionally, the model compression module 303 is specifically used to perform quantization compression on the parameters in the frequency domain representation by using the asymmetric quantization method.

[0131] Optionally, the model compression device 300 further includes a parameter compression module 304.

[0132] The parameter compression module 304 is used to compress the parameters in the frequency domain representation to reduce the sparsity of the parameter matrix in the frequency domain representation.

[0133] Optionally, the parameter compression module 304 is specifically used to compress the parameters in the frequency domain representation by using the gradient compression algorithm.

[0134] Optionally, the parameter compression module 304 is specifically used to calculate the absolute values ​​of the parameters expressed in the frequency domain. The absolute values ​​are sorted to obtain their indices. The sorted absolute values ​​are divided into a preset number of numerical intervals. Each absolute value is mapped to its corresponding numerical interval. Each index is mapped to its corresponding numerical interval index. The mapped indices are compressed into a single numerical interval to obtain the gradient compressed values. The gradient compressed values ​​are then reordered according to their original order before the absolute values ​​were sorted.

[0135] Example 4

[0136] This application provides a compressed model decompression device 400, such as... Figure 4 As shown, it includes a second parameter conversion module 401.

[0137] The second parameter conversion module 401 is used to convert the parameters of the compressed model in the low-precision frequency domain expression into parameters in the low-precision time domain expression, thereby obtaining the decompressed model. The compressed model is generated according to the model compression method in Example 1, and the conversion method between the parameters in the low-precision frequency domain expression and the parameters in the low-precision time domain expression is the inverse transformation of the parameter conversion method between the parameters and the parameters in the frequency domain expression in Example 1.

[0138] Optionally, the second parameter conversion module 401 is specifically used to convert the parameters expressed in the time domain to the parameters expressed in the frequency domain using the Fourier transform algorithm when the compressed model uses Fourier transform to convert the parameters expressed in the time domain to the parameters expressed in the frequency domain.

[0139] Example 5

[0140] This application provides an electronic device 500, such as... Figure 5 As shown, it includes a processor 501 and a memory 502. The processor 501 is used to execute one or more programs stored in the memory 502 to implement the method described in Embodiment 1 or Embodiment 2 above.

[0141] I understand. Figure 5 The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown.

[0142] For example, the processor 501 and the memory 502 may be connected via a communication bus. As another example, the electronic device may also include components such as a display, mouse, and keyboard.

[0143] In this embodiment, the processor 501 can be a central processing unit, a microprocessor, a microcontroller, etc., but this is not a limitation. The memory 502 can be a random access memory, a read-only memory, a programmable read-only memory, an erasable read-only memory, an electrically erasable read-only memory, etc., but this is not a limitation.

[0144] In the embodiments of this application, the electronic device can be, but is not limited to, physical devices such as desktop computers, laptops, smartphones, smart wearable devices, and in-vehicle devices, or virtual devices such as virtual machines. Furthermore, the electronic device is not necessarily a single device; it can be a combination of multiple devices, such as a server cluster, etc.

[0145] This embodiment also provides a computer-readable storage medium, such as a floppy disk, optical disk, hard disk, flash memory, USB flash drive, SD (Secure Digital Memory Card), MMC (Multimedia Card), etc., in which one or more programs implementing the above steps are stored. These one or more programs can be executed by one or more processors to implement the methods described in Embodiment 1 or Embodiment 2. Further details will not be elaborated here.

[0146] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0147] In the various embodiments of this application, the functional modules can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0148] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0149] In this article, "multiple" refers to two or more.

[0150] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A model compression method, characterized in that, include: Obtain a high-precision model; the high-precision model refers to a model whose parameters are expressed using floating-point numbers; The parameters of the model are converted into parameters expressed in the frequency domain; The parameters expressed in the frequency domain are compressed using a gradient compression algorithm to reduce the sparsity of the parameter matrix expressed in the frequency domain. The parameters expressed in the compressed frequency domain are quantized and compressed to obtain a compressed model of the parameters expressed in the low-precision frequency domain. The step of compressing the parameters expressed in the frequency domain using a gradient compression algorithm includes: Calculate the absolute values ​​of the parameters expressed in the frequency domain; The absolute values ​​are sorted to obtain the index of the sorted absolute values. The index records the subscript of the frequency domain expression parameter in the array before sorting. The sorted absolute values ​​are divided into a preset number of numerical intervals; the preset number is a positive integer greater than the number of elements in the absolute value array of the parameters expressed in the frequency domain. The absolute values ​​are mapped to their corresponding numerical ranges, and the indices are mapped to the indices of the numerical ranges corresponding to their respective absolute values. The mapped indices are compressed to a unit value range to obtain the gradient compressed value; The gradient compression values ​​are reordered according to the order before the absolute values ​​were sorted.

2. The model compression method as described in claim 1, characterized in that, The step of converting the parameters of the model into frequency domain parameters includes: The parameters of the model are converted into parameters expressed in the frequency domain using the Fourier transform algorithm.

3. The model compression method as described in claim 1, characterized in that, The quantization and compression of the parameters of the compressed frequency domain representation includes: The parameters of the compressed frequency domain representation are quantized and compressed using an asymmetric quantization method.

4. A method for decompressing a compressed model, characterized in that, include: The parameters of the compressed model expressed in low precision frequency domain are converted into parameters expressed in low precision time domain to obtain the decompressed model; the compressed model is generated by the model compression method according to claim 1, and the conversion method between the parameters expressed in low precision frequency domain and the parameters expressed in low precision time domain is the inverse transformation of the conversion method between the parameters expressed in claim 1 and the parameters expressed in frequency domain.

5. The compression model decompression method as described in claim 4, characterized in that, The compressed model uses Fourier transform to convert the parameters expressed in the time domain into parameters expressed in the frequency domain. The conversion of the low-precision frequency domain parameters of the compressed model into low-precision time domain parameters includes: The parameters of the compressed model expressed in low precision frequency domain are converted into parameters expressed in low precision time domain using the inverse Fourier transform algorithm.

6. A model compression device, characterized in that, include: The acquisition module is used to acquire a high-precision model; the high-precision model refers to a model whose parameters are expressed using floating-point numbers. The first parameter conversion module is used to convert the parameters of the model into parameters expressed in the frequency domain; The parameter compression module is used to compress the parameters expressed in the frequency domain using a gradient compression algorithm to reduce the sparsity of the parameter matrix expressed in the frequency domain. The model compression module is used to quantize and compress the parameters expressed in the compressed frequency domain to obtain a compressed model of the parameters expressed in the low-precision frequency domain. The step of compressing the frequency domain parameters using a gradient compression algorithm includes: Calculate the absolute values ​​of the parameters expressed in the frequency domain; The absolute values ​​are sorted to obtain the index of the sorted absolute values. The index records the subscript of the frequency domain expression parameter in the array before sorting. The sorted absolute values ​​are divided into a preset number of numerical intervals; the preset number is a positive integer greater than the number of elements in the absolute value array of the parameters expressed in the frequency domain. The absolute values ​​are mapped to their corresponding numerical ranges, and the indices are mapped to the indices of the numerical ranges corresponding to their respective absolute values. The mapped indices are compressed to a unit value range to obtain the gradient compressed value; The gradient compression values ​​are reordered according to the order before the absolute values ​​were sorted.

7. A decompression device for a compressed model, characterized in that, include: The second parameter conversion module is used to convert the parameters of the low-precision frequency domain expression of the compressed model into the parameters of the low-precision time domain expression, so as to obtain the decompressed model; the compressed model is generated by the model compression method according to claim 1, and the conversion method between the parameters of the low-precision frequency domain expression and the parameters of the low-precision time domain expression is the inverse transformation of the conversion method between the parameters and the parameters of the frequency domain expression in claim 1.

8. An electronic device, characterized in that, include: Processor and memory; The processor is used to execute a program stored in the memory to implement the method as described in any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the method as described in any one of claims 1 to 5.

Images