Post-training quantization methods, devices, and storage media for audio noise reduction

Abstract

This application discloses a post-training quantization method, apparatus, and storage medium for audio denoising, relating to the field of artificial intelligence technology. The method includes: acquiring network weight data of a trained floating-point network (Floating-Point Network), which is used to denoise audio data; quantizing the network weight data; inputting an audio dataset into the trained floating-point network; acquiring the network activation values ​​of each activation layer during the audio dataset denoising process; quantizing the network activation values; and determining a quantized floating-point network based on the quantized network weight data and the quantized network activation values, which is then used for audio data denoising. This solves the problem in existing technologies where the large time consumption required for floating-point network quantization leads to low audio data denoising efficiency.

Description

Technical Field

[0001] This invention relates to a post-training quantization method, apparatus, and storage medium for audio noise reduction, belonging to the field of artificial intelligence technology. Background Technology

[0002] Low-bit quantization of deep learning models can effectively reduce the overhead of storage, computation, and communication during model deployment, and is a common model compression and inference optimization technique. However, model quantization still faces many challenges in practical applications, the most common being a decrease in model accuracy. Introducing quantization-related constraints during the model training phase, i.e., Quantization-aware training (QAT), can better address the accuracy problem of model quantization.

[0003] An existing training-aware quantization method includes: (1) performing pseudo-quantization on the parts of the floating-point network that need to be quantized; (2) performing iterative training with pseudo-quantization; and (3) storing the quantized network parameters after obtaining the network model that meets the expectations.

[0004] In the above method, in order to ensure the noise reduction accuracy of the floating-point network for audio data, the floating-point network needs to be retrained, and pseudo-quantization and other steps need to be introduced during the training process. These will make the quantization step cumbersome and time-consuming, which means that the noise reduction efficiency of the audio data is low. Summary of the Invention

[0005] The purpose of this invention is to provide a post-training quantization method, apparatus, and storage medium for audio noise reduction, in order to solve the problems existing in the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] According to a first aspect, embodiments of the present invention provide an audio noise reduction method, the method comprising:

[0008] Obtain the network weight data of the trained floating-point network, which is used to denoise the audio data;

[0009] The network weight data is quantized;

[0010] The audio dataset is input into the trained floating-point network;

[0011] Obtain the network activation values ​​of each activation layer of the floating-point network during the noise reduction process of the audio dataset;

[0012] The network activation value is quantified;

[0013] Based on the quantized network weight data and the quantized network activation values, a quantized floating-point network is determined. The quantized floating-point network is used for noise reduction of audio data. Optionally, the network weight data includes i groups, where i is an integer greater than 1. Quantizing the network weight data includes:

[0014] Get the preset zero point value;

[0015] For each group of network weight data, obtain the scaling factor based on the network weight data of the group;

[0016] The network weight data of each group are quantized according to the preset zero point value and the scaling factor corresponding to each group.

[0017] Optionally, obtaining the scaling factor based on the network weight data of the group includes:

[0018] Obtain the maximum network weight data from the network weight data of the group;

[0019] The scaling factor is obtained based on the maximum network weight data.

[0020] Optionally, the network activation values ​​include j groups, where j is an integer greater than 1, and the quantization of the network activation values ​​includes:

[0021] For each group of network activation values, obtain the scaling factor and zero point value corresponding to the group based on the network activation value of the group;

[0022] The network activation value of each group is quantized based on the scaling factor and zero-point value corresponding to each group.

[0023] Optionally, obtaining the scaling factor and zero-point value corresponding to the group based on the network activation value of the group includes:

[0024] For the network activation values ​​of the group, each network activation value is clustered according to the data distribution of the network activation values, and K categories are obtained, where K is an integer greater than 1;

[0025] For each of the K categories, obtain the quantization center value of that category;

[0026] Based on the calculated values ​​of each quantization center, the scaling factor and zero point value corresponding to the group are obtained.

[0027] Optionally, obtaining the quantization center value for each of the K categories includes:

[0028] For each of the K categories, obtain the network activation value at the cluster center in that category;

[0029] The obtained network activation values ​​are mapped to K uniformly distributed quantization centers, which are the quantization center values ​​of each category.

[0030] Optionally, obtaining the scaling factor and zero-point value corresponding to the group based on the calculated quantization center values ​​includes:

[0031] The scaling factor corresponding to the group is obtained based on the maximum and minimum quantization center values ​​among the various quantization center values;

[0032] The zero-point value corresponding to the classification is obtained based on the minimum quantization center value in the classification.

[0033] Optionally, determining the quantized floating-point network based on the quantized network weight data and the quantized network activation values ​​includes:

[0034] Based on the quantized network weight data and the quantized network activation value, it is determined whether the quantized floating-point network meets the target conditions.

[0035] If not, update the quantized network weight data and the quantized network activation value, and execute the step of detecting whether the quantized floating-point network meets the target conditions based on the quantized network weight data and the quantized network activation value again.

[0036] In a second aspect, a post-training quantization apparatus for audio noise reduction is provided, the apparatus including a memory and a processor, the memory storing at least one program instruction, the processor loading and executing the at least one program instruction to implement the method as described in the first aspect.

[0037] Thirdly, a computer storage medium is provided, wherein at least one program instruction is stored therein, the at least one program instruction being loaded and executed by a processor to implement the method as described in the first aspect.

[0038] By acquiring the network weight data of a trained floating-point network, which is then used to denoise audio data, the following steps are taken: quantizing the network weight data; inputting the audio dataset into the trained floating-point network; obtaining the network activation values ​​of each activation layer during the audio dataset denoising process; quantizing the network activation values; and determining the quantized floating-point network based on the quantized network weight data and quantized network activation values. This quantized floating-point network is then used for audio data denoising. This approach solves the problem of low audio data denoising efficiency due to the large time consumption required for floating-point network quantization in existing technologies. It achieves the effect of reducing time consumption, simplifying the quantization step, and improving the denoising efficiency of audio data by eliminating the need for retraining the floating-point network and only requiring statistical quantization of the network weight data and network activation values.

[0039] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0040] Figure 1 A flowchart illustrating a post-training quantization method for audio noise reduction according to an embodiment of the present invention;

[0041] Figure 2 This is a possible mapping diagram for mapping the values ​​of non-uniformly distributed quantization centers according to an embodiment of the present invention. Detailed Implementation

[0042] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0044] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0045] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0046] Please refer to Figure 1 It illustrates a flowchart of a post-training quantization method for audio noise reduction provided in one embodiment of this application, as shown below. Figure 1 As shown, the method includes:

[0047] Step 101: Obtain the network weight data of the trained floating-point network;

[0048] After the floating-point network is trained, the network weight data of the floating-point network can be obtained and stored.

[0049] The obtained network weight data may include i groups, each group containing at least one or more network weight data. Here, i is an integer greater than 1.

[0050] Unless otherwise specified, this application uses a trained floating-point network for noise reduction of audio data as an example. In one possible embodiment, it is applied to noise reduction of voice call content.

[0051] Step 102: Quantize the network weight data;

[0052] After obtaining the network weight data, the network weight data can be quantized. Optionally, this step may include:

[0053] First, obtain the preset zero-point value;

[0054] In one possible implementation, the preset zero-point value can be a system default value or a custom value. Usually, since the distribution of the obtained network weight data is relatively concentrated, the unsaturated symmetric quantization method is used, that is, the preset zero-point value can be 0.

[0055] Second, for each group of network weight data, obtain the scaling factor based on the network weight data of the group;

[0056] Optionally, this step includes:

[0057] (1) Obtain the maximum network weight data in the network weight data of the group;

[0058] Typically, the distribution of the obtained network weight data is relatively concentrated. Therefore, in one possible implementation, it can be calculated using an unsaturated symmetric quantization method.

[0059] Optionally, for each group, the maximum network weight data max(x) in that group can be obtained. Here, x represents the network weight data of each group, and x is a floating-point number.

[0060] (2) Obtain the scaling factor based on the maximum network weight data.

[0061] Optionally, the scaling factor S is: Where n is the number of quantization bits.

[0062] Third, the network weight data of each group is quantized according to the preset zero point value and the scaling factor corresponding to each group.

[0063] For each network weight data in each group, the network weight data is quantized according to the scaling factor and zero point value of that group.

[0064] In one possible implementation, the quantized network weight data q is: Where r is the network weight data before quantization, S is the scaling factor, and Z is the zero point value.

[0065] The above quantization method can be used to convert floating-point network weight data into fixed-point network weight data.

[0066] Step 103: Input the audio dataset into the trained floating-point network;

[0067] The audio dataset described in this application is a representative and diverse dataset. In actual implementation, designers can set it according to actual needs, and its specific content is not limited.

[0068] Step 104: Obtain the network activation values ​​of each activation layer in the audio dataset denoising process of the floating-point network;

[0069] After inputting the audio dataset into the trained floating-point network, the data calculated after each activation layer is obtained, and this calculated data is used as the network activation value for that activation layer. In other words, the network activation values ​​for each activation layer can be obtained through the above steps. The network activation values ​​for each layer are grouped together; that is, when the floating-point network includes n activation layers, n groups of network activation values ​​can be obtained.

[0070] The obtained network activation values ​​can include j groups, and each group can contain one or more network activation values, without limitation. j is an integer greater than 1.

[0071] Step 105: Quantize the network activation value;

[0072] After obtaining the activation values ​​of each network, these values ​​can be quantified. Optionally, this step includes:

[0073] First, for each group of network activation values, obtain the scaling factor and zero point value corresponding to the group based on the network activation value of the group;

[0074] (1) For the network activation values ​​of the group, the network activation values ​​are clustered according to the data distribution of the network activation values, and K categories are obtained, where K is an integer greater than 1;

[0075] Because the network activation values ​​are widely and unevenly distributed, with significant differences in distribution between layers, this application quantifies them independently based on the distribution of activation values ​​in each layer. Optionally, the network activation values ​​of each group can be clustered using the K-means method. In one possible embodiment, K(2^3) is selected using the K-means method. n Starting with 0 initial network activation values, iterate repeatedly until convergence to obtain K network activation value centers and network activation value intervals. The network activation value centers represent the activation values ​​within the network activation value intervals. Each network activation value interval represents a category.

[0076] Optionally, before performing the above clustering, the activation values ​​of each network in the group can be filtered. For example, outliers that deviate far from the normal distribution can be truncated by setting a threshold T. The filtered activation values ​​can then be clustered. This application does not limit the filtering method.

[0077] (2) For each of the K categories, obtain the quantization center value of the category;

[0078] A. For each of the K categories, obtain the network activation value at the cluster center in that category;

[0079] Optionally, the network activation center for each category is obtained by clustering using the K-means method.

[0080] B. Map the obtained network activation values ​​to K uniformly distributed quantization centers, where the K quantization centers are the quantization center values ​​of each category.

[0081] Typically, the obtained K network activation values ​​are non-uniformly distributed. Therefore, in this application, each obtained network activation value can be mapped to a uniformly distributed K quantization centers. Optionally, the mapping can be performed using lines with different slopes to obtain the K quantization centers. Of course, in actual implementation, if the obtained K network activation values ​​are uniformly distributed, no mapping is necessary, and this application does not impose any limitations on this.

[0082] Specifically, the mapping steps may include:

[0083] 1. Divide the quantization center values ​​into N groups from smallest to largest;

[0084] The value of N is selected based on the distribution of the quantization center values. The more uniform the data distribution, the smaller the value of N; the more uneven the distribution, the larger the value of N.

[0085] 2. Calculate the proportion P of the quantization center value range in each group to the total quantization center value range. n (n = 1, 2, ... N);

[0086] 3. Calculate the slope K of the mapping line corresponding to the quantization center value of each group. n (n = 1, 2, ... N);

[0087] in, Since the mapping lines of each group need to be connected end to end, the constant term of each group of mapping lines can be obtained.

[0088] 4. Substitute the quantization center values ​​of each group into the mapping line formula for that group to calculate the uniformly distributed quantization center values ​​after mapping.

[0089] In one possible implementation, assuming K = 128 and N = 8, that is, the 128 quantization center values ​​are divided into 8 groups. The slope of the mapping line corresponding to each group's quantization center value is calculated based on the proportion of the quantization center value range in the total quantization center value range. Since the mapping lines of each group need to be connected end-to-end, a constant term can be obtained. For example, please refer to... Figure 2 The diagram illustrates one possible schematic of the calculated straight lines. By substituting the quantization center values ​​of each group into the corresponding mapping line formula, 128 mapped points are obtained, and these points are distributed in a basically uniform manner.

[0090] (3) Based on the calculated values ​​of each quantization center, obtain the scaling factor and zero point value corresponding to the group.

[0091] For each group, the scaling factor corresponding to the group is obtained based on the maximum and minimum quantization center values ​​among the quantization center values ​​in the group;

[0092] In one possible implementation, the scaling factor S is: Where max(x) is the maximum quantization center value, min(x) is the minimum quantization center value, and n is the number of quantization bits.

[0093] The zero-point value corresponding to the group is obtained based on the minimum quantization center value in the group.

[0094] For each group, the minimum quantization center value is obtained based on the data distribution of each quantization center value in the group, and the zero-point value corresponding to the group is obtained based on the minimum quantization center value in the group.

[0095] Specifically, the zero-point value Z is: Where min(x) is the minimum quantization center value, and S is the scaling factor for this group.

[0096] Second, the network activation value of the group is quantized based on the scaling factor and zero point value corresponding to each group.

[0097] Optionally, for each group, quantization is performed based on the scaling factor, zero point value, and quantization center value of that group.

[0098] Specifically, the network activation value q after quantization for each classification is: Where S is the scaling factor, Z is the zero-point value, and r is the quantization center value of the classification.

[0099] Step 106: Determine the quantized floating-point network based on the quantized network weight data and the quantized network activation value.

[0100] After calculating the quantized network weights and activation values, they can be substituted into the floating-point network to quantize it into a fixed-point network.

[0101] In actual implementation, the network performance of the quantized floating-point network can be tested. If the network efficiency is not up to standard, it can be requantized, and if it is up to standard, the data can be saved.

[0102] Optionally, this step may include:

[0103] First, based on the quantized network weight data and the quantized network activation value, it is detected whether the quantized floating-point network meets the target conditions;

[0104] The target condition is usually network performance, which designers can set according to actual needs. This application does not impose any restrictions on this.

[0105] Second, if the conditions are not met, the quantized network weight data and the quantized network activation value are updated, and the step of detecting whether the quantized floating-point network meets the target conditions based on the quantized network weight data and the quantized network activation value is executed again.

[0106] Since the quantized network weight data and network activation values ​​in the above steps are related to the scaling factor S and the zero-point value Z, in one possible implementation, the quantized network weight data and network activation values ​​can be adjusted by adjusting S and Z, so that the quantized floating-point network meets the target conditions.

[0107] Optionally, the adjustment methods for S and Z include taking values ​​near S or Z, such as updating S to S' or updating Z to Z'. The difference between S and S' is less than a first threshold, and the difference between Z and Z' is less than a second threshold. Furthermore, the relationship between S and S', and between Z and Z', is not fixed; that is, S' can be greater than or less than S, and similarly, Z' can be greater than or less than Z. This application does not impose any limitations on this. In actual implementation, one value can be kept constant while the other value is adjusted; for example, S can be kept constant while Z is adjusted, or Z can be kept constant while S is adjusted. This application does not limit the specific adjustment method.

[0108] It should be further explained that after determining the quantized floating-point network, the network data can be combined into a data structure that can be adapted to the accelerator interface, and then the accelerator interface can be integrated into the neural network engineering code. Here, the network data includes the quantized network weight data and network activation values. Furthermore, after using the accelerator, the speedup effect is significant when multiplying int8 data matrices, allowing for more efficient use of the quantized int8 data, significantly improving running speed, and reducing network computational cost.

[0109] Furthermore, after obtaining the quantized floating-point network, the audio data to be denoised can be denoised using the quantized floating-point network.

[0110] In summary, by acquiring the network weight data of a trained floating-point network, which is then used for audio data denoising; quantizing the network weight data; inputting the audio dataset into the trained floating-point network; obtaining the network activation values ​​of each activation layer during the audio dataset denoising process; quantizing the network activation values; and determining the quantized floating-point network based on the quantized network weight data and quantized network activation values, the quantized floating-point network is then used for audio data denoising. This solves the problem of low audio data denoising efficiency due to the large time consumption required for floating-point network quantization in existing technologies. It achieves the effect of reducing time consumption, simplifying the quantization step, and improving the audio data denoising efficiency by eliminating the need for retraining the floating-point network and only requiring statistical quantization of the network weight data and network activation values.

[0111] This application also provides a post-training quantization apparatus for audio noise reduction, the apparatus including a memory and a processor, the memory storing at least one program instruction, the processor loading and executing the at least one program instruction to implement the method described above.

[0112] This application also provides a computer storage medium storing at least one program instruction, which is loaded and executed by a processor to implement the method described above.

[0113] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0114] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A post-training quantization method for audio noise reduction, characterized in that, The method includes: Obtain the network weight data of the trained floating-point network, which is used to denoise the audio data; The network weight data is quantized; the network weight data includes i groups, where i is an integer greater than 1. The quantization of the network weight data includes: obtaining a preset zero-point value; for each group of network weight data, obtaining a scaling factor based on the network weight data of the group; and quantizing each network weight data of the group based on the preset zero-point value and the scaling factor corresponding to each group. The audio dataset is input into the trained floating-point network; Obtain the network activation values ​​of each activation layer of the floating-point network during the noise reduction process of the audio dataset; The network activation values ​​are quantized; the network activation values ​​include j groups, where j is an integer greater than 1. The quantization of the network activation values ​​includes: for each group of network activation values, obtaining the scaling factor and zero-point value corresponding to the group based on the network activation values ​​of the group; and quantizing the network activation value of the group based on the scaling factor and zero-point value corresponding to each group. Based on the quantized network weight data and the quantized network activation value, the quantized floating-point network is determined, and the quantized floating-point network is used to denoise the audio data.

2. The method according to claim 1, characterized in that, The step of obtaining the scaling factor based on the network weight data of the group includes: Obtain the maximum network weight data from the network weight data of the group; The scaling factor is obtained based on the maximum network weight data.

3. The method according to claim 1, characterized in that, The step of obtaining the scaling factor and zero-point value corresponding to the group based on the network activation value of the group includes: For the network activation values ​​of the group, each network activation value is clustered according to the data distribution of the network activation values, and K categories are obtained, where K is an integer greater than 1; For each of the K categories, obtain the quantization center value of that category; Based on the calculated values ​​of each quantization center, the scaling factor and zero point value corresponding to the group are obtained.

4. The method according to claim 3, characterized in that, For each of the K categories, obtaining the quantization center value of that category includes: For each of the K categories, obtain the network activation value at the cluster center in that category; The obtained network activation values ​​are mapped to K uniformly distributed quantization centers, which are the quantization center values ​​of each category.

5. The method according to claim 3, characterized in that, The step of obtaining the scaling factor and zero-point value corresponding to the group based on the calculated quantization center values ​​includes: The scaling factor corresponding to the group is obtained based on the maximum and minimum quantization center values ​​among the various quantization center values; The zero-point value corresponding to the group is obtained based on the minimum quantization center value in the group.

6. The method according to any one of claims 1 to 5, characterized in that, The step of determining the quantized floating-point network based on the quantized network weight data and the quantized network activation value includes: Based on the quantized network weight data and the quantized network activation value, it is determined whether the quantized floating-point network meets the target conditions. If not, update the quantized network weight data and the quantized network activation value, and execute the step of detecting whether the quantized floating-point network meets the target conditions based on the quantized network weight data and the quantized network activation value again.

7. A post-training quantization device for audio noise reduction, characterized in that, The device includes a memory and a processor, the memory storing at least one program instruction, and the processor loading and executing the at least one program instruction to implement the method as described in any one of claims 1 to 6.

8. A computer storage medium, characterized in that, The computer storage medium stores at least one program instruction, which is loaded and executed by a processor to implement the method as described in any one of claims 1 to 6.

Images