A loss function-based noise reduction method, device and system

Abstract

The application relates to the technical field of audio processing, and discloses a loss function-based noise reduction method, a noise reduction device and a noise reduction system. The method comprises the following steps: acquiring a time domain output sequence and a frequency domain time-frequency graph of noisy audio; respectively dividing the time domain output sequence and corresponding time domain labels into output sequence audio segments and corresponding label audio segments of different unit lengths to obtain an output sequence audio group and a label audio group; calculating a time domain error based on the output sequence audio group and the label audio group; calculating a frequency domain error based on the frequency domain time-frequency graph, the output sequence audio group and the label audio group; constructing a target loss function; outputting a trained deep learning model; and estimating noisy audio to obtain target audio. The application can improve the over-suppression phenomenon of a model on a speech signal and improve the noise reduction precision of the model.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to a noise reduction method, apparatus and system based on a loss function. Background Technology

[0002] Currently, deep learning-based audio noise suppression mainly consists of two stages: the training stage and the inference stage. The training stage involves obtaining the output from the model based on the input data, calculating the error between the output and the label using a loss function, and then iterating the model through backpropagation based on this error to finally obtain the inference model. The inference stage is to perform inference based on the inference model obtained in the training stage.

[0003] However, in existing solutions, the loss functions of the models are mostly MSE (Mean Square Error), SNR (Signal-to-Noise Ratio), or SI-SNR (Scale-invariant Signal-to-Noise Ratio). The features used to calculate the error are often amplitude spectrum, complex spectrum, or time-domain signal, and the error is always calculated only in the time or frequency domain, which is difficult to fully represent the error in estimating audio and labels. This results in a low objective evaluation index of the loss function of the trained model. In addition, in practical applications, in order to better suppress audio noise, there is often an over-suppression of the speech signal, which also leads to a low objective evaluation index of the loss function during the training of existing models. Consequently, the trained model has a serious over-suppression of the speech signal, a large model bias, and reduced noise reduction accuracy.

[0004] Regarding the aforementioned technologies, the inventors discovered that existing audio noise reduction methods suffer from severe over-suppression and low noise reduction accuracy. Summary of the Invention

[0005] To improve the oversuppression of speech signals and enhance noise reduction accuracy, this application provides a noise reduction method, apparatus, and system based on a loss function.

[0006] Firstly, this application provides a denoising method based on a loss function.

[0007] This application is achieved through the following technical solution:

[0008] A noise reduction method based on a loss function includes the following steps:

[0009] Obtain the time-domain output sequence and frequency-domain time-frequency plot of the noisy frequency;

[0010] The time-domain output sequence and the corresponding time-domain label are respectively divided into output sequence audio segments of different unit durations and corresponding label audio segments of different unit durations to obtain output sequence audio groups and label audio groups;

[0011] Calculate the temporal error based on the output sequence audio group and the tag audio group; and

[0012] The frequency domain error is calculated based on the frequency domain time-frequency diagram, the output sequence audio group, and the tag audio group;

[0013] Construct a target loss function based on the time-domain error and the frequency-domain error;

[0014] The noisy frequency is input into a preset deep learning model, and the model is trained by combining the target loss function. The denoised frequency is output until the preset training conditions are met, and then the trained deep learning model is output.

[0015] The target audio is obtained by estimating the noisy frequencies using a trained deep learning model.

[0016] In a preferred embodiment, this application can be further configured such that the step of calculating the temporal error based on the output sequence audio group and the tag audio group includes:

[0017] Based on the output sequence audio group and the tag audio group, the scale-invariant signal-to-noise ratio of each output sequence audio segment with a different unit duration and the corresponding tag audio segment is calculated to obtain the segmented scale-invariant signal-to-noise ratio;

[0018] The time-domain error is obtained by summing the scale-invariant signal-to-noise ratios of each scale.

[0019] In a preferred embodiment, this application can be further configured such that the step of calculating the frequency domain error based on the frequency domain time-frequency plot, the output sequence audio group, and the tag audio group includes:

[0020] Based on the output sequence audio group and the tag audio group, and combined with the frequency domain time-frequency diagram, the mean square error is calculated, and the mean square error includes the complex mean square error and the amplitude mean square error.

[0021] The complex mean square error is obtained by adding the real part mean square error of the output sequence audio segments with different unit durations and the corresponding tag audio segments to the imaginary part mean square error of the output sequence audio segments with different unit durations and the corresponding tag audio segments.

[0022] The predicted amplitude is obtained by adding the sum of the squares of the real part mean square errors of the corresponding output sequence audio segments of different unit durations to the sum of the squares of the imaginary part mean square errors of the corresponding output sequence audio segments.

[0023] The temporal tag amplitude is obtained by adding the sum of the squares of the real part mean square errors of the tag audio segments of different unit durations in the tag audio group to the sum of the squares of the imaginary part mean square errors of the corresponding tag audio segments.

[0024] Calculate the mean square error between the predicted amplitude and the temporal label amplitude to obtain the amplitude loss;

[0025] The frequency domain error is obtained based on the complex mean square error and the amplitude loss.

[0026] In a preferred embodiment, this application can be further configured such that, when obtaining the frequency domain error based on the complex mean square error and the amplitude loss, the following formula is used.

[0027] loss freq =0.9 * loss composite +0.1*loss mags

[0028] In the formula, loss freq Represents frequency domain error, loss composite The loss represents the complex mean square error. mags This indicates a loss in magnitude.

[0029] In a preferred embodiment, this application may be further configured such that, prior to the step of constructing the target loss function based on the time-domain error and the frequency-domain error, the following method is also included:

[0030] Calculate the asymmetric amplitude error in the low-frequency component, and perform gradient descent only when the time-domain label amplitude is greater than the predicted amplitude;

[0031] The target loss function is constructed based on the time-domain error, the frequency-domain error, and the asymmetric amplitude error.

[0032] In a preferred embodiment, this application can be further configured such that, in the step of calculating the asymmetric amplitude error of the low-frequency portion, the following formula is used.

[0033]

[0034] In the formula, loss mag_asym Indicates asymmetric amplitude error, label mags Indicates the magnitude of the time-domain label, pred mags Indicates the predicted magnitude, [:f low ] indicates that in f low The asymmetric amplitude error is calculated below the specified frequency.

[0035] In a preferred embodiment, this application can be further configured to include the following steps:

[0036] When calculating the asymmetric amplitude error in the low-frequency component, f low Use 1kHz.

[0037] In a preferred embodiment, this application can be further configured such that, when constructing the target loss function based on the time-domain error, the frequency-domain error, and the asymmetric amplitude error, the following formula is used.

[0038] loss = loss time +loss freq +0.1*loss mag_asym

[0039] In the formula, loss represents the target loss function. time Represents time-domain error, loss freq Represents frequency domain error, loss mag_asym This indicates asymmetric amplitude error.

[0040] Secondly, this application provides a noise reduction device based on a loss function.

[0041] This application is achieved through the following technical solution:

[0042] A noise reduction device based on a loss function includes,

[0043] The data acquisition module is used to acquire the time-domain output sequence and frequency-domain time-frequency plot of the noisy frequency.

[0044] The segmentation module is used to divide the time-domain output sequence and the corresponding time-domain label into output sequence audio segments of different unit durations and corresponding label audio segments of different unit durations, respectively, to obtain output sequence audio groups and label audio groups.

[0045] A time-domain error module is used to calculate the time-domain error based on the output sequence audio group and the tag audio group;

[0046] The frequency domain error module is used to calculate the frequency domain error based on the frequency domain time-frequency diagram, the output sequence audio group, and the tag audio group;

[0047] The loss function module is used to construct a target loss function based on the time-domain error and the frequency-domain error;

[0048] The training module is used to input the noisy frequency into a preset deep learning model, combine it with the target loss function to train the model, output the denoised frequency, and output the trained deep learning model when the preset training conditions are met.

[0049] The noise reduction module is used to estimate the noisy frequencies using a trained deep learning model to obtain the target audio.

[0050] Thirdly, this application provides a noise reduction system.

[0051] This application is achieved through the following technical solution:

[0052] A noise reduction system, comprising,

[0053] The acquisition unit is used to convert speech signals from the environment into digital signals;

[0054] The computing unit is used to receive the output signal of the acquisition unit and execute the steps of any of the above-described loss function-based noise reduction methods.

[0055] A data transmission unit is used to transmit the output signal of the computing unit;

[0056] A terminal unit is used to play the output signal of the data transmission unit.

[0057] In summary, compared with the prior art, the beneficial effects of the technical solution provided in this application include at least the following:

[0058] The model acquires the noisy output sequence and frequency-domain time-frequency plot in the time domain to calculate the time-domain and frequency-domain errors. It segments the time-domain output sequence and its corresponding time-domain label into audio segments of different unit durations and label audio segments of different unit durations, resulting in output sequence audio groups and label audio groups. This allows the model to obtain the difference between the label and the estimated audio at different positions within an audio segment, increasing its robustness to audio of different durations. The model calculates the time-domain error based on the output sequence audio groups and label audio groups, and the frequency-domain error based on the frequency-domain time-frequency plot, output sequence audio groups, and label audio groups. A target loss function is constructed based on the time-domain and frequency-domain errors, which can fully represent the error between the model output and the label. The objective evaluation metrics of the model's loss function meet the requirements. The noisy frequency is input into a pre-defined deep learning model, and the model is trained using the target loss function, outputting the denoised frequency until the pre-defined training conditions are met. The trained deep learning model is then used to estimate the noisy frequency to obtain the target audio. This improves the model's over-suppression of speech signals and enhances its denoising accuracy. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of the main process of a loss function-based denoising method provided as an exemplary embodiment of this application.

[0060] Figure 2 This is a structural block diagram of a noise reduction system provided as another exemplary embodiment of this application.

[0061] Figure 3A structural block diagram of a loss function-based noise reduction device is provided for another exemplary embodiment of this application. Detailed Implementation

[0062] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.

[0063] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0064] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0065] To overcome the problems in existing technologies where time-domain or frequency-domain loss functions alone are insufficient to fully represent the errors of model output and labels, resulting in severe oversuppression of speech signals, poor model noise reduction, and unpleasant actual listening experience, this solution proposes a composite loss function based on asymmetric MSE (mean squared error). This loss function, used in NS (Noise Suppression) scenarios, can simultaneously utilize the time-domain and frequency-domain errors of model output and labels, and adds asymmetric amplitude loss in the low-frequency part. Furthermore, based on the subjective perception of different frequencies by the human ear, the complex-domain MSE is weighted and adjusted to optimize the subjective listening experience, improve the oversuppression of speech signals by the model, and enhance the noise suppression effect.

[0066] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0067] Reference Figure 1 This application provides a noise reduction method based on a loss function, and the main steps of the method are described below.

[0068] S1: Obtain the time-domain output sequence and frequency-domain time-frequency plot of the noisy frequency;

[0069] S2: Divide the time-domain output sequence and the corresponding time-domain label into output sequence audio segments of different unit durations and corresponding label audio segments of different unit durations, respectively, to obtain output sequence audio groups and label audio groups;

[0070] S3: Calculate the temporal error based on the output sequence audio group and the tag audio group;

[0071] S4: Calculate the frequency domain error based on the frequency domain time-frequency diagram, the output sequence audio group, and the tag audio group;

[0072] S51: Construct a target loss function based on the time-domain error and the frequency-domain error;

[0073] S6: Input the noisy frequency into the preset deep learning model, combine it with the target loss function to train the model, output the noise-reduced frequency, until the preset training conditions are met, and output the trained deep learning model.

[0074] S7: Use a trained deep learning model to estimate the noisy frequencies and obtain the target audio.

[0075] Specifically, taking an audio clip tagged with 10 seconds as an example, the sampling rate is set to 16K, the length of the time-domain tag is 160000, the window length of the STFT (Short-Time Fourier Transform) is set to 320, the step size is set to 160, and the number of STFT points is set to 512, to obtain a time-frequency plot with a frequency domain length of 1000*256. The model outputs the time-frequency plot for speech estimation, with a frequency domain output length of 1000*256. After passing through the inverse short-time Fourier transform (iSTFT), a time-domain estimated speech with a length of 160000 is obtained.

[0076] While the model used for speech estimation outputs a time-domain sequence, it also converts the 1000*256 time-frequency data into a time-frequency sequence. Figure 1 It can be saved and output for calculating the complex field MSE.

[0077] The time-domain output sequence and its corresponding time-domain label are segmented into output sequence audio segments of different durations and corresponding label audio segments of different durations. For example, a 10-second time-domain output sequence and its corresponding time-domain label are divided into one 10-second audio segment, ten 1-second audio segments, and twenty 0.5-second audio segments, i.e., output sequence audio segments and label audio segments of different durations. These are then summarized to obtain output sequence audio groups and label audio groups. This allows us to obtain the difference between the label and the estimated audio at different positions within an audio segment, which is used to calculate SI-SNR / SNR segment by segment. This is completely different from the conventional method of directly calculating SI-SNR / SNR using an entire audio segment, and it increases the model's robustness to audio segments of different durations.

[0078] The temporal error is calculated based on the output sequence audio group and the tag audio group.

[0079] In one embodiment, the step of calculating the temporal error based on the output sequence audio group and the tag audio group includes,

[0080] Based on the output sequence audio group and the tag audio group, the scale-invariant signal-to-noise ratio of each output sequence audio segment with a different unit duration and the corresponding tag audio segment is calculated to obtain the segmented scale-invariant signal-to-noise ratio;

[0081] The time-domain error is obtained by summing the scale-invariant signal-to-noise ratios of each scale.

[0082] By calculating the SI-SNR / SNR of output sequence audio segments of different unit durations and their corresponding tag audio segments of different unit durations (e.g., the SI-SNR of 1 10-second audio segment, 10 1-second audio segments, and 20 0.5-second audio segments), i.e., the average SI-SNR error, and then summing the SI-SNR of audio segments of different unit durations, we obtain the total SI-SNR error, which is the time-domain error (temporal domain loss). The calculation formula is as follows:

[0083] loss time = seg_SI_SNR(pred, label)

[0084] In the formula, loss time This indicates the time-domain error, pred represents the output sequence audio group, and label represents the label audio group.

[0085] In addition, the frequency domain error is calculated based on the frequency domain time-frequency plot, the output sequence audio group, and the tag audio group.

[0086] In one embodiment, the step of calculating the frequency domain error based on the frequency domain time-frequency diagram, the output sequence audio group, and the tag audio group includes:

[0087] Based on the output sequence audio group and the tag audio group, and combined with the frequency domain time-frequency diagram, the mean square error is calculated, and the mean square error includes the complex mean square error and the amplitude mean square error.

[0088] The complex mean square error is obtained by adding the real part mean square error of the output sequence audio segments with different unit durations and the corresponding tag audio segments to the imaginary part mean square error of the output sequence audio segments with different unit durations and the corresponding tag audio segments.

[0089] The predicted amplitude is obtained by adding the sum of the squares of the real part mean square errors of the corresponding output sequence audio segments of different unit durations to the sum of the squares of the imaginary part mean square errors of the corresponding output sequence audio segments.

[0090] The temporal tag amplitude is obtained by adding the sum of the squares of the real part mean square errors of the tag audio segments of different unit durations in the tag audio group to the sum of the squares of the imaginary part mean square errors of the corresponding tag audio segments.

[0091] Calculate the mean square error between the predicted amplitude and the temporal label amplitude to obtain the amplitude loss;

[0092] The frequency domain error is obtained based on the complex mean square error and the amplitude loss.

[0093] The MSE is calculated using the frequency domain time-frequency diagram and the time-frequency diagram corresponding to the labeled audio group. The MSE mainly includes the complex MSE and the amplitude MSE.

[0094] The formula for calculating the complex number MSE is as follows:

[0095] loos real =mse(pred real ,label real )#

[0096] loss imag =mse(pred imag ,label imag )#

[0097] loss composite =loss real +loss imag #

[0098] In the formula, loos real Pred represents the real part mean square error between the output sequence audio segments of different unit durations and their corresponding labeled audio segments. real The label represents the real part of the output sequence audio segment. real The loss represents the real part of the labeled audio segment. imag The pred represents the imaginary mean square error between the output sequence audio segments of different unit durations and their corresponding labeled audio segments. imag The label represents the imaginary part of the output sequence audio segment. imag The loss represents the imaginary part of the labeled audio segment. composite It represents the complex mean square error.

[0099] The formula for calculating the amplitude MSE is as follows:

[0100] pred mags =(pred real ) 2 +(pred imag ) 2 #

[0101] label mags=(label) real ) 2 +(lable imag ) 2 #

[0102] loss mags =mse(pred mags ,label mags )#

[0103] loss freq =loss composite +loss mags

[0104] In the formula, pred mags The label represents the predicted amplitude, i.e., the amplitude of the audio group in the output sequence. mags Represents the magnitude of the time-domain label, loss mags Indicates amplitude loss, loss freq This represents the frequency domain error.

[0105] To address the issue that existing models, which rely solely on temporal or frequency domain loss during training, struggle to adequately represent the errors in model output and labels, this approach employs both a Seg-SI-SNR (Segment-Scale-invariant Signal-to-Noise Ratio) loss function to represent temporal error and an asymmetric complex-domain MSE to represent frequency domain error. The temporal error is represented by Seg-SI-SNR, calculated by segmenting the model's output sequence into different unit durations to increase robustness to audio of varying durations. The asymmetric complex-domain MSE consists of three parts: the real MSE, the imaginary MSE, and the amplitude MSE of the estimated output and label. The real and imaginary MSEs are calculated using the real and imaginary parts of the estimated speech and label, respectively. The amplitude MSE is obtained by taking the modulus of the real and imaginary parts of the estimated speech and label. Finally, the weights are adjusted according to specific requirements.

[0106] On the other hand, when calculating the MSE of the complex spectrum, the loss function has the same weight for different frequencies. This does not match the human ear's perception of sound in the frequency domain and does not conform to actual human perception. It results in severe oversuppression, as the human ear is often more sensitive to low-frequency sounds, leading to poor listening experience. In one embodiment, the weight of the complex mean square error is set to 0.9, and the weight of the amplitude loss is set to 0.1. The specific formula is as follows:

[0107] loss freq =0.9 * loss composite +0.1*lossmags

[0108] Setting the weights of the real and imaginary parts to 0.9 and the amplitude MSE weight to 0.1 is an experimentally verified setting scheme that effectively improves the oversuppression of speech signals, optimizes subjective auditory perception, and enhances noise suppression.

[0109] In one embodiment, before the step of constructing the target loss function based on the time-domain error and the frequency-domain error, the method further includes:

[0110] S521: Calculate the asymmetric amplitude error in the low-frequency part, and perform gradient descent only when the time-domain label amplitude is greater than the predicted amplitude;

[0111] S522: Construct a target loss function based on the time-domain error, the frequency-domain error, and the asymmetric amplitude error.

[0112] To improve the subjective listening experience, better preserve the speech signal, and prevent sound quality degradation, this application also adds an asymmetric amplitude loss calculation for the low-frequency portion, taking advantage of the fact that the fundamental frequency of the speech signal is usually distributed in the low frequency range. The low-frequency asymmetric MSE loss function mainly calculates the amplitude, and this loss function only performs gradient descent when the amplitude of the label is greater than the predicted amplitude.

[0113] In one embodiment, the step of calculating the asymmetric amplitude error of the low-frequency portion uses the following formula:

[0114] loss mag_asym =(max(0,(label)) mags [:f low ]-pred mags [:f low ]))) 2

[0115] In the formula, loss mag_asym Indicates asymmetric amplitude error, label mags Indicates the magnitude of the time-domain label, pred mags Indicates the predicted magnitude, [:f low ] indicates that in f low The asymmetric amplitude error is calculated below the specified frequency.

[0116] By introducing asymmetric amplitude loss in the low-frequency part, over-suppression in frequency bands where the predicted amplitude is less than the label amplitude is avoided to some extent. This prevents the estimated amplitude of the speech signal in the low-frequency part from being less than the amplitude of the label speech signal, thus preventing over-suppression in the low-frequency part and effectively reducing the over-suppression of the speech signal in the low-frequency part of the system.

[0117] In one embodiment, the following steps are also included:

[0118] When calculating the asymmetric amplitude error in the low-frequency range, f low Use 1kHz.

[0119] By calculating the asymmetric MSE loss for the portion below 1kHz, it is better able to meet the needs of practical application scenarios and also improves the computational efficiency of the model.

[0120] In one embodiment, the target loss function is constructed based on the time-domain error, the frequency-domain error, and the asymmetric amplitude error using the following formula:

[0121] loss = loss time +loss freq +0.1*loss mag_asym

[0122] In the formula, loss represents the target loss function. time Represents time-domain error, loss freq Represents frequency domain error, loss mag_asym This indicates asymmetric amplitude error.

[0123] Finally, the noisy frequency is input into the preset deep learning model, and the model is trained in combination with the target loss function to output the denoised frequency. When the preset training conditions are met, the trained deep learning model is output to estimate the noisy frequency and obtain the target audio.

[0124] The noisy and denoising frequencies of the model can come from the training dataset used to train the denoising model. The input is noisy human voice, and the output is clean human voice estimated by the AI ​​model.

[0125] In this embodiment, the deep learning model can adopt DeepMMSE (Deep Minimum Mean Squared Error) or CRN (Convolutional Recurrent Network), which can effectively enhance speech in various noisy environments, enabling the model to effectively reduce noise in both the frequency and time domains.

[0126] In summary, a loss function-based denoising method obtains the noisy time-domain output sequence and frequency-domain time-frequency plot for calculating time-domain and frequency-domain errors. It segments the time-domain output sequence and its corresponding time-domain labels into audio segments of different unit durations and label audio segments of different unit durations, obtaining output sequence audio groups and label audio groups. This allows the model to obtain the difference between the label and the estimated audio at different positions within an audio segment, increasing the model's robustness to audio of different durations. The method calculates the time-domain error based on the output sequence audio groups and label audio groups, and the error is calculated based on the frequency-domain time-frequency plot and the output... Frequency domain errors are calculated for both the sequence audio group and the labeled audio group. A target loss function is constructed based on the time domain error and the frequency domain error, which can fully represent the error between the model output and the label. The objective evaluation index of the model's loss function meets the requirements. Noisy frequencies are input into a pre-set deep learning model, and the model is trained in combination with the target loss function. Noise-reduced frequencies are output until the pre-set training conditions are met, at which point the trained deep learning model is output. The trained deep learning model is used to estimate the noisy frequencies to obtain the target audio. This improves the over-suppression phenomenon of the model for speech signals and enhances the noise reduction accuracy of the model.

[0127] A loss function-based denoising method employs time-domain and frequency-domain loss functions to iteratively optimize the model for convergence. In the time-domain loss function, SI-SNR is calculated after audio segmentation to increase the model's robustness to audio of different durations. In the frequency-domain loss function, real part MSE, imaginary part MSE, and amplitude MSE are used simultaneously to improve the over-suppression of speech signals, effectively alleviating the problem of speech over-suppression in deep learning and resulting in better denoising performance. Furthermore, to prevent speech over-suppression, low-frequency amplitude asymmetric MSE loss calculation is introduced to improve subjective speech quality.

[0128] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0129] Reference Figure 2 This application also provides a noise reduction system, including,

[0130] The acquisition unit is used to convert speech signals from the environment into digital signals;

[0131] The computing unit is used to receive the output signal of the acquisition unit and execute the steps of any of the above-described loss function-based noise reduction methods.

[0132] A data transmission unit is used to transmit the output signal of the computing unit;

[0133] A terminal unit is used to play the output signal of the data transmission unit.

[0134] The acquisition unit can consist of a microphone and an ADC hardware chip, which is used to convert the voice signals in the environment into digital signals that are easy for computers to store and calculate.

[0135] The computing unit can consist of a single microcontroller or a computing chip with an operating system, and is used for training and inference of AI models.

[0136] The data transmission unit can consist of a data-transmitting network system used to transmit computed voice data.

[0137] The terminal unit can consist of any real-time conferencing communication device with network access, used to play processed voice data.

[0138] The entire system can be flexibly deployed in real-time conferencing communication equipment to achieve real-time noise reduction and suppression of voice.

[0139] Reference Figure 3 This application also provides a loss function-based denoising device, which corresponds one-to-one with the loss function-based denoising method described in the above embodiments. This loss function-based denoising device includes...

[0140] The data acquisition module is used to acquire the time-domain output sequence and frequency-domain time-frequency plot of the noisy frequency.

[0141] The segmentation module is used to divide the time-domain output sequence and the corresponding time-domain label into output sequence audio segments of different unit durations and corresponding label audio segments of different unit durations, respectively, to obtain output sequence audio groups and label audio groups.

[0142] A time-domain error module is used to calculate the time-domain error based on the output sequence audio group and the tag audio group;

[0143] The frequency domain error module is used to calculate the frequency domain error based on the frequency domain time-frequency diagram, the output sequence audio group, and the tag audio group;

[0144] The loss function module is used to construct a target loss function based on the time-domain error and the frequency-domain error;

[0145] The training module is used to input the noisy frequency into a preset deep learning model, combine it with the target loss function to train the model, output the denoised frequency, and output the trained deep learning model when the preset training conditions are met.

[0146] The noise reduction module is used to estimate the noisy frequencies using a trained deep learning model to obtain the target audio.

[0147] A noise reduction device based on a loss function also includes,

[0148] The asymmetric amplitude error module is used to calculate the asymmetric amplitude error in the low-frequency part, and gradient descent is only performed when the time-domain label amplitude is greater than the predicted amplitude.

[0149] The loss function module constructs a target loss function based on the time-domain error, the frequency-domain error, and the asymmetric amplitude error.

[0150] For specific limitations on a loss function-based denoising device, please refer to the limitations on a loss function-based denoising method mentioned above, which will not be repeated here.

[0151] The modules in the aforementioned loss function-based noise reduction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0152] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements any of the aforementioned loss function-based noise reduction methods.

[0153] In one embodiment, a computer-readable storage medium is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the above-described loss function-based denoising methods.

[0154] In one embodiment, a computer program product is provided, the computer program product including a computer program that, when executed by a processor, implements any of the above-described loss function-based noise reduction methods.

[0155] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. When executed, the computer program may include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0156] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A denoising method based on a loss function, characterized in that, Includes the following steps, Obtain the time-domain output sequence and frequency-domain time-frequency plot of the noisy frequency; The time-domain output sequence and the corresponding time-domain label are respectively divided into output sequence audio segments of different unit durations and corresponding label audio segments of different unit durations to obtain output sequence audio groups and label audio groups; Calculate the temporal error based on the output sequence audio group and the tag audio group; as well as The frequency domain error is calculated based on the frequency domain time-frequency plot, the output sequence audio group, and the tag audio group. Specifically, this includes: calculating the mean square error (MSE) based on the output sequence audio group and the tag audio group, combined with the frequency domain time-frequency plot. The MSE includes a complex MSE and an amplitude MSE. The complex MSE is obtained by adding the real part MSE of the output sequence audio segments of different unit durations to the imaginary part MSE of the corresponding tag audio segments. The predicted amplitude is obtained by adding the sum of the squares of the real part MSEs of the output sequence audio segments of different unit durations to the sum of the squares of the imaginary part MSEs of the corresponding output sequence audio segments. The time domain tag amplitude is obtained by adding the sum of the squares of the real part MSEs of the tag audio segments of different unit durations to the sum of the squares of the imaginary part MSEs of the corresponding tag audio segments. The amplitude loss is obtained by calculating the MSE of the predicted amplitude and the time domain tag amplitude. The frequency domain error is obtained based on the complex MSE and the amplitude loss. Construct a target loss function based on the time-domain error and the frequency-domain error; The noisy frequency is input into a preset deep learning model, and the model is trained by combining the target loss function. The denoised frequency is output until the preset training conditions are met, and then the trained deep learning model is output. The target audio is obtained by estimating the noisy frequencies using a trained deep learning model.

2. The denoising method based on a loss function according to claim 1, characterized in that, The steps for calculating the temporal error based on the output sequence audio group and the tag audio group include: Based on the output sequence audio group and the tag audio group, the scale-invariant signal-to-noise ratio of each output sequence audio segment with a different unit duration and the corresponding tag audio segment is calculated to obtain the segmented scale-invariant signal-to-noise ratio; The time-domain error is obtained by summing the scale-invariant signal-to-noise ratios of each scale.

3. The denoising method based on a loss function according to claim 1, characterized in that, The frequency domain error is obtained using the following formula based on the complex mean square error and the amplitude loss. In the formula, Indicates frequency domain error. This represents the complex mean square error. This indicates a loss in magnitude.

4. The denoising method based on a loss function according to any one of claims 1-3, characterized in that, Before the step of constructing the target loss function based on the time-domain error and the frequency-domain error, the method further includes: Calculate the asymmetric amplitude error in the low-frequency component, and perform gradient descent only when the time-domain label amplitude is greater than the predicted amplitude; The target loss function is constructed based on the time-domain error, the frequency-domain error, and the asymmetric amplitude error.

5. The denoising method based on a loss function according to claim 4, characterized in that, The following formula is used in the step of calculating the asymmetric amplitude error of the low-frequency component. In the formula, Indicates asymmetric amplitude error. Indicates the amplitude of the time-domain label. Indicates the predicted magnitude. Indicates in The asymmetric amplitude error is calculated below the specified frequency.

6. The denoising method based on a loss function according to claim 5, characterized in that, It also includes the following steps, When calculating the asymmetric amplitude error of the low-frequency component... use Hz.

7. The denoising method based on a loss function according to claim 4, characterized in that, When constructing the target loss function based on the time-domain error, the frequency-domain error, and the asymmetric amplitude error, the following formula is used. In the formula, Represents the target loss function. Indicates time-domain error, Indicates frequency domain error. This indicates asymmetric amplitude error.

8. A noise reduction device based on a loss function, characterized in that, include, The data acquisition module is used to acquire the time-domain output sequence and frequency-domain time-frequency plot of the noisy frequency. The segmentation module is used to divide the time-domain output sequence and the corresponding time-domain label into output sequence audio segments of different unit durations and corresponding label audio segments of different unit durations, respectively, to obtain output sequence audio groups and label audio groups. A time-domain error module is used to calculate the time-domain error based on the output sequence audio group and the tag audio group; The frequency domain error module is used to calculate the frequency domain error based on the frequency domain time-frequency diagram, the output sequence audio group, and the tag audio group. Specifically, it includes: calculating the mean square error based on the output sequence audio group and the tag audio group, combined with the frequency domain time-frequency diagram. The mean square error includes a complex mean square error and an amplitude mean square error; adding the real part mean square error of the output sequence audio segment with the corresponding tag audio segment of different unit durations to the imaginary part mean square error of the output sequence audio segment with the corresponding tag audio segment of different unit durations to obtain the complex mean square error; and using the... The predicted amplitude is obtained by adding the sum of the squared real-part mean squared errors of the corresponding output audio segments of different unit durations in the output audio sequence group to the sum of the squared imaginary-part mean squared errors of the corresponding output audio segments. The temporal label amplitude is obtained by adding the sum of the squared real-part mean squared errors of the corresponding imaginary-part mean squared errors of the label audio segments of the label audio group. The amplitude loss is obtained by calculating the mean squared errors of the predicted amplitude and the temporal label amplitude. The frequency domain error is obtained based on the complex mean squared error and the amplitude loss. The loss function module is used to construct a target loss function based on the time-domain error and the frequency-domain error; The training module is used to input the noisy frequency into a preset deep learning model, combine it with the target loss function to train the model, output the denoised frequency, and output the trained deep learning model when the preset training conditions are met. The noise reduction module is used to estimate the noisy frequencies using a trained deep learning model to obtain the target audio.

9. A noise reduction system, characterized in that, include, The acquisition unit is used to convert speech signals from the environment into digital signals; A computing unit is configured to receive the output signal of the acquisition unit and execute the steps of the noise reduction method based on the loss function as described in any one of claims 1 to 7. A data transmission unit is used to transmit the output signal of the computing unit; A terminal unit is used to play the output signal of the data transmission unit.

Images