Noise reduction method, device and system based on loss function

Abstract

The invention relates to the technical field of audio processing, and discloses a noise reduction method, device and system based on a loss function, and the method comprises the steps: obtaining a time domain output sequence with noise frequency and a frequency domain time-frequency graph; respectively segmenting the time domain output sequence and the corresponding time domain label into output sequence audio segments with different unit durations and corresponding label audio segments with different unit durations to obtain an output sequence audio group and a label audio group; calculating a time domain error based on the output sequence audio group and the tag audio group; and calculating a frequency domain error based on the frequency domain time-frequency graph, the output sequence audio group and the tag audio group; constructing a target loss function; outputting the trained deep learning model; and estimating the audio with noise to obtain a target audio. According to the invention, the over-suppression phenomenon of the model on the voice signal can be improved, and the noise reduction precision of the model is improved.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a noise reduction method, device and noise reduction system based on a loss function. Background Art

[0002] At present, audio noise suppression based on deep learning mainly consists of two stages: training and inference. In the training stage, the output is obtained through the model based on the input data, the error is calculated with the label using the loss function, and the model is back-propagated iteratively based on the error to finally obtain the inference model; the inference stage is to perform inference based on the inference model obtained in the training stage.

[0003] However, in existing solutions, the loss function of the model mostly uses MSE (Mean Square Error), SNR (Signal-to-Noise Ratio) or SI-SNR (Scale-invariant Signal-to-Noise Ratio). The features used to calculate the error are often amplitude spectrum, complex spectrum or time domain signal, and the error is always calculated only in the time domain or frequency domain, which makes it difficult to fully represent the error of the estimated audio and label, resulting in a low objective evaluation index of the loss function of the training model; in addition, in practical applications, in order to better suppress audio noise, the speech signal is often over-suppressed, which also leads to a low objective evaluation index of the loss function during the training of the existing model, and then the trained model has serious over-suppression of the speech signal, the model deviation is large, and the noise reduction accuracy is reduced.

[0004] With respect to the above-mentioned related technologies, the inventors have found that the existing audio noise reduction methods have the problems of severe over-suppression and low noise reduction accuracy. Summary of the invention

[0005] In order to improve the over-suppression phenomenon of speech signals and enhance the accuracy of noise reduction, the present application provides a noise reduction method, device and noise reduction system based on a loss function.

[0006] In a first aspect, the present application provides a denoising method based on a loss function.

[0007] This application is achieved through the following technical solutions:

[0008] A denoising method based on a loss function comprises the following steps:

[0009] Get the time domain output sequence and frequency domain spectrogram with noise frequency;

[0010] Respectively dividing the time-domain output sequence and the corresponding time-domain label into output sequence audio segments of different unit durations and corresponding label audio segments of different unit durations to obtain an output sequence audio group and a label audio group;

[0011] Calculating a time domain error based on the output sequence audio group and the label audio group; and

[0012] Calculate a frequency domain error based on the frequency domain spectrogram, the output sequence audio group, and the label audio group;

[0013] Constructing a target loss function according to the time domain error and the frequency domain error;

[0014] Input the noisy frequency into a preset deep learning model, perform model training in combination with the target loss function, output the noise-reduced frequency, and output the trained deep learning model when the preset training conditions are met;

[0015] The trained deep learning model is used to estimate the noisy frequency and obtain the target audio.

[0016] In a preferred example, the present application may be further configured as follows: the step of calculating the time domain error based on the output sequence audio group and the label audio group includes:

[0017] According to the output sequence audio group and the label audio group, the scale-invariant signal-to-noise ratio of the output sequence audio segments of different unit time lengths and the corresponding label audio segments is calculated to obtain a piecewise scale-invariant signal-to-noise ratio;

[0018] The time domain error is obtained by summing the scale-invariant signal-to-noise ratios.

[0019] In a preferred example, the present application may be further configured as follows: the step of calculating the frequency domain error based on the frequency domain time-frequency diagram, the output sequence audio group and the label audio group includes:

[0020] According to the output sequence audio group and the label audio group, combined with the frequency domain time-frequency diagram, a mean square error is calculated, wherein the mean square error includes a complex mean square error and an amplitude mean square error;

[0021] The complex mean square error is obtained by adding the real mean square error between the output sequence audio segments of different unit time lengths and the corresponding label audio segments to the imaginary mean square error between the output sequence audio segments of different unit time lengths and the corresponding label audio segments;

[0022] The prediction amplitude is obtained by adding the square sum of the real mean square errors of the output sequence audio segments of different unit time lengths of the output sequence audio group to the square sum of the imaginary mean square errors of the corresponding output sequence audio segments;

[0023] The time domain label amplitude is obtained by adding the square sum of the real mean square errors of the label audio segments of different unit time lengths of the label audio group to the square sum of the imaginary mean square errors of the corresponding label audio segments;

[0024] Calculating the mean square error between the predicted amplitude and the time domain label amplitude to obtain the amplitude loss;

[0025] The frequency domain error is obtained based on the complex mean square error and the amplitude loss.

[0026] In a preferred example, the present application may be further configured as follows: when obtaining the frequency domain error based on the complex mean square error and the amplitude loss, the following formula is used:

[0027] loss freq =0.9*loss composite +0.1*loss mags

[0028] In the formula, loss freq Represents the frequency domain error, loss composite Represents the complex mean square error, loss mags Indicates amplitude loss.

[0029] In a preferred example, the present application may be further configured as follows: before the step of constructing the target loss function according to the time domain error and the frequency domain error, the step further includes:

[0030] Calculate the asymmetric amplitude error of the low-frequency part and perform gradient descent only when the time domain label amplitude is greater than the predicted amplitude;

[0031] A target loss function is constructed according to the time domain error, the frequency domain error and the asymmetric amplitude error.

[0032] In a preferred example, the present application may be further configured as follows: in the step of calculating the asymmetric amplitude error of the low-frequency part, the following formula is used:

[0033]

[0034] In the formula, loss mag_asym Indicates asymmetric amplitude error, label mags Indicates the time domain label amplitude, pred mags represents the prediction amplitude, [:f low ] indicates that in f low The calculation of the asymmetric amplitude error is performed below the frequency.

[0035] In a preferred example, the present application can be further configured as follows: it also includes the following steps:

[0036] When calculating the asymmetric amplitude error of the low-frequency part, f low Using 1kHz.

[0037] In a preferred example, the present application can be further configured as follows: when constructing a target loss function according to the time domain error, the frequency domain error and the asymmetric amplitude error, the following formula is used:

[0038] loss=loss time +loss freq +0.1*loss mag_asym

[0039] In the formula, loss represents the target loss function, loss time Represents the time domain error, loss freq Represents the frequency domain error, loss mag_asym Indicates the asymmetric amplitude error.

[0040] In a second aspect, the present application provides a noise reduction device based on a loss function.

[0041] This application is achieved through the following technical solutions:

[0042] A noise reduction device based on a loss function, comprising:

[0043] A data acquisition module is used to obtain a time domain output sequence with noise frequency and a frequency domain time-frequency diagram;

[0044] A segmentation module, used to segment the time domain output sequence and the corresponding time domain label into output sequence audio segments of different unit time lengths and corresponding label audio segments of different unit time lengths, respectively, to obtain an output sequence audio group and a label audio group;

[0045] A time domain error module, configured to calculate a time domain error based on the output sequence audio group and the label audio group;

[0046] A frequency domain error module, used for calculating a frequency domain error based on the frequency domain time-frequency diagram, the output sequence audio group and the label audio group;

[0047] A loss function module, used to construct a target loss function according to the time domain error and the frequency domain error;

[0048] A training module, used to input the noisy frequency into a preset deep learning model, perform model training in combination with the target loss function, and output the noise-reduced frequency, until the preset training conditions are met, and then output the trained deep learning model;

[0049] The denoising module is used to estimate the noisy frequency using the trained deep learning model to obtain the target audio.

[0050] In a third aspect, the present application provides a noise reduction system.

[0051] This application is achieved through the following technical solutions:

[0052] A noise reduction system, comprising:

[0053] A collection unit, used for converting voice signals in the environment into digital signals;

[0054] A computing unit, configured to receive an output signal of the acquisition unit and execute any one of the steps of the above-mentioned denoising method based on the loss function;

[0055] A data transmission unit, used for transmitting the output signal of the calculation unit;

[0056] The terminal unit is used to play the output signal of the data transmission unit.

[0057] In summary, compared with the prior art, the technical solution provided by this application has at least the following beneficial effects:

[0058] The time domain output sequence and frequency domain time-frequency graph of the noisy audio are obtained for calculating the time domain error and the frequency domain error; the time domain output sequence and the corresponding time domain label are respectively divided into output sequence audio segments of different unit time lengths and corresponding label audio segments of different unit time lengths to obtain output sequence audio groups and label audio groups, thereby obtaining the difference between labels and estimated audios at different positions in a segment of audio, thereby increasing the robustness of the model to audios of different time lengths; the time domain error is calculated based on the output sequence audio group and the label audio group, and the frequency domain error is calculated based on the frequency domain time-frequency graph, the output sequence audio group and the label audio group, and the target loss function is constructed according to the time domain error and the frequency domain error, which can fully represent the error between the model output and the label, and the objective evaluation index of the loss function of the model can meet the requirements; the noisy audio is input into a preset deep learning model, and the model is trained in combination with the target loss function, and the noise-reduced audio is output until the preset training conditions are met, and the trained deep learning model is output; the noisy audio is estimated using the trained deep learning model to obtain the target audio; the over-suppression phenomenon of the model on the speech signal is improved, and the noise reduction accuracy of the model is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of the main process of a denoising method based on a loss function is provided for an exemplary embodiment of the present application.

[0060] Figure 2 A structural block diagram of a noise reduction system provided for yet another exemplary embodiment of the present application.

[0061] Figure 3A structural block diagram of a noise reduction device based on a loss function is provided for another exemplary embodiment of the present application. DETAILED DESCRIPTION

[0062] This specific embodiment is merely an explanation of the present application and is not a limitation of the present application. After reading this specification, those skilled in the art may make modifications to the present embodiment without any creative contribution as needed, but such modifications are protected by the patent law as long as they are within the scope of the claims of the present application.

[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0064] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0065] In order to overcome the problems in the prior art that the time domain or frequency domain loss function alone cannot fully represent the errors of the model output and the label, resulting in serious over-suppression of the speech signal, poor model noise reduction effect, and poor actual listening experience, this solution proposes a composite loss function based on asymmetric MSE (mean square error). This loss function is used in the NS (Noise Suppression) scenario. It can simultaneously use the time domain and frequency domain errors of the model output and the label, and increase the asymmetric amplitude loss of the low-frequency part. It also weights the complex domain MSE based on the subjective perception of the human ear for different frequencies, thereby optimizing the subjective auditory perception, improving the over-suppression phenomenon of the model on the speech signal, and improving the noise suppression effect.

[0066] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0067] Reference Figure 1 , an embodiment of the present application provides a denoising method based on a loss function, and the main steps of the method are described as follows.

[0068] S1: Obtain the time domain output sequence with noise frequency and the frequency domain time-frequency diagram;

[0069] S2: respectively dividing the time domain output sequence and the corresponding time domain label into output sequence audio segments of different unit durations and corresponding label audio segments of different unit durations to obtain an output sequence audio group and a label audio group;

[0070] S3: Calculating a time domain error based on the output sequence audio group and the label audio group;

[0071] S4: Calculating a frequency domain error based on the frequency domain time-frequency diagram, the output sequence audio group and the label audio group;

[0072] S51: constructing a target loss function according to the time domain error and the frequency domain error;

[0073] S6: inputting the noisy frequency into a preset deep learning model, performing model training in combination with the target loss function, and outputting the noise-reduced frequency, until the preset training conditions are met, and then outputting the trained deep learning model;

[0074] S7: Use the trained deep learning model to estimate the noisy frequency and obtain the target audio.

[0075] Specifically, taking the audio with a label of 10s as an example, the sampling rate is set to 16K, the length of the time domain label is 160000, the window length of STFT (Short-Time Fourier Transform) is set to 320, the step size is set to 160, and the number of STFT points is set to 512, so as to obtain a time-frequency diagram with a frequency domain of 1000*256. The model outputs a time-frequency diagram for estimating speech, and the frequency domain output length is 1000*256. After the inverse short-time Fourier transform iSTFT, the time domain estimated speech with a length of 160000 is obtained.

[0076] While the model used to estimate speech outputs the time domain sequence, the 1000*256 time-frequency Figure 1 Save and output together for calculating complex domain MSE.

[0077] The time domain output sequence and the corresponding time domain label are respectively divided into output sequence audio segments of different unit durations and corresponding label audio segments of different unit durations. For example, the time domain output sequence of 10s duration and the corresponding time domain label are divided into 1 segment of 10s audio, 10 segments of 1s audio and 20 segments of 0.5s audio, that is, output sequence audio segments of different unit durations and label audio segments of different unit durations, and the output sequence audio group and the label audio group are summarized. In this way, the difference between the label and the estimated audio at different positions in a segment of audio can be obtained, which is used to calculate SI-SNR / SNR in segments, which is completely different from the conventional method of directly using a whole segment of audio to calculate SI-SNR / SNR, and can increase the robustness of the model to audio of different durations.

[0078] The time domain error is calculated based on the output sequence audio group and the label audio group.

[0079] In one embodiment, the step of calculating the time domain error based on the output sequence audio group and the label audio group includes:

[0080] According to the output sequence audio group and the label audio group, the scale-invariant signal-to-noise ratio of the output sequence audio segments of different unit time lengths and the corresponding label audio segments is calculated to obtain a piecewise scale-invariant signal-to-noise ratio;

[0081] The time domain error is obtained by summing the scale-invariant signal-to-noise ratios.

[0082] By calculating the SI-SNR / SNR of the output sequence audio segments of different unit time lengths and the corresponding label audio segments of different unit time lengths, such as the SI-SNR of 1 segment of 10s audio, 10 segments of 1s audio, and 20 segments of 0.5s audio, that is, the average SI-SNR error, and then summing the SI-SNR of the audios of different unit time lengths, we can get the total SI-SNR error, that is, the time domain error (time domain loss), which is calculated as follows:

[0083] loss time =seg_SI_SNR(pred,label)

[0084] In the formula, loss time Represents the time domain error, pred represents the output sequence audio group, and lable represents the label audio group.

[0085] And, calculating the frequency domain error based on the frequency domain spectrogram, the output sequence audio group and the label audio group.

[0086] In one embodiment, the step of calculating the frequency domain error based on the frequency domain time-frequency diagram, the output sequence audio group and the label audio group includes:

[0087] According to the output sequence audio group and the label audio group, combined with the frequency domain time-frequency diagram, a mean square error is calculated, wherein the mean square error includes a complex mean square error and an amplitude mean square error;

[0088] The complex mean square error is obtained by adding the real mean square error between the output sequence audio segments of different unit time lengths and the corresponding label audio segments to the imaginary mean square error between the output sequence audio segments of different unit time lengths and the corresponding label audio segments;

[0089] The prediction amplitude is obtained by adding the square sum of the real mean square errors of the output sequence audio segments of different unit time lengths of the output sequence audio group to the square sum of the imaginary mean square errors of the corresponding output sequence audio segments;

[0090] The time domain label amplitude is obtained by adding the square sum of the real mean square errors of the label audio segments of different unit time lengths of the label audio group to the square sum of the imaginary mean square errors of the corresponding label audio segments;

[0091] Calculating the mean square error between the predicted amplitude and the time domain label amplitude to obtain the amplitude loss;

[0092] The frequency domain error is obtained based on the complex mean square error and the amplitude loss.

[0093] The MSE is calculated through the frequency domain time-frequency diagram and the time-frequency diagram corresponding to the labeled audio group, mainly including complex MSE and amplitude MSE.

[0094] The calculation formula for complex MSE is as follows:

[0095] loos real = mse(pred real ,label real )#

[0096] loss imag = mse(pred imag ,label imag )#

[0097] loss composite =loss real +loss imag #

[0098] In the formula, loos real Represents the real mean square error between the output sequence audio segments of different unit lengths and the corresponding label audio segments, pred real Represents the real part of the output sequence audio segment, label real Represents the real part of the labeled audio segment, loss imag Represents the imaginary mean square error between the output sequence audio segments of different unit lengths and the corresponding label audio segments, pred imag Represents the imaginary part of the output sequence audio segment, label imag Represents the imaginary part of the labeled audio segment, loss composite represents the complex mean square error.

[0099] The calculation formula for the amplitude MSE is as follows:

[0100] pred mags =(pred real ) 2 +(pred imag ) 2 #

[0101] label mags=(lable real ) 2 +(lable imag ) 2 #

[0102] loss mags = mse(pred mags ,label mags )#

[0103] loss freq =loss composite +loss mags

[0104] Where pred mags Indicates the predicted amplitude, that is, the amplitude of the output sequence audio group, lable mags Indicates the time domain label amplitude, loss mags Indicates amplitude loss, loss freq represents the frequency domain error.

[0105] In view of the problem that the existing models use only time domain loss or frequency domain loss during training, which makes it difficult to fully represent the errors of model output and labels, this solution uses the Seg-SI-SNR (Segment-Scale-invariant Signal-to-Noise Ratio) loss function to represent the time domain error and the asymmetric complex domain MSE to represent the frequency domain error. The time domain error is represented by Seg-SI-SNR (segment-scale-invariant signal-to-noise ratio), and the output sequence of the model is divided into different unit time lengths to calculate SI-SNR, and the segmented calculation is performed to increase the robustness of the model to audio of different time lengths; the asymmetric complex domain MSE mainly consists of the real MSE, imaginary MSE, and amplitude MSE of the estimated output and label. The real MSE and imaginary MSE are mainly calculated using the real and imaginary parts of the estimated speech and the real and imaginary parts of the label. The amplitude MSE is obtained by taking the modulus of the real and imaginary parts of the estimated language and label respectively. Finally, the weight is adjusted according to actual needs.

[0106] On the other hand, when calculating the MSE of the complex spectrum, the weights of the loss functions of different frequencies are the same, which is not compatible with the human ear's perception of sound in the frequency domain, does not conform to the actual human ear perception, and has serious over-suppression. The human ear is often more sensitive to low-frequency sounds, resulting in poor hearing. In one embodiment, the weight of the complex mean square error is set to 0.9, and the weight of the amplitude loss is set to 0.1. The specific formula is as follows:

[0107] loss freq =0.9*loss composite +0.1*lossmags

[0108] Setting the weight of the real and imaginary parts to 0.9 and the weight of the amplitude MSE to 0.1 is a setting scheme verified by experiments to effectively improve the over-suppression of speech signals, optimize the subjective auditory experience, and enhance the noise suppression effect.

[0109] In one embodiment, before the step of constructing a target loss function according to the time domain error and the frequency domain error, the method further includes:

[0110] S521: Calculate the asymmetric amplitude error of the low-frequency part, and perform gradient descent only when the time domain label amplitude is greater than the predicted amplitude;

[0111] S522: Construct a target loss function based on the time domain error, the frequency domain error and the asymmetric amplitude error.

[0112] In order to improve the subjective listening effect, better preserve the speech signal, and prevent the deterioration of sound quality, this application also adds an asymmetric amplitude loss calculation for the low-frequency part based on the fact that the fundamental frequency of the speech signal is usually distributed in the low frequency. The low-frequency asymmetric MSE loss function mainly calculates the amplitude, and the loss function performs gradient descent only when the amplitude of the label is greater than the predicted amplitude.

[0113] In one embodiment, the step of calculating the asymmetric amplitude error of the low-frequency part uses the following formula:

[0114] loss mag_asym =(max(0,(lable mags [:f low ]-pred mags [:f low ]))) 2

[0115] In the formula, loss mag_asym Indicates asymmetric amplitude error, label mags Indicates the time domain label amplitude, pred mags represents the prediction amplitude, [:f low ] indicates that in f low The calculation of the asymmetric amplitude error is performed below the frequency.

[0116] By introducing asymmetric amplitude loss in the low-frequency part, over-suppression in the frequency band where the predicted amplitude is smaller than the label amplitude can be avoided to a certain extent, and the estimated speech signal amplitude in the low-frequency part can be prevented from being smaller than the label speech signal amplitude, and over-suppression in the low-frequency part can be prevented, which can effectively reduce the system's over-suppression of the speech signal in the low-frequency part.

[0117] In one embodiment, the following steps are also included:

[0118] When calculating the asymmetric amplitude error of the low-frequency part, f low Using 1kHz.

[0119] By calculating the asymmetric MSE loss for the part below 1Khz, it can better meet the actual application scenarios and improve the calculation efficiency of the model.

[0120] In one embodiment, when constructing a target loss function based on the time domain error, the frequency domain error and the asymmetric amplitude error, the following formula is used:

[0121] loss=loss time +loss freq +0.1*loss mag_asym

[0122] In the formula, loss represents the target loss function, loss time Represents the time domain error, loss freq Represents the frequency domain error, loss mag_asym Indicates the asymmetric amplitude error.

[0123] Finally, the noisy audio is input into a preset deep learning model, and the model is trained in combination with the target loss function to output the noise-reduced audio. When the preset training conditions are met, the trained deep learning model is output to estimate the noisy audio and obtain the target audio.

[0124] The noisy audio and denoised audio of the model can come from the training data set used to train the denoising model. The input is the noisy human voice, and the output is the clean human voice estimated by the AI ​​model.

[0125] In this embodiment, the deep learning model can use DeepMMSE (Deep Minimum Mean Squared Error) or CRN (Convolutional Recurrent Network), which can well enhance speech in various noise environments, so that the model can effectively reduce noise in the frequency domain and time domain.

[0126] In summary, a denoising method based on a loss function obtains a time domain output sequence and a frequency domain time-frequency diagram of a noisy audio frequency to calculate the time domain error and the frequency domain error; the time domain output sequence and the corresponding time domain label are respectively divided into output sequence audio segments of different unit time lengths and corresponding label audio segments of different unit time lengths to obtain output sequence audio groups and label audio groups, thereby obtaining the difference between labels and estimated audios at different positions in an audio segment, thereby increasing the robustness of the model to audios of different time lengths; the time domain error is calculated based on the output sequence audio group and the label audio group, and the frequency domain time-frequency diagram, the output The frequency domain error is calculated for the sequence audio group and the label audio group, and the target loss function is constructed according to the time domain error and the frequency domain error, which can fully represent the error between the model output and the label, and the objective evaluation index of the model's loss function can meet the requirements; the noisy audio is input into the preset deep learning model, and the model is trained in combination with the target loss function, and the noise-reduced audio is output until the preset training conditions are met, and the trained deep learning model is output; the noisy audio is estimated using the trained deep learning model to obtain the target audio; the over-suppression phenomenon of the model on the speech signal is improved, and the noise reduction accuracy of the model is improved.

[0127] A denoising method based on loss function uses loss functions in time domain and frequency domain to perform convergence iterative optimization on the model. In the time domain loss function part, SI-SNR calculation is performed after the audio is segmented to increase the robustness of the model to audio of different durations; in the frequency domain loss function part, real MSE, imaginary MSE and amplitude MSE are used at the same time to improve the over-suppression of speech signals, which can effectively alleviate the problem of speech over-suppression in deep learning, and the denoising effect of the model is better; and in order to prevent speech over-suppression, low-frequency amplitude asymmetric MSE loss calculation is introduced to improve subjective speech quality.

[0128] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0129] Reference Figure 2 , the embodiment of the present application also provides a noise reduction system, including,

[0130] A collection unit, used for converting voice signals in the environment into digital signals;

[0131] A computing unit, configured to receive an output signal of the acquisition unit and execute any one of the steps of the above-mentioned denoising method based on the loss function;

[0132] A data transmission unit, used for transmitting the output signal of the calculation unit;

[0133] The terminal unit is used to play the output signal of the data transmission unit.

[0134] Among them, the acquisition unit can be composed of 1 microphone and 1 ADC hardware chip, which is used to convert the voice signal in the environment into a digital signal that is easy for computer storage and calculation.

[0135] The computing unit can consist of a single-chip microcomputer or a computing chip with an operating system, which is used for training and reasoning of AI models.

[0136] The data transmission unit may be composed of a network system capable of transmitting data, and is used for transmitting the voice data after calculation.

[0137] The terminal unit can be composed of any real-time conference communication equipment with network access, and is used to play the processed voice data.

[0138] The entire system can be flexibly deployed in real-time conference communication equipment to achieve real-time noise reduction and suppression of voice.

[0139] Reference Figure 3 The embodiment of the present application also provides a denoising device based on a loss function, which corresponds one-to-one to a denoising method based on a loss function in the above embodiment. The denoising device based on a loss function includes:

[0140] A data acquisition module is used to obtain a time domain output sequence with noise frequency and a frequency domain time-frequency diagram;

[0141] A segmentation module, used to segment the time domain output sequence and the corresponding time domain label into output sequence audio segments of different unit time lengths and corresponding label audio segments of different unit time lengths, respectively, to obtain an output sequence audio group and a label audio group;

[0142] A time domain error module, configured to calculate a time domain error based on the output sequence audio group and the label audio group;

[0143] A frequency domain error module, used for calculating a frequency domain error based on the frequency domain time-frequency diagram, the output sequence audio group and the label audio group;

[0144] A loss function module, used to construct a target loss function according to the time domain error and the frequency domain error;

[0145] A training module, used to input the noisy frequency into a preset deep learning model, perform model training in combination with the target loss function, and output the noise-reduced frequency, until the preset training conditions are met, and then output the trained deep learning model;

[0146] The denoising module is used to estimate the noisy frequency using the trained deep learning model to obtain the target audio.

[0147] A noise reduction device based on a loss function also includes:

[0148] The asymmetric amplitude error module is used to calculate the asymmetric amplitude error of the low-frequency part and perform gradient descent only when the time domain label amplitude is greater than the predicted amplitude;

[0149] The loss function module constructs a target loss function according to the time domain error, the frequency domain error and the asymmetric amplitude error.

[0150] For the specific definition of a denoising device based on a loss function, please refer to the definition of a denoising method based on a loss function in the above text, which will not be repeated here.

[0151] Each module in the above-mentioned noise reduction device based on loss function can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0152] In one embodiment, a computer device is provided, which may be a server. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, any one of the above-mentioned noise reduction methods based on the loss function is implemented.

[0153] In one embodiment, a computer-readable storage medium is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, any one of the above-mentioned denoising methods based on the loss function is implemented.

[0154] In one embodiment, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, any one of the above-mentioned denoising methods based on the loss function is implemented.

[0155] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. When the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0156] Those skilled in the art will clearly understand that for the sake of convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

Claims

1. A denoising method based on a loss function, characterized in that: The following steps are included: Get the time domain output sequence and frequency domain spectrogram with noise frequency; Respectively dividing the time-domain output sequence and the corresponding time-domain label into output sequence audio segments of different unit durations and corresponding label audio segments of different unit durations to obtain an output sequence audio group and a label audio group; Calculating a time domain error based on the output sequence audio group and the label audio group; and Calculate a frequency domain error based on the frequency domain spectrogram, the output sequence audio group, and the label audio group; Constructing a target loss function according to the time domain error and the frequency domain error; Input the noisy frequency into a preset deep learning model, perform model training in combination with the target loss function, output the noise-reduced frequency, and output the trained deep learning model when the preset training conditions are met; The trained deep learning model is used to estimate the noisy frequency and obtain the target audio.

2. The denoising method based on loss function according to claim 1, characterized in that: The step of calculating the time domain error based on the output sequence audio group and the label audio group includes: According to the output sequence audio group and the label audio group, the scale-invariant signal-to-noise ratio of the output sequence audio segments of different unit time lengths and the corresponding label audio segments is calculated to obtain a piecewise scale-invariant signal-to-noise ratio; The time domain error is obtained by summing the scale-invariant signal-to-noise ratios.

3. The denoising method based on loss function according to claim 1, characterized in that: The step of calculating the frequency domain error based on the frequency domain time-frequency diagram, the output sequence audio group and the label audio group includes: According to the output sequence audio group and the label audio group, combined with the frequency domain time-frequency diagram, a mean square error is calculated, wherein the mean square error includes a complex mean square error and an amplitude mean square error; The complex mean square error is obtained by adding the real mean square error between the output sequence audio segments of different unit time lengths and the corresponding label audio segments to the imaginary mean square error between the output sequence audio segments of different unit time lengths and the corresponding label audio segments; The prediction amplitude is obtained by adding the square sum of the real mean square errors of the output sequence audio segments of different unit time lengths of the output sequence audio group to the square sum of the imaginary mean square errors of the corresponding output sequence audio segments; The time domain label amplitude is obtained by adding the square sum of the real mean square errors of the label audio segments of different unit time lengths of the label audio group to the square sum of the imaginary mean square errors of the corresponding label audio segments; Calculating the mean square error between the predicted amplitude and the time domain label amplitude to obtain the amplitude loss; The frequency domain error is obtained based on the complex mean square error and the amplitude loss.

4. The denoising method based on loss function according to claim 3, characterized in that: When the frequency domain error is obtained based on the complex mean square error and the amplitude loss, the following formula is used: loss freq =0.9*loss composite +0.1*loss mags In the formula, loss freq Represents the frequency domain error, loss composite Represents the complex mean square error, loss mags Indicates amplitude loss.

5. The denoising method based on loss function according to any one of claims 1 to 4, characterized in that: Before the step of constructing the target loss function according to the time domain error and the frequency domain error, the method further includes: Calculate the asymmetric amplitude error of the low-frequency part and perform gradient descent only when the time domain label amplitude is greater than the predicted amplitude; A target loss function is constructed according to the time domain error, the frequency domain error and the asymmetric amplitude error.

6. The denoising method based on loss function according to claim 5, characterized in that: The step of calculating the asymmetric amplitude error of the low-frequency part adopts the following formula: loss mag_asym =(max(0,(lable mags [:f low ]-pred mags [:f low ]))) 2 In the formula, loss mag_asym Indicates asymmetric amplitude error, label mags Indicates the time domain label amplitude, pred mags represents the prediction amplitude, [:f low ] indicates that in f low The calculation of the asymmetric amplitude error is performed below the frequency.

7. The denoising method based on loss function according to claim 6, characterized in that: The following steps are also included: When calculating the asymmetric amplitude error of the low-frequency part, f low Using 1kHz.

8. The denoising method based on loss function according to claim 5, characterized in that: When constructing the target loss function according to the time domain error, the frequency domain error and the asymmetric amplitude error, the following formula is used: loss=loss time +loss freq +0.1*loss mag_asym In the formula, loss represents the target loss function, loss time Represents the time domain error, loss freq Represents the frequency domain error, loss mag_asym Indicates the asymmetric amplitude error.

9. A noise reduction device based on a loss function, characterized in that: include, A data acquisition module is used to obtain a time domain output sequence with noise frequency and a frequency domain time-frequency diagram; A segmentation module, used to segment the time domain output sequence and the corresponding time domain label into output sequence audio segments of different unit time lengths and corresponding label audio segments of different unit time lengths, respectively, to obtain an output sequence audio group and a label audio group; A time domain error module, configured to calculate a time domain error based on the output sequence audio group and the label audio group; A frequency domain error module, used for calculating a frequency domain error based on the frequency domain time-frequency diagram, the output sequence audio group and the label audio group; A loss function module, used to construct a target loss function according to the time domain error and the frequency domain error; A training module, used to input the noisy frequency into a preset deep learning model, perform model training in combination with the target loss function, and output the noise-reduced frequency, until the preset training conditions are met, and then output the trained deep learning model; The denoising module is used to estimate the noisy frequency using the trained deep learning model to obtain the target audio.

10. A noise reduction system, characterized in that: include, A collection unit, used for converting voice signals in the environment into digital signals; A computing unit, configured to receive the output signal of the acquisition unit and execute the steps of the denoising method based on the loss function according to any one of claims 1 to 8; A data transmission unit, used for transmitting the output signal of the calculation unit; The terminal unit is used to play the output signal of the data transmission unit.

Images