A delay-controllable speech noise reduction method

Abstract

The present application relates to a method for speech noise reduction with controllable time delay, comprising the following steps: framing the noisy speech, transforming the noisy speech into a complex spectrum of the noisy speech in the time-frequency domain; determining a gain function based on the complex spectrum of the noisy speech; the gain function being a real number or a complex number; determining a time domain filter based on the gain function, wherein the order of the time domain filter is set according to the time delay requirement; inputting the noisy speech into the time domain filter for noise reduction processing that meets the time delay requirement to obtain pure speech. The method proposed in the embodiment of the present application can achieve advanced speech noise reduction performance under low-latency conditions, reduce computational complexity, and improve robustness.

Description

Technical Field

[0001] The present application relates to the field of multimedia, and more specifically, to a method for voice noise reduction with controllable delay. Background Art

[0002] Speech noise reduction has crucial applications in voice communication, speech recognition, hearing aids, and cochlear implants, significantly improving communication quality and interactive experience. Speech noise reduction can be categorized into unsupervised and supervised methods. Unsupervised methods include spectral subtraction and subspace methods, while supervised methods include non-negative matrix factorization, dictionary learning, and deep neural networks. Currently, the vast majority of speech noise reduction approaches are performed in the time-frequency domain: data is first framed and windowed, followed by a Fourier transform. A gain function is then inferred using unsupervised methods or estimated using supervised methods. This gain function is then applied to the complex spectrum of the noisy signal, and the time domain signal is reconstructed using the overlap-add (OLA) method. With this approach, the latency is determined by the frame length. Latency is strictly constrained in many systems. For example, in hearing aid systems, all signal processing delays must be controlled within 4ms to reduce comb effects while meeting the just-noticeable-time difference requirement. Current TWS (True Wireless Stereo) headphones all have a pass-through feature. When this feature is enabled, similar to hearing aids, latency must also be controlled within 4ms. In sound reinforcement systems, noise suppression of microphone-pickup signals places even higher latency requirements. Excessive introduction of electrical signal algorithm delays can lead to significant latency in the sound reinforcement system, and in severe cases, may even cause echo. Therefore, researching and implementing a low-latency, high-performance speech noise reduction method is of great application value.

[0003] One way to reduce latency is to shorten the frame length, for example, to 4ms. However, studies have shown that using an excessively short frame length results in low frequency resolution of the Fourier transformed spectrum. Unsupervised methods cannot effectively suppress noise between speech harmonics, while supervised methods can reduce the ability to distinguish between noise and speech features, severely impacting the performance of supervised methods and, in severe cases, preventing convergence of supervised learning model training.

[0004] Another way to reduce latency is to use long frames to reduce frame shift, such as reducing it to 4ms or even 2ms. However, most existing delay-controllable speech noise reduction methods use frequency domain analysis and synthesis, and use the overlap-addition method to reconstruct the enhanced speech time domain signal. The latency is still determined by the frame length. It is worth mentioning that existing speech separation methods use an end-to-end approach to time domain signals. In theory, the latency is determined by the frame shift. However, their performance is inferior to the frequency domain analysis and synthesis method, and they have higher computational complexity and insufficient stability. Summary of the Invention

[0005] The purpose of this application is to set the delay according to the actual application to achieve the goal of controllable delay, reduce computational complexity and improve robustness.

[0006] To achieve the above-mentioned objectives, the present application provides a delay-controllable speech noise reduction method comprising the following steps: framing the noisy speech, and performing time-frequency domain transformation to obtain a complex spectrum of the noisy speech; determining a gain function based on the complex spectrum of the noisy speech; the gain function being a real number or a complex number; determining a time domain filter based on the gain function, wherein the order of the time domain filter is set according to the delay requirement; and inputting the noisy speech into the time domain filter for noise reduction processing that meets the delay requirement to obtain clean speech.

[0007] As a preferred embodiment, determining the gain function based on the complex spectrum of the noisy speech includes: determining the amplitude spectrum of the complex spectrum of the noisy speech; inputting the amplitude spectrum of the complex spectrum of the noisy speech into the deep learning network, wherein the deep learning network is a real network; and determining the gain function based on the mapping target of the real network, wherein the gain function is a real number.

[0008] As a preferred embodiment, the gain function is determined according to the mapping target of the real network, including: when the mapping target of the deep learning network is the amplitude spectrum of clean speech, the gain function is the ratio of the amplitude spectrum of the clean speech to the amplitude spectrum of the noisy speech; when the mapping target of the deep learning network is the amplitude spectrum compression value of clean speech, the gain function is the ratio of the amplitude spectrum compression value of the clean speech to the amplitude spectrum of the noisy speech.

[0009] As a preferred embodiment, determining the gain function on the complex spectrum includes: determining the real part and imaginary part of the complex spectrum of the noisy speech, and inputting the real part and imaginary part of the complex spectrum of the noisy speech into the deep learning network, where the deep learning network is a complex network; or determining the real part and imaginary part of the compressed complex spectrum of the noisy speech; inputting the real part and imaginary part of the compressed complex spectrum of the noisy speech into the complex network; determining the gain function according to the mapping target of the complex network; the gain function is a complex number.

[0010] As a preferred embodiment, determining the gain function according to the mapping target of the complex network includes: when the mapping target of the complex network is the complex spectrum of clean speech, obtaining the gain function according to the ratio of the complex spectrum of the clean speech to the noisy speech, and the gain function is a complex number; or when the mapping target of the complex network is the compressed complex spectrum of clean speech, obtaining the gain function according to the ratio of the compressed complex spectrum of the clean speech to the noisy speech, and the gain function is a complex number.

[0011] As a preferred embodiment, the time domain filter is determined according to the gain function, and the order of the time domain filter is set according to the delay control requirements, including: using a finite impulse response time domain filter to approximate the gain function, and the gain function is the fitting value of the finite impulse response time domain filter; determining the order of the finite impulse response time domain filter according to the delay control requirements.

[0012] As a preferred embodiment, the time domain filter is determined according to the gain function, and the order of the time domain filter is set according to the delay control requirement, including: using an infinite impulse response time domain filter to approximate the gain function, the gain function is a fitting value of the infinite impulse response time domain filter; determining the amplitude-frequency response of the infinite impulse response time domain filter; determining the order of the infinite impulse response time domain filter according to the amplitude value of the gain function and the amplitude-frequency response, wherein the order of the infinite impulse response time domain filter meets the delay control requirement.

[0013] As a preferred embodiment, the inputting the noisy speech into the time domain filter for noise reduction processing that meets the delay requirements to obtain pure speech includes: inputting the noisy speech into a finite impulse response time domain filter or an infinite impulse response time domain filter, performing noise reduction processing that meets the delay requirements, and obtaining pure speech.

[0014] As a preferred embodiment, the noisy speech is input into the time domain filter for noise reduction processing that meets the delay requirements to obtain pure speech, including: dividing the noisy speech to obtain first and second sub-band signals; passing the first sub-band signal through an infinite impulse response time domain filter to obtain estimated mid- and low-frequency speech; passing the second sub-band signal through a finite impulse response time domain filter to obtain estimated high-frequency speech; and synthesizing the mid- and low-frequency speech signals and the high-frequency speech signal to obtain pure speech.

[0015] As a preferred embodiment, the noisy speech time domain signal is input into the time domain filter to perform noise reduction processing that meets the delay requirements to obtain pure speech, including: determining a finite impulse response time domain filter mapped by a deep learning network, whose order is 2t d f s / 1000; where t d is the delay requirement, f s is the sampling frequency; the noisy speech of the lth frame is convolved with the finite impulse response time domain filter mapped by the deep learning network in the lth frame to obtain the clean speech of the lth frame; l is a natural number; the clean speech of the lth frame is arranged in time to obtain the clean speech that meets the delay requirement.

[0016] The method proposed in the embodiment of the present application can achieve advanced speech noise reduction performance under low latency conditions, reduce computational complexity, and improve robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the multiple embodiments disclosed in this specification, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings described below are only the multiple embodiments disclosed in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 This is a flow chart of the delay-controllable speech noise reduction method proposed in an embodiment of the present application;

[0019] Figure 2 This is a flow chart for designing an FIR time-domain filter in the delay-controllable speech noise reduction method proposed in an embodiment of the present application;

[0020] Figure 3 This is a graph showing the effect of the spectrogram test before and after Babble noise processing provided in the embodiment of the present application. DETAILED DESCRIPTION

[0021] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0022] In the following description, the terms "first\second\third, etc." or module A, module B, module C, etc. are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that the specific order or sequence can be interchanged where permitted so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0023] In the following description, the numbers representing the steps, such as S110, S120, etc., do not necessarily mean that the steps must be executed in this manner. If permitted, the order of the steps can be interchanged or they can be executed simultaneously.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0025] The following will describe the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application and are not intended to limit the scope of protection of the present application.

[0026] It should be noted that the term "first" in the description and claims of the embodiments of the present application is used to distinguish different objects, rather than to describe the specific order of objects. For example, the first voice segment is used to distinguish different voice segments, rather than to describe the specific order of target objects. In the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0027] First, the principle of a delay-controllable speech noise reduction method proposed in an embodiment of the present application is introduced.

[0028] Assume that the time domain signal of the noisy speech picked up by the microphone is x(n):

[0029] x(n)=s(n)+d s (n)+d t (n) (1)

[0030] In formula (1), s(n) is pure speech, d s (n) is the stationary noise, d t (n) is transient noise. After short-time Fourier transform (STFT), the above time domain signal model can be expressed as:

[0031] X(k,l)=S(k,l)+D s (k,l)+D t (k,l) (2)

[0032] In formula (2), k and l represent the kth frequency point and the lth frame respectively, S(k,l) represents the complex spectrum of pure speech, and D s (k,l) and D t (k, l) represent the complex spectrum of stationary noise and transient noise respectively, and X(k, l) represents the complex spectrum of noisy speech. Taking X(k, l) as an example, the short-time Fourier transform formula is:

[0033]

[0034] In formula (3), R is the frame shift and N is the frame length.

[0035] When only the noisy speech time domain signal x(n) or its complex spectrum X(k,l) picked up by the microphone is available, the purpose of single-channel speech noise reduction is to estimate the clean speech s(n) or the clean speech complex spectrum S(k,l) through various speech noise reduction methods, which can generally be written as:

[0036]

[0037] Among them, G(k,l) is a gain function. In existing methods, the value of G(k,l) is usually a real number and ranges from 0 to 1.

[0038] In order to reduce spectral leakage, existing methods usually use subband analysis to decompose noisy speech, but subband decomposition will introduce a certain delay and increase computational complexity. Therefore, some existing methods estimate the gain function at each time-frequency point, or estimate the noise power at each time-frequency point, and then calculate the gain function based on the noisy speech power and noise power at that time-frequency point. Since these existing methods use speech endpoint detection methods or methods based on minimum statistical characteristics to estimate noise power, it is difficult to accurately estimate non-stationary noise power, resulting in difficulty in accurately estimating the gain function G(k,l); and the accuracy of the estimated value of the gain function G(k,l) can directly affect the domain filter h l Therefore, although the existing method can control the time delay, its performance in suppressing non-stationary noise is poor.

[0039] In some cases, a speech denoising method based on a deep learning network can be used. Instead of explicitly estimating G(k,l), it directly maps the complex spectrum of the clean speech or the amplitude spectrum of the clean speech, such as:

[0040]

[0041] Regardless of whether equation (4) or equation (5) is used, the pure speech must be reconstructed through the overlap-add method, and its delay is determined by the frame length N.

[0042] The embodiment of the present application proposes a delay-controllable speech noise reduction method, which comprises the following steps: framing the noisy speech and performing time-frequency transformation to obtain a complex spectrum of the noisy speech; determining a gain function of the complex spectrum of the noisy speech; the gain function being a real number or a complex number; determining a time domain filter based on the gain function, the order of the time domain filter being determined according to the delay control requirement; and inputting the noisy speech into the time domain filter for noise reduction processing to obtain clean speech that meets the delay requirement.

[0043] Figure 1 This is a flow chart of the delay-controllable speech noise reduction method proposed in the embodiment of this application. Figure 1As shown, delay-controllable speech noise reduction can be achieved through the following steps S1-S4.

[0044] S1, dividing the noisy speech into frames, and performing time-frequency domain transformation to obtain a complex spectrum of the noisy speech.

[0045] In one feasible implementation, the noisy speech x(n) picked up by the microphone may be passed through a sub-band analysis filter to output a sub-band signal of the noisy speech at the k-th frequency point and the l-th frame.

[0046] According to the above embodiment, the noisy speech x(n) can be passed through a sub-band analysis filter to divide the full-band signal into at least two sub-band time domain signals, which are respectively denoted as x and n. L (n) and x H (n), where x L (n) is the sub-band speech in the low-medium frequency band, such as below 4000 Hz, x H (n) is the sub-band speech above 4000 Hz.

[0047] In one feasible implementation, the noisy speech x(n) picked up by the microphone can be subjected to frequency band analysis to output the complex spectrum of the noisy speech at the k-th frequency point and the l-th frame, where the complex spectrum of the noisy speech at the k-th frequency point and the l-th frame can be marked as X(k, l).

[0048] In comparison, using frequency band analysis, namely short-time Fourier transform, to calculate the complex spectrum of noisy speech can achieve the goal of speech noise reduction with controllable delay. Compared with the subband analysis scheme, it has lower computational complexity and can achieve more satisfactory performance.

[0049] S2, determining a gain function according to the complex spectrum of the noisy speech, where the gain function is a real number or a complex number.

[0050] In one feasible implementation, a traditional speech noise reduction estimation gain function G(k,l) is used for the subband signal of the noisy speech at the kth frequency point and the lth frame.

[0051] In one feasible implementation, the gain function G(k, l) may be obtained by mapping the complex spectrum of the noisy speech at the k-th frequency point and the l-th frame using a deep learning network.

[0052] S3, determine the time domain filter h according to the gain function G(k,l) l (n), the order of the time domain filter is determined according to the delay control requirements.

[0053] S4, input the noisy speech x(n) into the time domain filter h l (n) Filtering to obtain enhanced pure speech

[0054] In the delay-controlled speech noise reduction method proposed in the embodiments of this application, step S2 employs a deep learning network-based speech noise reduction method, where G(k, l) can be either a real number or a complex number. In low signal-to-noise ratio scenarios, a complex gain function is used, which typically exhibits better performance due to the ability to simultaneously estimate the pure speech phase.

[0055] In one feasible implementation, step S2 is implemented by the following steps.

[0056] S21, deep learning network mapping is used to obtain the gain function G(k,l), that is:

[0057] G(k,l)=DL_gNet{X(k,l)} (6)

[0058] In formula (6), the gain function G(k,l) can be a real number or a complex number. Correspondingly, the deep learning network can be a real network or a complex network. When the deep learning network adopts a real network, its input is the amplitude spectrum |X(k,l)| of the complex spectrum of the noisy speech, the compressed complex spectrum |X(k,l)| β Or mel-frequency cepstrum coefficients (MFCC), etc.

[0059] The mapping target of the deep learning network is real G(k,l). The cost function of the deep learning network can be determined by SA (signal approximation), that is:

[0060]

[0061] The value of β is between 0 and 1, and the typical value is 0.5.

[0062] In one feasible implementation, when the deep learning network adopts a real network, the amplitude spectrum of the complex spectrum of the noisy speech can be input into the deep learning network, and the output mapping target is a gain function, which is a real number.

[0063] In the case where the real network does not implicitly map the real gain function G(k,l), the mapping target is the amplitude spectrum of the pure speech or its compressed value |S(k,l)| β , then the gain function is determined by the ratio of the mapping target and the noisy speech amplitude spectrum, that is:

[0064]

[0065] In this embodiment, when the mapping target of the deep learning network is the amplitude spectrum of clean speech, the gain function is the ratio of the amplitude spectrum of clean speech to the amplitude spectrum of noisy speech; when the mapping target of the deep learning network is the amplitude spectrum compression value of clean speech, the gain function is the ratio of the amplitude spectrum compression value of clean speech to the amplitude spectrum of noisy speech.

[0066] In one feasible implementation, when the deep learning network adopts a complex network, the input of the deep learning network is the real part and imaginary part of the complex spectrum X(k,l) of the noisy speech, such as X(k,l)=X r (k,l)+jX i (k,l), that is, X r (k,l) and X i (k,l) are the real and imaginary parts of X(k,l) respectively, which can be used as the input of the complex network.

[0067] In this embodiment, the real and imaginary parts of the compressed complex spectrum of the noisy speech can also be used as inputs of the deep learning network. The compressed complex spectrum is:

[0068]

[0069] in, Right now and is the compressed complex spectrum X (c) The real and imaginary parts of (k,l).

[0070] Obviously, the compressed complex spectrum X (c) (k,l) changes the amplitude of the original complex spectrum X(k,l), and the amplitude changes to |X(k,l)| β , but does not change its phase. Using compressed complex spectrum generally has better noise reduction performance.

[0071] The general mapping target is complex G(k,l), and the cost function of deep learning can also be SA (Signal Approximation), that is:

[0072] Loss all =αLoss_mag+(1-α)Loss_complex (10)

[0073] Among them, α takes a value between 0 and 1, and the complex domain loss function Loss_complex is:

[0074]

[0075] in, and is G(k,l)X (c) The real and imaginary parts of (k,l). and is the compressed complex spectrum of the pure speech complex spectrum S. In the speech denoising task, α is typically set to 0.5, which can better balance the speech distortion and noise reduction.

[0076] If the complex network does not implicitly map the complex G(k,l), that is, the mapping target is the complex spectrum of the pure speech or the compressed complex spectrum, then the gain function can be obtained by the ratio of the mapping target to the noisy speech, that is:

[0077]

[0078] In one feasible implementation, when the mapping target of the deep learning network is the complex spectrum of clean speech, a gain function can be obtained according to the ratio of the complex spectrum of clean speech to the noisy speech, and the gain function is a complex number.

[0079] In one feasible implementation, when the mapping target of the deep learning network is the compressed complex spectrum of clean speech, a gain function is obtained according to the ratio of the compressed complex spectrum of clean speech to the noisy speech, and the gain function is a complex number.

[0080] The deep learning network used in the delay-controlled speech noise reduction method proposed in the embodiment of the present application can adopt a fully connected network (FC), a convolutional neural network (CNN), a long-short term memory network (LSTM), etc. The model size can be determined according to the computing resources and storage resources of the chip or platform, and the type of model used can be determined according to the acceleration core used by the chip and platform. In one feasible embodiment, step S3 of the time-domain filter design can be implemented by the following steps.

[0081] S31 , when a finite impulse response (FIR) filter is used, the gain function G(k, l) is used as the fitting value of the FIR time domain filter.

[0082] In one feasible embodiment, a finite impulse response (FIR) filter with a linear phase may be used to approximate the gain function G(k, l).

[0083] S32: Determine the order of the FIR filter according to the delay control requirement.

[0084] Since the FIR filter with linear phase has symmetry, when the FIR filter filters the time domain signal of the noisy speech, the time delay introduced by the FIR filter is exactly equal to half of the order of the FIR filter.

[0085] For example, when the delay requirement is t d milliseconds, with a sampling rate of f s (Unit: Hz), the maximum order of the FIR filter is 2t d f s / 1000, such as time delay t d Required in 4 milliseconds, for sampling rate f s The frequency is 16000 Hz and the maximum length of the FIR filter is 128 points.

[0086] Figure 2 Design flow chart for FIR time domain filter. Figure 2 As shown, the window function method can be used to design an FIR time domain filter, and step S32 can be implemented by the following steps S321-S323.

[0087] S321, the gain function G(k, l) of the lth frame is transformed back to the time domain through the inverse short-time Fourier transform to obtain the gain function G of the lth frame signal in the nth time domain l (n).

[0088] S322, the gain function G of the lth frame signal in the nth time domain l (n) Perform right-shift processing in the time domain to obtain the gain function G in the n-n0th time domain l (n-n0).

[0089] S323, the gain function G in the n-n0 time domain is l (n-n0) is truncated to obtain the FIR time domain filter h obtained by mapping the nth time domain in the lth frame of the deep learning network l (n).

[0090] The FIR time-domain filter designed using the above-mentioned window function method has the advantages of low computational complexity and stable performance.

[0091] In one feasible implementation, a minimum phase design can be used to obtain an infinite impulse response (IIR) time-domain filter. Since an IIR time-domain filter can be designed as a minimum phase filter, its delay is shorter than that of an FIR time-domain filter. The disadvantage of the IIR time-domain filter approximation is that it cannot guarantee linear phase, that is, the delay at each frequency point may not be the same. The higher the order of the IIR time-domain filter, the more accurate the approximation, but also the higher the complexity. If the order of the IIR time-domain filter is too low, it may cause excessive approximation errors, resulting in unstable speech noise reduction performance.

[0092] In this embodiment, step S3 can be implemented by the following steps S31 ′-S33 ′.

[0093] S31′: When an IIR time domain filter is used, the gain function is used as the fitting value of the IIR time domain filter, and the IIR time domain filter is obtained by using a minimum phase.

[0094] S32′, calculating the amplitude-frequency response according to the IIR time-domain filter coefficients, and determining the amplitude-frequency response of the IIR time-domain filter.

[0095] S33′, determining the order of the infinite impulse response time domain filter according to the amplitude value and the amplitude-frequency response of the gain function, where the order of the infinite impulse response time domain filter meets the delay control requirement.

[0096] In one feasible implementation, the amplitude-frequency response of the IIR time-domain filter can be compared with the amplitude value |G(k,l)| of the gain function. When the error is large, the order of the IIR time-domain filter can be increased to reduce the approximation error and meet the delay control requirements.

[0097] The delay-controllable speech noise reduction method proposed in the embodiment of the present application adopts an IIR time-domain filter to fit the gain function G(k,l), and its delay can be controlled within a smaller range.

[0098] Because the computational complexity of IIR time-domain filter design and filtering is directly related to the IIR order, IIR time-domain filters are more suitable for approximating peaks and valleys, while FIR time-domain filters are more suitable for approximating smooth amplitude-frequency response curves. For speech signals, peaks and valleys are generally only evident in low- and mid-frequency bands, such as voiced speech below 4000Hz, but not in high frequencies above 4000Hz.

[0099] In a feasible implementation, a hybrid design of FIR and IIR time domain filters may be adopted, and step S3 may be implemented through the following steps S31″-S32″.

[0100] S31″, for the gain function G(k,l) of the noisy speech in the frequency band below 4000 Hz, an IIR filter is used to fit.

[0101] S32″, for the gain function G(k,l) of the noisy speech in the frequency band above 4000 Hz, an FIR filter is used to fit the gain function G(k,l) to reduce the algorithm complexity.

[0102] In one feasible implementation, step S4 can be implemented by the following steps S41-S42.

[0103] S41, performing frequency division on the noisy speech to obtain first and second sub-band signals;

[0104] In an achievable implementation, the noisy speech time domain signal x(n) can be passed through a subband analysis filter to divide the full-band signal into two sub-band time domain signals, which are respectively recorded as the first sub-band signal x and the second sub-band signal x. L (n) and the second subband signal x H (n).

[0105] S42: Pass the first subband signal through an infinite impulse response time domain filter to obtain estimated mid- and low-frequency speech.

[0106] In an achievable implementation, the first sub-band signal x L (n) The estimated low- and medium-frequency speech is obtained through the IIR time domain filter

[0107] S43, passing the second subband signal through a finite impulse response time domain filter to obtain estimated high-frequency speech.

[0108] In an achievable implementation, the second sub-band signal x H (n) The estimated high-frequency speech is obtained through the FIR time domain filter

[0109] S44 combines mid- and low-frequency speech with high-frequency speech to obtain pure speech across the entire frequency band.

[0110] In one possible implementation, the speech of the two sub-bands can be and The final estimated pure speech of the full frequency band in the time domain is obtained by synthesis

[0111] In one feasible embodiment, step S4 can be implemented by an FIR time-domain filter based on deep learning network mapping, comprising the following steps:

[0112] S41′, determine the FIR time domain filter mapped by the deep learning network, whose order is 2t d f s / 1000 2 .

[0113] The enhanced speech time domain signal is subjected to STFT transformation, and the cost function of formula (10) can also be used to train the deep learning network parameters for realizing FIR time domain filter mapping.

[0114] S42′, convolve the noisy speech time domain signal of the first frame with the finite impulse response time domain filter mapped by the deep learning network in the first frame to obtain the clean speech of the first frame that meets the delay requirement. The clean speech output of the first frame is:

[0115]

[0116] In formula (13), x l (n) is the time domain signal of the noisy speech in the first frame, h l (n) is the FIR time domain filter mapped by the deep learning network at the first frame, To meet the delay requirement, the first frame is pure speech.

[0117] The time domain convolution in Equation (13) can also be quickly implemented by frequency domain multiplication. In order to ensure linear convolution, h l (n) should be padded with zeros before Fourier transform, x l (n) It is also necessary to take the signal of the previous frame and make it up with the zero-filled h l (n) The same length.

[0118] Example 1

[0119] Embodiment 1 of the present application proposes a delay-controllable speech noise reduction method, which includes:

[0120] S51, the noisy speech time domain signal x l (n) Frame the data with a frame length of N and a frame shift of R, and perform short-time Fourier transform to obtain the complex spectrum X(k,l) of the noisy speech.

[0121] S52, using a real network to directly estimate the real gain function G(k, l) or a complex network to directly estimate the complex gain function G(k, l);

[0122] S53, using a window function method to design an FIR time-domain filter, the order of which is determined by the time delay; or directly using an IIR time-domain filter to approximate the gain function G(k,l);

[0123] S54, the noisy speech time domain signal x(n) is passed through an FIR time domain filter or an IIR time domain filter to obtain a pure speech of full frequency band in the time domain

[0124] Example 2

[0125] Example 2 of the present application proposes a delay-controllable speech noise reduction method, which includes:

[0126] S61, the noisy speech time domain signal x(n) is framed with a frame length of N and a frame shift of R, and a short-time Fourier transform is performed to obtain the noisy speech complex spectrum X(k,l);

[0127] S62, using real network to estimate the amplitude spectrum of pure speech Or the complex network estimates the complex spectrum of the clean speech The real and imaginary parts of

[0128] S63, using formula (7) or formula (11) to calculate the gain function G(k, l);

[0129] S64, using a window function method to design an FIR time-domain filter, the order of which is determined by the time delay; or directly using an IIR time-domain filter to approximate the gain function G(k,l);

[0130] S65, the noisy speech time domain signal x(n) is passed through an FIR time domain filter or an IIR time domain filter to obtain a pure speech of full frequency band in the time domain.

[0131] Example 3

[0132] Example 3 of the present application proposes a delay-controllable speech noise reduction method, which includes:

[0133] S71, the noisy speech time domain signal x(n) is framed with a frame length of N and a frame shift of R, and a short-time Fourier transform is performed to obtain the noisy speech complex spectrum X(k,l);

[0134] S72, directly estimating or indirectly estimating the gain function G(k, l) using deep learning;

[0135] S73, approximating the gain function G(k, l) corresponding to frequencies below 4000 Hz using an IIR time-domain filter, and approximating the gain function G(k, l) corresponding to frequencies above 4000 Hz using an FIR time-domain filter;

[0136] S74, pass the noisy speech time domain signal x(n) through the subband analysis filter to obtain the subband signal x below 4000 Hz L (n) and the sub-band signal x above 4000 Hz H (n);

[0137] S75,x L (n) After passing through the IIR time domain filter, the sub-band enhanced speech below 4000Hz is obtained x H (n) After passing through the FIR time domain filter, the sub-band enhanced speech above 4000Hz is obtained

[0138] S76, the voice of the two sub-bands and The final estimated pure speech of the full frequency band in the time domain is synthesized

[0139] Example 4

[0140] Example 4 of the present application proposes a delay-controllable speech noise reduction method, which includes:

[0141] S81, the noisy speech time domain signal x(n) is framed with a frame length of N and a frame shift of R, and a short-time Fourier transform is performed to obtain the noisy speech complex spectrum X(k,l);

[0142] S82, directly estimating or indirectly estimating the gain function G(k, l) using deep learning;

[0143] S83, the noisy speech complex spectrum X(k, l) and the gain function G(k, l) are simultaneously used as the deep learning network input to map the l-th frame FIR time domain filter h l (n);

[0144] S84, using formula (12) to calculate the enhanced speech time domain signal of the first frame Or use frequency domain multiplication instead of time domain convolution to obtain the enhanced speech time domain signal The time series of the output signals of each frame is the pure speech of the full frequency band in the time domain.

[0145] In response to the low latency requirements of single-channel speech noise reduction, the embodiments of this application propose a delay-controllable deep learning speech noise reduction solution. This solution retains the advantages of deep learning speech enhancement methods in suppressing strong non-stationary noise and restoring speech in low signal-to-noise ratio scenarios, while also achieving the goal of controllable latency. If the latency requirement is not to exceed the frame length, the overlap-addition method can be used to directly synthesize the enhanced clean speech. If the latency requirement is much smaller than the frame length, such as 4 milliseconds, the gain function must be approximated in the time domain. This can be done using an FIR time domain filter, an IIR time domain filter, or a hybrid of the two. Alternatively, a deep learning network can be used to directly map the time domain filter.

[0146] Figure 3 This is a graph showing the effect of the spectrogram test before and after Babble noise processing provided in the embodiment of this application. Figure 3 Figure 2 shows a comparison of spectrograms before and after Babble noise processing: (a) noisy speech; (b) using traditional methods to estimate the gain function and design a time-domain filter using a window function method for filtering; (c) using deep learning methods to estimate the gain function and design a time-domain filter using a window function method for filtering; and (d) using deep learning methods to estimate the gain function and map the time-domain filter using a deep learning method for filtering. Test results show that the method proposed in this embodiment can achieve advanced speech noise reduction performance under low latency conditions.

[0147] The delay-controllable speech noise reduction method proposed in the embodiment of the present application performs single-channel speech noise reduction based on delay-controllable deep learning. This method combines supervised learning methods such as deep learning and time-domain filter methods.

[0148] The delay-controllable speech noise reduction method proposed in the embodiment of the present application uses a deep learning method to infer the gain function of each time-frequency point in the time-frequency domain, performs time-domain filter optimization in each frame, and performs filtering enhancement in the time domain to achieve speech noise reduction.

[0149] The embodiments of the present application propose to use three methods: infinite impulse response (IIR), finite impulse response (FIR) filter and deep learning network mapping fitting gain function to achieve the optimal design of time domain filter.

[0150] In order to reduce computational complexity, improve robustness, and consider speech characteristics, the embodiment of the present application also proposes the use of a subband analysis and synthesis method. First, the full-band signal is divided into two sub-bands through a sub-band analysis filter, and an IIR time domain filter is used at the low frequency, and an FIR time domain filter is used at the high frequency for fitting. Then, the two sub-bands are respectively subjected to time domain filtering enhancement, and finally, sub-band synthesis is used to reconstruct the full-band speech time domain signal.

[0151] Those skilled in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0152] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0153] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of this application. It should be understood that the above description is only the specific implementation methods of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A delay-controllable speech noise reduction method, characterized in that: The following steps are involved: The noisy speech is divided into frames, and a time-frequency domain transform is performed to obtain a complex spectrum of the noisy speech; Determining a gain function based on the complex spectrum of the noisy speech; the gain function is a real number or a complex number; comprising: inputting the complex spectrum of the noisy speech into a deep learning network; when the deep learning network is a real network, determining the gain function based on a mapping target of the real network; when the deep learning network is a complex network, obtaining the gain function based on the ratio of the complex spectrum of the clean speech to the noisy speech; the gain function is a complex number; Determining a time domain filter according to the gain function, where the order of the time domain filter is set according to a delay requirement, includes: Using a finite impulse response time domain filter to approximate the gain function, the gain function is a fitting value of the finite impulse response time domain filter; determining the order of the finite impulse response time domain filter according to a delay control requirement; An infinite impulse response time domain filter is used to approximate the gain function, where the gain function is a fitted value of the infinite impulse response time domain filter; an amplitude-frequency response of the infinite impulse response time domain filter is determined; and an order of the infinite impulse response time domain filter is determined based on the amplitude value of the gain function and the amplitude-frequency response, where the order of the infinite impulse response time domain filter meets a delay control requirement. Inputting the noisy speech into the time domain filter to perform noise reduction processing that meets the delay requirement to obtain pure speech, including: The noisy speech is subjected to frequency division to obtain first and second sub-band signals; the first sub-band signal is subjected to an infinite impulse response time-domain filter to obtain estimated low- and medium-frequency speech; the second sub-band signal is subjected to a finite impulse response time-domain filter to obtain estimated high-frequency speech; and the low- and medium-frequency speech signals and the high-frequency speech signal are synthesized to obtain pure speech.

2. The method according to claim 1, characterized in that The step of determining a gain function according to the complex spectrum of the noisy speech comprises: Determining an amplitude spectrum of a complex spectrum of the noisy speech; Inputting the amplitude spectrum of the complex spectrum of the noisy speech into the deep learning network, wherein the deep learning network is a real network; A gain function is determined according to a mapping target of the real network, where the gain function is a real number.

3. The method according to claim 2, characterized in that The determining of the gain function according to the mapping target of the real network includes: When the mapping target of the deep learning network is the amplitude spectrum of clean speech, the gain function is the ratio of the amplitude spectrum of the clean speech to the amplitude spectrum of the noisy speech; When the mapping target of the deep learning network is the amplitude spectrum compression value of clean speech, the gain function is the ratio of the amplitude spectrum compression value of the clean speech to the amplitude spectrum of the noisy speech.

4. The method according to claim 1, wherein The step of determining a gain function according to the complex spectrum of the noisy speech comprises: Determine the real part and the imaginary part of the complex spectrum of the noisy speech, and input the real part and the imaginary part of the complex spectrum of the noisy speech into the deep learning network, wherein the deep learning network is a complex network; or Determining the real part and the imaginary part of the compressed complex spectrum of the noisy speech; inputting the real part and the imaginary part of the compressed complex spectrum of the noisy speech into the complex network; A gain function is determined according to a mapping target of the complex network; the gain function is a complex number.

5. The method according to claim 4, characterized in that The determining of a gain function according to a mapping target of the complex network comprises: When the mapping target of the complex network is the complex spectrum of clean speech, a gain function is obtained according to the ratio of the complex spectrum of the clean speech to the noisy speech, and the gain function is a complex number; or When the mapping target of the complex network is the compressed complex spectrum of clean speech, a gain function is obtained according to the ratio of the compressed complex spectrum of clean speech to the noisy speech, and the gain function is a complex number.

6. The method according to any one of claims 1 to 5, characterized in that The step of inputting the noisy speech into the time domain filter to perform noise reduction processing that meets the time delay requirement to obtain pure speech includes: The noisy speech is input into a finite impulse response time domain filter or an infinite impulse response time domain filter, and noise reduction processing that meets the time delay requirement is performed to obtain pure speech.

7. The method according to claim 1, characterized in that The step of inputting the noisy speech time domain signal into the time domain filter to perform noise reduction processing that meets the time delay requirement to obtain pure speech includes: Determine the finite impulse response time domain filter mapped by the deep learning network, whose order is ;in, For delay requirements, is the sampling frequency; The first Frame noisy speech and deep learning network The finite impulse response time domain filter obtained by frame mapping is convolved to obtain the first Frame-clean speech; is a natural number; Will The clean speech of the frames is arranged in time to obtain the clean speech that meets the delay requirement.

Images