A Low-Complexity Multi-Channel De-reverberation and Noise Reduction Method Based on Kalman Filtering

Abstract

This invention provides a low-complexity multi-channel denoising and noise reduction method based on Kalman filtering, comprising: acquiring a signal and preprocessing the acquired signal to obtain a signal in the short-time Fourier domain; calculating a multi-channel noise covariance matrix; estimating multi-channel autoregressive parameters using the delayed, noiseless reverberant signal estimated in the previous frame and the acquired signal in the current frame, and determining the variance value of the Kalman state noise based on the sound source change detection result of the previous frame; estimating a noiseless reverberant signal using the estimated autoregressive parameters, the acquired signal in the current frame, and the estimated multi-channel noise covariance matrix; delaying the estimated noiseless reverberant signal and calculating the estimated noiseless late reverberant signal using the autoregressive coefficients; subtracting the noiseless late reverberant signal from the noiseless reverberant signal to obtain the desired direct sound and early reverberant signal. This invention reduces computational complexity and enables real-time applications in embedded products.

Description

Technical Field

[0001] This invention relates to the field of audio processing technology, and in particular to a low-complexity multi-channel dereverberation and noise reduction method based on Kalman filtering. Background Technology

[0002] When a microphone is used to capture a speaker's voice signal in a room, reflected sound from the room walls is also captured simultaneously; this reflected sound is called reverberation. When the reverberation time is long, it affects the clarity of speech in voice communication and reduces the recognition rate of a speech recognition system.

[0003] Spectral subtraction can be used to achieve speech dereverberation. For example, the existing literature "Lebart K, Boucher JM, Denbigh P NA New Method Based on Spectral Subtraction for Speech Dereverberation[J]. Acta Acustica United with Acustica, 2001, 87(3):359-366." uses short-time Fourier transform to transform the single-channel speech signal to the time-frequency domain, and then uses spectral subtraction to subtract the power spectrum of the speech signal in the current frame from the estimated late reverberation power spectrum to obtain the power spectrum of the dereverberated signal. Finally, the time-domain dereverberated speech signal is obtained through inverse short-time Fourier transform. However, this dereverberation method based on spectral subtraction causes a significant loss of speech quality.

[0004] Kalman filtering is an adaptive filtering method. Combining Kalman filtering with a multi-channel prediction model can be used as an adaptive dereverberation method. For example, the literature "Braun S, Habets EAP. Online Dereverberation for Dynamic Scenarios Using a Kalman Filter With an Autoregressive Model[J].IEEE Signal Processing Letters,2016,23(12):1741-1745" indicates that Kalman filtering has good dereverberation performance.

[0005] None of the aforementioned multi-channel dereverberation algorithms assume the presence of environmental noise in their modeling. In practical applications, environmental noise significantly impacts the performance of these algorithms. Low-frequency speech signals often exhibit predictability due to the presence of environmental noise, leading to over-suppression of low-frequency speech signals. The paper "Masahito Togami, MULTICHANNEL ONLINE SPEECH DEREVERBERATION UNDERNOISY ENVIRONMENTS" proposes not using noisy reverberation signals to estimate late-stage reverberation signals. Instead, it optimizes the dereverberation filter using noise-free microphone input signals, thus achieving a good dereverberation filter even in noisy environments. For microphone-acquired signals, a multi-channel Wiener filter is first used to estimate the noise-free input signal; then, the multi-channel dereverberation filter is updated to predict the late-stage reverberation signal. The Wiener filter in this algorithm relies on the autoregressive parameters of the dereverberation filter. However, the reverberation path is time-varying, and the dereverberation filter parameters from the previous frame are no longer applicable to the current frame's environment. Therefore, this algorithm suffers from causal errors, resulting in insufficient noise reduction capabilities.

[0006] The existing paper "Sebastian Braun, Linear prediction based online dereverberation and noise reduction using alternating Kalman filters" considers the time-varying nature of the reverberation path and addresses the causal problem in multi-channel denoising and denoising algorithms. It proposes a sequential structure that first estimates the autoregressive reverberation parameters and then performs noise suppression. This approach can effectively denoise and reduce noise in low signal-to-noise ratio environments. However, this algorithm utilizes two alternating Kalman filters, resulting in high computational complexity and making it difficult to implement in real-time embedded devices. Furthermore, it does not provide a method for estimating the multi-channel noise covariance matrix, and because it uses a bit matrix as the state transition matrix, the performance of the Kalman filter significantly degrades after abrupt changes in the sound source location.

[0007] The existing literature, "T. Dietzen, S. Doclo, A. Spriet, W. Tirry, M. Moonen, and T. van Waterschoot, 'Low-Complexity Kalman filter for multi-channel linear-prediction-based blind speech dereverberation,' in the 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2017, pp. 284–288," mentions using a constant less than 1 multiplied by the identity matrix as the state transition matrix. While it doesn't show a significant difference in algorithm performance before and after changes in the sound source position, the overall algorithm performance is poor. In practical applications, speaker movement during speech breaks or changes in speaker position can lead to abrupt changes in the sound source position. Therefore, when using Kalman filtering for reverberation removal, it's crucial to address the issues arising from these abrupt changes in sound source position.

[0008] In summary, existing multi-channel dereverberation algorithms have high computational complexity, making them difficult to apply in real-time in embedded products. They also perform poorly in low signal-to-noise ratio environments and have poor ability to track reverberation when the reverberation environment changes, requiring a long time to converge. Summary of the Invention

[0009] The technical problem to be solved by this invention is to provide a low-complexity multi-channel dereverberation and noise reduction method based on Kalman filtering, which achieves rapid convergence of the reverberation path, solves the dereverberation problem in noisy environments, reduces computational complexity, and can meet the requirements of real-time applications in embedded products.

[0010] The technical problem to be solved by this invention is achieved as follows: a low-complexity multi-channel dereverberation and noise reduction method based on Kalman filtering, comprising the following steps:

[0011] Step S1: Acquire the signal and preprocess the acquired signal to obtain the signal in the short-time Fourier domain;

[0012] Step S2: Calculate the multi-channel noise covariance matrix using the acquired signal in the short-time Fourier domain;

[0013] Step S3: Using the delayed, reverberant, noise-free signal estimated in the previous frame and the acquired signal in the current frame, estimate the time-varying multi-channel autoregressive parameters based on the Kalman filter algorithm; the variance of the Kalman state noise used in the estimation of the multi-channel autoregressive parameters is adjusted according to the sound source change detection results of the previous frame.

[0014] Step S4: Estimate the noise-free band reverberant signal using the estimated autoregressive parameters, the acquired signal of the current frame, and the estimated multi-channel noise covariance matrix;

[0015] Step S5: Delay the estimated noiseless band reverberation signal and calculate the estimated noiseless late reverberation signal using autoregressive coefficients; subtract the estimated noiseless late reverberation signal from the estimated noiseless band reverberation signal to obtain the desired direct sound and early reverberation signal.

[0016] Furthermore, step S1 specifically includes:

[0017] Assuming an unknown number of sound sources in a reverberant environment, and using M sub-microphones fixed at arbitrary positions for acquisition, give the Struts-Tex domain representation of the acquired signal:

[0018] y(k,n)=[Y1(k,n),…,Y M (k,n)] T

[0019] Among them, Y m (k,n) is the frequency domain representation of the kth sub-band of the mth signal in the nth frame;

[0020] Assume the multi-channel microphone signal consists of two parts:

[0021] y(k,n)=x(k,n)+v(k,n)

[0022] Where vectors x(k,n) and v(k,n) represent the reverberant speech signal and additive noise collected by the microphones on the array, respectively;

[0023] The expression for the reverberant speech signal x(k,n) is as follows:

[0024]

[0025] Where, vector s(k,n)=[S1(k,n),…,S M (k,n)] T S represents the Strut domain of the acquired signal, which contains the desired direct sound and early reverberation signals. m (k,n) represents the frequency domain representation of the k-th sub-band of the n-th frame of the m-th sub-microphone, and matrix C l (k,n)∈C M×M Let x(k,nl)∈C represent the Strft domain of the acquired signal for the nth frame, l∈[D,D+1,…,L]. M,1 The filtering parameters are: D is the delay parameter, L represents the filter length, and r(k,n) is the late reverberation signal.

[0026] Furthermore, the calculation process of the multi-channel noise covariance matrix in step S2 is as follows:

[0027] Step a1: Preset the instantaneous posterior signal-to-noise ratio threshold φ0 and the long-term posterior signal-to-noise ratio threshold.

[0028] Step a2: Initialize the covariance matrix of the acquired signal and noise covariance matrix

[0029] Step a3, in the first L of the algorithm init Within a frame, assuming the initial acquired signal contains only noise, the L... init The number of frames representing the initial pure noise in the audio;

[0030] Iterative calculation of the covariance matrix of the acquired signal and the noise covariance matrix:

[0031]

[0032]

[0033] Where, α v The iteration coefficients of the noise signal; α y The iteration coefficients for the acquired signal are H, where H represents the matrix conjugate transpose operation.

[0034] Step a4, in L init After the frame, the following calculations are performed:

[0035] Step a41: Iteratively estimate the covariance matrix of the acquired signal:

[0036]

[0037] Step a42: Considering the non-correlation between the speech signal and the noise signal, estimate the covariance matrix of the speech signal:

[0038]

[0039] Step a43: Calculate the instantaneous posterior signal-to-noise ratio:

[0040]

[0041] Step a44: Calculate the long-term posterior signal-to-noise ratio:

[0042]

[0043] Where tr{.} represents the trace operation of a matrix;

[0044] Step a45: Calculate the prior signal-to-noise ratio:

[0045]

[0046] Where M represents the number of channels, i.e., the number of sub-microphones;

[0047]

[0048] Step a46: Calculate the probability of the existence of smooth iterative speech:

[0049] Calculate the probability that speech does not exist at the local scale:

[0050]

[0051] Calculate the posterior signal-to-noise ratio after windowing smoothing and the probability that the smoothed speech does not exist:

[0052]

[0053] Where w global This represents the Hanning window function, with a window length defined as 2K1+1;

[0054]

[0055] Calculate the posterior signal-to-noise ratio mean of each frequency point in the nth frame, and calculate the probability of speech non-existence at the frame scale:

[0056]

[0057]

[0058] Calculate the probability of speech not existing by combining three scales

[0059]

[0060] Based on the estimated probability of speech absence, the probability of multi-channel prior speech presence is calculated.

[0061]

[0062] Step a47: Calculate the probability of the existence of smooth iterative speech:

[0063]

[0064] Where, α p A smoothing coefficient representing the probability of speech occurrence;

[0065] Step a48: Determine the smoothing coefficient for the noise covariance matrix estimation based on the speech presence probability, and update the multi-channel noise covariance matrix:

[0066]

[0067]

[0068] in, The noise covariance matrix is... This is the noise covariance matrix estimated from the previous frame;

[0069] At this point, the noise covariance matrix estimation is complete.

[0070] Furthermore, the estimation of the autoregressive parameters in step 3 is as follows:

[0071] Step 31: Establish the first Kalman model:

[0072] The Kalman observation matrix is ​​constructed as follows:

[0073]

[0074] in I represents the Kronecker product. M Let represent the M-dimensional identity matrix, with the superscript T indicating the vector transpose operation, and x(n) represent the noise-free reverberant signal of the nth frame; simultaneously, the autoregressive parameters are defined as the state vector of the Kalman module as follows:

[0075] c(n)=Vec{[C L (n)…C D (n)] T};

[0076] C L (n) is a part of the state vector. The subscript L indicates that this part is the autoregressive parameters for the (nL)th frame. The ellipsis indicates that the autoregressive parameters for frames (nL) to (nD) are omitted. Vec{.} is a matrix straightening operation, which concatenates the columns of the matrix within the curly braces from left to right to obtain a new vector c(n), where the length of c(n) is L. c = M×M×(LD); X(n) is a shape of M×L c sparse matrix;

[0077] Step 32, Calculation steps for the first Kalman filter module:

[0078] Step 321: Calculate the prior state error covariance:

[0079]

[0080] Where, φ w (n) represents the state noise covariance;

[0081] Step 322: Calculate the state error e(n):

[0082] e(n) = y(n) - X(nD)c(n-1);

[0083] Where y(n) represents the microphone acquisition signal, X(nD) represents the observation matrix, and c(n-1) represents the autoregressive parameters calculated in the previous frame;

[0084] Step 323: Calculate the Kalman gain K(n):

[0085]

[0086] in, Represents the observation noise covariance;

[0087] Step 324: Calculate the posterior state error covariance.

[0088]

[0089] Step 325: Calculate the autoregressive parameter c(n):

[0090] c(n) = c(n-1) + K(n)e(n);

[0091] Step 326: Calculate the observation noise

[0092]

[0093] Furthermore, the state noise covariance φ w (n) is obtained in the following way:

[0094] State noise covariance φ w The value of (n) is determined by the amount of change of the autoregressive parameters between two adjacent frames, and a very small positive number is added to simulate the actual continuous change when the estimated autoregressive parameters between two adjacent frames do not change.

[0095]

[0096] φ w Let L be the variance of the state noise. c Let c(n) be the length of the autoregressive parameter. For L c An identity matrix of order 1;

[0097] The observation noise covariance φ u (n) is obtained in the following way:

[0098] Calculate the prior observation noise covariance matrix:

[0099]

[0100] Combined with the posterior observation noise covariance matrix calculated in the previous frame Calculate the observation noise covariance matrix of the current frame:

[0101]

[0102] Update the posterior observation noise covariance matrix:

[0103]

[0104] Where α represents the iteration coefficient, and the initial value of the posterior observation noise covariance matrix is ​​defined as a matrix with all zeros. To observe noise;

[0105] Calculate the observation noise covariance φ u (n):

[0106]

[0107] Furthermore, in step S3, "the variance of the Kalman state noise used in the estimation of the multi-channel autoregressive parameters is adjusted according to the sound source change detection result of the previous frame" is as follows:

[0108] The system calculates the energy of the acquired signal and the energy ratio of the estimated direct sound and the earlier signal. When the energy ratio changes abruptly, it is determined that the sound source in the current frame has changed (because the time difference between frames is approximately 32ms; if a change in the sound source position is detected 32ms ago, then the sound source position in the current frame is likely to have changed as well). When a change in the sound source is detected, the state noise variance value in the Kalman derivation process of the deriver module is briefly increased by ten times until the ratio of the two recovers to above the threshold, thereby enhancing the ability to track state changes.

[0109] The energy calculation formula for the acquired signal is as follows:

[0110]

[0111] The formula for calculating the energy of the estimated dereverberated signal is as follows:

[0112]

[0113] α py ,α ps P represents the smoothing coefficients for the acquired signal energy and the dederenced signal energy, respectively. y (n) represents the energy of the signal acquired in the nth frame, P s(n) represents the energy of the dereverberated signal (i.e., the direct sound and early reverberation) of the nth frame. K represents the number of frequency points in the STFT domain. y(k,n) refers to the acquired signal at frequency point k in the nth frame. Represents the kth frequency point of the calculated direct sound signal in the nth frame signal; when the ratio P y (n) / P s (n) < threshold (the threshold is set as needed, such as taking 0.65) is too low, indicating reverberation leakage, and it is determined that the sound source of the current frame has changed.

[0114] Furthermore, the anechoic band reverberation signal in step S4 is estimated by creating a second Kalman model, specifically as follows:

[0115] Step S41: Establish the second Kalman model:

[0116] According to the anechoic band reverberation signal x(n), construct the state vector of the second Kalman filter x (n), which represents the signal after removing noise in the acquired signal within L frames, is a one-dimensional vector with a length of L×M, where M refers to the number of channels;

[0117] x (n) = [x T (n - L + 1), …, x T (n)] T ;

[0118] Among them, x Each x(l) in (n) represents the anechoic reverberation signal of the lth frame, which is a vector with a length of M;

[0119] At the same time, define the observation noise of the second Kalman module as s (n), and its construction method is:

[0120] s (n) = [0 1×M(L-1) s T (n)] T ;

[0121] Construct the state transition matrix based on the autoregressive parameters estimated by the Kalman filter in the previous stage:

[0122]

[0123] (It means there are still symbols between C_L and C_D, which are C_{L - 1}, C_{L - 1}..... until C_D)

[0124] Construct the observation matrix H in the observation equation:

[0125] H = [0 M×M(L-1) I M ];

[0126] Among them, I M Represents the identity matrix of order M; 0 M×M(L-1) This represents a matrix of size M rows and M(L-1) columns, consisting entirely of zeros.

[0127] Therefore, the state transition equation and observation equation for the second Kalman filter are constructed as follows:

[0128] x (n)=F(n) x (n-1)+ s (n);

[0129] y(n)=H x (n)+v(n);

[0130] Where v(n) represents the noise signal collected by the microphone; y(n) represents the collected signal;

[0131] Step S42: Calculate the second Kalman filter:

[0132] Calculate the prior state error covariance matrix

[0133]

[0134] Where, Φ s (n) represents the covariance matrix of the state noise; This represents the prior state error covariance matrix of the second Kalman module. This represents the posterior state error covariance matrix of the previous frame of the second Kalman module;

[0135] Calculate the prior state vector x (n|n-1):

[0136] x (n|n-1)=F(n) x (n-1);

[0137] Where F(n) represents the state transition matrix, x (n-1) represents the estimated state vector value of the previous frame. x (n|n-1) represents the prior state vector of the current frame;

[0138] Calculate the Kalman gain K x (n):

[0139]

[0140] Calculate the state error e x (n):

[0141] e x (n)=y(n)-H x (n|n-1);

[0142] Among them, K x (n) represents the Kalman gain of the second Kalman module. Let y(n) represent the noise covariance matrix, and e represent the microphone signal. x (n) represents the state error of the second Kalman module;

[0143] Calculate the posterior state error covariance matrix

[0144]

[0145] Calculate the state vector x (n):

[0146] x (n)= x (n|n-1)+K x (n)e x (n);

[0147] from x Obtain the estimated noise-free reverberation signal from (n).

[0148] x (n) is a one-dimensional vector of length L×M. Taking the last M lengths of this vector yields the estimated noise-free reverberation signal.

[0149] Furthermore, the covariance matrix Φ of the state noise in the second Kalman filter calculation s (n) is obtained in the following way:

[0150] For each frame of signal, estimate the prior covariance matrix; then combine it with the posterior covariance matrix calculated in the previous frame to estimate the covariance matrix of the current frame.

[0151]

[0152] Wherein, γ is the weighting parameter for balancing the prior and posterior fits, Φ s (n) represents the covariance matrix of the state noise. This represents the posterior state noise covariance matrix of the previous frame. This represents the prior state noise covariance matrix estimated for the current frame;

[0153] The posterior covariance matrix is ​​obtained through smooth iteration between time frames:

[0154]

[0155] Where α represents the smoothing coefficient. This represents the estimated early reverberation signal and the direct sound signal;

[0156] The prior covariance matrix is ​​obtained using a multi-channel Wiener filtering algorithm:

[0157]

[0158] in The Wiener filter weight matrix of size [M×M] is calculated as follows:

[0159]

[0160] in, Let be the covariance matrix of the noise signal;

[0161] Φ r (n) is the covariance matrix of the noise-free late-stage reverberation signal, which can be iteratively calculated using the product of the first Kalman filter; Φ y (n) is the covariance matrix of the microphone-acquired signal, which is also obtained through smooth iteration between time frames.

[0162] Furthermore, step 5 specifically includes:

[0163] The estimated noiseless band reverberation signal (i.e., the noise-free acquisition signal) is obtained after a D-frame delay.

[0164] The delayed noiseless reverberation signal and reverberation autoregressive parameters The product yields the predicted late-stage reverberation signal:

[0165]

[0166] From the estimated noiseless band reverberation signal Subtract the estimated late reverberation signal The estimated early reverberation signal and direct sound signal were obtained.

[0167]

[0168] The present invention has the following advantages:

[0169] 1. This invention proposes a simplified algorithm that avoids large-scale matrix inversion by approximately diagonalizing the first Kalman filter state vector error covariance matrix and the observation noise covariance matrix, thereby greatly reducing computational complexity and enabling real-time denoising and noise reduction in embedded products.

[0170] 2. This invention proposes a technique for detecting changes in the reverberation environment. Based on the energy ratio before and after the de-reverberation module, a sound source change detection module is implemented. When the position changes, the state transition noise variance is increased to accelerate the convergence speed and enhance the reverberation tracking capability of the algorithm, enabling the algorithm to complete the convergence of the reverberation path within 100ms.

[0171] 3. This invention combines the time-varying nature of noise and reverberation paths, constructs a cascaded sequence that prioritizes reverberation path estimation before noise reduction, and solves the dereverberation problem in low signal-to-noise ratio environments. Attached Figure Description

[0172] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0173] Figure 1 This is a flowchart illustrating the execution of a low-complexity multi-channel denoising and noise reduction method based on Kalman filtering according to the present invention.

[0174] Figure 2 This is a schematic diagram of the overall signal flow of the present invention.

[0175] Figure 3 This is a schematic diagram illustrating the principle of the dual Kalman denoising algorithm of the present invention. Detailed Implementation

[0176] like Figures 1 to 3 As shown, the present invention provides a low-complexity multi-channel denoising and noise reduction method based on Kalman filtering, comprising the following steps:

[0177] Step S1: Acquire the signal and preprocess the acquired signal to obtain the signal in the short-time Fourier domain;

[0178] Step S2: Calculate the multi-channel noise covariance matrix using the acquired signal in the short-time Fourier domain;

[0179] Step S3: Using the delayed, reverberant, noise-free signal estimated in the previous frame and the acquired signal in the current frame, estimate the time-varying multi-channel autoregressive parameters based on the Kalman filter algorithm; the variance of the Kalman state noise used in the estimation of the multi-channel autoregressive parameters is adjusted according to the sound source change detection results of the previous frame.

[0180] Step S4: Estimate the noise-free band reverberant signal using the estimated autoregressive parameters, the acquired signal of the current frame, and the estimated multi-channel noise covariance matrix;

[0181] Step S5: Delay the estimated noiseless band reverberation signal and calculate the estimated noiseless late reverberation signal using autoregressive coefficients; subtract the estimated noiseless late reverberation signal from the estimated noiseless band reverberation signal to obtain the desired direct sound and early reverberation signal.

[0182] Specifically, step S1 is as follows:

[0183] Assuming an unknown number of sound sources in a reverberant environment, and using M sub-microphones fixed at arbitrary positions for acquisition, give the Struts-Tex domain representation of the acquired signal:

[0184] y(k,n)=[Y1(k,n),…,Y M (k,n)] T

[0185] Among them, Y m (k,n) is the frequency domain representation of the kth sub-band of the mth signal in the nth frame;

[0186] Assume the multi-channel microphone signal consists of two parts:

[0187] y(k,n)=x(k,n)+v(k,n)

[0188] Where vectors x(k,n) and v(k,n) represent the reverberant speech signal and additive noise collected by the microphones on the array, respectively;

[0189] The expression for the reverberant speech signal x(k,n) is as follows:

[0190]

[0191] Where, vector s(k,n)=[S1(k,n),…,S M (k,n)] T S represents the Strut domain of the acquired signal, which contains the desired direct sound and early reverberation signals. m (k,n) represents the frequency domain representation of the k-th sub-band of the n-th frame of the m-th sub-microphone, and matrix C l (k,n)∈C M×M Let x(k,nl)∈C represent the Strft domain of the acquired signal for the nth frame, l∈[D,D+1,…,L]. M,1 The filtering parameters are: D is the delay parameter, L represents the filter length, and r(k,n) is the late reverberation signal.

[0192] Preferably, the calculation process of the multi-channel noise covariance matrix in step S2 is as follows:

[0193] Step a1: Preset the instantaneous posterior signal-to-noise ratio threshold φ0 and the long-term posterior signal-to-noise ratio threshold.

[0194] Step a2: Initialize the covariance matrix of the acquired signal and noise covariance matrix

[0195] Step a3, in the first L of the algorithm init Within a frame, assuming the initial acquired signal contains only noise, the L... init The number of frames representing the initial pure noise in the audio;

[0196] Iterative calculation of the covariance matrix of the acquired signal and the noise covariance matrix:

[0197]

[0198]

[0199] Where, α v The iteration coefficients of the noise signal; α y The iteration coefficients for the acquired signal are H, where H represents the matrix conjugate transpose operation.

[0200] Step a4, in L init After the frame, the following calculations are performed:

[0201] Step a41: Iteratively estimate the covariance matrix of the acquired signal:

[0202]

[0203] Step a42: Considering the non-correlation between the speech signal and the noise signal, estimate the covariance matrix of the speech signal:

[0204]

[0205] Step a43: Calculate the instantaneous posterior signal-to-noise ratio:

[0206]

[0207] Step a44: Calculate the long-term posterior signal-to-noise ratio:

[0208]

[0209] Where tr{.} represents the trace operation of a matrix;

[0210] Step a45: Calculate the prior signal-to-noise ratio:

[0211]

[0212] Where M represents the number of channels, i.e., the number of sub-microphones;

[0213]

[0214] Step a46: Calculate the probability of the existence of smooth iterative speech:

[0215] Calculate the probability that speech does not exist at the local scale:

[0216]

[0217] Calculate the posterior signal-to-noise ratio after windowing smoothing and the probability that the smoothed speech does not exist:

[0218]

[0219] Where w global This represents the Hanning window function, with a window length defined as 2K1+1;

[0220]

[0221] Calculate the posterior signal-to-noise ratio mean of each frequency point in the nth frame, and calculate the probability of speech non-existence at the frame scale:

[0222]

[0223]

[0224] Calculate the probability of speech not existing by combining three scales

[0225]

[0226] Based on the estimated probability of speech absence, the probability of multi-channel prior speech presence is calculated.

[0227] Step a47: Calculate the probability of the existence of smooth iterative speech:

[0228]

[0229] Where, α p A smoothing coefficient representing the probability of speech occurrence;

[0230] Step a48: Determine the smoothing coefficient for the noise covariance matrix estimation based on the speech presence probability, and update the multi-channel noise covariance matrix:

[0231]

[0232]

[0233] in, The noise covariance matrix is... This is the noise covariance matrix estimated from the previous frame;

[0234] At this point, the estimation of the noise covariance matrix (which is the observation noise covariance matrix in the second Kalman model) is complete.

[0235] Preferably, the estimation of the autoregressive parameters in step 3 is as follows:

[0236] Step 31: Establish the first Kalman model:

[0237] The Kalman observation matrix is ​​constructed as follows:

[0238]

[0239] in I represents the Kronecker product. M Let represent the M-dimensional identity matrix, with the superscript T indicating the vector transpose operation, and x(n) represent the noise-free reverberant signal of the nth frame; simultaneously, the autoregressive parameters are defined as the state vector of the Kalman module as follows:

[0240] c(n)=Vec{[C L (n)…C D (n)] T};

[0241] C L (n) is a part of the state vector. The subscript L indicates that this part is the autoregressive parameters for the (nL)th frame. The ellipsis indicates that the autoregressive parameters for frames (nL) to (nD) are omitted. Vec{.} is a matrix straightening operation, which concatenates the columns of the matrix within the curly braces from left to right to obtain a new vector c(n), where the length of c(n) is L. c = M×M×(LD); X(n) is a shape of M×L c sparse matrix;

[0242] Step 32, Calculation steps for the first Kalman filter module:

[0243] Step 321: Calculate the prior state error covariance:

[0244]

[0245] Where, φ w (n) represents the state noise covariance;

[0246] Step 322: Calculate the state error e(n):

[0247] e(n) = y(n) - X(nD)c(n-1);

[0248] Where y(n) represents the microphone acquisition signal, X(nD) represents the observation matrix, which is the observation matrix for the Kalman module and the noiseless reverberation in a practical sense, and c(n-1) represents the autoregressive parameters calculated in the previous frame;

[0249] Step 323: Calculate the Kalman gain K(n):

[0250]

[0251] in, Represents the observation noise covariance;

[0252] Step 324: Calculate the posterior state error covariance.

[0253]

[0254] Step 325: Calculate the autoregressive parameter c(n):

[0255] c(n) = c(n-1) + K(n)e(n);

[0256] Step 326: Calculate the observation noise

[0257]

[0258] Preferably, the state noise covariance φ w (n) is obtained in the following way:

[0259] State noise covariance φ w The value of (n) is determined by the amount of change of the autoregressive parameters between two adjacent frames, and a very small positive number is added to simulate the actual continuous change when the estimated autoregressive parameters between two adjacent frames do not change.

[0260]

[0261] φ w Let L be the variance of the state noise. c Let c(n) be the length of the autoregressive parameter. For L c An identity matrix of order 1;

[0262] The observation noise covariance φ u (n) is obtained in the following way:

[0263] Calculate the prior observation noise covariance matrix:

[0264]

[0265] Combined with the posterior observation noise covariance matrix calculated in the previous frame Calculate the observation noise covariance matrix of the current frame:

[0266]

[0267] Update the posterior observation noise covariance matrix:

[0268]

[0269] Where α represents the iteration coefficient, and the initial value of the posterior observation noise covariance matrix is ​​defined as a matrix with all zeros. To observe noise;

[0270] Calculate the observation noise covariance φ u (n):

[0271]

[0272] Because the computation of the first Kalman model described above has been simplified in this application, the computational speed is effectively improved while ensuring the final computational accuracy. The simplified computation of the first Kalman model in this application includes diagonalizing the posterior state error covariance matrix, the prior state error covariance matrix, and the speech signal covariance matrix to reduce computational complexity. The intermediate derivation process for this simplified computation is as follows:

[0273] First, the prior state error covariance matrix is ​​approximately diagonalized; then, it is approximated as an identity matrix multiplied by a coefficient. Define the state prediction covariance matrix as:

[0274]

[0275] in, Indicates that the order of the matrix is ​​L c The identity matrix, This represents the prior state error covariance;

[0276] Similarly, the posterior state error covariance matrix can be approximated as:

[0277]

[0278] in, Indicates that the order of the matrix is ​​L c The identity matrix, Indicates the posterior state covariance;

[0279] Update the existing Kalman filter's golden five rules to the matrix. Approximation:

[0280]

[0281] Where tr{} is the trace symbol;

[0282] Then, the observation noise covariance matrix is ​​approximately diagonalized: that is, the observation noise covariance matrix in the first Kalman model is approximated as follows:

[0283]

[0284] in, I represents the observation noise covariance. M Let M denote the identity matrix of order M, and e(n) denote the state error, where This indicates that the square is obtained after taking the L2 norm.

[0285] The update of the posterior observation noise covariance matrix was also approximated, and is approximated as follows:

[0286]

[0287] in, I represents the posterior observation noise covariance. M It is an identity matrix of order M;

[0288] Therefore, the simplified Kalman filter calculation process of the present invention is as follows:

[0289] Calculate the prior state error covariance

[0290]

[0291] Calculate the state error e(n):

[0292] e(n) = y(n) - X(nD)c(n-1);

[0293] Calculate the Kalman gain K(n):

[0294]

[0295] Calculate the posterior state error covariance

[0296]

[0297] Calculate the autoregressive parameter c(n): c(n) = c(n|n-1) + K(n)e(n);

[0298] And calculate the observation noise.

[0299]

[0300] The above is the detailed derivation process of the simplified process of the first Kalman filter in step S3.

[0301] Preferably, in step S3, "the variance value of the Kalman state noise used in the estimation process of the multi-channel autoregressive parameters is adjusted according to the sound source change detection result of the previous frame" is specifically as follows:

[0302] Calculate the energy of the collected signal and the energy ratio of the estimated direct sound and early signals. When the energy ratio changes suddenly, it is determined that the sound source has changed in the current frame (because the time difference between frames is about 32 ms. If the sound source position change is determined 32 ms before, then it is very likely that the sound source position in the current frame has also changed); and when it is detected that the sound source has changed, the variance value of the state noise in the Kalman derivation process of the dereverberation module is briefly increased to ten times the original until the ratio between the two returns above the threshold, thereby strengthening the tracking ability of state changes;

[0303] Among them, the energy calculation formula of the collected signal is as follows:

[0304]

[0305] The energy calculation formula of the estimated dereverberated signal is as follows:

[0306]

[0307] α py ,α ps are the smoothing coefficients of the energy of the collected signal and the dereverberated signal respectively, P y (n) represents the energy of the nth frame of the collected signal, P s (n) represents the energy of the dereverberated signal (that is, the direct sound and early reverberation) of the nth frame, K represents the number of frequency points in the stft domain, P y (n - 1) refers to the energy of the (n - 1)th frame of the collected signal, y(k, n) refers to the collected signal of the nth frame at the frequency point k, represents the kth frequency point of the calculated direct sound signal, the nth frame signal. Here, since all frequency points k need to be accumulated, therefore, the frequency point k is introduced for calculation. If the frequency point k is not introduced, it means that all frequency points k are calculated in the same way; when the ratio between the two P y (n) / P s (n) < threshold (the threshold is set as needed, such as taking 0.65) is too low, it indicates reverberation leakage, and it is determined that the sound source has changed in the current frame.

[0308] The parameter description table for the first Kalman module of this invention is shown in Table 1 below:

[0309] Table 1

[0310]

[0311] Preferably, in step S4, the noiseless reverberant signal is estimated by creating a second Kalman model, as follows:

[0312] Step S41: Establish the second Kalman model:

[0313] Based on the noiseless band reverberant signal x(n), construct the state vector of the second Kalman filter. x (n) represents the noise-removed signal in the acquired signal within L frames. It is a one-dimensional vector with a length of L×M, where M refers to the number of channels.

[0314] x (n)=[x T (n-L+1),…,x T (n)] T ;

[0315] in, x Each x(l) in (n) represents the noise-free reverberation signal of the l-th frame, which is a vector of length M;

[0316] Meanwhile, the observation noise of the second Kalman module is defined as s (n), which is constructed as follows:

[0317] s(n)=[0 1×M(L-1) s T (n)] T ;

[0318] Construct the state transition matrix c(n) based on the autoregressive parameters estimated by the Kalman filter in the previous stage:

[0319]

[0320] (This indicates that there are more symbols between C_L and C_D, namely C_{L-1}, C_{L-1}, ... up to C_D.)

[0321] Construct the observation matrix H in the observation equation:

[0322] H = [0 M×M(L-1) I M ];

[0323] Among them, I M Represents the identity matrix of order M; 0 M×M(L-1)This represents a matrix of size M rows and M(L-1) columns, consisting entirely of zeros.

[0324] Therefore, the state transition equation and observation equation for the second Kalman filter are constructed as follows:

[0325] x (n)=F(n) x (n-1)+ s (n);

[0326] y(n)=H x (n)+v(n);

[0327] Where v(n) represents the noise signal collected by the microphone; y(n) represents the collected signal;

[0328] Step S42: Calculate the second Kalman filter:

[0329] Calculate the prior state error covariance matrix

[0330]

[0331] Where, Φ s (n) represents the covariance matrix of the state noise; This represents the prior state error covariance matrix of the second Kalman module. This represents the posterior state error covariance matrix of the previous frame of the second Kalman module;

[0332] Calculate the prior state vector x (n|n-1):

[0333] x (n|n-1)=F(n) x (n-1);

[0334] Where F(n) represents the state transition matrix, x (n-1) represents the estimated state vector value of the previous frame. x (n|n-1) represents the prior state vector of the current frame;

[0335] Calculate the Kalman gain K x (n):

[0336]

[0337] Calculate the state error e x (n):

[0338] e x (n)=y(n)-H x (n|n-1);

[0339] Among them, K x (n) represents the Kalman gain of the second Kalman module. Let y(n) represent the noise covariance matrix, and e represent the microphone signal. x (n) represents the state error of the second Kalman module;

[0340] Calculate the posterior state error covariance matrix

[0341]

[0342] Calculate the state vector x (n):

[0343] x (n)= x (n|n-1)+K x (n)e x (n);

[0344] from x Obtain the estimated noise-free reverberation signal from (n).

[0345] x (n) is a one-dimensional vector of length L×M. Taking the last M lengths of this vector yields the estimated noise-free reverberation signal.

[0346] Preferably, the covariance matrix Φ of the state noise in the second Kalman filter calculation s (n) is obtained in the following way:

[0347] For each frame of signal, estimate the prior covariance matrix; then combine it with the posterior covariance matrix calculated in the previous frame to estimate the covariance matrix of the current frame.

[0348]

[0349] Wherein, γ is the weighting parameter for balancing the prior and posterior fits, Φ s (n) represents the covariance matrix of the state noise. This represents the posterior state noise covariance matrix of the previous frame. This represents the prior state noise covariance matrix estimated for the current frame;

[0350] The posterior covariance matrix is ​​obtained through smooth iteration between time frames:

[0351]

[0352] Where α represents the smoothing coefficient. This represents the estimated early reverberation signal and the direct sound signal;

[0353] The prior covariance matrix is ​​obtained using a multi-channel Wiener filtering algorithm:

[0354]

[0355] in The Wiener filter weight matrix of size [M×M] is calculated as follows:

[0356]

[0357] in, Let be the covariance matrix of the noise signal;

[0358] Φ r (n) is the covariance matrix of the noise-free late-stage reverberation signal, which can be iteratively calculated using the product of the first Kalman filter; Φ y (n) is the covariance matrix of the microphone-acquired signal, which is also obtained through smooth iteration between time frames.

[0359] Preferably, step 5 specifically comprises:

[0360] The estimated noiseless band reverberation signal (i.e., the noise-free acquisition signal) is obtained after a D-frame delay.

[0361] The delayed noiseless reverberation signal and reverberation autoregressive parameters The product yields the predicted late-stage reverberation signal:

[0362]

[0363] From the estimated noiseless band reverberation signal Subtract the estimated late reverberation signal The estimated early reverberation signal and direct sound signal were obtained.

[0364]

[0365] The technical solutions provided by the above embodiments of the present invention have at least the following advantages:

[0366] The algorithm is simplified by approximately fully diagonalizing the first Kalman filter state vector error covariance matrix and the observation noise covariance matrix, avoiding large-scale matrix inversion steps and greatly reducing computational complexity. This enables real-time dereverberation and noise reduction in embedded products. A technique for detecting changes in the reverberation environment is proposed. A sound source change detection module is implemented based on the energy ratio before and after the dereverberation module. When the position changes, the state transition noise variance is increased to accelerate the convergence speed and enhance the algorithm's reverberation tracking capability, enabling the algorithm to complete the reverberation path convergence within 100ms. Combining the time-varying nature of noise and reverberation path, a cascaded sequence is constructed that prioritizes reverberation path estimation before noise reduction, solving the dereverberation problem in low signal-to-noise ratio environments.

[0367] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A low-complexity multi-channel dereverberation and noise reduction method based on Kalman filtering, characterized in that: Includes the following steps: Step S1: Acquire the signal and preprocess the acquired signal to obtain the signal in the short-time Fourier domain; Step S2: Calculate the multi-channel noise covariance matrix using the acquired signal in the short-time Fourier domain; Step S3: Using the delayed, reverberant, noise-free signal estimated in the previous frame and the acquired signal in the current frame, estimate the time-varying multi-channel autoregressive parameters based on the Kalman filter algorithm; the variance of the Kalman state noise used in the estimation of the multi-channel autoregressive parameters is adjusted according to the sound source change detection results of the previous frame, as follows: The system calculates the energy of the acquired signal and the energy ratio of the estimated direct sound and the early signal. When the energy ratio changes abruptly, it is determined that the sound source in the current frame has changed. When a change in the sound source is detected, the state noise variance value in the Kalman derivation process of the deriver module is briefly increased by ten times. The energy calculation formula for the acquired signal is as follows: ; The formula for calculating the energy of the estimated dereverberated signal is as follows: ; These are the smoothing coefficients for the acquired signal energy and the dedeverted signal energy, respectively. This represents the energy of the signal acquired in the nth frame. Let y(k,n) represent the energy of the dederenced signal in the nth frame, K represent the number of frequency points in the STRIT domain, and y(k,n) refer to the acquired signal in the nth frame at frequency k. This represents the k-th frequency point of the calculated direct sound signal, and the n-th frame signal; when the ratio of the two is... When this happens, it is determined that the sound source in the current frame has changed; Step S4: Estimate the noise-free band reverberant signal using the estimated autoregressive parameters, the acquired signal of the current frame, and the estimated multi-channel noise covariance matrix; Step S5: Delay the estimated noiseless band reverberation signal and calculate the estimated noiseless late reverberation signal using autoregressive coefficients; subtract the estimated noiseless late reverberation signal from the estimated noiseless band reverberation signal to obtain the desired direct sound and early reverberation signal.

2. The method according to claim 1, characterized in that: Step S1 specifically involves: Assuming an unknown number of sound sources exist in a reverberant environment, and M sub-microphones fixed at arbitrary positions are used for sound acquisition, then give the acquired signal. Representation of a domain: ; in, It is the frequency domain representation of the k-th sub-band of the m-th signal in the n-th frame; Assume the multi-channel microphone signal consists of two parts: ; Where, vector as well as These represent the reverberant speech signal and additive noise collected by the microphones on the array, respectively. The reverberant speech signal The expression is as follows: ; Where, vector This indicates the direct sound and early reverberation signals that are desired to be acquired from the collected signal. domain, Indicates the first The first of the wheat Frame number Sub-band frequency domain representation, matrix , indicating that for the first frame acquisition signal domain Filtering parameters; Here, L represents the delay parameter, and L represents the filter length. This is a late-stage reverberation signal.

3. The method according to claim 1, characterized in that: The calculation process of the multi-channel noise covariance matrix in step S2 is as follows: Step a1: Preset instantaneous posterior signal-to-noise ratio threshold and long-term posterior signal-to-noise ratio threshold ; Step a2: Initialize the covariance matrix of the acquired signal and noise covariance matrix ; Step a3, before the algorithm Within a frame, assuming the initial acquired signal contains only noise, the... The number of frames representing the initial pure noise in the audio; Iterative calculation of the covariance matrix of the acquired signal and the noise covariance matrix: ; ; in, The iteration coefficients for the acquired signal are H, where H represents the matrix conjugate transpose operation. Step a4, in After the frame, the following calculations are performed: Step a41: Iteratively estimate the covariance matrix of the acquired signal: ; Step a42: Considering the non-correlation between the speech signal and the noise signal, estimate the covariance matrix of the speech signal: ; Step a43: Calculate the instantaneous posterior signal-to-noise ratio: ; Step a44: Calculate the long-term posterior signal-to-noise ratio: ; Where tr{.} represents the trace operation of a matrix; Step a45: Calculate the prior signal-to-noise ratio: ; in, This indicates the number of channels, i.e., the number of sub-channels; ； Step a46: Calculate the probability of the existence of smooth iterative speech: Calculate the probability that speech does not exist at the local scale: , ; Other calculations include the posterior signal-to-noise ratio for windowed smoothing and the probability that the smoothed speech does not exist. ; in This represents the Hanning window function, with the window length defined as... ; ； Calculate the posterior signal-to-noise ratio mean of each frequency point in the nth frame, and calculate the probability of speech non-existence at the frame scale: ; ; Calculate the probability of speech not existing by combining three scales : ; Based on the estimated probability of speech absence, the probability of multi-channel prior speech presence is calculated. : ; Step a47: Calculate the probability of the existence of smooth iterative speech: ; in, A smoothing coefficient representing the probability of speech occurrence; Step a48: Determine the smoothing coefficient for the noise covariance matrix estimation based on the speech presence probability, and update the multi-channel noise covariance matrix: ; ; in The iteration coefficients of the signal, The noise covariance matrix is... This is the noise covariance matrix estimated from the previous frame; At this point, the noise covariance matrix estimation is complete.

4. The method according to claim 1, characterized in that: The estimation of the autoregressive parameters in step S3 is as follows: Step 31: Establish the first Kalman model: The Kalman observation matrix is ​​constructed as follows: ; in Represents the Kronecker product. express The identity matrix is ​​3D, with the superscript T indicating the vector transpose operation, and x(n) represents the noise-free reverberant signal of the nth frame; the autoregressive parameters are defined as the state vector of the Kalman module as follows: ; It is a part of the state vector. The subscript L indicates that this part is the autoregressive parameters for the (nL)th frame. The ellipsis in the middle indicates that the autoregressive parameters for the corresponding (nL) to (nD) frames are omitted. This is a matrix straightening operation, which means concatenating the columns of the matrix within the curly braces from left to right to obtain a new vector. , length ; It is a shape of sparse matrix; Step 32, Calculation steps for the first Kalman filter module: Step 321: Calculate the prior state error covariance: ; in, Represents the state noise covariance; Step 322: Calculate the state error e(n): ; Where y(n) represents the microphone acquisition signal, X(nD) represents the observation matrix, and c(n-1) represents the autoregressive parameters calculated in the previous frame; Step 323: Calculate the Kalman gain K(n): ; in, Represents the observation noise covariance; Step 324: Calculate the posterior state error covariance. : ; Step 325: Calculate the autoregressive parameter c(n): ; Step 326: Calculate the observation noise : .

5. The method according to claim 4, characterized in that: The state noise covariance Obtained through the following methods: State noise covariance The size is determined by the amount of change in the autoregressive parameters between two adjacent frames, while a very small positive number is added to simulate the actual continuous change when the estimated autoregressive parameters between two adjacent frames do not change. ; For state noise variance, The length of the autoregressive parameter c(n); The observation noise covariance Obtained through the following methods: Calculate the prior observation noise covariance matrix: ; Combined with the posterior observation noise covariance matrix calculated in the previous frame Calculate the observation noise covariance matrix of the current frame: ; in, The initial value of the posterior observation noise covariance matrix is ​​defined as a matrix consisting entirely of zeros, representing the iteration coefficients. To observe noise; Calculate the observation noise covariance : .

6. The method according to claim 1, characterized in that: In step S4, the noiseless reverberant signal is estimated by creating a second Kalman model, as follows: Step S41: Establish the second Kalman model: Based on the noiseless band reverberant signal x(n), construct the state vector of the second Kalman filter. , characterization The noise-removed signal in the intra-frame acquisition signal is a one-dimensional vector with a length of Where M refers to the number of channels; ; in, Each x(l) in the vector represents the noise-free reverberant signal of the l-th frame, which is a vector of length M; Meanwhile, the observation noise of the second Kalman module is defined as Its construction method is as follows: ; Based on the autoregressive parameter estimation of the previous stage Kalman filter, Construct the state transition matrix: ; Constructing the observation matrix in the observation equation : ; in, Represents the identity matrix of order M; This represents a matrix of size M rows and M(L-1) columns, consisting entirely of zeros. Therefore, the state transition equation and observation equation for the second Kalman filter are constructed as follows: ; ; in, This represents the noise signal captured by the microphone; Indicates the acquired signal; Step S42: Calculate the second Kalman filter: Calculate the prior state error covariance matrix ; ； in, The covariance matrix representing the state noise; This represents the prior state error covariance matrix of the second Kalman module. This represents the posterior state error covariance matrix of the second Kalman module in the previous frame; Calculate the prior state vector : ; Where F(n) represents the state transition matrix, This represents the estimated state vector value from the previous frame. This represents the prior state vector of the current frame; Calculate Kalman gain : ; Calculate state error : ; in, For the Kalman gain of the second Kalman module, Let represent the noise covariance matrix, and y(n) represent the microphone signal. This indicates the state error of the second Kalman module; Calculate the posterior state error covariance matrix : ; Calculate the state vector : ; from Obtain the estimated noise-free reverberation signal : Given a one-dimensional vector of length L×M, the predicted noise-free reverberation signal can be obtained by taking the last M lengths of this vector. .

7. The method according to claim 6, characterized in that: The covariance matrix of the state noise in the second Kalman filter calculation is obtained as follows: For each frame of signal, estimate the prior covariance matrix; then combine it with the posterior covariance matrix calculated in the previous frame to estimate the covariance matrix of the current frame. ; in, The parameters are weighted parameters that weigh the prior and posterior matching. The covariance matrix represents the state noise. This represents the posterior state noise covariance matrix of the previous frame. This represents the prior state noise covariance matrix estimated for the current frame; The posterior covariance matrix is ​​obtained through smooth iteration between time frames: ; in, Represents the smoothing coefficient. This represents the estimated early reverberation signal and the direct sound signal; The prior covariance matrix is ​​obtained using a multi-channel Wiener filtering algorithm: ; in for The Wiener filter weight matrix of the specified size is calculated as follows: ; in, Let be the covariance matrix of the noise signal; The covariance matrix of the noise-free late reverberation signal can be iteratively calculated using the product of the first Kalman filter. The covariance matrix of the signal acquired by the microphone is also obtained through smooth iteration between time frames.

8. The method according to claim 1, characterized in that: Step S5 specifically involves: The estimated noiseless band reverberation signal Obtained after D-frame delay ; The delayed noiseless reverberation signal and reverberation autoregressive parameters The product yields the predicted late-stage reverberation signal: ; From the estimated noiseless band reverberation signal Subtract the estimated late reverberation signal The estimated early reverberation signal and direct sound signal were obtained. : .

Images