Voice de-reverberation method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to the technical field of audio processing, and provides a voice de-reverberation method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: carrying out two-stage filtering dereverberation operation according to a multi-frame voice frequency domain delay signal, a first-stage observation equation and a second-stage observation equation to obtain an estimated value of a second-stage current frame voice frequency domain dereverberation signal so as to obtain a current frame voice time domain dereverberation signal; when the filtering reverberation removal operation at the stage is carried out, obtaining a state space observation value of the current frame at the stage according to a multi-frame voice frequency domain delay signal, an observation equation at the stage and a spatial regression coefficient required by the stage; according to the posterior estimation value of the error covariance diagonal matrix of the previous frame in the current stage, obtaining a priori estimation value of the error covariance diagonal matrix of the current frame in the current stage; and according to the state space observation value of the stage, obtaining an estimated value of the current frame voice frequency domain de-reverberation signal of the stage. By adopting the method, the occupied storage resources can be reduced.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to a speech dereverberation method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] In indoor call scenarios, such as indoor voice calls or remote video conferences, due to the reflection of indoor objects or walls, the sound signal picked up by the sound pickup device contains reverberation sound components in addition to the direct sound components. The late reflection sound in the reverberation sound component will seriously damage the voice quality and intelligibility, causing the performance of the sound processing algorithm of the sound pickup system to drop significantly. Therefore, speech dereverberation has important practical significance.

[0003] Speech dereverberation can be performed based on a spectral enhancement method, an inverse filtering method, or a multi-channel linear prediction method. Among these methods, the multi-channel linear prediction method is one of the most popular technologies. This method first uses the microphone array to receive signals to predict the reverberation components in these signals, and then subtracts the estimated reverberation components from the received signals to achieve the purpose of dereverberation. In practical applications, multi-channel linear prediction filters are usually estimated in an adaptive manner. In order to reduce the computational complexity, the filter coefficient matrix can be decomposed into multiple low-dimensional matrices by using the Kronecker integral decomposition method. However, there is still the problem of needing to update multiple sub-matrices, which results in the need to cache the above sub-matrices during the algorithm processing, resulting in the problem of excessive storage resource usage. Summary of the invention

[0004] Based on this, it is necessary to provide a speech dereverberation method, apparatus, computer device, computer-readable storage medium and computer program product to address the above technical problems.

[0005] In a first aspect, the present application provides a speech dereverberation method, comprising:

[0006] Obtaining a multi-frame audio-domain delayed signal according to the audio-domain signal of the current frame and the audio-domain signals of multiple historical frames;

[0007] A two-stage filtering and dereverberation operation is performed according to the multi-frame speech frequency domain delay signal, the first-stage observation equation and the second-stage observation equation to obtain an estimated value of the second-stage current frame speech frequency domain dereverberation signal, and then a posterior update operation is performed to form a speech time domain dereverberation signal of the current frame; the first-stage observation equation and the second-stage observation equation are obtained by performing Kronecker integral decomposition on the spatial regression coefficients in the initial observation equation;

[0008] Wherein, when performing the filtering and dereverberation operation of this stage, the state space observation value of this stage corresponding to the current frame is obtained according to the multi-frame speech frequency domain delay signal, the observation equation of this stage and the spatial regression coefficient required by this stage; according to the posterior estimation value of the error covariance diagonal matrix of the previous frame in this stage, the prior estimation value of the error covariance diagonal matrix of the current frame in this stage is updated; the error covariance diagonal matrix of the current frame in this stage is obtained by approximately fully diagonalizing the error covariance matrix of the current frame in this stage; according to the state space observation value of this stage, the estimation value of the speech frequency domain dereverberation signal of the current frame in this stage is updated;

[0009] If this stage is a two-stage stage, the spatial regression coefficient required for this stage is the estimated value of the spatial regression coefficient of the current frame of the first stage; the estimated value of the spatial regression coefficient of the current frame of the first stage is obtained by the following steps: according to the estimated value of the speech frequency domain dereverberation signal of the current frame of the first stage, the estimated value of the variance diagonal matrix of the speech frequency domain dereverberation signal of the current frame of the first stage is updated; the variance diagonal matrix is ​​obtained by approximately fully diagonalizing the variance matrix; according to the state space observation value of the first stage corresponding to the current frame, the diagonal multiplication observation value of the first stage corresponding to the current frame, and the prior estimate of the error covariance diagonal matrix of the current frame of the first stage The calculated value and the estimated value of the variance diagonal matrix of the speech-domain dereverberation signal of the current frame in the first stage are used to update the Kalman gain coefficient of the current frame in the first stage; the diagonal multiplication observation value corresponding to the current frame in the first stage is obtained by multiplying the state-space observation value corresponding to the current frame in the first stage and the conjugate transposed observation value of the state-space corresponding to the current frame in the first stage and performing approximate full diagonalization; the estimated value of the speech-domain dereverberation signal of the current frame in the first stage, the Kalman gain coefficient of the current frame in the first stage and the estimated value of the spatial regression coefficient of the previous frame in the first stage are used to update the estimated value of the spatial regression coefficient of the current frame in the first stage.

[0010] In one embodiment, if the current stage is the first stage, the spatial regression coefficient required in the current stage is the estimated value of the spatial regression coefficient of the upper frame of the second stage; obtaining the estimated value of the spatial regression coefficient of the upper frame of the second stage specifically includes:

[0011] According to the estimated value of the speech-domain dereverberation signal of the second-stage upper frame, an estimated value of the variance diagonal matrix of the speech-domain dereverberation signal of the second-stage upper frame is updated; the variance diagonal matrix is ​​obtained by approximately fully diagonalizing the variance matrix;

[0012] According to the two-stage state space observation value corresponding to the previous frame, the two-stage diagonal multiplication observation value corresponding to the previous frame, the prior estimation value of the two-stage upper frame error covariance diagonal matrix and the estimation value of the variance diagonal matrix of the two-stage upper frame speech domain dereverberation signal, the two-stage upper frame Kalman gain coefficient is updated; the two-stage diagonal multiplication observation value corresponding to the previous frame is obtained by multiplying the two-stage state space observation value corresponding to the previous frame and the two-stage state space conjugate transposed observation value corresponding to the previous frame and performing approximate full diagonalization;

[0013] The second-stage upper frame spatial regression coefficient estimate is updated according to the second-stage upper frame speech domain dereverberation signal estimate, the second-stage upper frame Kalman gain coefficient and the second-stage upper-upper frame spatial regression coefficient estimate.

[0014] In one embodiment, obtaining a multi-frame audio-domain delayed signal according to the audio-domain signal of the current frame and the audio-domain signals of multiple historical frames includes:

[0015] Acquire the speech time domain signal of the current frame through the microphone array;

[0016] Performing windowing processing on the speech time domain signal to obtain a windowed speech time domain signal;

[0017] Performing Fourier transform on the windowed speech time domain signal to obtain a speech frequency domain signal of the current frame;

[0018] Obtaining a multi-frame speech frequency domain signal according to the speech frequency domain signal of the current frame and the speech frequency domain signals of the historical multi-frames;

[0019] According to the time delay coefficient and the multi-frame voice frequency domain signal, a multi-frame voice frequency domain delay signal is obtained.

[0020] In one embodiment, forming a speech time-domain dereverberation signal of a current frame according to an estimated value of a speech time-domain dereverberation signal of a current frame in the second stage includes:

[0021] According to the estimated value of the second-stage current frame speech domain dereverberation signal, the second-stage state space observation value corresponding to the current frame and the second-stage current frame Kalman gain coefficient, a posteriori filtering update is performed to obtain the speech domain dereverberation signal of the current frame;

[0022] Performing inverse Fourier transform on the speech audio-domain dereverberation signal of the current frame to obtain the speech time-domain dereverberation signal of the current frame.

[0023] In one embodiment, the updating of the prior estimation value of the current frame error covariance diagonal matrix at this stage according to the a posteriori estimation value of the frame error covariance diagonal matrix at this stage includes:

[0024] The spatial regression coefficient required in the present stage is modeled as a first-order Markov process, and Kronecker integral decomposition is performed to obtain the state equation of the present stage;

[0025] According to the complex Gaussian disturbance noise of the current frame of the current stage in the state equation of the current stage, the variance of the complex Gaussian disturbance noise of the current frame of the current stage is obtained;

[0026] Obtaining a posterior estimate of the frame error covariance diagonal matrix at the current stage;

[0027] According to the a posteriori estimate of the frame error covariance diagonal matrix at the current stage and the sum of the variance of the complex Gaussian disturbance noise of the current frame at the current stage, the a priori estimate of the current frame error covariance diagonal matrix at the current stage is updated.

[0028] In one embodiment, obtaining the a posteriori estimate of the frame error covariance diagonal matrix at the current stage includes:

[0029] Multiply the state space observation value of this stage corresponding to the previous frame and the Kalman gain coefficient of the previous frame of this stage to obtain the multiplication matrix of this stage;

[0030] Subtract the current stage unit matrix from the current stage multiplication matrix to obtain the current stage subtraction matrix;

[0031] Approximately fully diagonalizing the subtraction matrix of this stage to obtain a subtraction diagonal matrix of this stage;

[0032] According to the a priori estimated value of the subtraction diagonal matrix of the current stage and the previous frame error covariance diagonal matrix of the current stage, the a posteriori estimated value of the previous frame error covariance diagonal matrix of the current stage is obtained.

[0033] In one embodiment, the one-stage state space observation value corresponding to the current frame and the one-stage state space conjugate transposed observation value corresponding to the current frame are obtained, including:

[0034] According to the one-stage state space observation value corresponding to the current frame, the one-stage state space conjugate transposed observation value corresponding to the current frame is obtained;

[0035] Multiply the one-stage state space observation value corresponding to the current frame and the one-stage state space conjugate transposed observation value corresponding to the current frame to obtain the one-stage multiplied observation value corresponding to the current frame;

[0036] The one-stage multiplication observation values ​​corresponding to the current frame are approximately fully diagonalized to obtain the one-stage diagonal multiplication observation values ​​corresponding to the current frame.

[0037] In a second aspect, the present application also provides a speech dereverberation device, comprising:

[0038] A frequency domain delayed signal acquisition module, used to obtain a multi-frame frequency domain delayed signal according to the frequency domain signal of the current frame and the frequency domain signals of multiple historical frames;

[0039] A time domain dereverberation signal acquisition module is used to perform two-stage filtering dereverberation operations according to the multi-frame audio domain delay signals, the first-stage observation equation and the second-stage observation equation to obtain an estimated value of the second-stage current frame audio domain dereverberation signal to form a speech time domain dereverberation signal of the current frame; the first-stage observation equation and the second-stage observation equation are obtained by performing Kronecker integral decomposition on the spatial regression coefficients in the initial observation equation;

[0040] The filtering and dereverberation operation module of this stage is used to obtain the state space observation value of this stage corresponding to the current frame according to the multi-frame speech frequency domain delay signal, the observation equation of this stage and the spatial regression coefficient required by this stage when performing the filtering and dereverberation operation of this stage; update the prior estimation value of the error covariance diagonal matrix of the current frame of this stage according to the posterior estimation value of the error covariance diagonal matrix of the previous frame of this stage; the error covariance diagonal matrix of the current frame of this stage is obtained by approximately fully diagonalizing the error covariance matrix of the current frame of this stage; update the estimation value of the speech frequency domain dereverberation signal of the current frame of this stage according to the state space observation value of this stage;

[0041] The spatial regression coefficient acquisition module is used for, if the current stage is the two-stage stage, then the spatial regression coefficient required in this stage is the estimated value of the spatial regression coefficient of the current frame of the first stage; the estimated value of the spatial regression coefficient of the current frame of the first stage is obtained by the following steps: according to the estimated value of the speech frequency domain dereverberation signal of the current frame of the first stage, the estimated value of the variance diagonal matrix of the speech frequency domain dereverberation signal of the current frame of the first stage is updated; the variance diagonal matrix is ​​obtained by approximately fully diagonalizing the variance matrix; according to the state space observation value of the first stage corresponding to the current frame, the diagonal multiplication observation value of the first stage corresponding to the current frame, the error covariance matrix of the current frame of the first stage The prior estimated value of the diagonal matrix and the estimated value of the variance diagonal matrix of the speech-domain dereverberation signal of the current frame in the first stage are used to update the Kalman gain coefficient of the current frame in the first stage; the diagonal multiplication observation value corresponding to the current frame in the first stage is obtained by multiplying the state-space observation value corresponding to the current frame in the first stage and the conjugate transposed observation value of the state-space corresponding to the current frame in the first stage and performing approximate full diagonalization; the estimated value of the spatial regression coefficient of the current frame in the first stage is updated according to the estimated value of the speech-domain dereverberation signal of the current frame in the first stage, the Kalman gain coefficient of the current frame in the first stage and the estimated value of the spatial regression coefficient of the previous frame in the first stage.

[0042] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor executes the above method.

[0043] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to execute the above method.

[0044] In a fifth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and the computer program is executed by a processor to execute the above method.

[0045] The speech dereverberation method, apparatus, computer device, computer-readable storage medium and computer program product obtain a multi-frame audio-domain delayed signal according to the audio-domain signal of the current frame and the audio-domain signals of multiple historical frames; perform two-stage filtering dereverberation operations according to the multi-frame audio-domain delayed signal, the first-stage observation equation and the second-stage observation equation to obtain an estimated value of the second-stage audio-domain dereverberation signal of the current frame, and then form a speech time-domain dereverberation signal of the current frame through a posterior update operation; the first-stage observation equation and the second-stage observation equation are to perform spatial regression coefficients in the initial observation equation. The Kronecker integral is decomposed; wherein, when performing the filtering and reverberation operation at this stage, the state space observation value of this stage corresponding to the current frame is obtained according to the multi-frame speech frequency domain delay signal, the observation equation of this stage and the spatial regression coefficient required at this stage; according to the posterior estimate of the diagonal matrix of the error covariance of the previous frame at this stage, the prior estimate of the diagonal matrix of the error covariance of the current frame at this stage is updated; the diagonal matrix of the error covariance of the current frame at this stage is obtained by approximately fully diagonalizing the error covariance matrix of the current frame at this stage; according to the state space observation value of this stage, the speech frequency domain of the current frame at this stage is updated. Estimated value of dereverberation signal; if this stage is a two-stage stage, the spatial regression coefficient required for this stage is the estimated value of the spatial regression coefficient of the current frame of the first stage; the estimated value of the spatial regression coefficient of the current frame of the first stage is obtained by the following steps: according to the estimated value of the speech domain dereverberation signal of the current frame of the first stage, the estimated value of the variance diagonal matrix of the speech domain dereverberation signal of the current frame of the first stage is updated; the variance diagonal matrix is ​​obtained by approximately fully diagonalizing the variance matrix; according to the state space observation value of the first stage corresponding to the current frame, the diagonal multiplication observation value of the first stage corresponding to the current frame, and the error covariance diagonal matrix of the current frame of the first stage The prior estimate of the matrix and the estimate of the variance diagonal matrix of the speech-domain dereverberation signal of the current frame in one stage are used to update the Kalman gain coefficient of the current frame in one stage; the one-stage diagonal multiplication observation corresponding to the current frame is obtained by multiplying the one-stage state-space observation corresponding to the current frame and the one-stage state-space conjugate transposed observation corresponding to the current frame and performing approximate full diagonalization; the one-stage current frame spatial regression coefficient estimate is updated according to the estimate of the speech-domain dereverberation signal of the current frame in one stage, the one-stage current frame Kalman gain coefficient and the one-stage upper frame spatial regression coefficient estimate.The present application uses a full diagonal Kalman filter to replace the original Kalman filter, approximates the full diagonalization of the error covariance matrix of the current frame in this stage to obtain the error covariance diagonal matrix of the current frame in this stage, approximates the full diagonalization of the variance matrix to obtain the variance diagonal matrix, multiplies the state space observation value corresponding to the current frame and the conjugate transposed observation value of the state space of the first stage corresponding to the current frame and approximates the full diagonalization to obtain the diagonal multiplication observation value of the first stage corresponding to the current frame, updates the Kalman gain coefficient of the current frame in one stage according to the state space observation value corresponding to the current frame, the diagonal multiplication observation value corresponding to the current frame, the prior estimate of the error covariance diagonal matrix of the current frame in one stage and the estimate of the variance diagonal matrix of the speech domain dereverberation signal of the current frame in one stage, reduces the dimensions of the error covariance matrix, the variance matrix and the Kalman gain coefficient, and only needs to store the diagonal elements of the error covariance matrix, the variance matrix and the Kalman gain coefficient, which greatly reduces the amount of parameters to be stored, thereby reducing the occupied storage resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A diagram showing an application environment of a speech dereverberation method in an embodiment;

[0048] Figure 2 A schematic diagram of a reverberation application environment of a speech dereverberation method in one embodiment;

[0049] Figure 3 A schematic diagram of a flow chart of a speech dereverberation method in one embodiment;

[0050] Figure 4 A schematic flow chart of a speech dereverberation method according to another embodiment;

[0051] Figure 5 A schematic diagram of a flow chart of a first-stage filtering and dereverberation operation in an embodiment;

[0052] Figure 6 A schematic diagram of a flow chart of a two-stage filtering and dereverberation operation in an embodiment;

[0053] Figure 7 is a structural block diagram of a speech dereverberation device in one embodiment;

[0054] Figure 8FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0056] The present application embodiment provides a method for voice dereverberation, which can be executed by a computer device, such as Figure 1 As shown, the computer device can obtain the speech frequency domain signal of the current frame and the speech frequency domain signals of multiple historical frames, and then obtain the speech time domain dereverberation signal of the current frame. It is understandable that the computer device can be implemented by a server, or by a terminal, or by an interactive system between a terminal and a server. The speech dereverberation method provided in the present application is applicable to a microphone array system composed of a single or multiple microphones. Assuming that there is a uniform linear array composed of M omnidirectional microphones in a reverberation environment, the application scenarios of the speech dereverberation method are as follows: Figure 2 In this embodiment, the speech dereverberation method includes: Figure 3 The steps shown are:

[0057] Step S301 : obtaining a multi-frame delayed speech-domain signal according to a speech-domain signal of a current frame and speech-domain signals of multiple historical frames.

[0058] The speech time domain signal of the current frame and the speech time domain signals of the historical multiple frames may be subjected to Fourier transformation to obtain the speech frequency domain signal of the current frame and the speech frequency domain signals of the historical multiple frames.

[0059] Assuming that the speech reverberation signal is generated by a multi-channel autoregressive (MAR) process, according to the approximate relationship of the convolution transfer function, the speech time domain signal received by the microphone array is After short-time Fourier transform, the obtained speech domain signal can be expressed as formula (1).

[0060] (1)

[0061] in, represents the speech domain signal of the microphone array, represents the autoregressive coefficient, represents a pure speech signal, is the frequency band index, is the frame number index, superscript represents the transpose operation, Represents the delay coefficient, which is used to distinguish early reflected sound from late reflected sound. The delay coefficient is related to the frame length. Represents the linear convolution length.

[0062] A multi-frame speech frequency domain signal can be obtained according to the speech frequency domain signal of the current frame and the speech frequency domain signals of multiple historical frames; a multi-frame speech frequency domain delayed signal can be obtained according to the delay coefficient D and the multi-frame speech frequency domain signal.

[0063] Step S302, performing two-stage filtering and dereverberation operations according to the multi-frame audio-domain delayed signals, the first-stage observation equation and the second-stage observation equation to obtain an estimated value of the second-stage current frame audio-domain dereverberation signal, and then forming the speech time-domain dereverberation signal of the current frame through a posteriori update operation; the first-stage observation equation and the second-stage observation equation are obtained by performing Kronecker integral decomposition on the spatial regression coefficients in the initial observation equation.

[0064] You can define the following variables:

[0065] (2)

[0066] (3)

[0067] in, represent The identity matrix of is the Kronecker product, is a column vector stacking operation. Formula (2) represents the stacking of historical multi-frame speech domain signals, and the Kronecker product is used to ensure that different channels use the same multi-frame speech domain delay signal. The spatial regression coefficient in formula (3) corresponds to formula (2) one by one. Formula (2) and (3) can be used to rewrite formula (1) as follows:

[0068] (4)

[0069] According to the adaptive filtering theory, formula (4) can be regarded as the observation equation in the Kalman filtering process, and the observation equation of formula (4) is used as the initial observation equation.

[0070] The spatial regression coefficient in the initial observation equation shown in formula (4) can be decomposed by Kronecker integral. By using the properties of Kronecker integral decomposition, the spatial regression coefficient can be decomposed into two low-dimensional coefficients. ,in and The lengths are and Substituting the vector into formula (1), the initial observation equation can be rewritten to obtain the initial observation rewritten equation shown in formula (5).

[0071] (5)

[0072] According to the properties of the Kronecker product, we can obtain equivalent relations as shown in equations (6) and (7).

[0073] (6)

[0074] (7)

[0075] Substitute formula (6) and formula (7) into formula (5) and define and The observation equations shown in equations (8) and (9) can be obtained.

[0076] (8)

[0077] (9)

[0078] According to formula (8) and formula (9), two different observation equations can be obtained, and these two observation equations can correspond to two different filter update processes. Although there are differences in the observation equations, there is no order of precedence between the two filter update processes. For a certain update process, the other update process can be regarded as the update process after the observation equation changes. Without loss of generality, this embodiment can use formula (8) as a first-stage observation equation and formula (9) as a second-stage observation equation, or use formula (9) as a first-stage observation equation and formula (8) as a second-stage observation equation, and the two stages are interrelated.

[0079] A first-stage filtering and dereverberation operation can be performed based on the multi-frame audio-domain delayed signal and the first-stage observation equation to obtain the spatial regression coefficient required for the second stage; a second-stage filtering and dereverberation operation can be performed based on the multi-frame audio-domain delayed signal, the second-stage observation equation and the second-stage required spatial regression coefficient to obtain an estimated value of the audio-domain dereverberation signal of the current frame in the second stage, and then a posterior update operation is performed to form a speech time-domain dereverberation signal of the current frame.

[0080] Step S303, when performing the filtering and dereverberation operation of this stage, the state space observation value of this stage corresponding to the current frame is obtained according to the multi-frame speech frequency domain delay signal, the observation equation of this stage and the spatial regression coefficient required by this stage; according to the posterior estimate of the error covariance diagonal matrix of the previous frame in this stage, the prior estimate of the error covariance diagonal matrix of the current frame in this stage is updated; the error covariance diagonal matrix of the current frame in this stage is obtained by approximately fully diagonalizing the error covariance matrix of the current frame in this stage; according to the state space observation value of this stage, the estimate of the speech frequency domain dereverberation signal of the current frame in this stage is updated.

[0081] If this stage is one stage, when performing one stage filtering and reverberation operation, according to the multi-frame speech domain delay signal , the first-stage observation equation and the spatial regression coefficient required for the first stage, and obtain the first-stage state space observation value corresponding to the current frame , as shown in formula (10).

[0082] (10)

[0083] Among them, the one-stage observation equation is calculated as shown in formula (8): .

[0084] The posterior estimate of the diagonal matrix of the frame error covariance in the previous stage can be , update the prior estimate of the diagonal matrix of the current frame error covariance , as shown in formula (11).

[0085] (11)

[0086] in, Represents the variance of the complex Gaussian perturbation noise of the current frame in one stage; the diagonal matrix of the error covariance of the current frame in one stage is the covariance matrix of the error covariance of the current frame in one stage The result obtained by approximate full diagonalization is shown in formula (12).

[0087] (12)

[0088] in, express The identity matrix of .

[0089] According to the observation value of the state space in one stage , update the estimated value of the dereverberation signal in the speech domain of the current frame in the first stage , as shown in formula (13).

[0090] (13)

[0091] in, Represents the current frame speech domain signal; Represents the estimated value of the spatial regression coefficient of the previous frame in one stage.

[0092] If this stage is a two-stage filter, when performing a two-stage filter demeverberation operation, according to the multi-frame speech frequency domain delay signal , the two-stage observation equation and the spatial regression coefficient required for the second stage, and obtain the two-stage state space observation value corresponding to the current frame , as shown in formula (14).

[0093] (14)

[0094] Among them, the two-stage observation equation is calculated as shown in formula (9): .

[0095] The posterior estimate of the diagonal matrix of the frame error covariance in the second stage can be , update the prior estimate of the diagonal matrix of the current frame error covariance in the second stage , as shown in formula (15).

[0096] (15)

[0097] in, Represents the variance of the complex Gaussian perturbation noise of the current frame in the second stage; the diagonal matrix of the error covariance of the current frame in the second stage is the covariance matrix of the error covariance of the current frame in the second stage The result obtained by approximate full diagonalization is shown in formula (16).

[0098] (16)

[0099] in, express The identity matrix of .

[0100] According to the two-stage state space observation , update the estimated value of the speech-domain dereverberation signal of the current frame in the second stage , as shown in formula (17).

[0101] (17)

[0102] in, Represents the current frame speech domain signal; Represents the estimated value of the spatial regression coefficient of the second stage upper frame.

[0103] Step S304, if this stage is two stages, then the spatial regression coefficient required for this stage is the estimated value of the spatial regression coefficient of the current frame of the first stage; the estimated value of the spatial regression coefficient of the current frame of the first stage is obtained by the following steps: according to the estimated value of the speech frequency domain dereverberation signal of the current frame of the first stage, the estimated value of the variance diagonal matrix of the speech frequency domain dereverberation signal of the current frame of the first stage is updated; the variance diagonal matrix is ​​obtained by approximately fully diagonalizing the variance matrix; according to the state space observation value of the first stage corresponding to the current frame, the diagonal multiplication observation value of the first stage corresponding to the current frame, the error covariance diagonal matrix of the current frame of the first stage The prior estimate of the matrix and the estimate of the variance diagonal matrix of the speech-domain dereverberation signal of the current frame in one stage are used to update the Kalman gain coefficient of the current frame in one stage; the one-stage diagonal multiplication observation corresponding to the current frame is obtained by multiplying the one-stage state-space observation corresponding to the current frame and the one-stage state-space conjugate transposed observation corresponding to the current frame and performing approximate full diagonalization; the one-stage current frame spatial regression coefficient estimate is updated according to the estimate of the speech-domain dereverberation signal of the current frame in one stage, the one-stage current frame Kalman gain coefficient and the one-stage previous frame spatial regression coefficient estimate.

[0104] If this stage is the second stage, the spatial regression coefficient required for this stage is the estimated value of the spatial regression coefficient of the current frame in the first stage. .

[0105] One-stage current frame spatial regression coefficient estimate The following steps are used to obtain the estimated value of the dereverberation signal in the current frame of the speech domain: , update the estimated value of the variance diagonal matrix of the dereverberation signal in the speech domain of the current frame in the first stage , as shown in formula (18).

[0106] (18)

[0107] in, represents the forgetting factor, which can be expressed as ; Represents the estimated value of the variance diagonal matrix of the dereverberation signal in the speech domain of the previous frame in one stage; the variance diagonal matrix is ​​the variance matrix The result obtained by approximate full diagonalization is shown in formula (19).

[0108] (19)

[0109] According to the one-stage state space observation value corresponding to the current frame , the one-stage diagonal multiplication observation corresponding to the current frame , the prior estimate of the diagonal matrix of the current frame error covariance in one stage and the estimated value of the variance diagonal matrix of the dereverberation signal in the speech domain of the current frame in the first stage , update the Kalman gain coefficient of the current frame in the first stage , as shown in formula (20).

[0110] (20)

[0111] Among them, the diagonal multiplication observation value corresponding to the current frame It is based on the one-stage state space observation value corresponding to the current frame , and the conjugate transposed observation of the one-stage state space corresponding to the current frame The result obtained by multiplying and approximately fully diagonalizing is shown in formula (21).

[0112] (twenty one)

[0113] According to the estimated value of the speech-domain dereverberation signal of the current frame in the first stage , Kalman gain coefficient of the current frame in the first stage and the estimated regression coefficients of the first-stage upper frame space , update the estimated value of the spatial regression coefficient of the current frame in the first stage , as shown in formula (22).

[0114] (twenty two)

[0115] In the above-mentioned speech dereverberation method, a full diagonal Kalman filter is used to replace the original Kalman filter, the error covariance matrix of the current frame in this stage is approximately fully diagonalized to obtain the error covariance diagonal matrix of the current frame in this stage, the variance matrix is ​​approximately fully diagonalized to obtain the variance diagonal matrix, and the one-stage state space observation value corresponding to the current frame and the one-stage state space conjugate transposed observation value corresponding to the current frame are multiplied to obtain the one-stage diagonal multiplication observation value corresponding to the current frame. According to the one-stage state space observation value corresponding to the current frame, the one-stage diagonal multiplication observation value corresponding to the current frame, the one-stage current frame error covariance diagonal matrix prior estimation value and the one-stage current frame speech domain dereverberation signal variance diagonal matrix estimation value, the one-stage current frame Kalman gain coefficient is updated to obtain, thereby reducing the dimensions of the error covariance matrix, the variance matrix and the Kalman gain coefficient, and only the diagonal elements of the error covariance matrix, the variance matrix and the Kalman gain coefficient need to be stored, which greatly reduces the amount of parameters to be stored, thereby reducing the occupied storage resources.

[0116] In one of the embodiments, if the current stage is the first stage, the spatial regression coefficient required for the current stage is the estimated value of the spatial regression coefficient of the second-stage upper frame; the estimated value of the spatial regression coefficient of the second-stage upper frame is obtained by the following specific steps: according to the estimated value of the speech frequency domain dereverberation signal of the second-stage upper frame, the estimated value of the variance diagonal matrix of the speech frequency domain dereverberation signal of the second-stage upper frame is updated; the variance diagonal matrix is ​​obtained by approximately fully diagonalizing the variance matrix; according to the second-stage state space observation value corresponding to the upper frame, the second-stage diagonal multiplication observation value corresponding to the upper frame, and the second-stage upper frame error covariance matrix ... The prior estimate of the diagonal matrix and the estimate of the variance diagonal matrix of the two-stage upper frame speech domain dereverberation signal are used to update the two-stage upper frame Kalman gain coefficient; the two-stage diagonal multiplication observation corresponding to the upper frame is obtained by multiplying the two-stage state space observation corresponding to the upper frame and the two-stage state space conjugate transposed observation corresponding to the upper frame and approximating full diagonalization; the two-stage upper frame space regression coefficient estimate is updated according to the estimate of the two-stage upper frame speech domain dereverberation signal, the two-stage upper frame Kalman gain coefficient and the two-stage upper-upper frame space regression coefficient estimate.

[0117] If this stage is the first stage, the spatial regression coefficient required for this stage is the estimated value of the spatial regression coefficient of the second stage frame. If the current frame is the initial frame, then the estimated value of the spatial regression coefficient of the second stage upper frame is the estimated value of the initial spatial regression coefficient of the second stage, and the estimated value of the initial spatial regression coefficient of the second stage is ,in Represents a vector of all 1s.

[0118] The estimated value of the spatial regression coefficient of the second stage upper frame is obtained. The specific steps are as follows: According to the estimated value of the speech domain dereverberation signal of the second stage upper frame , update the estimated value of the variance diagonal matrix of the two-stage upper frame speech domain dereverberation signal , as shown in formula (23).

[0119] (twenty three)

[0120] in, represents the forgetting factor, which can be expressed as ; Represents the estimated value of the variance diagonal matrix of the two-stage upper frame speech domain dereverberation signal; the variance diagonal matrix is ​​the variance matrix The result obtained by approximate full diagonalization is shown in formula (24).

[0121] (twenty four)

[0122] According to the two-stage state space observation value corresponding to the previous frame , the two-stage diagonal multiplication observation corresponding to the previous frame , the prior estimate of the diagonal matrix of the frame error covariance in the second stage and the estimated value of the variance diagonal matrix of the two-stage upper frame speech domain dereverberation signal , update the second-stage upper frame Kalman gain coefficient , as shown in formula (25).

[0123] (25)

[0124] Among them, the two-stage diagonal multiplication observation value corresponding to the previous frame It is based on the two-stage state space observation value corresponding to the previous frame , and the conjugate transposed observations of the two-stage state space corresponding to the previous frame The result obtained by multiplying and approximately fully diagonalizing is shown in formula (26).

[0125] (26)

[0126] Estimation of the dereverberation signal in the speech domain based on the second-stage upper frame , the second stage upper frame Kalman gain coefficient and the estimated coefficients of the two-stage upper frame spatial regression , update the estimated value of the spatial regression coefficient of the second stage upper frame , as shown in formula (27).

[0127] (27)

[0128] In this embodiment, if this stage is the first stage, the spatial regression coefficient required for this stage is the estimated value of the spatial regression coefficient of the second-stage upper frame. The estimated value of the spatial regression coefficient of the second-stage upper frame can be updated according to the estimated value of the speech domain dereverberation signal of the second-stage upper frame, the Kalman gain coefficient of the second-stage upper frame and the estimated value of the spatial regression coefficient of the second-stage upper-upper frame.

[0129] In one of the embodiments, a multi-frame speech frequency domain delay signal is obtained according to a speech frequency domain signal of a current frame and a multi-frame speech frequency domain signal of a historical number of frames, and the specific steps are as follows: a speech time domain signal of a current frame is acquired through a microphone array; a speech time domain signal is windowed to obtain a windowed speech time domain signal; a windowed speech time domain signal is Fourier transformed to obtain a speech frequency domain signal of a current frame; a multi-frame speech frequency domain signal is obtained according to the speech frequency domain signal of the current frame and a multi-frame speech frequency domain signal of a historical number of frames; a multi-frame speech frequency domain delay signal is obtained according to a delay coefficient and the multi-frame speech frequency domain signal.

[0130] Can be based on Figure 2The microphone array shown in the figure obtains the speech time domain signal of the current frame. The speech time domain signal can be windowed to obtain the windowed speech time domain signal; the windowed speech time domain signal is Fourier transformed to obtain the speech time domain signal of the current frame.

[0131] The speech-frequency domain signal of the current frame is cached in sequence, and when the number of cached frames is greater than the preset number of delayed frames, the speech-frequency domain signal of multiple historical frames is obtained.

[0132] A multi-frame speech domain signal is obtained according to the speech domain signal of the current frame and the speech domain signals of multiple historical frames.

[0133] According to the delay coefficient D, the multi-frame speech frequency domain signal is delayed to obtain the multi-frame speech frequency domain delayed signal.

[0134] In this embodiment, a relatively accurate speech frequency domain signal of the current frame can be obtained by performing Fourier transform on the windowed speech time domain signal; a multi-frame speech frequency domain signal can be obtained based on the speech frequency domain signal of the current frame and the speech frequency domain signals of multiple historical frames; a multi-frame speech frequency domain delayed signal can be obtained based on the delay coefficient and the multi-frame speech frequency domain signal.

[0135] In one of the embodiments, a speech time-domain dereverberation signal of a current frame is formed according to an estimated value of a two-stage current frame speech frequency-domain dereverberation signal, and the specific steps are as follows: a posteriori filtering update is performed according to the estimated value of the two-stage current frame speech frequency-domain dereverberation signal, the two-stage state space observation value corresponding to the current frame, and the two-stage current frame Kalman gain coefficient to obtain the speech frequency-domain dereverberation signal of the current frame; and an inverse Fourier transform is performed on the speech frequency-domain dereverberation signal of the current frame to obtain the speech time-domain dereverberation signal of the current frame.

[0136] According to the estimated value of the speech-domain dereverberation signal of the current frame in the second stage , the two-stage state space observation corresponding to the current frame and the second-stage current frame Kalman gain coefficient Perform a posteriori filtering update to obtain the speech domain dereverberation signal of the current frame , as shown in formula (28).

[0137] (28)

[0138] De-reverberate the speech domain signal of the current frame Perform inverse Fourier transform to obtain the speech time domain dereverberation signal of the current frame.

[0139] In this embodiment, a posteriori filtering update is performed based on the estimated value of the two-stage current frame speech frequency domain dereverberation signal, the two-stage state space observation value corresponding to the current frame, and the two-stage current frame Kalman gain coefficient, and a dereverberation operation is further performed to obtain a more accurate speech frequency domain dereverberation signal of the current frame; and an inverse Fourier transform is performed on the speech frequency domain dereverberation signal of the current frame to obtain a speech time domain dereverberation signal of the current frame.

[0140] In one of the embodiments, based on the posterior estimate of the frame error covariance diagonal matrix in the current stage, the prior estimate of the frame error covariance diagonal matrix in the current stage is updated, and the specific steps are as follows: the spatial regression coefficient required in the current stage is modeled as a first-order Markov process, and the Kronecker integral is decomposed to obtain the state equation of the current stage; based on the complex Gaussian perturbation noise of the current frame in the current stage in the state equation of the current stage, the variance of the complex Gaussian perturbation noise of the current frame in the current stage is obtained; the posterior estimate of the frame error covariance diagonal matrix in the current stage is obtained; based on the sum of the posterior estimate of the frame error covariance diagonal matrix in the current stage and the variance of the complex Gaussian perturbation noise of the current frame in the current stage, the prior estimate of the frame error covariance diagonal matrix in the current stage is updated.

[0141] According to the multi-channel autoregressive process, the spatial regression coefficient required in this stage can be modeled as a first-order Markov process, and the Kronecker integral decomposition is performed to obtain two different state equations as shown in Equation (29) and Equation (30).

[0142] (29)

[0143] (30)

[0144] in, and Represent the state transfer matrices of the first-stage filter and the second-stage filter respectively, and the unit matrix is ​​usually selected. and Represent the complex Gaussian perturbation noise in the state transition process of the first-stage filtering and the second-stage filtering, and the variance of the complex Gaussian perturbation noise is respectively expressed as and It means that the update of disturbance noise directly affects the final effect of the filtering algorithm, which can be obtained through equations (31) and (32). and Estimate separately,

[0145] (31)

[0146] (32)

[0147] in represents a positive minimum, characterizing the change of disturbance; represents mathematical expectation.

[0148] If this stage is one stage, then Equation (29) is the one-stage state equation; according to the one-stage current frame complex Gaussian disturbance noise in the one-stage state equation , get the variance of the complex Gaussian perturbation noise of the current frame in one stage .

[0149] Get the posterior estimate of the diagonal matrix of the frame error covariance in the first stage ; Based on the posterior estimate of the diagonal matrix of the frame error covariance in the first stage and the variance of the complex Gaussian perturbation noise of the current frame in one stage The sum of the current frame error covariance diagonal matrix is ​​updated to obtain the prior estimate of the current frame error covariance diagonal matrix , as shown in formula (11).

[0150] If this stage is the second stage, then Equation (30) is the second stage state equation; according to the second stage current frame complex Gaussian disturbance noise in the second stage state equation , get the variance of the complex Gaussian perturbation noise of the current frame in the second stage .

[0151] Get the posterior estimate of the diagonal matrix of the two-stage upper frame error covariance ; Based on the posterior estimate of the diagonal matrix of the frame error covariance in the second stage and the variance of the complex Gaussian perturbation noise of the current frame in the second stage The sum of the two-stage current frame error covariance diagonal matrix is ​​updated to obtain the prior estimate , as shown in formula (16).

[0152] In this embodiment, the spatial regression coefficient required in this stage is modeled as a first-order Markov process to obtain the state equation of this stage; based on the complex Gaussian perturbation noise of the current frame in this stage in the state equation of this stage, the variance of the complex Gaussian perturbation noise of the current frame in this stage is obtained; based on the posterior estimate of the diagonal matrix of the error covariance of the previous frame in this stage and the sum of the variance of the complex Gaussian perturbation noise of the current frame in this stage, the prior estimate of the diagonal matrix of the error covariance of the current frame in this stage is updated.

[0153] In one of the embodiments, the posterior estimate of the error covariance diagonal matrix of the previous frame in this stage is obtained, and the specific steps are as follows: multiply the state space observation value of the current stage corresponding to the previous frame and the Kalman gain coefficient of the previous frame in this stage to obtain the multiplication matrix of this stage; subtract the unit matrix of this stage from the multiplication matrix of this stage to obtain the subtraction matrix of this stage; approximately fully diagonalize the subtraction matrix of this stage to obtain the subtraction diagonal matrix of this stage; obtain the posterior estimate of the error covariance diagonal matrix of the previous frame in this stage according to the prior estimate of the subtraction diagonal matrix of this stage and the error covariance diagonal matrix of the previous frame in this stage.

[0154] If this stage is one stage, the state space observation value of the previous frame corresponding to the one stage and the first-stage upper frame Kalman gain coefficient Multiply them to get a one-stage multiplication matrix ; The first-stage identity matrix And the one-stage multiplication matrix Subtract and get a one-stage subtraction matrix .

[0155] The one-stage subtraction matrix is ​​approximately fully diagonalized to obtain the one-stage subtraction diagonal matrix, as shown in formula (33).

[0156] (33)

[0157] According to the prior estimate of the one-stage subtraction diagonal matrix and the one-stage up-up frame error covariance diagonal matrix , and obtain the posterior estimate of the diagonal matrix of the frame error covariance in the first stage , as shown in formula (34).

[0158] (34)

[0159] If this stage is the second stage, the second stage state space observation value corresponding to the previous frame and the second-stage upper frame Kalman gain coefficient Multiply them to get the two-stage multiplication matrix ; The second stage identity matrix And the two-stage multiplication matrix Subtract and get the two-stage subtraction matrix .

[0160] The two-stage subtraction matrix is ​​approximately fully diagonalized to obtain the two-stage subtraction diagonal matrix, as shown in formula (35).

[0161] (35)

[0162] According to the prior estimate of the two-stage subtraction diagonal matrix and the two-stage up-up frame error covariance diagonal matrix , and obtain the posterior estimate of the diagonal matrix of the frame error covariance in the second stage , as shown in formula (36).

[0163] (36)

[0164] In this embodiment, the subtraction diagonal matrix of this stage is obtained based on the state space observation values ​​of this stage corresponding to the previous frame, the Kalman gain coefficients of the previous frame of this stage and the unit matrix of this stage; the posterior estimate of the error covariance diagonal matrix of the previous frame of this stage is obtained based on the subtraction diagonal matrix of this stage and the prior estimate of the error covariance diagonal matrix of the previous frame of this stage.

[0165] In one of the embodiments, based on the one-stage state space observation value corresponding to the current frame and the one-stage state space conjugate transposed observation value corresponding to the current frame, the one-stage diagonal multiplication observation value corresponding to the current frame is obtained, and the specific steps are as follows: based on the one-stage state space observation value corresponding to the current frame, the one-stage state space conjugate transposed observation value corresponding to the current frame is obtained; the one-stage state space observation value corresponding to the current frame and the one-stage state space conjugate transposed observation value corresponding to the current frame are multiplied to obtain the one-stage multiplication observation value corresponding to the current frame; the one-stage multiplication observation value corresponding to the current frame is approximately fully diagonalized to obtain the one-stage diagonal multiplication observation value corresponding to the current frame.

[0166] According to the one-stage state space observation value corresponding to the current frame , and obtain the conjugate transposed observation value of the state space corresponding to the current frame ; The state space observation value corresponding to the current frame and the conjugate transposed observation of the one-stage state space corresponding to the current frame Multiply to obtain the one-stage multiplication observation value corresponding to the current frame; approximately fully diagonalize the one-stage multiplication observation value corresponding to the current frame to obtain the one-stage diagonal multiplication observation value corresponding to the current frame.

[0167] In this embodiment, the one-stage state space observation value corresponding to the current frame and the one-stage state space conjugate transposed observation value corresponding to the current frame are multiplied to obtain the one-stage multiplied observation value corresponding to the current frame; the one-stage multiplied observation value corresponding to the current frame is approximately fully diagonalized to obtain the one-stage diagonal multiplied observation value corresponding to the current frame.

[0168] In order to better understand the above method, an application example of the speech dereverberation method of the present application is described in detail below. The specific steps are as follows: Figure 4As shown. The speech dereverberation method provided in the present application is applicable to a microphone array system having a single or multiple microphones. Assuming that there is a uniform linear array consisting of M omnidirectional microphones in a reverberation environment, the application scenario of the speech dereverberation method is as follows: Figure 2 shown.

[0169] Step S401: Acquire a speech time domain signal of a current frame through a microphone array.

[0170] Step S402, performing windowing processing on the speech time domain signal to obtain a windowed speech time domain signal; performing Fourier transform on the windowed speech time domain signal to obtain a speech time domain signal of the current frame.

[0171] Step S403, sequentially cache the speech frequency domain signal of the current frame, and determine whether the current cache frame number is greater than the preset delay frame. If so, obtain the speech frequency domain signal of the historical multiple frames, otherwise return to step S401. According to the speech frequency domain signal of the current frame and the speech frequency domain signal of the historical multiple frames, obtain the multiple frames of speech frequency domain signal. According to the delay coefficient D, perform signal delay on the multiple frames of speech frequency domain signal to obtain the multiple frames of speech frequency domain delayed signal.

[0172] Step S404 can perform a first-stage filtering and reverberation operation based on the multi-frame speech frequency domain delay signal and the first-stage observation equation to obtain the spatial regression coefficient required for the second stage. The specific process is as follows: Figure 5 As shown in Figure 2, the first-stage observation equation and the second-stage observation equation are obtained by performing Kronecker decomposition on the spatial regression coefficients in the initial observation equation.

[0173] Step S405 can perform a two-stage filtering and dereverberation operation based on the multi-frame audio-domain delayed signal, the two-stage observation equation and the spatial regression coefficient required for the two-stage, and obtain an estimated value of the audio-domain dereverberation signal of the current frame in the two-stage. The specific process is as follows: Figure 6 shown.

[0174] Step S406, performing a posteriori filtering update based on the estimated value of the second-stage current frame speech domain dereverberation signal, the second-stage state space observation value corresponding to the current frame, and the second-stage current frame Kalman gain coefficient to obtain the speech domain dereverberation signal of the current frame; performing an inverse Fourier transform on the speech domain dereverberation signal of the current frame to obtain the speech time domain dereverberation signal of the current frame The speech time domain dereverberation signal of the current frame is Indicates the first The speech time-domain dedeverberation signal of the current frame in three independent frequency bands.

[0175] The speech frequency domain dereverberation signal of the current frame processed by independent filtering of each frequency band of one of the microphones in the microphone array can be subjected to inverse Fourier transform, and then the speech time domain dereverberation signal of the current frame of one of the microphones in the microphone array can be obtained by using overlap-addition method. .

[0176] In this embodiment, the Kronecker integral decomposition is first used to decompose the autoregressive coefficients to be estimated in the Kalman filtering process into two vectors with lower dimensions, and the original Kalman update process is transformed into two mutually related filtering and reverberation operations. The current frame error covariance matrix, the variance matrix and the one-stage state space observation value corresponding to the current frame, and the multiplication value of the one-stage state space conjugate transposed observation value corresponding to the current frame in the filtering and reverberation operation are approximately fully diagonalized, which greatly reduces the amount of parameters to be stored, thereby reducing the occupied storage resources.

[0177] In addition, the Kronecker decomposition can decouple the space-time correlation to a certain extent. In the multi-channel dereverberation scenario, by reasonably designing the two-stage filter weights of the Kronecker decomposition, for example, decomposing it into the "number of channels" dimension and the "prediction order" dimension, the filtering dereverberation operation can be regarded as filtering in two stages in the spatial dimension and the temporal dimension respectively, and finally combining them through the Kronecker product. When the channel-related autoregressive coefficients tend to be stable, only the prediction-related autoregressive coefficients can be updated.

[0178] Furthermore, in the two-stage filtering and dereverberation operation, the autoregressive coefficient estimation in each stage can select the original Kalman filter, block diagonal or full diagonal Kalman filter, or a combination of the two according to the computing and storage capabilities of the current device. At the same time, the length of the Kronecker integral decomposition of the autoregressive coefficient can also be flexibly selected. In addition to the selection method of space-time decoupling, it can also be used as a two-stage iterative process of dimensionality reduction.

[0179] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0180] Based on the same inventive concept, the embodiment of the present application also provides a speech dereverberation device for implementing the speech dereverberation method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more speech dereverberation device embodiments provided below can refer to the limitations of the speech dereverberation method above, and will not be repeated here.

[0181] In an exemplary embodiment, Figure 7 As shown, a speech dereverberation device is provided, wherein:

[0182] The frequency domain delayed signal acquisition module 701 is used to obtain a multi-frame frequency domain delayed signal according to the frequency domain signal of the current frame and the frequency domain signals of multiple historical frames;

[0183] The time domain dereverberation signal acquisition module 702 is used to perform a two-stage filtering dereverberation operation according to the multi-frame audio domain delay signal, the first-stage observation equation and the second-stage observation equation to obtain an estimated value of the second-stage current frame audio domain dereverberation signal to form a speech time domain dereverberation signal of the current frame; the first-stage observation equation and the second-stage observation equation are obtained by performing Kronecker integral decomposition on the spatial regression coefficients in the initial observation equation;

[0184] The filtering and dereverberation operation module 703 of this stage is used to obtain the state space observation value of this stage corresponding to the current frame according to the multi-frame speech frequency domain delay signal, the observation equation of this stage and the spatial regression coefficient required by this stage when performing the filtering and dereverberation operation of this stage; update the prior estimation value of the error covariance diagonal matrix of the current frame of this stage according to the posterior estimation value of the error covariance diagonal matrix of the previous frame of this stage; the error covariance diagonal matrix of the current frame of this stage is obtained by approximately fully diagonalizing the error covariance matrix of the current frame of this stage; update the estimation value of the speech frequency domain dereverberation signal of the current frame of this stage according to the state space observation value of this stage;

[0185] The spatial regression coefficient acquisition module 704 is used for, if the current stage is the second stage, then the spatial regression coefficient required for the current stage is the estimated value of the spatial regression coefficient of the current frame of the first stage; the estimated value of the spatial regression coefficient of the current frame of the first stage is obtained by the following steps: according to the estimated value of the speech frequency domain dereverberation signal of the current frame of the first stage, the estimated value of the variance diagonal matrix of the speech frequency domain dereverberation signal of the current frame of the first stage is updated; the variance diagonal matrix is ​​obtained by approximately fully diagonalizing the variance matrix; according to the state space observation value of the first stage corresponding to the current frame, the diagonal multiplication observation value of the first stage corresponding to the current frame, the error covariance of the current frame of the first stage The prior estimated value of the diagonal matrix and the estimated value of the variance diagonal matrix of the speech-domain dereverberation signal of the current frame in the first stage are used to update the Kalman gain coefficient of the current frame in the first stage; the diagonal multiplication observation value corresponding to the current frame in the first stage is obtained by multiplying the state-space observation value corresponding to the current frame in the first stage and the conjugate transposed observation value of the state-space corresponding to the current frame in the first stage and performing approximate full diagonalization; the estimated value of the spatial regression coefficient of the current frame in the first stage is updated according to the estimated value of the speech-domain dereverberation signal of the current frame in the first stage, the Kalman gain coefficient of the current frame in the first stage and the estimated value of the spatial regression coefficient of the previous frame in the first stage.

[0186] In one embodiment, the spatial regression coefficient acquisition module 704 is further used for: if the current stage is the first stage, the spatial regression coefficient required for the current stage is the estimated value of the spatial regression coefficient of the second stage upper frame; the estimated value of the spatial regression coefficient of the second stage upper frame is obtained by the following steps: according to the estimated value of the speech frequency domain dereverberation signal of the second stage upper frame, the estimated value of the variance diagonal matrix of the speech frequency domain dereverberation signal of the second stage upper frame is updated; the variance diagonal matrix is ​​obtained by approximately fully diagonalizing the variance matrix; according to the second stage state space observation value corresponding to the upper frame, the second stage diagonal multiplication observation value corresponding to the upper frame, the second stage upper frame The prior estimate of the frame error covariance diagonal matrix and the estimate of the variance diagonal matrix of the two-stage upper frame speech domain dereverberation signal are used to update the two-stage upper frame Kalman gain coefficient; the two-stage diagonal multiplication observation value corresponding to the upper frame is obtained by multiplying the two-stage state space observation value corresponding to the upper frame and the two-stage state space conjugate transposed observation value corresponding to the upper frame and performing approximate full diagonalization; the two-stage upper frame space regression coefficient estimate is updated according to the estimate of the two-stage upper frame speech domain dereverberation signal, the two-stage upper frame Kalman gain coefficient and the two-stage upper-upper frame space regression coefficient estimate.

[0187] In one embodiment, the frequency domain delayed signal acquisition module 701 is further used to: acquire a speech time domain signal of a current frame through a microphone array; perform windowing processing on the speech time domain signal to obtain a windowed speech time domain signal; perform Fourier transform on the windowed speech time domain signal to obtain a speech frequency domain signal of a current frame; obtain multiple frames of speech frequency domain signals based on the speech frequency domain signal of the current frame and multiple historical frames of speech frequency domain signals; and obtain multiple frames of speech frequency domain delayed signals based on a delay coefficient and the multiple frames of speech frequency domain signals.

[0188] In one of the embodiments, the time domain dereverberation signal acquisition module 702 is further used to: perform a posteriori filtering update based on the estimated value of the second-stage current frame speech domain dereverberation signal, the second-stage state space observation value corresponding to the current frame, and the second-stage current frame Kalman gain coefficient to obtain the speech domain dereverberation signal of the current frame; and perform inverse Fourier transform on the speech domain dereverberation signal of the current frame to obtain the speech time domain dereverberation signal of the current frame.

[0189] In one embodiment, the device also includes a priori estimate acquisition module, which is used to: model the spatial regression coefficients required in the current stage as a first-order Markov process, and perform Kronecker integral decomposition to obtain the state equation of the current stage; obtain the variance of the complex Gaussian perturbation noise of the current frame in the current stage based on the complex Gaussian perturbation noise of the current frame in the current stage in the state equation of the current stage; obtain the posterior estimate of the diagonal matrix of the frame error covariance in the current stage; update the prior estimate of the diagonal matrix of the error covariance of the current frame in the current stage based on the sum of the posterior estimate of the diagonal matrix of the frame error covariance in the current stage and the variance of the complex Gaussian perturbation noise of the current frame in the current stage.

[0190] In one embodiment, the prior estimation value acquisition module is also used to: multiply the state space observation value of the current stage corresponding to the previous frame and the Kalman gain coefficient of the previous frame of the current stage to obtain the multiplication matrix of the current stage; subtract the unit matrix of the current stage from the multiplication matrix of the current stage to obtain the subtraction matrix of the current stage; approximately fully diagonalize the subtraction matrix of the current stage to obtain the subtraction diagonal matrix of the current stage; obtain the posterior estimation value of the error covariance diagonal matrix of the previous frame of the current stage based on the prior estimation value of the subtraction diagonal matrix of the current stage and the error covariance diagonal matrix of the previous frame of the current stage.

[0191] In one embodiment, the spatial regression coefficient acquisition module 704 is also used to: obtain a conjugate transposed observation value of a stage state space corresponding to the current frame based on a stage state space observation value corresponding to the current frame; multiply the one-stage state space observation value corresponding to the current frame and the conjugate transposed observation value of the one-stage state space corresponding to the current frame to obtain a one-stage multiplied observation value corresponding to the current frame; approximately fully diagonalize the one-stage multiplied observation value corresponding to the current frame to obtain a one-stage diagonal multiplied observation value corresponding to the current frame.

[0192] Each module in the above-mentioned speech dereverberation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0193] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data of an embodiment of a speech dereverberation method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a speech dereverberation method is implemented.

[0194] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0195] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0196] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0197] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0198] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0199] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0200] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0201] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A speech dereverberation method, characterized in that: The method comprises: Obtaining a multi-frame audio-frequency delayed signal in the audio-frequency domain according to the audio-frequency domain signal of the current frame and the audio-frequency domain signals of the historical multi-frames; A two-stage filtering and dereverberation operation is performed according to the multi-frame speech frequency domain delay signal, the first-stage observation equation and the second-stage observation equation to obtain an estimated value of the second-stage current frame speech frequency domain dereverberation signal, and then a posterior update operation is performed to form a speech time domain dereverberation signal of the current frame; the first-stage observation equation and the second-stage observation equation are obtained by performing Kronecker integral decomposition on the spatial regression coefficients in the initial observation equation; Wherein, when performing the filtering and dereverberation operation of this stage, the state space observation value of this stage corresponding to the current frame is obtained according to the multi-frame speech frequency domain delay signal, the observation equation of this stage and the spatial regression coefficient required by this stage; according to the posterior estimation value of the error covariance diagonal matrix of the previous frame in this stage, the prior estimation value of the error covariance diagonal matrix of the current frame in this stage is updated; the error covariance diagonal matrix of the current frame in this stage is obtained by approximately fully diagonalizing the error covariance matrix of the current frame in this stage; according to the state space observation value of this stage, the estimation value of the speech frequency domain dereverberation signal of the current frame in this stage is updated; If this stage is a two-stage stage, the spatial regression coefficient required for this stage is the estimated value of the spatial regression coefficient of the current frame of the first stage; the estimated value of the spatial regression coefficient of the current frame of the first stage is obtained by the following steps: according to the estimated value of the speech frequency domain dereverberation signal of the current frame of the first stage, the estimated value of the variance diagonal matrix of the speech frequency domain dereverberation signal of the current frame of the first stage is updated; the variance diagonal matrix is ​​obtained by approximately fully diagonalizing the variance matrix; according to the state space observation value of the first stage corresponding to the current frame, the diagonal multiplication observation value of the first stage corresponding to the current frame, and the prior estimate of the error covariance diagonal matrix of the current frame of the first stage The calculated value and the estimated value of the variance diagonal matrix of the speech-domain dereverberation signal of the current frame in the first stage are used to update the Kalman gain coefficient of the current frame in the first stage; the diagonal multiplication observation value corresponding to the current frame in the first stage is obtained by multiplying the state-space observation value corresponding to the current frame in the first stage and the conjugate transposed observation value of the state-space corresponding to the current frame in the first stage and performing approximate full diagonalization; the estimated value of the speech-domain dereverberation signal of the current frame in the first stage, the Kalman gain coefficient of the current frame in the first stage and the estimated value of the spatial regression coefficient of the previous frame in the first stage are used to update the estimated value of the spatial regression coefficient of the current frame in the first stage.

2. The method according to claim 1, characterized in that If the current stage is the first stage, the spatial regression coefficient required in the current stage is the estimated value of the spatial regression coefficient of the upper frame of the second stage; obtaining the estimated value of the spatial regression coefficient of the upper frame of the second stage specifically includes: According to the estimated value of the speech-domain dereverberation signal of the second-stage upper frame, an estimated value of the variance diagonal matrix of the speech-domain dereverberation signal of the second-stage upper frame is updated; the variance diagonal matrix is ​​obtained by approximately fully diagonalizing the variance matrix; According to the two-stage state space observation value corresponding to the previous frame, the two-stage diagonal multiplication observation value corresponding to the previous frame, the prior estimation value of the two-stage upper frame error covariance diagonal matrix and the estimation value of the variance diagonal matrix of the two-stage upper frame speech domain dereverberation signal, the two-stage upper frame Kalman gain coefficient is updated; the two-stage diagonal multiplication observation value corresponding to the previous frame is obtained by multiplying the two-stage state space observation value corresponding to the previous frame and the two-stage state space conjugate transposed observation value corresponding to the previous frame and performing approximate full diagonalization; The second-stage upper frame spatial regression coefficient estimate is updated according to the second-stage upper frame speech domain dereverberation signal estimate, the second-stage upper frame Kalman gain coefficient and the second-stage upper-upper frame spatial regression coefficient estimate.

3. The method according to claim 1, characterized in that The step of obtaining a multi-frame audio-frequency delayed signal according to the audio-frequency signal of the current frame and the audio-frequency signals of multiple historical frames includes: Acquire the speech time domain signal of the current frame through the microphone array; Performing windowing processing on the speech time domain signal to obtain a windowed speech time domain signal; Performing Fourier transform on the windowed speech time domain signal to obtain a speech frequency domain signal of the current frame; Obtaining a multi-frame speech frequency domain signal according to the speech frequency domain signal of the current frame and the speech frequency domain signals of the historical multi-frames; According to the time delay coefficient and the multi-frame voice frequency domain signal, a multi-frame voice frequency domain delay signal is obtained.

4. The method according to claim 1, characterized in that: According to the estimated value of the speech dereverberation signal in the audio domain of the current frame in the second stage, a speech dereverberation signal in the time domain of the current frame is formed, including: According to the estimated value of the second-stage current frame speech domain dereverberation signal, the second-stage state space observation value corresponding to the current frame and the second-stage current frame Kalman gain coefficient, a posteriori filtering update is performed to obtain the speech domain dereverberation signal of the current frame; Performing inverse Fourier transform on the speech audio-domain dereverberation signal of the current frame to obtain the speech time-domain dereverberation signal of the current frame.

5. The method according to claim 1, characterized in that The updating of the prior estimation value of the current frame error covariance diagonal matrix at this stage according to the posterior estimation value of the frame error covariance diagonal matrix at this stage comprises: The spatial regression coefficient required in the present stage is modeled as a first-order Markov process, and Kronecker integral decomposition is performed to obtain the state equation of the present stage; According to the complex Gaussian disturbance noise of the current frame of the current stage in the state equation of the current stage, the variance of the complex Gaussian disturbance noise of the current frame of the current stage is obtained; Obtaining a posterior estimate of the frame error covariance diagonal matrix at the current stage; According to the a posteriori estimate of the frame error covariance diagonal matrix at the current stage and the sum of the variance of the complex Gaussian disturbance noise of the current frame at the current stage, the a priori estimate of the current frame error covariance diagonal matrix at the current stage is updated.

6. The method according to claim 5, characterized in that The obtaining of the a posteriori estimate of the frame error covariance diagonal matrix at the current stage includes: Multiply the state space observation value of this stage corresponding to the previous frame and the Kalman gain coefficient of the previous frame of this stage to obtain the multiplication matrix of this stage; Subtract the current stage unit matrix from the current stage multiplication matrix to obtain the current stage subtraction matrix; Approximately fully diagonalizing the subtraction matrix of this stage to obtain a subtraction diagonal matrix of this stage; According to the a priori estimated value of the subtraction diagonal matrix of the current stage and the previous frame error covariance diagonal matrix of the current stage, the a posteriori estimated value of the previous frame error covariance diagonal matrix of the current stage is obtained.

7. The method according to claim 1, characterized in that According to the one-stage state space observation value corresponding to the current frame and the one-stage state space conjugate transposed observation value corresponding to the current frame, the one-stage diagonal multiplication observation value corresponding to the current frame is obtained, including: According to the one-stage state space observation value corresponding to the current frame, the one-stage state space conjugate transposed observation value corresponding to the current frame is obtained; Multiply the one-stage state space observation value corresponding to the current frame and the one-stage state space conjugate transposed observation value corresponding to the current frame to obtain the one-stage multiplied observation value corresponding to the current frame; The one-stage multiplication observation values ​​corresponding to the current frame are approximately fully diagonalized to obtain the one-stage diagonal multiplication observation values ​​corresponding to the current frame.

8. A speech dereverberation device, characterized in that: The device comprises: A frequency domain delayed signal acquisition module, used to obtain a multi-frame frequency domain delayed signal according to the frequency domain signal of the current frame and the frequency domain signals of multiple historical frames; A time domain dereverberation signal acquisition module is used to perform two-stage filtering dereverberation operations according to the multi-frame audio domain delay signals, the first-stage observation equation and the second-stage observation equation to obtain an estimated value of the second-stage current frame audio domain dereverberation signal to form a speech time domain dereverberation signal of the current frame; the first-stage observation equation and the second-stage observation equation are obtained by performing Kronecker integral decomposition on the spatial regression coefficients in the initial observation equation; The filtering and dereverberation operation module of this stage is used to obtain the state space observation value of this stage corresponding to the current frame according to the multi-frame speech frequency domain delay signal, the observation equation of this stage and the spatial regression coefficient required by this stage when performing the filtering and dereverberation operation of this stage; update the prior estimation value of the error covariance diagonal matrix of the current frame of this stage according to the posterior estimation value of the error covariance diagonal matrix of the previous frame of this stage; the error covariance diagonal matrix of the current frame of this stage is obtained by approximately fully diagonalizing the error covariance matrix of the current frame of this stage; update the estimation value of the speech frequency domain dereverberation signal of the current frame of this stage according to the state space observation value of this stage; The spatial regression coefficient acquisition module is used for, if the current stage is the two-stage stage, then the spatial regression coefficient required in this stage is the estimated value of the spatial regression coefficient of the current frame of the first stage; the estimated value of the spatial regression coefficient of the current frame of the first stage is obtained by the following steps: according to the estimated value of the speech frequency domain dereverberation signal of the current frame of the first stage, the estimated value of the variance diagonal matrix of the speech frequency domain dereverberation signal of the current frame of the first stage is updated; the variance diagonal matrix is ​​obtained by approximately fully diagonalizing the variance matrix; according to the state space observation value of the first stage corresponding to the current frame, the diagonal multiplication observation value of the first stage corresponding to the current frame, the error covariance matrix of the current frame of the first stage The prior estimated value of the diagonal matrix and the estimated value of the variance diagonal matrix of the speech-domain dereverberation signal of the current frame in the first stage are used to update the Kalman gain coefficient of the current frame in the first stage; the diagonal multiplication observation value corresponding to the current frame in the first stage is obtained by multiplying the state-space observation value corresponding to the current frame in the first stage and the conjugate transposed observation value of the state-space corresponding to the current frame in the first stage and performing approximate full diagonalization; the estimated value of the spatial regression coefficient of the current frame in the first stage is updated according to the estimated value of the speech-domain dereverberation signal of the current frame in the first stage, the Kalman gain coefficient of the current frame in the first stage and the estimated value of the spatial regression coefficient of the previous frame in the first stage.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images