A method for estimating the time difference of arrival of speech signals based on the maximum expectation algorithm

Abstract

The present invention relates to a method for estimating the time difference of arrival of a speech signal based on a maximum expectation algorithm. The method utilizes the sparsity of the speech signal distribution in the time-frequency domain to preprocess the speech signal and construct a mixed probability density function. The maximum expectation algorithm is then used to estimate the time difference of arrival and the probability of speech presence. The present invention not only realizes the estimation of the time difference of arrival, but also can use the probability of speech signal presence to estimate the accuracy of the time difference of arrival.

Description

Technical Field

[0001] The present invention relates to the field of audio signal and speech signal processing, and in particular to a speech signal arrival time difference estimation method based on a maximum expectation algorithm. Background Art

[0002] In applications such as video conferencing, public safety monitoring, and intelligent interaction, the accuracy of speech source location estimation is a key factor affecting the feasibility of these applications. However, in most speech source location estimation methods, the time difference between the speech signal reaching two microphones is the key measurement value. Common methods for time difference estimation include generalized correlation function, maximum likelihood, channel estimation, and spatial spectrum methods. These methods are either unable to adapt to the broadband characteristics of speech signals or lack modeling of speech signals, making it impossible to achieve high-precision time difference estimation. Deep learning-based methods that have emerged in recent years can fully learn the characteristics of speech signals, significantly improving estimation accuracy. However, these methods lack stability when the training environment does not match the real environment. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method for estimating the time difference of arrival of a speech signal based on a maximum expectation algorithm in a noisy and reverberation-free environment.

[0004] The technical solution adopted by the present invention is a method for estimating the time difference of arrival of a speech signal based on a maximum expectation algorithm, comprising a pair of microphones for receiving far-field speech signals, and comprising the following steps:

[0005] S1. Preprocessing the voice signal received by the microphone to obtain a frequency domain signal;

[0006] S2. Construct a mixed probability density function of the frequency domain signal based on the sparse representation of the speech signal;

[0007] S3. Use the maximum expectation algorithm to estimate the arrival time difference in the mixture probability density function.

[0008] The beneficial effects of the present invention are as follows: the present invention utilizes the sparsity of the speech signal distribution in the time-frequency domain to preprocess the speech signal, construct a mixed probability density function, and then uses the maximum expectation algorithm to estimate the arrival time difference and the probability of speech presence; the present invention not only realizes the estimation of the arrival time difference, but also can use the probability of speech signal presence to estimate the accuracy of the arrival time difference.

[0009] Preferably, the S1 comprises the following steps:

[0010] S101. The time domain signal of the noisy speech received by a pair of microphones is represented as:

[0011] x1(t)=s(t)+n1(t)

[0012] x2(t)=s(t-τ)+n2(t)

[0013] Where t is the sampling time; s(t) is the speech source signal; x1(t) and x2(t) are the noisy speech signals received by the two microphones respectively; n1(t) and n2(t) are the noise signals; τ is the arrival time difference;

[0014] S102. Windowing and framing the time domain signal to obtain a speech signal frame;

[0015] S103. Window and frame the time domain signal x1(t) and the time domain signal x2(t) respectively to obtain several frames of short time domain signals, and perform short-time Fourier transform on each frame of the short time domain signal to obtain frequency domain signals of several frequency bands:

[0016] X1(l,f)=S(l,f)+N1(l,f)

[0017] X2(l,f)=e -j2πfτ S(l,f)+N2(l,f)

[0018] Where l represents the number of the speech frame; f represents the number of the frequency band;

[0019] S104. Simplify the frequency domain signal X(f) to:

[0020] X(f)=A(f)S(f)+N(f)

[0021] Where:

[0022] X(f)=[X1(f),X2(f)] T

[0023] A(f)=[1,e -j2πfτ ] T

[0024] N(f)=[N1(f),N2(f)] T .

[0025] Preferably, the S2 includes:

[0026] Based on the sparsity of the speech signal distribution in the time-frequency domain, the Gaussian mixture model is used to construct the mixture probability density function of the frequency domain signal X(f):

[0027] p(X(f))=p0g0(X(f))+p1g1(X(f))

[0028] Where:

[0029] The probability of the existence of a speech signal represents the sparsity of the speech signal in the time-frequency domain. p0 is the probability that the speech signal does not exist at a time-frequency point; p1 is the probability that the speech signal exists at a time-frequency point.

[0030] p1+p0=1:

[0031] g0(X(f)) is the probability density function of the time-frequency domain signal X(f) when the speech signal does not exist:

[0032]

[0033] where ∑(f) represents the diagonal covariance matrix:

[0034]

[0035] and is the noise power spectrum received by the two microphones;

[0036] g1(X(f)) is the probability density function of the frequency domain signal X(f) when the speech signal exists:

[0037]

[0038] Where Γ(f) is the covariance matrix,

[0039] is the speech power spectrum,

[0040]

[0041]

[0042] Preferably, the noise power spectrum and It is obtained using short-time domain signal estimation. Thus, after obtaining the noise power spectrum and speech power spectrum, the mixed probability density function only depends on two unknown parameters: arrival time difference and speech existence probability.

[0043] Preferably, the step S3 uses the maximum expectation algorithm to estimate the arrival time difference and the probability of existence of the voice signal, including:

[0044] S301. Initialize the time difference and the probability of the presence of a voice signal;

[0045] S302. Calculate the posterior probability of speech presence using E-step;

[0046]

[0047] S303 uses M-step to calculate the probability of the presence of the voice signal and the time difference of arrival;

[0048]

[0049]

[0050] Where, the superscript ι represents the number of iteration steps; F is the number of frequency bands contained in a frame of speech signal; represents the probability of the posterior speech not existing (k=0) or the probability of the posterior speech existing (k=1); represents the probability of the prior speech not existing (k=0) or the probability of the prior speech existing (k=1); τ(i) is the estimated value of the time difference of arrival;

[0051] S304. Loop through the E-step and M-step iterations until the maximum number of iterations is estimated to be reached. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A simplified flow chart of a method for estimating the time difference of arrival of a speech signal according to the present invention;

[0053] Figure 2 Detailed flow chart of the method for estimating the time difference of arrival of speech signals according to the present invention;

[0054] Figure 3 This is the experimental result of the speech signal arrival time difference estimation method of the present invention. DETAILED DESCRIPTION

[0055] The invention will be further described below with reference to the accompanying drawings and in combination with specific implementations, so that those skilled in the art can implement the invention with reference to the description. The protection scope of the invention is not limited to the specific implementations.

[0056] Those skilled in the art should understand that, in the disclosure of the present invention, the orientation or positional relationship indicated by terms such as "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", and "outside" are based on the orientation or positional relationship shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the above terms should not be understood as limiting the present invention.

[0057] A method for estimating the time difference of arrival of a speech signal based on a maximum expectation algorithm includes a pair of microphones for receiving far-field speech signals and comprises the following steps:

[0058] 1. Preprocess the voice signal received by the microphone to obtain a frequency domain signal;

[0059] Step 1. The time domain signal of the noisy speech received by a pair of microphones is expressed as:

[0060] x1(t)=s(t)+n1(t)

[0061] x2(t)=s(t-τ)+n2(t)

[0062] Where t is the sampling time; s(t) is the speech source signal; x1(t) and x2(t) are the noisy speech signals received by the two microphones respectively; n1(t) and n2(t) are the noise signals; τ is the arrival time difference;

[0063] The time domain signal x1(t) and the time domain signal x2(t) are windowed and framed respectively to obtain several frames of short time domain signals, and each frame of short time domain signal is subjected to short-time Fourier transform to obtain frequency domain signals of several frequency bands:

[0064] X1(l,f)=S(l,f)+N1(l,f)

[0065] X2(l,f)=e -j2πff S(l,f)+N2(l,f)

[0066] Where l represents the number of the speech frame; f represents the number of the frequency band; the simplified frequency domain signal X(f) is:

[0067] X(f)=A(f)S(f)+N(f)

[0068] Where,

[0069] X(f)=[X1(f),X2(f)] T

[0070] A(f)=[1,e -j2πfτ ] T

[0071] N(f)=[N1(f),N2(f)] T

[0072] This specific embodiment uses a Hamming window with a length of 512 to frame the time domain signal, and the overlap rate of the two frames is 50%; after the short-time Fourier transform, a frequency domain signal with 257 frequency bands is obtained;

[0073] Step 2. Estimate the noise power spectrum based on the first five frames of the frequency domain signal of each frequency band. In this specific embodiment, it is assumed that the first five frames of the frequency domain signal do not contain speech, and the average power of the first five frames of the frequency domain signal of each frequency band is taken as the noise power spectrum of the frequency domain signal of the frequency band.

[0074] Step 3. Estimate the initial value of the arrival time difference τ based on the generalized correlation function;

[0075] 2. Based on the sparsity of the speech signal distribution in the time-frequency domain, that is, the presence or absence of a speech signal at a certain time-frequency point, a Gaussian mixture model is used to construct the mixture probability density function of the frequency domain signal X(f):

[0076] p(X(f))=p0g0(X(f))+p1g1(X(f))

[0077] Where, the probability of the existence of the speech signal represents the sparsity of the speech signal in the time-frequency domain, p0 is the probability that the speech signal does not exist, p1 is the probability that the speech signal exists, and p1+p0=1;

[0078] g0(X(f)) is the probability density function of the time-frequency domain signal X(f) when the speech signal does not exist. It is specifically expressed as a two-dimensional complex Gaussian distribution function with a mean of zero:

[0079]

[0080] where ∑(f) represents the diagonal covariance matrix:

[0081]

[0082] and It is represented as the noise power spectrum received by the two microphones;

[0083] g1(X(f)) is the probability density function of the time-frequency domain signal X(f) when the speech signal exists, specifically expressed as a two-dimensional complex Gaussian distribution function with a mean of zero:

[0084]

[0085] Where Γ(f) is the covariance matrix:

[0086] To estimate the speech power spectrum at using the maximum likelihood algorithm:

[0087]

[0088]

[0089] 3. Use the maximum expectation algorithm to estimate the arrival time difference τ in the mixture probability density function, specifically including:

[0090] Step 1. Initialize the arrival time difference and the probability of the existence of the speech signal. The initial value of the arrival time difference τ has been estimated in the preprocessing stage, and the initial value of the probability of the existence of the speech signal p1 is set to p1 = 0.5;

[0091] Step 2. Use E-step to calculate the posterior probability of speech presence;

[0092]

[0093] Step 3. Use M-step to calculate the probability of the existence of the speech signal and the arrival time difference;

[0094]

[0095]

[0096] Where, the superscript i represents the number of iteration steps; F is the number of frequency bands contained in a frame of speech signal; represents the probability of the posterior speech not existing (k=0) or the probability of the posterior speech existing (k=1); represents the probability of the prior speech not existing (k=0) or the probability of the prior speech existing (k=1); τ (i) is the estimated arrival time difference;

[0097] Step 4. Loop through the E-step and M-step iterations until the maximum number of iterations is reached. The maximum number of iterations in this embodiment can be set to 2. The maximum expectation algorithm ends and the final estimated values ​​of the arrival time difference and the probability of speech presence are obtained.

[0098] The present invention uses 320 speech signal samples collected by two microphones. The sampling frequency of the signal is 16kHz. During the acquisition process, the microphones and the speech source are at different positions. The root mean square error of the arrival time difference estimation value under different signal-to-noise ratios (5dB, 10dB and 20dB) is calculated to be 3.700*10 -4 , 3.300*10 -4 and 2.750*10 -4 . Figure 3 The relationship between arrival time difference and speech presence probability is shown. It can be seen that the greater the probability of speech presence in the frame, the smaller the error of the estimated arrival time difference.

Claims

1. A method for estimating the time difference of arrival of a speech signal based on a maximum expectation algorithm, comprising a pair of microphones for receiving far-field speech signals, characterized in that: The following steps are involved: S1. Preprocessing the voice signal received by the microphone to obtain a frequency domain signal; S2. Construct a mixed probability density function of the frequency domain signal based on the sparse representation of the speech signal; S3. Estimating the arrival time difference in the mixture probability density function using the maximum expectation algorithm; including: S301. Initialize the time difference and the prior probability of the presence of speech; S302. Calculate the posterior probability of speech presence using E-step; ； Where, is the probability that the speech signal does not exist at the time-frequency point; is the probability of the existence of the speech signal at the time-frequency point; There is no time-frequency domain signal for the speech signal The probability density function of There is a time-frequency domain signal for the speech signal The probability density function of Represents the number of iteration steps; is the estimated arrival time difference; S303 uses M-step to calculate the prior probability and arrival time difference of speech presence; ； ； Where, The number of frequency bands contained in a frame of speech signal; Represents the probability that the posterior speech does not exist or the posterior probability of speech existence ; Represents the probability that the prior speech does not exist or the prior probability of speech existence ; S304. Loop through the E-step and M-step iterations until the maximum number of iterations is estimated.

2. The method for estimating the time difference of arrival of a speech signal based on the maximum expectation algorithm according to claim 1, wherein: Said S1 comprises the following steps: S101. The time domain signal of the noisy speech received by a pair of microphones is represented as: ； ； Where, is the sampling time; is the speech source signal; and are the noisy speech signals received by the two microphones respectively; and is the noise signal; It is the arrival time difference; S102. respectively for the time domain signal and time domain signals Perform windowing and framing to obtain several frames of short-time domain signals, and perform short-time Fourier transform on each frame of short-time domain signal to obtain frequency domain signals of several frequency bands: ； ； Where, Represents the number of the speech frame; The number representing the frequency band; S103. Simplify to obtain frequency domain signal for: ； Where: ； ； 。 3. The method for estimating the time difference of arrival of a speech signal based on the maximum expectation algorithm according to claim 2, wherein: The S2 includes: Based on the sparsity of speech signal distribution in the time-frequency domain, a Gaussian mixture model is used to construct the frequency domain signal. The probability density function of the mixture is: ； In the formula, the probability of the existence of the speech signal represents the sparsity of the speech signal in the time-frequency domain. is the probability that the speech signal does not exist at the time-frequency point; is the probability of the existence of the speech signal at the time-frequency point; ; There is no time-frequency domain signal for the speech signal The probability density function of : ； in, represents the diagonal covariance matrix: ； and is the noise power spectrum received by the two microphones; There is a time-frequency domain signal for the speech signal The probability density function of : ； in, is the covariance matrix, , is the speech power spectrum, ; .

4. The method for estimating the time difference of arrival of a speech signal based on the maximum expectation algorithm according to claim 3, wherein: The noise power spectrum and Obtained based on frequency domain signal estimation.

Images