Noise estimation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a noise estimation method and device, electronic equipment and a storage medium. The noise estimation method comprises the following steps: carrying out time delay alignment on a first original voice signal and a second original voice signal to obtain a first target voice signal and a second target voice signal; determining a main path signal and an auxiliary path signal from the first target voice signal and the second target voice signal; and calculating an energy standard ratio and a dual-channel weighted feature of the main path signal and the auxiliary path signal, and performing noise estimation on the main path signal based on the energy standard ratio and the dual-channel weighted feature to obtain a noise power spectrum of the main path signal. According to the embodiment of the invention, when noise estimation is carried out on the main path signal, the energy standard ratio of the dual-channel voice signals and the dual-channel weighted features are added, so that the correlation information between the dual-channel voice signals is combined during noise estimation, the accuracy of noise estimation can be improved, and the method is more suitable for dual-channel noise estimation.

Description

Technical Field

[0001] The present application relates to the technical field of speech signal processing, and in particular to a noise estimation method, device, electronic device and storage medium. Background Art

[0002] Noise estimation is mainly used to estimate the noise in speech signals. Noise estimation has a wide range of applications in signal processing, communication systems, audio engineering, speech recognition, image processing and other fields. Accurate noise estimation is crucial for designing effective signal processing algorithms and systems.

[0003] IMCRA (Improved Minimal Controlled Recursive Averaging) is a noise estimation algorithm based on the idea of ​​time recursion, that is, the noise power spectrum estimation value obtained in the previous frame and the noisy speech power spectrum of the current frame are weighted to obtain the noise power spectrum estimation value of the current frame. However, IMCRA only performs noise estimation on single-channel speech signals, and cannot perform noise estimation on dual-channel speech signal scenarios such as Bluetooth headsets. Summary of the invention

[0004] In view of the above problems, the embodiments of the present application propose a noise estimation method, device, electronic device and storage medium to solve the noise estimation problem of dual-channel speech signals.

[0005] According to one aspect of an embodiment of the present application, a noise estimation method is provided, the method comprising:

[0006] Performing time delay alignment on the first original speech signal and the second original speech signal to obtain a first target speech signal and a second target speech signal;

[0007] Determine a main channel signal and an auxiliary channel signal from the first target voice signal and the second target voice signal;

[0008] The energy standard ratio and dual-channel weighted features of the main signal and the auxiliary signal are calculated, and noise estimation is performed on the main signal based on the energy standard ratio and the dual-channel weighted features to obtain a noise power spectrum of the main signal.

[0009] According to another aspect of an embodiment of the present application, a noise estimation device is provided, the device comprising:

[0010] An alignment module, used for performing time delay alignment on the first original voice signal and the second original voice signal to obtain a first target voice signal and a second target voice signal;

[0011] A determination module, used to determine a main channel signal and an auxiliary channel signal from the first target voice signal and the second target voice signal;

[0012] The estimation module is used to calculate the energy standard ratio and dual-channel weighted characteristics of the main signal and the auxiliary signal, and perform noise estimation on the main signal based on the energy standard ratio and the dual-channel weighted characteristics to obtain the noise power spectrum of the main signal.

[0013] According to another aspect of an embodiment of the present application, an electronic device is provided, comprising a processor and a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium; when the computer program is executed by the processor, the processor executes the noise estimation method as described in any one of the above items.

[0014] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the processor executes the noise estimation method as described in any one of the above items.

[0015] In an embodiment of the present application, time delay alignment is performed on the dual-channel voice signal, and the main signal and the auxiliary signal are determined. When the noise of the main signal is estimated, the energy standard ratio and the dual-channel weighted characteristics of the dual-channel voice signal are added. Therefore, the correlation information between the dual-channel voice signals is combined during the noise estimation. Compared with the method of performing noise estimation on a single-channel voice signal alone, the accuracy of the noise estimation can be improved, and it is more suitable for dual-channel noise estimation.

[0016] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solution of the embodiment of the present application, the drawings required for use in the description of the embodiment of the present application will be briefly introduced below. Obviously, the drawings described below are only some drawings of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a flowchart of the steps of a noise estimation method according to an embodiment of the present application;

[0019] Figure 2 It is a schematic diagram of a Bluetooth headset application scenario of an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a delay alignment process of an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of a noise estimation process of an embodiment of the present application;

[0022] Figure 5 is a structural block diagram of a noise estimation device according to an embodiment of the present application;

[0023] Figure 6 is a structural block diagram of an electronic device according to an embodiment of the present application;

[0024] Figure 7 It is a structural block diagram of a computer-readable storage medium in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] When using a Bluetooth headset to make a call, in order to reduce the impact of environmental noise on call quality and fluency, it is necessary to use voice enhancement technology to filter out interfering noise and give users a better call experience. As users' requirements for miniaturization and portability of Bluetooth headsets continue to increase, researchers are also constantly optimizing Bluetooth transmission technology to make real-time data interaction between wide-distance Bluetooth devices possible. On this basis, a solution of placing a single microphone on each side of the Bluetooth headset was proposed.

[0027] In the field of dual-channel speech enhancement, previous research has mostly focused on algorithms based on beamforming, which uses the time difference between the signals received by the two channels to time-calibrate these signals and then weight them to form a beam that has a significant enhancement effect in the target direction and a suppression effect in other directions. However, in the application scenario of Bluetooth headsets, the two microphones of the dual-microphone array are located on the two-ear Bluetooth headsets, with a distance of more than 15 cm, which is far beyond the criterion required by the spatial sampling theorem that the microphone distance should be less than half the minimum wavelength of the speech, making the beamforming algorithm unusable in the application scenario of Bluetooth headsets.

[0028] In view of the above problems, it is proposed in the embodiments of the present application to first perform noise estimation, and then perform speech enhancement based on the noise estimation. Considering that the pure single-channel VAD (Voice Activity Detection) algorithm in the IMCRA noise estimation algorithm is not robust enough, when the signal-to-noise ratio is low, the VAD algorithm in the IMCRA is less accurate, resulting in a large error in the noise power spectrum estimation, and directly affecting the effect of speech enhancement. In view of the above problems, in the embodiments of the present application, the single-channel IMCRA noise estimation algorithm is expanded to dual-channel, and a VAD algorithm that integrates dual-channel information is designed, so that it is more robust in complex acoustic environments with wide spacing and low signal-to-noise ratio. The following is a detailed introduction.

[0029] The noise estimation method of the embodiment of the present application can be applied to dual-channel or multi-channel speech signal processing scenarios such as Bluetooth headsets.

[0030] Reference Figure 1 , shows a step flow chart of a noise estimation method in an embodiment of the present application.

[0031] like Figure 1 As shown, the noise estimation method may include the following steps:

[0032] Step 101: perform time delay alignment on a first original speech signal and a second original speech signal to obtain a first target speech signal and a second target speech signal.

[0033] In the embodiment of the present application, two channels are used as an example for explanation. The first microphone corresponds to the first channel, and the second microphone corresponds to the second channel. The first original voice signal is the original voice signal of the first channel collected by the first microphone, and the second original voice signal is the original voice signal of the second channel collected by the second microphone. The first original voice signal and the second original voice signal may contain only the target voice, or only noise, or both the target voice and noise.

[0034] Reference Figure 2 , shows a schematic diagram of a Bluetooth headset application scenario in an embodiment of the present application. Figure 2 As shown, the two microphones (mic1 and mic2) on the two ear sides of the Bluetooth headset record the two earphones respectively. The target voice is the sound emitted by the mouth of the headset user, and other sounds are defined as noise, including near-field directional noise and far-field diffuse noise.

[0035] After the first original voice signal and the second original voice signal are collected, the first original voice signal and the second original voice signal are time-delay aligned to obtain the first target voice signal and the second target voice signal, thereby facilitating subsequent signal analysis.

[0036] Exemplarily, performing time delay alignment on the first original speech signal and the second original speech signal to obtain the first target speech signal and the second target speech signal may include the following steps A1 to A3:

[0037] Step A1: convert the first original speech signal into a first frequency domain speech signal, and convert the second original speech signal into a second frequency domain speech signal.

[0038] The first original voice signal and the second original voice signal are both time domain voice signals. The first original voice signal is windowed and framed, and Fourier transform, such as STFT (Short-Time Fourier Transform, short-time Fourier transform), is performed to obtain a first frequency domain voice signal. The second original voice signal is windowed and framed, and Fourier transform is performed to obtain a second frequency domain voice signal. Therefore, the first frequency domain voice signal and the second frequency domain voice signal are both multi-frames.

[0039] When the time-domain speech signal is processed by windowing and framing, a movable window of finite length can be used to weight the time-domain speech signal to realize the framing operation, that is, the time-domain speech signal is multiplied by a window function (which can be any applicable window function such as a Hamming window) to obtain the speech signal after the windowing. Because noise estimation and noise reduction processing are both to analyze the frequency domain of the speech signal, it is also necessary to use Fourier transform to obtain the frequency domain characteristics of the speech signal. For the specific process of windowing and framing and Fourier transform, it can be processed according to actual experience, and the present embodiment will not be discussed in detail at this.

[0040] Step A2, when the first original speech signal and the second original speech signal are initial speech signals, calculate the target delay based on the first frequency domain speech signal and the second frequency domain speech signal, and store the target delay; when the first original speech signal and the second original speech signal are not initial speech signals, obtain the stored target delay.

[0041] In the embodiment of the present application, considering that the user of the Bluetooth headset is generally fixed, the initial voice signal after the Bluetooth headset is turned on is usually a pure voice signal without noise, so the target delay is calculated based on the initial voice signal within the first period of time after the Bluetooth headset is turned on each time, and the target delay is no longer calculated subsequently. This method can reduce the amount of calculation.

[0042] Exemplarily, the process of calculating the target delay based on the first frequency domain voice signal and the second frequency domain voice signal includes: performing delay estimation on the first frequency domain voice signal and the second frequency domain voice signal to obtain an initial delay; determining multiple candidate delays based on the initial delay, and selecting the target delay from the multiple candidate delays. This method takes into account that the initial delay obtained by the delay estimation method may have a certain deviation, so a delay analysis is further performed on the basis of the initial delay, thereby improving the accuracy of the target delay.

[0043] When estimating the time delay of the first frequency domain speech signal and the second frequency domain speech signal, any applicable method such as GCC-PHAT (Generalized Cross-Correlation Phase Transform), cross-correlation function, machine learning model, etc. can be used to estimate the time delay and obtain the initial time delay. This embodiment will not be discussed in detail here.

[0044] When determining multiple candidate delays based on the initial delay, the candidate delays may be determined based on the initial delay according to actual needs. The multiple candidate delays include a delay less than the initial delay, the initial delay, and a delay greater than the initial delay. The specific values ​​of the candidate delays may be set according to actual needs, and this embodiment does not limit this.

[0045] Exemplarily, the process of selecting the target time delay from the multiple candidate time delays includes: for each candidate time delay, obtaining the candidate frequency domain voice signal corresponding to the first frequency domain voice signal under the candidate time delay; for each candidate frequency domain voice signal, calculating the first correlation between the first frequency domain voice signal and the candidate frequency domain voice signal; judging whether there is an obvious difference between the first correlations; when there is, taking the candidate time delay corresponding to the candidate frequency domain voice signal with the largest first correlation as the target time delay; when there is not, for each candidate frequency domain voice signal among the multiple candidate frequency domain voice signals with the largest first correlation, converting the candidate frequency domain voice signal into a candidate time domain voice signal, calculating the Pearson coefficient between the first original voice signal and the candidate time domain voice signal, and taking the candidate time delay corresponding to the candidate time domain voice signal with the largest Pearson coefficient as the target time delay.

[0046] Through the above method, first select the frequency domain judgment with less calculation amount, calculate the first correlation judgment in the frequency domain judgment, if there is an obvious difference in the first correlation, no longer perform the time domain judgment, select the candidate delay when the correlation is the largest as the target delay; if there is no obvious difference in the first correlation, then enter the time domain judgment, use the normalized Pearson coefficient to characterize the similarity of the dual-channel time domain speech signal, and select the candidate delay when the Pearson coefficient is used as the target delay. This method can reduce the amount of calculation and improve efficiency.

[0047] Exemplarily, the process of determining whether there is a significant difference between the first correlations includes: calculating a first ratio between the second largest first correlation and the largest first correlation; when the first ratio is less than or equal to a first preset ratio, determining that there is a significant difference between the first correlations; otherwise (the first ratio is greater than the first preset ratio), determining that there is no significant difference between the first correlations. Alternatively, calculating a second ratio between the largest first correlation and the second largest first correlation; when the second ratio is greater than or equal to a second preset ratio, determining that there is a significant difference between the first correlations; otherwise (the second ratio is less than the second preset ratio), determining that there is no significant difference between the first correlations. This method is based on comparison of the two largest first correlations, which can simplify the judgment process. For the specific values ​​of the first preset ratio and the second preset ratio, empirical values ​​can be set according to actual needs, and this embodiment does not limit this.

[0048] When the first frequency domain speech signal and the second frequency domain speech signal are noisy speech signals:

[0049] The expression of the first frequency domain speech signal can be as follows:

[0050] X1(n,k)=M(n,k)+N(n,k)Formula 1

[0051] The expression of the second frequency domain speech signal can be as follows:

[0052] X2(n,k)=M(n,k)e -iωt +N(n,k)e -iωt Formula 2

[0053] When the first frequency domain speech signal and the second frequency domain speech signal are pure speech signals:

[0054] The expression of the first frequency domain speech signal can be as follows:

[0055] X1(n,k)=M(n,k)Formula 3

[0056] The expression of the second frequency domain speech signal can be as follows:

[0057] X2(n,k)=M(n,k)e -iωt Formula 4

[0058] Among them, n represents the frame identifier, k represents the frequency point identifier, X1(n,k) represents the first frequency domain speech signal, X2(n,k) represents the second frequency domain speech signal, M(n,k) represents the target speech, N(n,k) represents the noise, i represents the imaginary unit used to represent the phase operation, ω represents the angular frequency, and t represents the delay.

[0059] In the embodiment of the present application, the human voice frequency band from 300 Hz to 3400 Hz can be intercepted to perform delay estimation based on GCC-PHAT to obtain the initial delay.

[0060] In the following, the first frequency domain speech signal is X1(n,k)=M(n,k), the initial delay is t1, and the multiple candidate delays are t1, t1-1, and t1+1.

[0061] X1(n,k) corresponds to the candidate frequency domain speech signal X at t1 20 (n, k) is as shown in the following formula 5:

[0062]

[0063] X1(n,k) corresponds to the candidate frequency domain speech signal X at t1+1 21 (n, k) is as shown in the following formula 6:

[0064]

[0065] X1(n,k) corresponds to the candidate frequency domain speech signal X at t1-1 22 (n, k) is as shown in the following formula 7:

[0066]

[0067] Calculate X1(n,k) and X respectively 20 The first correlation COH of (n,k) 20 , X1(n,k) and X 21 The first correlation COH of (n,k) 21 , X1(n,k) and X 22 The first correlation COH of (n,k) 22 .

[0068] X1(n,k) and X 20 The first correlation COH of (n,k) 20 For example, as shown in Formula 8:

[0069]

[0070] Among them, i is the frequency mark corresponding to the lower limit of the frequency interval (for example, the frequency mark corresponding to 300 Hz is 5), j is the frequency mark corresponding to the upper limit of the frequency interval (for example, the frequency mark corresponding to 3400 Hz is 54), P 120 (n,k) represents the difference between X1(n,k) and X 20 (n,k), P1(n,k) represents the actual power spectrum of X1(n,k), P 20 (n,k) represents X 20 The actual power spectrum of (n, k). The calculation of the actual power spectrum and the cross power spectrum can be processed according to actual experience, and will not be discussed in detail in this embodiment.

[0071] COH is calculated in the same way as above 21 and COH 22 .

[0072] Compare COH 20 , COH 21 and COH 22 size.

[0073] For example, COH 20 >COH 21 >COH 22 ,calculate

[0074] like Then the target delay is determined to be t1. β1 is the first preset ratio.

[0075] like Then X 20 (n, k) is converted into a candidate time domain speech signal x through an inverse Fourier transform, such as ISTFT (Inverse Short-Time Fourier Transform). 20 , X 21 (n,k) is converted into a candidate time-domain speech signal x through inverse Fourier transform 21 , the time domain speech signal (ie, the first original speech signal) corresponding to X1(n,k) is x1.

[0076] Calculate x1 and x by formula 9 20 Pearson's coefficient r 20 :

[0077]

[0078] Calculate x1 and x by formula 10 21 Pearson's coefficient r 21 :

[0079]

[0080] Wherein, m is the number of sampling points selected for one calculation (for example, 256), and the sum is the sum of m sampling points.

[0081] If | r 20 |>|r 21 |, the target delay is t1, otherwise, the target delay is t1+1.

[0082] Step A3: align the first frequency domain speech signal and the second frequency domain speech signal by using the target delay to obtain a first target speech signal and a second target speech signal.

[0083] The first frequency domain speech signal X1(n,k) is used as the first target speech signal. According to Formula 4, the target delay is added to the first frequency domain speech signal X1(n,k) to obtain the second target speech signal X2′(n,k)=X1(n,k)e -iωt′ , t ′ Indicates the target delay.

[0084] Figure 3 FIG. 1 is a schematic diagram of a delay alignment process according to an embodiment of the present application. Figure 3 As shown, the delay of the dual-channel speech signal is estimated to obtain an initial delay n; candidate delay n-1, candidate delay n, and candidate delay n+1 are determined; the first correlation under each candidate delay is calculated; the first correlations are sorted; it is determined whether there is an obvious difference between the first correlations; if so, the candidate delay corresponding to the largest first correlation is used as the target delay; if not, an inverse Fourier transform is performed; the Pearson coefficient is calculated; and the candidate delay corresponding to the largest Pearson coefficient is used as the target delay.

[0085] Step 102: determine a main channel signal and an auxiliary channel signal from the first target voice signal and the second target voice signal.

[0086] Since the energy of the target speech arriving at the two channels is approximately equal, and there is an energy difference between the noise arriving at the two channels, the signal with a smaller actual power spectrum has a higher signal-to-noise ratio and is selected as the main signal, and the other is used as the auxiliary signal.

[0087] Exemplarily, the process of determining the main channel signal and the auxiliary channel signal from the first target voice signal and the second target voice signal may include: calculating the actual power spectrum of the first target voice signal and the actual power spectrum of the second target voice signal; when the actual power spectrum of the first target voice signal is greater than or equal to the actual power spectrum of the second target voice signal, taking the second target voice signal as the main channel signal and the first target voice signal as the auxiliary channel signal; when the actual power spectrum of the first target voice signal is less than the actual power spectrum of the second target voice signal, taking the first target voice signal as the main channel signal and the second target voice signal as the auxiliary channel signal. The calculation of the actual power spectrum can be processed based on actual experience and will not be discussed in detail in this embodiment.

[0088] The energy difference between the first target speech signal and the second target speech signal is calculated by the following formula 11:

[0089] PLD(n,k)=|P1(n,k)|-|P2(n,k)|Formula 11

[0090] Wherein, PLD(n,k) represents the energy difference between the first target speech signal and the second target speech signal, P1(n,k) represents the actual power spectrum of the first target speech signal, and P2(n,k) represents the actual power spectrum of the second target speech signal.

[0091] If PLD(n,k)≥0, the main channel signal is the first target speech signal; otherwise, the main channel signal is the second target speech signal.

[0092] In the embodiment of the present application, the main channel signal is taken as the first target voice signal, and the auxiliary channel signal is taken as the second target voice signal as an example for description.

[0093] Step 103, calculating the energy standard ratio and dual-channel weighted features of the main signal and the auxiliary signal, performing noise estimation on the main signal based on the energy standard ratio and the dual-channel weighted features, and obtaining the noise power spectrum of the main signal.

[0094] Exemplarily, the energy standard ratio and dual-channel weighted features of the main signal and the auxiliary signal are calculated, and the noise power spectrum of the main signal is obtained by performing noise estimation on the main signal based on a preset fixed search window length, the energy standard ratio and the dual-channel weighted features. The fixed search window length can be set according to actual needs.

[0095] In the above method, a fixed search window length is used, so that the estimated noise power spectrum cannot track the actual noise changes in time, which will produce a large delay and result in low accuracy of noise estimation. In response to the above problem, an adaptive search window length can be used for processing. Exemplarily, the energy standard ratio and dual-channel weighted characteristics of the main signal and the auxiliary signal are calculated, and the search window length of the main signal is calculated based on the energy standard ratio. The main signal is estimated based on the search window length, the energy standard ratio and the dual-channel weighted characteristics to obtain the noise power spectrum of the main signal.

[0096] Exemplarily, calculating the energy standard ratio and dual-channel weighted features of the main signal and the auxiliary signal, performing noise estimation on the main signal based on the energy standard ratio and the dual-channel weighted features, and obtaining the noise power spectrum of the main signal includes the following steps B1 to B6:

[0097] Step B1, calculating the energy standard ratio of the main channel signal to the auxiliary channel signal.

[0098] Exemplarily, the energy standard ratio of the main channel signal to the auxiliary channel signal is calculated by the following formula 12:

[0099]

[0101] Where n represents the frame identifier, k represents the frequency point identifier, and P b (n) represents the energy standard ratio of the main signal of the nth frame to the auxiliary signal of the nth frame, P1(n,k) represents the actual power spectrum of the main signal of the nth frame, and P2(n,k) represents the actual power spectrum of the auxiliary signal of the nth frame. represents the average value of the actual power spectrum of the main signal of the nth frame and the actual power spectrum of the auxiliary signal of the nth frame, P1(n-1,k) represents the actual power spectrum of the main signal of the n-1th frame, and P2(n-1,k) represents the actual power spectrum of the auxiliary signal of the n-1th frame. It represents the average value of the actual power spectrum of the main signal in the n-1th frame and the actual power spectrum of the auxiliary signal in the n-1th frame.

[0102] Since the frequency range of human voice is mainly from 300 Hz to 3400 Hz, the energy standard ratio of the n-th frame and the n-1-th frame speech signal in the range of 300 Hz to 3400 Hz can be calculated.

[0103] Step B2: Calculate the search window length of the main channel signal based on the energy standard ratio.

[0104] Exemplarily, calculating the search window length of the main path signal based on the energy standard ratio includes:

[0105] When the energy standard ratio is greater than or equal to the preset maximum energy standard ratio, the preset minimum window length is used as the search window length. b (n)≥α max , the noise energy changes dramatically, and a smaller window length is selected to track the noise change in time, that is, U(n,k)=U min , α max represents the preset maximum energy standard ratio, U(n,k) represents the search window length of the main channel signal of the nth frame, and U min Indicates the preset minimum window length.

[0106] When the energy standard ratio is less than or equal to the preset minimum energy standard ratio, the preset maximum window length is used as the search window length. b (n)≤α min , the noise energy is sharply reduced, and a longer window length is selected to ensure the stability of the estimation, that is, U(n,k)=U max , α min Indicates the preset minimum energy standard ratio, U max Indicates the preset maximum window length.

[0107] When the energy standard ratio is greater than the preset minimum energy standard ratio and less than the preset maximum energy standard ratio, that is, α min <P b (n)<α max , the noise energy is relatively stable, but there are still slight changes. The window length needs to be fine-tuned to ensure the accuracy of noise estimation.

[0108] Formula 13 calculates the search window length of the main channel signal:

[0109]

[0110] Where n represents the frame identifier, k represents the frequency identifier, U(n,k) represents the search window length of the main signal of the nth frame, U(n-1,k) represents the search window length of the main signal of the n-1th frame, P b (n) represents the energy standard ratio of the first target speech signal of the nth frame to the second target speech signal of the nth frame, U max represents the maximum window length, U min represents the minimum window length, Indicates rounding up.

[0111] The specific values ​​of the minimum energy standard ratio, the maximum energy standard ratio, the minimum window length, and the maximum window length can be set according to actual needs, and this embodiment does not limit this. For example, the minimum window length is 4, the maximum window length is 16, and so on.

[0112] Step B3, performing a first smoothing operation on the main channel signal, performing a first iterative operation on the main channel signal based on the energy standard ratio and the search window length, and obtaining a first speech detection result of the main channel signal.

[0113] In the process of performing the first smoothing operation on the main signal, formula 14 and formula 15 are calculated:

[0114]

[0115] S1(n,k)=α s S(n-1,k)+(1-α s )S f1 (n,k) Formula 15

[0116] Among them, S f1 (n, k) represents the average power spectrum of the first frequency point of the main signal of the nth frame, S1(n, k) represents the first smoothed power spectrum of the main signal of the nth frame, b(i) represents the standard window function, Y1(n, ki) represents the amplitude spectrum of the main signal of the nth frame at different frequencies, ω represents the angular frequency, α s represents the spectrum smoothing factor, and S1(n-1,k) represents the first smoothed power spectrum of the main channel signal of the n-1th frame.

[0117] The process of performing a first iterative operation on the main channel signal based on the energy standard ratio and the search window length to obtain a first speech detection result of the main channel signal includes: performing a first iterative operation on the main channel signal based on the search window length to calculate a first judgment index and a second judgment index; when the first judgment index is less than a first index threshold and the second judgment index is less than a second index threshold, or when the energy standard ratio is greater than a preset maximum energy standard ratio, determining that the first speech detection result of the main channel signal is that speech is absent; otherwise, determining that speech is present.

[0118] The first iteration operation may include a first minimum search and a first VAD.

[0119] The first minimum value search is performed based on the search window length calculated above, as shown in Formula 16:

[0120] S min (n,k)=min{S min (n-1,k),S1(n,k)}Formula 16

[0121] Among them, S min (n, k) is the first minimum power spectrum of the main signal of the nth frame, S min(n-1, k) represents the first minimum power spectrum of the main signal of the n-1th frame, and S1(n, k) represents the first smoothed power spectrum of the main signal of the nth frame.

[0122] The first VAD is performed based on the first minimum power spectrum and energy standard ratio obtained from the first minimum search, as shown below:

[0123]

[0124] Among them, γ min (n, k) represents the first judgment index, ζ(n, k) represents the second judgment index, Y1(n, k) represents the amplitude spectrum of the main signal of the nth frame, S1(n, k) represents the first smoothed power spectrum of the main signal of the nth frame, S min (n, k) is the first minimum power spectrum of the main signal of the nth frame, B min For bias compensation.

[0125] If γ min (n, k) < γ0 (first index threshold) and ζ(n, k) < ζ0 (second index threshold), or, P b (n)>α max , it is determined that there is no voice in the main channel signal of the nth frame; in other cases, it is determined that there is voice in the main channel signal of the nth frame.

[0126] Step B4: performing a smoothing operation on the main channel signal again based on the first speech detection result, and calculating dual-channel weighted features of the main channel signal and the auxiliary channel signal.

[0127] In the process of re-smoothing the main signal, formula 19 and formula 20 are calculated:

[0128]

[0129] in, Indicates the average power spectrum of the second frequency point of the main signal of the nth frame, represents the second smoothed power spectrum of the main signal of the nth frame, b(i) represents the standard window function, I(n,ki) represents the shift function, Y1(n,ki) represents the amplitude spectrum of the main signal of the nth frame at different frequencies, ω represents the angular frequency, α s represents the spectral smoothing factor, Represents the second smoothed power spectrum of the main channel signal of the n-1th frame.

[0130] The dual-channel weighted characteristics of the main channel signal and the auxiliary channel signal are calculated by the following formula 21:

[0131]

[0132] Alternatively, the dual-channel weighted characteristics of the main channel signal and the auxiliary channel signal are calculated by the following formula 22:

[0133]

[0134] Alternatively, the dual-channel weighted characteristics of the main channel signal and the auxiliary channel signal are calculated by the following formula 23:

[0135]

[0136] Wherein, n represents the frame identifier, k represents the frequency identifier, D(n,k) represents the dual-channel weighted characteristics of the main signal of the nth frame and the auxiliary signal of the nth frame, α1 represents the dynamic weighting factor, COH(n-1,k) represents the first correlation between the main signal of the n-1th frame and the auxiliary signal of the n-1th frame, COH(n,k) represents the first correlation between the main signal of the nth frame and the auxiliary signal of the nth frame, P1(n,k) represents the actual power spectrum of the main signal of the nth frame, P2(n,k) represents the actual power spectrum of the auxiliary signal of the nth frame, P1(n-1,k) represents the actual power spectrum of the main signal of the n-1th frame, P 2(n-1,k) represents the actual power spectrum of the auxiliary signal of the n-1th frame, COH′(n-1,k) represents the second correlation between the main signal of the n-1th frame and the auxiliary signal of the n-1th frame, COH′(n,k) represents the second correlation between the main signal of the n-1th frame and the auxiliary signal of the n-1th frame, S1(n,k) represents the first smoothed power spectrum of the main signal of the n-1th frame, S2(n,k) represents the first smoothed power spectrum of the auxiliary signal of the n-1th frame, S1(n-1,k) represents the first smoothed power spectrum of the main signal of the n-1th frame, S2(n-1,k) represents the first smoothed power spectrum of the auxiliary signal of the n-1th frame, Indicates the third correlation between the main channel signal of the n-1th frame and the auxiliary channel signal of the n-1th frame, Indicates the third correlation between the main channel signal of the nth frame and the auxiliary channel signal of the nth frame, Represents the second smoothed power spectrum of the main signal of the nth frame, represents the second smoothed power spectrum of the auxiliary signal of the nth frame, represents the second smoothed power spectrum of the main signal of the n-1th frame, Represents the second smoothed power spectrum of the auxiliary channel signal of the n-1th frame.

[0137] For the above S2(n,k), The calculation process can refer to S1(n,k), The first correlation is calculated based on P1(n,k) and P2(n,k) according to Formula 8, the second correlation is calculated based on S1(n,k) and S2(n,k) according to Formula 8, and the third correlation is calculated based on and Calculated according to formula 8.

[0138] Step B5, performing an iterative operation on the main channel signal again based on the dual-channel weighted features and the search window length to obtain a second speech detection result of the main channel signal.

[0139] Exemplarily, the process of performing an iterative operation on the main signal based on the dual-channel weighted feature and the search window length to obtain the second speech detection result of the main signal includes: performing an iterative operation on the main signal based on the search window length to calculate a third judgment index and a fourth judgment index; when the third judgment index is less than or equal to 1 and the fourth judgment index is less than a fourth index threshold, or when the main signal of the previous frame of the main signal is a noise signal and the dual-channel weighted feature is greater than 1, determining that the second speech detection result of the main signal is a noise signal; when the third judgment index is greater than 1 and less than the third indicator threshold, and the fourth judgment index is less than the fourth indicator threshold, determining that the second speech detection result of the main signal is a noisy speech signal; when the third judgment index is greater than or equal to the third indicator threshold and the fourth judgment index is greater than or equal to the fourth indicator threshold, or when the main signal of the previous frame of the main signal is a pure speech signal and the dual-channel weighted feature is less than or equal to 1, determining that the second speech detection result of the main signal is a pure speech signal.

[0140] The re-iterative operation may include re-minimum search and re-VAD.

[0141] Based on the search window length calculated above, a minimum value search is performed again, as shown in Formula 24:

[0142]

[0143] in, is the second minimum power spectrum of the main signal of the nth frame, Indicates the second minimum power spectrum of the main signal of the n-1th frame, Represents the second smoothed power spectrum of the main channel signal of the nth frame.

[0144] Based on the second minimum power spectrum obtained by the second minimum search and the dual-channel weighted features, VAD is performed again as shown below:

[0145]

[0146] in, represents the third judgment index, represents the fourth judgment index, Y1(n,k) represents the amplitude spectrum of the main channel signal of the nth frame, Represents the second smoothed power spectrum of the main signal of the nth frame, is the second minimum power spectrum of the main signal of the nth frame, B min For bias compensation.

[0147] like and (the fourth indicator threshold, which is equal to the second indicator threshold here), or, the main channel signal of the n-1th frame is a noise signal and D(n,k)>1, then the main channel signal of the nth frame is a noise signal.

[0148] like (Third indicator threshold) and Then the main signal of the nth frame is a noisy language signal.

[0149] like and , or, the main channel signal of the n-1th frame is a pure speech signal and D(n,k)≤1, then the main channel signal of the nth frame is a pure speech signal.

[0150] Step B6: Calculate the noise power spectrum of the main signal based on the second speech detection result.

[0151] Calculate the probability of speech loss:

[0152]

[0153] Wherein, q(n,k) represents the speech missing probability of the main channel signal of the nth frame, and other parameters refer to the above related descriptions.

[0154] Calculate the probability of speech presence:

[0155]

[0157] Among them, p(n,k) represents the probability of speech existence in the main channel signal of the nth frame, ξ(n,k) represents the priori signal-to-noise ratio, γ(n,k) represents the posterior signal-to-noise ratio, and other parameters refer to the relevant description above.

[0158] In order to track the noise variation more accurately, an adaptive smoothing factor is calculated:

[0159]

[0160] in, Represents the adaptive smoothing factor of the main signal of the nth frame, α d Indicates the set fixed smoothing factor. For other parameters, refer to the above description.

[0161] Compute the noise power spectrum:

[0162]

[0163] in, represents the noise power spectrum of the main signal of the nth frame, Indicates the noise power spectrum of the main signal of the n-1th frame. For other parameters, refer to the above related descriptions.

[0164] Figure 4 is a schematic diagram of a noise estimation process in an embodiment of the present application. Figure 4 As shown, the acoustic estimation process includes: Fourier transform of dual-channel speech signals; delay estimation; calculation of target delay; delay alignment; main channel signal selection; calculation of energy standard ratio based on actual power spectrum of main channel signal and actual power spectrum of auxiliary channel signal; calculation of search window length; first iteration operation; calculation of dual-channel weighted features based on smoothed power spectrum of main channel signal and smoothed power spectrum of auxiliary channel signal; second iteration operation; speech probability estimation to obtain noise power spectrum. For the specific process, refer to the detailed description above.

[0165] In an embodiment of the present application, after obtaining the noise power spectrum of the main signal, the main signal can also be subjected to speech enhancement processing based on the noise power spectrum of the main signal. Specifically, the obtained noise power spectrum can be combined with a speech enhancement algorithm to obtain a final enhanced speech signal. The speech enhancement algorithm includes but is not limited to spectral subtraction, Wiener filtering, and optimal logarithmic amplitude spectrum estimation. The specific speech enhancement process can be processed based on actual experience and will not be discussed in detail here.

[0166] Reference Figure 5 , shows a structural block diagram of a noise estimation device according to an embodiment of the present application.

[0167] like Figure 5 As shown, the noise estimation device may include the following modules:

[0168] An alignment module 501 is used to perform time delay alignment on the first original speech signal and the second original speech signal to obtain a first target speech signal and a second target speech signal;

[0169] A determination module 502, configured to determine a primary channel signal and a secondary channel signal from the first target voice signal and the second target voice signal;

[0170] The estimation module 503 is used to calculate the energy standard ratio and dual-channel weighted characteristics of the main signal and the auxiliary signal, and perform noise estimation on the main signal based on the energy standard ratio and the dual-channel weighted characteristics to obtain the noise power spectrum of the main signal.

[0171] Optionally, the alignment module 501 includes: a signal conversion unit, used to convert the first original speech signal into a first frequency domain speech signal, and convert the second original speech signal into a second frequency domain speech signal; a delay acquisition unit, used to calculate the target delay based on the first frequency domain speech signal and the second frequency domain speech signal when the first original speech signal and the second original speech signal are initial speech signals, and store the target delay; when the first original speech signal and the second original speech signal are not initial speech signals, obtain the stored target delay; a signal alignment unit, used to use the target delay to perform delay alignment on the first frequency domain speech signal and the second frequency domain speech signal to obtain the first target speech signal and the second target speech signal.

[0172] Optionally, the delay acquisition unit includes: a delay estimation subunit, used to perform delay estimation on the first frequency domain speech signal and the second frequency domain speech signal to obtain an initial delay; a delay selection subunit, used to determine multiple candidate delays based on the initial delay, and select the target delay from the multiple candidate delays; the multiple candidate delays include a delay less than the initial delay, the initial delay, and a delay greater than the initial delay.

[0173] Optionally, the time delay selection subunit is specifically used to obtain, for each candidate time delay, a candidate frequency domain voice signal corresponding to the first frequency domain voice signal under the candidate time delay; for each candidate frequency domain voice signal, calculate the first correlation between the first frequency domain voice signal and the candidate frequency domain voice signal; determine whether there is an obvious difference between the first correlations; when there is, take the candidate time delay corresponding to the candidate frequency domain voice signal with the largest first correlation as the target time delay; when not there, for each candidate frequency domain voice signal among the multiple candidate frequency domain voice signals with the largest first correlation, convert the candidate frequency domain voice signal into a candidate time domain voice signal, calculate the Pearson coefficient between the first original voice signal and the candidate time domain voice signal, and take the candidate time delay corresponding to the candidate time domain voice signal with the largest Pearson coefficient as the target time delay.

[0174] Optionally, the estimation module 503 is specifically used to calculate the energy standard ratio and dual-channel weighted characteristics of the main signal and the auxiliary signal, calculate the search window length of the main signal based on the energy standard ratio, and perform noise estimation on the main signal based on the search window length, the energy standard ratio and the dual-channel weighted characteristics to obtain the noise power spectrum of the main signal.

[0175] Optionally, the estimation module 503 includes: a first calculation unit, used to calculate the energy standard ratio of the main signal to the auxiliary signal; a second calculation unit, used to calculate the search window length of the main signal based on the energy standard ratio; a first operation unit, used to perform a first smoothing operation on the main signal, and perform a first iterative operation on the main signal based on the energy standard ratio and the search window length to obtain a first speech detection result of the main signal; a third calculation unit, used to perform a second smoothing operation on the main signal based on the first speech detection result, and calculate the dual-channel weighted characteristics of the main signal and the auxiliary signal; a second operation unit, used to perform a second iterative operation on the main signal based on the dual-channel weighted characteristics and the search window length to obtain a second speech detection result of the main signal; and a fourth calculation unit, used to calculate the noise power spectrum of the main signal based on the second speech detection result.

[0176] Optionally, the energy standard ratio of the main channel signal to the auxiliary channel signal is calculated by the following formula:

[0177]

[0178] Where n represents the frame identifier, k represents the frequency point identifier, and P b (n) represents the energy standard ratio of the main signal of the nth frame to the auxiliary signal of the nth frame, P1(n,k) represents the actual power spectrum of the main signal of the nth frame, and P2(n,k) represents the actual power spectrum of the auxiliary signal of the nth frame. represents the average value of the actual power spectrum of the main signal of the nth frame and the actual power spectrum of the auxiliary signal of the nth frame, P1(n-1,k) represents the actual power spectrum of the main signal of the n-1th frame, and P2(n-1,k) represents the actual power spectrum of the auxiliary signal of the n-1th frame. It represents the average value of the actual power spectrum of the main signal in the n-1th frame and the actual power spectrum of the auxiliary signal in the n-1th frame.

[0179] Optionally, the second calculation unit is specifically used to use the preset minimum window length as the search window length when the energy standard ratio is greater than or equal to the preset maximum energy standard ratio; when the energy standard ratio is less than or equal to the preset minimum energy standard ratio, use the preset maximum window length as the search window length; when the energy standard ratio is greater than the preset minimum energy standard ratio and less than the preset maximum energy standard ratio, calculate the search window length of the main channel signal by the following formula:

[0180]

[0181] Where n represents the frame identifier, k represents the frequency identifier, U(n,k) represents the search window length of the main signal of the nth frame, U(n-1,k) represents the search window length of the main signal of the n-1th frame, P b (n) represents the energy standard ratio of the first target speech signal of the nth frame to the second target speech signal of the nth frame, U max represents the maximum window length, U min represents the minimum window length, Indicates rounding up.

[0182] Optionally, the dual-channel weighted characteristics of the main channel signal and the auxiliary channel signal are calculated by the following formula:

[0183]

[0184] or,

[0185]

[0186] or,

[0187]

[0188] Wherein, n represents the frame identifier, k represents the frequency identifier, D(n,k) represents the dual-channel weighted characteristics of the main signal of the nth frame and the auxiliary signal of the nth frame, α1 represents the dynamic weighting factor, COH(n-1,k) represents the first correlation between the main signal of the n-1th frame and the auxiliary signal of the n-1th frame, COH(n,k) represents the first correlation between the main signal of the nth frame and the auxiliary signal of the nth frame, P1(n,k) represents the actual power spectrum of the main signal of the nth frame, P2(n,k) represents the actual power spectrum of the auxiliary signal of the nth frame, P1(n-1,k) represents the actual power spectrum of the main signal of the n-1th frame, P 2(n-1,k) represents the actual power spectrum of the auxiliary signal of the n-1th frame, COH′(n-1,k) represents the second correlation between the main signal of the n-1th frame and the auxiliary signal of the n-1th frame, COH′(n,k) represents the second correlation between the main signal of the n-1th frame and the auxiliary signal of the n-1th frame, S1(n,k) represents the first smoothed power spectrum of the main signal of the n-1th frame, S2(n,k) represents the first smoothed power spectrum of the auxiliary signal of the n-1th frame, S1(n-1,k) represents the first smoothed power spectrum of the main signal of the n-1th frame, S2(n-1,k) represents the first smoothed power spectrum of the auxiliary signal of the n-1th frame, Indicates the third correlation between the main channel signal of the n-1th frame and the auxiliary channel signal of the n-1th frame, Indicates the third correlation between the main channel signal of the nth frame and the auxiliary channel signal of the nth frame, Represents the second smoothed power spectrum of the main signal of the nth frame, represents the second smoothed power spectrum of the auxiliary signal of the nth frame, represents the second smoothed power spectrum of the main signal of the n-1th frame, Represents the second smoothed power spectrum of the auxiliary channel signal of the n-1th frame.

[0189] Optionally, the first operation unit is specifically used to perform a first iterative operation on the main channel signal based on the search window length, and calculate a first judgment index and a second judgment index; when the first judgment index is less than a first index threshold and the second judgment index is less than a second index threshold, or when the energy standard ratio is greater than a preset maximum energy standard ratio, determine that the first speech detection result of the main channel signal is that there is no speech; otherwise, determine that the first speech detection result of the main channel signal is that there is speech.

[0190] Optionally, the second operation unit is specifically used to perform an iterative operation on the main channel signal based on the search window length to calculate a third judgment index and a fourth judgment index; when the third judgment index is less than or equal to 1 and the fourth judgment index is less than a fourth index threshold, or when the main channel signal in a previous frame of the main channel signal is a noise signal and the dual-channel weighted feature is greater than 1, determine that the second speech detection result of the main channel signal is a noise signal; when the third judgment index is greater than 1 and less than a third indicator threshold, and the fourth judgment index is less than a fourth indicator threshold, determine that the second speech detection result of the main channel signal is a noisy speech signal; when the third judgment index is greater than or equal to the third indicator threshold and the fourth judgment index is greater than or equal to the fourth indicator threshold, or when the main channel signal in a previous frame of the main channel signal is a pure speech signal and the dual-channel weighted feature is less than or equal to 1, determine that the second speech detection result of the main channel signal is a pure speech signal.

[0191] Optionally, the device further includes: an enhancement module, configured to perform speech enhancement processing on the main channel signal based on a noise power spectrum of the main channel signal.

[0192] In an embodiment of the present application, time delay alignment is performed on the dual-channel voice signal, and the main signal and the auxiliary signal are determined. When the noise of the main signal is estimated, the energy standard ratio and the dual-channel weighted characteristics of the dual-channel voice signal are added. Therefore, the correlation information between the dual-channel voice signals is combined during the noise estimation. Compared with the method of performing noise estimation on a single-channel voice signal alone, the accuracy of the noise estimation can be improved, and it is more suitable for dual-channel noise estimation.

[0193] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0194] In an embodiment of the present application, an electronic device is also provided. The electronic device may include a processor and a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium; when the computer program is executed by the processor, the processor executes the noise estimation method of any of the above embodiments.

[0195] Reference Figure 6 , shows a structural block diagram of an electronic device according to an embodiment of the present application. Figure 6 As shown, the electronic device 70 includes a processor 701 and a computer-readable storage medium 702 , on which a computer program 7021 is stored.

[0196] The processor 701 is used to execute the computer program 7021 stored on the computer-readable storage medium 702. When the processor 701 executes the computer program 7021, it implements the noise estimation method of any of the above embodiments and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0197] The processor 701 mentioned above may include but is not limited to: a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0198] The computer-readable storage medium 702 mentioned above may include but is not limited to: read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), electronic erasable programmable read-only memory (EEPROM), hard disk, floppy disk, flash memory, etc.

[0199] In an embodiment of the present application, a computer-readable storage medium is further provided, on which a computer program is stored. The computer program can be executed by a processor of an electronic device. When the computer program is executed by the processor, the processor executes the noise estimation method described in any of the above embodiments.

[0200] Reference Figure 7 , shows a block diagram of a computer-readable storage medium according to an embodiment of the present application. Figure 7 As shown, a computer program 801 is stored on a computer-readable storage medium 80. When the computer program 801 is executed by a processor, the processor executes the noise estimation method described in any of the above embodiments and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0201] The various embodiments in this specification are interrelated, and each embodiment is described in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referenced to each other.

[0202] It should be noted that all actions to obtain signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the location and with the authorization given by the owner of the corresponding device.

[0203] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0204] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0205] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0206] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0207] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0208] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0209] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0210] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0211] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A noise estimation method, characterized in that: The method comprises: Performing time delay alignment on the first original speech signal and the second original speech signal to obtain a first target speech signal and a second target speech signal; Determine a main channel signal and an auxiliary channel signal from the first target voice signal and the second target voice signal; The energy standard ratio and dual-channel weighted features of the main signal and the auxiliary signal are calculated, and noise estimation is performed on the main signal based on the energy standard ratio and the dual-channel weighted features to obtain a noise power spectrum of the main signal.

2. The method according to claim 1, characterized in that The method of performing time delay alignment on the first original voice signal and the second original voice signal to obtain the first target voice signal and the second target voice signal includes: Convert the first original speech signal into a first frequency domain speech signal, and convert the second original speech signal into a second frequency domain speech signal; When the first original voice signal and the second original voice signal are initial voice signals, a target delay is calculated based on the first frequency domain voice signal and the second frequency domain voice signal, and the target delay is stored; when the first original voice signal and the second original voice signal are not initial voice signals, the stored target delay is obtained; The first frequency domain speech signal and the second frequency domain speech signal are time-delay aligned using the target time delay to obtain the first target speech signal and the second target speech signal.

3. The method according to claim 2, characterized in that The calculating the target delay based on the first frequency domain voice signal and the second frequency domain voice signal includes: Performing time delay estimation on the first frequency domain speech signal and the second frequency domain speech signal to obtain an initial time delay; A plurality of candidate delays are determined based on the initial delay, and the target delay is selected from the plurality of candidate delays; the plurality of candidate delays include a delay less than the initial delay, the initial delay, and a delay greater than the initial delay.

4. The method according to claim 3, characterized in that The selecting the target delay from the multiple candidate delays includes: For each candidate time delay, obtaining a candidate frequency domain speech signal corresponding to the first frequency domain speech signal at the candidate time delay; For each candidate frequency-domain speech signal, calculating a first correlation between the first frequency-domain speech signal and the candidate frequency-domain speech signal; determining whether there is a significant difference between the first correlations; When present, taking the candidate time delay corresponding to the candidate frequency-domain speech signal with the largest first correlation as the target time delay; When it does not exist, for each candidate frequency domain voice signal among the multiple candidate frequency domain voice signals with the largest first correlation, the candidate frequency domain voice signal is converted into a candidate time domain voice signal, the Pearson coefficient between the first original voice signal and the candidate time domain voice signal is calculated, and the candidate time delay corresponding to the candidate time domain voice signal with the largest Pearson coefficient is taken as the target time delay.

5. The method according to claim 1, characterized in that The calculating the energy standard ratio and the dual-channel weighted feature of the main signal and the auxiliary signal, and performing noise estimation on the main signal based on the energy standard ratio and the dual-channel weighted feature to obtain the noise power spectrum of the main signal includes: Calculate the energy standard ratio and dual-channel weighted features of the main signal and the auxiliary signal, calculate the search window length of the main signal based on the energy standard ratio, perform noise estimation on the main signal based on the search window length, the energy standard ratio and the dual-channel weighted features, and obtain the noise power spectrum of the main signal.

6. The method according to claim 1, characterized in that The calculating the energy standard ratio and the dual-channel weighted feature of the main signal and the auxiliary signal, and performing noise estimation on the main signal based on the energy standard ratio and the dual-channel weighted feature to obtain the noise power spectrum of the main signal includes: Calculating the energy standard ratio of the main channel signal to the auxiliary channel signal; Calculating a search window length of the main channel signal based on the energy standard ratio; Performing a first smoothing operation on the main channel signal, and performing a first iterative operation on the main channel signal based on the energy standard ratio and the search window length to obtain a first speech detection result of the main channel signal; Based on the first speech detection result, the main channel signal is smoothed again, and dual-channel weighted features of the main channel signal and the auxiliary channel signal are calculated; Performing another iterative operation on the main channel signal based on the dual-channel weighted feature and the search window length to obtain a second speech detection result of the main channel signal; The noise power spectrum of the main channel signal is calculated based on the second speech detection result.

7. The method according to any one of claims 1 to 6, characterized in that: The energy standard ratio of the main signal to the auxiliary signal is calculated by the following formula: Where n represents the frame identifier, k represents the frequency point identifier, and P b (n) represents the energy standard ratio of the main signal of the nth frame to the auxiliary signal of the nth frame, P1(n, k) represents the actual power spectrum of the main signal of the nth frame, and P2(n, k) represents the actual power spectrum of the auxiliary signal of the nth frame. represents the average value of the actual power spectrum of the main signal of the nth frame and the actual power spectrum of the auxiliary signal of the nth frame, P1(n-1, k) represents the actual power spectrum of the main signal of the n-1th frame, and P2(n-1, k) represents the actual power spectrum of the auxiliary signal of the n-1th frame. It represents the average value of the actual power spectrum of the main signal in the n-1th frame and the actual power spectrum of the auxiliary signal in the n-1th frame.

8. The method according to claim 5 or 6, characterized in that: The calculating the search window length of the main path signal based on the energy standard ratio includes: When the energy standard ratio is greater than or equal to a preset maximum energy standard ratio, using a preset minimum window length as the search window length; When the energy standard ratio is less than or equal to a preset minimum energy standard ratio, using a preset maximum window length as the search window length; When the energy standard ratio is greater than a preset minimum energy standard ratio and less than a preset maximum energy standard ratio, the search window length of the main channel signal is calculated by the following formula: Where n represents the frame identifier, k represents the frequency point identifier, U(n, k) represents the search window length of the main channel signal of the nth frame, U(n-1, k) represents the search window length of the main channel signal of the n-1th frame, P b (n) represents the energy standard ratio of the first target speech signal of the nth frame to the second target speech signal of the nth frame, U max represents the maximum window length, U min represents the minimum window length, Indicates rounding up.

9. The method according to claim 6, characterized in that The dual-channel weighted characteristics of the main channel signal and the auxiliary channel signal are calculated by the following formula: or, or, Wherein, n represents the frame identifier, k represents the frequency identifier, D(n, k) represents the dual-channel weighted characteristics of the main signal of the nth frame and the auxiliary signal of the nth frame, α1 represents the dynamic weighting factor, COH(n-1, k) represents the first correlation between the main signal of the n-1th frame and the auxiliary signal of the n-1th frame, COH(n, k) represents the first correlation between the main signal of the nth frame and the auxiliary signal of the nth frame, P1(n, k) represents the actual power spectrum of the main signal of the nth frame, P2(n, k) represents the actual power spectrum of the auxiliary signal of the nth frame, P1(n-1, k) represents the actual power spectrum of the main signal of the n-1th frame, P 2(n-1, k) represents the actual power spectrum of the auxiliary signal of the n-1th frame, COH′(n-1, k) represents the second correlation between the main signal of the n-1th frame and the auxiliary signal of the n-1th frame, COH′(n, k) represents the second correlation between the main signal of the n-1th frame and the auxiliary signal of the n-1th frame, S1(n, k) represents the first smoothed power spectrum of the main signal of the n-1th frame, S2(n, k) represents the first smoothed power spectrum of the auxiliary signal of the n-1th frame, S1(n-1, k) represents the first smoothed power spectrum of the main signal of the n-1th frame, S2(n-1, k) represents the first smoothed power spectrum of the auxiliary signal of the n-1th frame, Indicates the third correlation between the main channel signal of the n-1th frame and the auxiliary channel signal of the n-1th frame, Indicates the third correlation between the main channel signal of the nth frame and the auxiliary channel signal of the nth frame, Represents the second smoothed power spectrum of the main signal of the nth frame, represents the second smoothed power spectrum of the auxiliary signal of the nth frame, represents the second smoothed power spectrum of the main signal of the n-1th frame, Represents the second smoothed power spectrum of the auxiliary channel signal of the n-1th frame.

10. The method according to claim 6, characterized in that The first iterative operation is performed on the main channel signal based on the energy standard ratio and the search window length to obtain a first speech detection result of the main channel signal, including: Performing a first iterative operation on the main channel signal based on the search window length, and calculating a first judgment index and a second judgment index; When the first judgment indicator is less than the first indicator threshold and the second judgment indicator is less than the second indicator threshold, or when the energy standard ratio is greater than the preset maximum energy standard ratio, the first voice detection result of the main channel signal is determined to be the absence of voice; otherwise, the first voice detection result of the main channel signal is determined to be the presence of voice.

11. The method according to claim 6, characterized in that The iterative operation on the main channel signal based on the dual-channel weighted feature and the search window length to obtain a second speech detection result of the main channel signal includes: Performing an iterative operation on the main channel signal again based on the search window length to calculate a third judgment index and a fourth judgment index; When the third judgment index is less than or equal to 1 and the fourth judgment index is less than the fourth index threshold, or when the main signal of the previous frame of the main signal is a noise signal and the dual-channel weighted feature is greater than 1, it is determined that the second voice detection result of the main signal is a noise signal; When the third judgment index is greater than 1 and less than the third index threshold, and the fourth judgment index is less than the fourth index threshold, determining that the second voice detection result of the main channel signal is a noisy voice signal; When the third judgment index is greater than or equal to the third index threshold and the fourth judgment index is greater than or equal to the fourth index threshold, or when the main signal of the previous frame of the main signal is a pure speech signal and the dual-channel weighted feature is less than or equal to 1, it is determined that the second speech detection result of the main signal is a pure speech signal.

12. The method according to claim 1, characterized in that After performing noise estimation on the main channel signal based on the energy standard ratio and the dual-channel weighted feature to obtain the noise power spectrum of the main channel signal, the method further includes: Based on the noise power spectrum of the main channel signal, speech enhancement processing is performed on the main channel signal.

13. A noise estimation device, characterized in that: The device comprises: An alignment module, used for performing time delay alignment on the first original voice signal and the second original voice signal to obtain a first target voice signal and a second target voice signal; A determination module, used to determine a main channel signal and an auxiliary channel signal from the first target voice signal and the second target voice signal; The estimation module is used to calculate the energy standard ratio and dual-channel weighted characteristics of the main signal and the auxiliary signal, and perform noise estimation on the main signal based on the energy standard ratio and the dual-channel weighted characteristics to obtain the noise power spectrum of the main signal.

14. An electronic device, characterized in that: The electronic device comprises a processor and a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program; When the computer program is executed by the processor, the processor is caused to perform the noise estimation method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the noise estimation method according to any one of claims 1 to 12.

Images