Delay estimation method, device and storage medium
By calculating the voice activity status and credibility in the delay estimation, combined with Hanning window and bandpass weighting, the problem of inaccurate delay estimation under the influence of noise is solved, and the accuracy and speed of delay estimation are improved.
Patent Information
- Application Number
- CN202111682261.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-29
- Publication Date
- 2024-12-20
- Estimated Expiration
- 2041-12-29
AI Technical Summary
In actual use environments, due to the presence of noise, the delay estimation results are easily affected by pseudo-peaks, resulting in inaccurate delay estimation.
By acquiring the current frame signal and the reference signal, the target reference signal within the signal search range is determined, the confidence is calculated based on the voice activity state, and the correlation value is calculated when the confidence threshold is above the confidence threshold to determine the delay estimate value. Options include initializing the reference signal with the last delay estimate, Hanning window weighting, bandpass weighting, and chaotic mapping to generate random verification delays for verification.
It improves the accuracy of delay estimation, saves computing resources, weakens the impact of noise and frequency band interference, avoids local peak misunderstandings, and ensures the speed and accuracy of delay estimation.
Smart Images
Figure CN114171061B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of computer technology, and specifically relates to a delay estimation method, device and storage medium. Background Art
[0002] Delay estimation refers to estimating the time difference between the same-source signals received at different times. Delay estimation is widely used in scenarios such as echo cancellation and sound source localization.
[0003] A traditional delay estimation method includes: obtaining a current frame speech signal and a speech signal to be aligned; determining a signal search range of the speech signal to be aligned, wherein the signal search range includes at least one frame of speech signal to be aligned; calculating a correlation value between the current frame speech signal and at least one frame of speech signal to be aligned; and determining the time difference between a frame of speech signal to be aligned and the current frame speech signal corresponding to a peak value of the correlation value as a delay estimation value.
[0004] However, in actual usage environments, due to the presence of noise (not limited to smooth background noise, but also including reverberation and the voices of other people talking during the meeting), the peak of the correlation value is not obvious, and the delay estimation result will be affected by the pseudo peak, resulting in inaccurate delay estimation results. Summary of the invention
[0005] The present application provides a delay estimation method, device and storage medium, which can solve the problem that in actual use environment, due to the presence of noise, the peak value of the correlation value is not obvious, and the delay estimation result is affected by the pseudo peak, resulting in inaccurate delay estimation results. The present application provides the following technical solutions:
[0006] In a first aspect, a delay estimation method is provided, comprising: obtaining a current frame signal and a reference signal corresponding to the current frame signal; determining a signal search range of the reference signal to obtain n frames of target reference signals within the signal search range, wherein n is a positive integer; determining a first voice activity state of the current frame signal and a second voice activity state of the target reference signal; determining the credibility of the current frame signal based on the first voice activity state and the second voice activity state; when the credibility value is greater than a credibility threshold, respectively determining a correlation value between the current frame signal and the at least one frame of target reference signal; and using the correlation value to determine a delay estimation value corresponding to the current frame signal.
[0007] Optionally, determining the credibility of the current frame signal based on the first voice activity state and the second voice activity state includes: when the first voice activity state indicates that there is a voice signal in the current frame signal, determining a pre-alignment reference signal from the n-frame target reference signals, and a nearby reference signal whose time difference with the pre-alignment reference signal is less than or equal to a preset threshold based on the delay estimation value determined last time; wherein the delay estimation value determined last time is initialized to 0; and determining the credibility of the current frame signal based on the second voice activity state of the pre-alignment reference signal and the second voice activity state of the nearby reference signal.
[0008] Optionally, determining the credibility of the current frame signal based on the second voice activity state of the pre-aligned reference signal and the second voice activity state of the nearby reference signal includes: weighting the second voice activity state of the pre-aligned reference signal using a Hanning window of a preset length to obtain a first weighted value; weighting the second voice activity state of the nearby reference signal using the Hanning window of the preset length to obtain a second weighted value; determining the sum of the first weighted value and the second weighted value to obtain the credibility; wherein the preset length is determined based on the preset threshold.
[0009] Optionally, after using the correlation value to determine the delay estimation value corresponding to the current frame signal, it also includes: determining verification reference signals corresponding to m random verification delays from the search range, where m is an integer greater than 2; calculating the correlation value between the current frame signal and the verification reference signal; determining the target random verification delays corresponding to the two verification reference signals with the largest correlation values with the current frame signal to obtain a delay verification range; the upper limit value of the delay verification range is the larger value of the target random verification delay, and the lower limit value of the delay verification range is the smaller value of the target random verification delay; and using the delay verification range to verify the delay estimation value.
[0010] Optionally, the use of the delay verification range to verify the delay estimation value includes: when the delay estimation value is within the delay verification range, verifying that the delay estimation value is correct; when the delay estimation value is not within the delay verification range, determining whether a correlation value corresponding to the delay estimation value is greater than a correlation value corresponding to the target random verification delay; when the correlation value corresponding to the delay estimation value is greater than the correlation value corresponding to the target random verification delay, verifying that the delay estimation value is correct; when the correlation value corresponding to the delay estimation value is less than the correlation value corresponding to the target random verification delay, verifying that the delay estimation value is incorrect, and determining the maximum correlation value corresponding to the target random verification delay as the final delay estimation value.
[0011] Optionally, determining the verification reference signals corresponding to m random verification delays within the search range includes: generating the m random verification delays using a chaotic mapping method; extracting a verification reference signal whose delay with the current frame signal is the m random verification delays from the search range.
[0012] Optionally, respectively determining the correlation value between the current frame signal and the at least one frame target reference signal includes: performing bandpass weighting on the current frame signal and the target reference signal according to the distribution frequency of the speech signal; and calculating the correlation value between the bandpass weighted current frame signal and the bandpass weighted target reference signal.
[0013] Optionally, the signal search range is determined based on a preset maximum delay estimation value.
[0014] In a second aspect, an electronic device is provided, comprising a memory, a controller, and a computer program stored in the memory and executable on the controller, wherein the controller implements the steps of the above-mentioned delay estimation method when executing the computer program.
[0015] According to a third aspect, a computer-readable storage medium is provided, wherein a program is stored in the storage medium, and when the program is executed by a processor, the program is used to implement the delay estimation method provided in the first aspect.
[0016] The beneficial effects of the present application include at least: obtaining the current frame signal and the reference signal corresponding to the current frame signal; determining the signal search range of the reference signal, and obtaining the n-frame target reference signal within the signal search range; determining the first voice activity state of the current frame signal and the second voice activity state of the target reference signal; determining the credibility of the current frame signal based on the first voice activity state and the second voice activity state; determining the correlation value between the current frame signal and at least one frame of the target reference signal respectively when the credibility value is greater than the credibility threshold; using the correlation value to determine the delay estimation value corresponding to the current frame signal. The problem that the peak value of the correlation value is not obvious due to the presence of noise and the delay estimation result is affected by the pseudo peak, resulting in inaccurate delay estimation results, can be solved; by determining the credibility of the current frame signal based on the first voice activity state of the current frame signal and the second voice state of the target reference signal, and determining the delay estimation value by the correlation value when the credibility is greater than the credibility threshold, the accuracy of the delay estimation can be improved. At the same time, when the first voice activity state indicates that there is a voice signal in the current frame signal, the credibility calculation is performed again, which can save computing resources.
[0017] In addition, by determining which reference signals to use to calculate the credibility using the time delay estimation value determined last time, there is no need to use the second voice activity states of all reference signals within the signal search range to calculate the credibility, which can improve the calculation speed of the credibility.
[0018] In addition, by using the Hanning window to perform weighted calculation of the credibility of the second voice activity state, jitter can be removed, the impact of missed detection and false detection in voice detection can be reduced, and the accuracy of credibility calculation can be improved.
[0019] In addition, by performing bandpass weighting on the current frame signal and the target reference signal according to the distribution frequency of the speech signal when calculating the correlation value; and calculating the correlation value between the bandpass weighted current frame signal and the bandpass weighted target reference signal; the influence of noise and interfering sounds in other frequency bands can be further avoided, thereby improving the accuracy of the correlation value calculation.
[0020] In addition, by determining the verification reference signals corresponding to m random verification delays within the search range, the correlation value between the current frame signal and the verification reference signal is calculated; the target random verification delays corresponding to the two verification reference signals with the largest correlation values with the current frame signal are determined to obtain the delay verification range; the delay estimation value is verified using the delay verification range; it can be determined whether delay jitter occurs. Once delay jitter occurs, the delay range can be quickly determined and fine delay estimation can be performed in time, which can not only improve the accuracy of delay estimation, but also ensure the speed of delay estimation.
[0021] In addition, by generating m random verification delays through chaotic mapping, the global scope can be monitored, which not only avoids falling into the local peak optimal solution, but also reduces the consumption of global search, improves the accuracy of delay estimation, and ensures the speed of delay estimation, reducing computing consumption.
[0022] In addition, the signal search range is determined based on the preset maximum delay estimation value, and there is no need to perform speech detection on all reference signals, thereby ensuring the speed of delay estimation. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0024] Figure 1 is a flowchart of a delay estimation method provided by an embodiment of the present application;
[0025] Figure 2 is a schematic diagram of a delay estimation process provided by an embodiment of the present application;
[0026] Figure 3 is a block diagram of a delay estimation device provided by an embodiment of the present application;
[0027] Figure 4 It is a block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0028] The technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.
[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0030] In the present application, unless otherwise specified, the directional words used, such as "up, down, top, bottom", usually refer to the directions shown in the drawings, or to the components themselves in the vertical, perpendicular or gravity directions; similarly, for ease of understanding and description, "inside and outside" refer to the inside and outside relative to the outline of each component itself, but the above-mentioned directional words are not used to limit the present application.
[0031] The delay estimation method provided in this application is described in detail below.
[0032] Optionally, the execution subject of each embodiment in the present application may be an electronic device, which may be a terminal such as a computer, a mobile phone, a tablet computer, a personal computer, a video conferencing terminal, or a server. This embodiment does not limit the implementation method of the electronic device.
[0033] Optionally, the application scenarios of the delay estimation method provided in this application include but are not limited to:
[0034] The first one is echo cancellation scenario. In this case, the delay estimation result is used to eliminate the far-end echo.
[0035] The second type: sound source localization scenario. In this case, the delay estimation result is used to calculate the sound source position.
[0036] In actual implementation, the application scenario of the delay estimation method may also be other scenarios, and this embodiment does not list the application scenarios of the delay estimation method one by one.
[0037] like Figure 1 As shown, an embodiment of the present application provides a delay estimation method, which includes at least the following steps:
[0038] Step 101: Acquire a current frame signal and a reference signal corresponding to the current frame signal.
[0039] Schematically, the audio acquisition end acquires local audio and reference audio corresponding to the local audio, wherein the current frame signal is obtained by framing, windowing and time-frequency conversion of the local audio; the reference signal is obtained by framing, windowing and time-frequency conversion of the reference audio. The audio acquisition end may be an electronic device, or other device connected to the electronic device for communication. This embodiment does not limit the implementation of the audio acquisition end.
[0040] The reference audio and the local audio are audio data emitted by the same sound source and collected at different collection times. That is, this embodiment is used to estimate the time delay between the reference audio and the local audio.
[0041] Framing refers to dividing audio data into at least two short segments of equal length, each of which is called a frame.
[0042] Since audio signals usually need to be converted to the frequency domain for processing, and the time-frequency conversion requires the input signal to be stable. The speech signal is not stable in the macro sense, but stable in the micro sense, and has short-term stability. Therefore, when the application scenario of delay estimation includes speech output, considering the short-term stability of speech, the local audio and the reference audio are framed separately, including: framing with a preset frame shift. That is, there is an overlap between two adjacent frame signals.
[0043] Optionally, the starting position difference between two adjacent frames is called a frame shift, and the preset frame shift may be 50%, or a value less than 50%. This embodiment does not limit the value of the preset frame shift.
[0044] Windowing refers to the process of multiplying the window function with the original time domain signal.
[0045] Since the time-frequency conversion requires the input signal to be periodic, if the selected current frame signal is not a periodic signal, it is also necessary to perform windowing on the local voice signal and the remote voice signal after framing. The local audio and the reference audio are windowed separately, including: using the Hanning window function w0 for windowing.
[0046] The Hanning window function is expressed by the following formula:
[0047] w0(n)=1 / 2[1+cos(2*PI*n / (N-1))], n∈[-(N-1) / 2,(N-1) / 2]
[0048] Wherein, N is the length of the Hanning window, for example, the value of N is 1024. In actual implementation, the value of N may also be other values, and this embodiment does not limit the implementation method of the value of N.
[0049] The current frame signal after windowing is expressed by the following formula:
[0050] x(n)=x(n)w0(n)
[0051] The windowed reference signal is expressed as follows:
[0052] d(n)=d(n)w0(n)
[0053] After windowing, the middle part of the speech signal is highlighted and the two ends are weakened. In this embodiment, by repeatedly taking a section of the signal during framing, the influence of the window function can be weakened to ensure the integrity of the information.
[0054] Time-frequency conversion processing refers to transforming complex time signals into a structural form represented by frequency components.
[0055] The local audio and the reference audio are respectively subjected to time-frequency conversion processing, including: performing fast Fourier transform on the windowed signal to obtain a frequency domain signal.
[0056] Among them, the fast Fourier transform of the reference audio is expressed by the following formula:
[0057]
[0058] Among them, N is the length of the Hanning window, l is the current frame, k is the frequency of the lth frame signal, and x(·) represents the time domain signal corresponding to the reference signal.
[0059] The fast Fourier transform of local audio is expressed as follows:
[0060]
[0061] Wherein, N is the length of the Hanning window, l is the current frame, k is the frequency of the lth frame signal, and d(·) represents the time domain signal corresponding to the current frame signal.
[0062] The current frame signal is a frame of frequency domain signal in the local audio for which time delay estimation is currently to be performed, and the reference signal is each frame of frequency domain signal in the reference audio.
[0063] Step 102, determine a signal search range of a reference signal, and obtain n frames of target reference signals within the signal search range, where n is a positive integer.
[0064] Optionally, the signal search range is determined based on a preset maximum delay estimation value. Specifically, the maximum delay between the target reference signal and the reference signal in the signal search range is greater than or equal to the maximum delay estimation value.
[0065] In other embodiments, the signal search range may also be all reference signals. This embodiment does not limit the setting method of the signal search range.
[0066] Step 103: Determine a first voice activity state of the current frame signal and a second voice activity state of the target reference signal.
[0067] Optionally, the electronic device detects a first voice activity state of the current frame signal and a second voice activity state of the target reference signal by using a voice activity detection algorithm (Voice Activity Detection, VAD).
[0068] In one example, when there is speech in the current frame signal or the target reference signal, the first voice activity state and the second voice activity state can be represented by 1; when there is no speech in the current frame or the target reference signal, the first voice activity state and the second voice activity state can be represented by 0.
[0069] After obtaining the second voice activity state of the target reference signal, the electronic device stores the second voice activity state in the first cache queue QueueVad. Taking the upper limit value of the signal search range as the maximum delay estimation value as an example, the length of the first cache queue QueueVad is the maximum delay estimation value MAXDELAY, and the initialization value of each cache position in the first cache queue QueueVad is 0.
[0070] The first cache queue QueueVad is represented by the following formula:
[0071] QueueVad=[vad_x QueueRef(1:(MAXDELAY-1)]
[0072] Wherein, vad_x is used to represent the second voice activity state of the target reference signal.
[0073] In addition, the electronic device stores n target reference signals in the second cache queue QueueRef. Taking the upper limit value of the signal search range as the maximum delay estimation value MAXDELAY as an example, at this time, the value of n is MAXDELAY, and accordingly, the length of the second cache queue QueueRef is MAXDELAY*N / 2, and the initialization value of each cache position in the second cache queue QueueRef is 0.
[0074] The second cache queue QueueRef is represented by the following formula:
[0075] QueueRef=[X(l,k)QueueVad(1:(MAXDELAY-1)*N / 2]
[0076] Wherein, X(l,k) is used to represent the target reference signal, l represents the current frame, k represents the data of the target reference signal corresponding to the current frame, MAXDELAY represents the maximum delay estimation value, and N is the length of the Hanning window.
[0077] Step 104: Determine the credibility of the current frame signal based on the first voice activity state and the second voice activity state.
[0078] The credibility is used to indicate the credibility of the correlation value calculated using the current frame signal. When the credibility is low, it means that even if the correlation value is calculated using the current frame signal, the correlation value may not be accurate enough; in this case, the correlation may not be used for delay estimation.
[0079] Optionally, determining the credibility of the current frame signal based on the first voice activity state and the second voice activity state includes the following situations:
[0080] First, when the first voice activity state is 0, the credibility of the current frame signal is 0.
[0081] When the first voice activity state is 0, it means that there is no voice signal in the current frame signal. At this time, it is meaningless to estimate the delay of the current frame signal, and the credibility of the current frame signal is determined to be 0.
[0082] The second one is that when the first voice activity state is not 0, the voice state near the pre-aligned reference signal in the target reference signal is used to represent the credibility of the current frame signal.
[0083] Specifically, the credibility of the current frame signal is determined based on the first voice activity state and the second voice activity state, including: when the first voice activity state indicates that there is a voice signal in the current frame signal, based on the delay estimate value determined last time, determining a pre-alignment reference signal from n-frame target reference signals, and a nearby reference signal whose time difference with the pre-alignment reference signal is less than or equal to a preset threshold; based on the second voice activity state of the pre-alignment reference signal and the second voice activity state of the nearby reference signal, determining the credibility of the current frame signal.
[0084] The delay estimation value determined last time is initialized to 0.
[0085] The pre-alignment reference signal is a reference signal of one frame whose delay value is the last delay estimation value in the target reference signal of n frames.
[0086] The preset threshold is determined based on the selected length stored in the electronic device. Schematically, the selected length is deltaT, and the preset threshold is (deltaT-1) / 2. The selected length can be 5, 6 or adjusted according to actual needs. This embodiment does not limit the implementation method of the selected length.
[0087] In one example, the electronic device performs a weighted summation of the second voice activity state of the pre-aligned reference signal and the second voice activity state of the nearby reference signal to obtain the credibility.
[0088] However, due to the jitter of the speech signal, in another example, the credibility of the current frame signal is determined based on the second speech activity state of the pre-aligned reference signal and the second speech activity state of the nearby reference signal, including: using a Hanning window of a preset length to weight the second speech activity state of the pre-aligned reference signal to obtain a first weighted value; using a Hanning window of a preset length to weight the second speech activity state of the nearby reference signal to obtain a second weighted value; determining the sum of the first weighted value and the second weighted value to obtain credibility; wherein the preset length is determined based on a preset threshold.
[0089] Since the Hanning window can remove jitter, it can reduce the impact of missed detection and false detection in speech detection and improve the accuracy of credibility calculation. The length of the Hanning window w1 is the selected length.
[0090] Specifically, the credibility calculation process is expressed by the following formula:
[0091]
[0092]
[0093] Among them, deltaT is the selected length, R is the credibility of the current frame signal, and lastdelay is the last delay estimation value.
[0094] Step 105 : when the credibility value is greater than the credibility threshold, respectively determine the correlation value between the current frame signal and at least one frame target reference signal.
[0095] The value of the credibility threshold is 1 or other values, and this embodiment does not limit the value of the credibility threshold.
[0096] When the value of the credibility is greater than the credibility threshold, it means that both the current frame signal and the target reference frame signal are voice signals. At this time, the accuracy of the delay estimation value determined based on the correlation value is relatively high. Therefore, in this embodiment, only when the value of the credibility is greater than the credibility threshold, the correlation value between the current frame signal and at least one frame of the target reference signal is calculated to perform delay estimation, which can avoid calculating meaningless correlation values, thereby causing the problem of inaccurate delay estimation; it can save computing resources and improve the accuracy of delay estimation.
[0097] In one example, the electronic device may use a correlation function to directly calculate the current frame signal and the target reference signal.
[0098] However, since the range of speech is mainly distributed in 300-3400 Hz, in order to further avoid the influence of noise and interference sounds in other frequency bands, in another example, the electronic device determines the correlation value between the current frame signal and at least one frame of target reference signal, including: performing bandpass weighting on the current frame signal and the target reference signal according to the distribution frequency of the speech signal; calculating the correlation value between the bandpass weighted current frame signal and the bandpass weighted target reference signal.
[0099] In this embodiment, by performing bandpass weighting on the current frame signal and the target reference signal according to the distribution frequency of the speech signal, the influence of interference sounds in other frequency bands can be further avoided, thereby improving the accuracy of delay estimation.
[0100] Among them, the current frame signal and the target reference signal are band-pass weighted according to the distribution frequency of the speech signal, and the correlation value between the band-pass weighted current frame signal and the band-pass weighted target reference signal is calculated, which is expressed by the following formula:
[0101]
[0102]
[0103] Among them, X(l,k) is used to represent the target reference signal, l represents the current frame, k represents the data of the target reference signal corresponding to the current frame, D(l,k) is used to represent the current frame signal, l represents the current frame, k represents the data corresponding to the current frame signal, lastdelay is the last delay estimation value, N is the length of the Hanning window, M is the sampling rate of the current frame signal, 300 is the minimum frequency in the distribution frequency of the speech signal, and 3400 is the maximum frequency in the distribution frequency of the speech signal.
[0104] Step 106: Use the correlation value to determine the delay estimation value corresponding to the current frame signal.
[0105] Among them, using the correlation value to determine the delay estimation value corresponding to the current frame signal includes: determining the reference signal with the largest correlation value with the current frame signal among the pre-aligned reference signal and the nearby reference signals; and determining the time difference between the reference signal with the largest correlation value and the current frame signal as the delay estimation value.
[0106] For example, the electronic device takes reference signals in the second cache queue QueueRef within the range of deltaT with lastdelay as the center to calculate the correlation value Within this range, the position where the maximum correlation value is located is searched as the delay estimation value Delay, and the coherent value Corr=Corr(l,Delay) is updated.
[0107] In summary, the delay estimation method provided by the present embodiment obtains the current frame signal and the reference signal corresponding to the current frame signal; determines the signal search range of the reference signal to obtain the n-frame target reference signal within the signal search range; determines the first voice activity state of the current frame signal and the second voice activity state of the target reference signal; determines the credibility of the current frame signal based on the first voice activity state and the second voice activity state; when the credibility value is greater than the credibility threshold, determines the correlation value between the current frame signal and at least one frame of the target reference signal; uses the correlation value to determine the delay estimation value corresponding to the current frame signal; and can solve the delay estimation problem obtained by the existing delay estimation method. The problem of inaccurate estimation results; since the credibility can indicate the credibility of using the current frame signal to calculate the correlation value, the credibility is calculated based on the voice activity state. When the credibility is low, it means that the current frame signal and / or the reference signal are not related to the voice signal. Even if the correlation value is calculated using the current frame signal, the correlation value may not be accurate enough. At this time, the correlation can be used without using the delay estimation. Only when the credibility is high, the correlation value between the current frame signal and at least one frame of target reference signal is calculated to perform the delay estimation, which can avoid calculating meaningless correlation values, thereby causing the problem of inaccurate delay estimation. It can save computing resources and improve the accuracy of delay estimation.
[0108] In addition, by determining which reference signals to use to calculate the credibility using the time delay estimation value determined last time, there is no need to use the second voice activity states of all reference signals within the signal search range to calculate the credibility, which can improve the calculation speed of the credibility.
[0109] In addition, by using the Hanning window to perform weighted calculation of the credibility of the second voice activity state, jitter can be removed, the impact of missed detection and false detection in voice detection can be reduced, and the accuracy of credibility calculation can be improved.
[0110] In addition, by performing bandpass weighting on the current frame signal and the target reference signal according to the distribution frequency of the speech signal when calculating the correlation value; and calculating the correlation value between the bandpass weighted current frame signal and the bandpass weighted target reference signal; the influence of noise and interfering sounds in other frequency bands can be further avoided, thereby improving the accuracy of the correlation value calculation.
[0111] In order to reduce the impact of pseudo-peaks, it is usually necessary to increase the search range limit for the current delay estimate, but due to the instability of the audio acquisition system, there is often a problem of large delay jitter. At this time, the increased search range limit makes it easier for the delay estimation result to fall into a local optimal solution, and it is impossible to track the change of the delay in time, resulting in the inability to accurately and quickly align the echo signal with the reference. In this way, the subsequent echo cancellation or sound source localization effect will be poor. Based on this, in this embodiment, after calculating the delay estimate, it is also necessary to verify the delay estimate. Specifically, after step 106, it also includes: determining the verification reference signals corresponding to m random verification delays from the search range; calculating the correlation value between the current frame signal and the verification reference signal; determining the target random verification delay corresponding to the two verification reference signals with the largest correlation value with the current frame signal, and obtaining the delay verification range; using the delay verification range to verify the delay estimate.
[0112] Among them, m is an integer greater than 2; the upper limit value of the delay verification range is the larger value of the target random verification delay, and the lower limit value of the delay verification range is the smaller value of the target random verification delay.
[0113] Among them, the delay estimation value is verified using the delay verification range, including: when the delay estimation value is within the delay verification range, verifying that the delay estimation value is correct; when the delay estimation value is not within the delay verification range, determining whether the correlation value corresponding to the delay estimation value is greater than the correlation value corresponding to the target random verification delay; when the correlation value corresponding to the delay estimation value is greater than the correlation value corresponding to the target random verification delay, verifying that the delay estimation value is correct; when the correlation value corresponding to the delay estimation value is less than the correlation value corresponding to the target random verification delay, verifying that the delay estimation value is wrong, and determining the maximum correlation value corresponding to the target random verification delay as the final delay estimation value.
[0114] Schematically, verification reference signals corresponding to m random verification delays are determined within a search range, including: generating m random verification delays using a chaotic mapping method; extracting a verification reference signal whose delay with the current frame signal is m random verification delays from the search range.
[0115] Optionally, using a chaotic mapping method to generate m random verification delays includes: using a worm population mapping method to generate m random verification delays. Specifically, the worm population mapping method is represented by the following formula:
[0116] Z l,m+1 =μZ l,m (1-Z l,m )
[0117]
[0118] in, μ∈[0,4],Z l,0 =Z l-1,m .
[0119] The method of calculating the correlation value between the current frame signal and the verification reference signal is referred to step 105, and will not be described in detail in this embodiment.
[0120] Assume that the target random verification delays corresponding to the verification reference signals CorrMax1 and CorrMax2 with the largest correlation values are T1 and T2. The electronic device determines whether the delay estimate Delay is within the range of T1 and T2; if so, the delay estimate Delay is directly output. Otherwise, continue to determine whether the correlation value Corr corresponding to the delay estimate is greater than CorrMax1. If so, it means that the correlation value calculated by the experimental estimation value is larger. At this time, the calculated delay estimate is more accurate, and the delay estimate Delay is directly output. Otherwise, determine that the range of T1 and T2 is the range of delay jitter, and calculate the correlation values corresponding to each delay value within the range, and perform peak search to determine the delay value T12 corresponding to the maximum peak value. The delay estimation result can be expressed as DelayOut=T12, and directly output the final delay estimate DelayOut=T12.
[0121] In other embodiments, the electronic device may also use other chaotic mapping algorithms to generate m random verification delays. This embodiment does not limit the generation method of the m random verification delays.
[0122] In this embodiment, by determining the verification reference signals corresponding to m random verification delays within the search range, the correlation value between the current frame signal and the verification reference signal is calculated; the target random verification delays corresponding to the two verification reference signals with the largest correlation values with the current frame signal are determined to obtain the delay verification range; the delay estimation value is verified using the delay verification range; it can be determined whether delay jitter occurs. Once delay jitter occurs, the delay range can be quickly determined and fine delay estimation can be performed in time, which can not only improve the accuracy of delay estimation, but also ensure the speed of delay estimation.
[0123] In addition, by generating m random verification delays through chaotic mapping, the global scope can be monitored, which not only avoids falling into the local peak optimal solution, but also reduces the consumption of global search, improves the accuracy of delay estimation, and ensures the speed of delay estimation, reducing computing consumption.
[0124] In order to more clearly understand the delay estimation method provided by the present application, an example is given below to illustrate the method. Figure 2 ,The delay estimation method includes three stages, namely the preprocessing stage, the delay estimation stage and the delay estimation ,verification stage.
[0125] In the preprocessor stage, the electronic device finds the pre-alignment reference signal and the nearby reference signal in the remote cache according to the delay estimate value determined last time, obtains the reference signal x(n) of [-deltaT-1) / 2, +deltaT-1) / 2 near the pre-alignment reference signal, and uses the voice activity detection result of x(n) and the current frame signal d(n) to determine the credibility R of the delay estimate value of the current frame.
[0126] In the delay estimation stage, the electronic device decides whether to perform delay estimation based on the credibility R output in the preprocessing stage. Specifically, when R is less than 1, the delay estimation value determined last time is kept unchanged and output to the next stage. Otherwise, within the reference signal of [-deltaT-1) / 2, +deltaT-1) / 2], the correlation function is used to calculate the correlation value between the current frame signal and each reference signal in [-deltaT-1) / 2, +deltaT-1) / 2], and the peak value of the correlation value is searched to determine the delay estimation value Delay.
[0127] In the delay estimation verification phase, the electronic device combines the chaos concept to obtain random global scattered points within the preset MAXDELAY range, and calculates the correlation function of the signal corresponding to the scattered delay, so as to determine the scattered area [T1, T2] with the largest correlation value. It is determined whether the delay value output by the delay estimator is within the current area. If so, the current delay estimate is output; if not, it is considered that a delay jitter problem may occur at the current moment. It is determined whether the corresponding correlation value Corr output by the delay estimator is greater than the maximum correlation value corresponding to the scattered delay point. If so, the current delay estimate is output to exclude delay jitter; if not, it can be determined that a large delay jitter phenomenon occurs, and the coherence value within the scattered area [T1, T2] should be calculated in time, and the position corresponding to the maximum correlation value should be output as the final delay estimation result.
[0128] Among them, the output of the preprocessing stage determines the reliability of the estimation result of the delay estimator; at the same time, the delay estimation verification stage generates random delay scattering points based on the chaos idea to monitor the delay estimation value and global delay of the previous stage, ensuring the timeliness and accuracy of the delay result output.
[0129] This embodiment provides a delay estimation device, such as Figure 3 The device comprises at least the following modules: a signal acquisition module 310 , a range determination module 320 , a state determination module 330 , a credibility determination module 340 , a correlation value determination module 350 and a delay determination module 360 .
[0130] The signal acquisition module 310 is used to acquire the current frame signal and the reference signal corresponding to the current frame signal;
[0131] A range determination module 320 is used to determine a signal search range of a reference signal and obtain n frames of target reference signals within the signal search range;
[0132] A state determination module 330, configured to determine a first voice activity state of a current frame signal and a second voice activity state of a target reference signal;
[0133] A credibility determination module 340, configured to determine the credibility of a current frame signal based on the first voice activity state and the second voice activity state;
[0134] A correlation value determination module 350, configured to determine the correlation value between the current frame signal and at least one frame target reference signal respectively when the credibility value is greater than the credibility threshold;
[0135] The delay determination module 360 is used to determine the delay estimation value corresponding to the current frame signal using the correlation value.
[0136] For relevant details, refer to the above method embodiment.
[0137] It should be noted that: when the delay estimation device provided in the above embodiment performs delay estimation, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the delay estimation device is divided into different functional modules to complete all or part of the functions described above. In addition, the delay estimation device provided in the above embodiment and the delay estimation method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
[0138] This embodiment provides an electronic device, such as Figure 4 The electronic device at least includes a processor 401 and a memory 402 .
[0139] The processor 401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 401 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 401 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
[0140] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 402 is used to store at least one instruction, which is used to be executed by the processor 401 to implement the delay estimation method provided in the method embodiment of the present application.
[0141] In some embodiments, the electronic device may further optionally include: a peripheral device interface and at least one peripheral device. The processor 401, the memory 402 and the peripheral device interface may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface via a bus, a signal line or a circuit board. Schematically, the peripheral devices include but are not limited to: a radio frequency circuit, a touch display screen, an audio circuit, and a power supply.
[0142] Of course, the electronic device may also include fewer or more components, which is not limited in this embodiment.
[0143] Optionally, the present application also provides a computer-readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the delay estimation method of the above method embodiment.
[0144] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0145] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, ordinary technicians in this field can make other different forms of changes or modifications without making creative work, which should all fall within the scope of protection of the present application.
Claims
1. A delay estimation method, characterized in that: The method comprises: Acquire a current frame signal and a reference signal corresponding to the current frame signal; Determine a signal search range of the reference signal, and obtain n frames of target reference signals within the signal search range, where n is a positive integer; Determining a first voice activity state of the current frame signal and a second voice activity state of the target reference signal; Determining the credibility of the current frame signal based on the first voice activity state and the second voice activity state includes: In a case where the first voice activity state indicates that there is a voice signal in the current frame signal, determining a pre-alignment reference signal and a nearby reference signal whose time difference with the pre-alignment reference signal is less than or equal to a preset threshold from the n-frame target reference signals based on the delay estimation value determined last time; wherein the delay estimation value determined last time is initialized to 0; Determining the credibility of the current frame signal based on the second voice activity state of the pre-aligned reference signal and the second voice activity state of the nearby reference signal; When the value of the credibility is greater than the credibility threshold, respectively determining the correlation value between the current frame signal and at least one frame of the target reference signal; The correlation value is used to determine a delay estimation value corresponding to the current frame signal.
2. The method according to claim 1, characterized in that: The determining the credibility of the current frame signal based on the second voice activity state of the pre-aligned reference signal and the second voice activity state of the nearby reference signal comprises: Using a Hanning window of a preset length to weight the second voice activity state of the pre-aligned reference signal to obtain a first weighted value; Using the Hanning window of the preset length to weight the second voice activity state of the nearby reference signal to obtain a second weighted value; The sum of the first weighted value and the second weighted value is determined to obtain the credibility; wherein the preset length is determined based on the preset threshold.
3. The method according to claim 1, characterized in that After using the correlation value to determine the delay estimation value corresponding to the current frame signal, the method further includes: Determine verification reference signals corresponding to m random verification delays from the search range, where m is an integer greater than 2; Calculating a correlation value between the current frame signal and the verification reference signal; Determine the target random verification delays corresponding to the two verification reference signals having the largest correlation values with the current frame signal, and obtain a delay verification range; the upper limit value of the delay verification range is the larger value of the target random verification delays, and the lower limit value of the delay verification range is the smaller value of the target random verification delays; The delay estimation value is verified using the delay verification range.
4. The method according to claim 3, characterized in that The using the delay verification range to verify the delay estimation value includes: When the delay estimation value is within the delay verification range, verifying that the delay estimation value is correct; In a case where the delay estimation value is not within the delay verification range, determining whether a correlation value corresponding to the delay estimation value is greater than a correlation value corresponding to the target random verification delay; When the correlation value corresponding to the delay estimation value is greater than the correlation value corresponding to the target random verification delay, verifying that the delay estimation value is correct; When the correlation value corresponding to the delay estimation value is less than the correlation value corresponding to the target random verification delay, the delay estimation value is verified to be wrong, and the maximum correlation value corresponding to the target random verification delay is determined as the final delay estimation value.
5. The method according to claim 3, characterized in that: The determining, within the search range, verification reference signals corresponding to m random verification delays includes: Generating the m random verification delays using a chaotic mapping method; A verification reference signal whose time delay with the current frame signal is the m random verification delays is extracted from the search range.
6. The method according to any one of claims 1 to 5, characterized in that: The respectively determining the correlation value between the current frame signal and the at least one frame target reference signal comprises: Performing bandpass weighting on the current frame signal and the target reference signal respectively according to the distribution frequency of the speech signal; Calculate the correlation value between the bandpass weighted current frame signal and the bandpass weighted target reference signal.
7. The method according to any one of claims 1 to 5, characterized in that: The signal search range is determined based on a preset maximum delay estimation value.
8. An electronic device, characterized in that: The device includes a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement the delay estimation method according to any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that: The storage medium stores a program, and when the program is executed by the processor, it is used to implement the delay estimation method according to any one of claims 1 to 7.