Method for estimating echo delay and system therefor

Abstract

The application provides an echo delay estimation method and system, wherein the method comprises the following steps: obtaining remote continuous speech data and near-end speech data sent by a remote terminal; saving the remote continuous speech data into a first-in-first-out queue by frame through fast Fourier transform; performing frame processing and fast Fourier transform on the near-end speech data to obtain M groups of remote frequency vectors and one group of near-end frequency vectors; sequentially calculating at most M cosine values of angles in Euclidean space according to the remote frequency vectors, the near-end frequency vectors and the order of the remote frequency vectors entering the first-in-first-out queue, and performing weighted average processing on each cosine value of the angle to sequentially obtain at most M stable cosine values; and calculating the echo delay according to the maximum value in the M stable cosine values corresponding to the index number of the first-in-first-out queue and the time length of a single remote frequency vector. The application can accurately and stably estimate the echo delay.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a method and system for estimating echo delay. Background Technology

[0002] In real-time communication (RTC), the audio signal from a distant device played by the audio playback device of the near-end device can easily be captured back by the audio acquisition device of the near-end device. This results in the near-end audio signal acquired by the near-end device containing the far-end audio signal captured back by the audio acquisition device. Consequently, the distant user will hear their own spoken audio being played back. This phenomenon is called echo. Because echoes severely affect communication quality, it is necessary to eliminate them.

[0003] Existing methods eliminate echoes through time-delay estimation. The classic method for time-delay estimation is to perform time-domain autocorrelation calculations, but due to the large computational cost in the time domain and its susceptibility to noise, the accuracy and stability of the estimation are not ideal.

[0004] Therefore, how to accurately and stably estimate the echo delay has become an urgent problem to be solved. Summary of the Invention

[0005] To address the aforementioned problems, the echo delay estimation method and system provided by this invention converts speech into frequency domain information, calculates the angle between high-dimensional vector spaces, and filters out noise using an average accumulation method, thereby greatly improving the accuracy and stability of echo delay estimation.

[0006] In a first aspect, the present invention provides a method for estimating echo delay, the method comprising:

[0007] Acquire remote continuous voice data sent from a remote location via a communication network and near-end voice data acquired by a near-end voice acquisition module;

[0008] The far-end continuous speech data is segmented into frames, and each frame of the frequency-divided far-end continuous speech data is subjected to a Fast Fourier Transform (FFT) to obtain a set of far-end frequency vectors from each frame of far-end continuous speech data. The obtained set of far-end frequency vectors is then stored in a first-in-first-out (FIFO) queue. The near-end speech data is also segmented into frames, and each frame of the segmented near-end speech data is subjected to a FFT to obtain a set of near-end frequency vectors. The depth of the FIFO queue is M, and the frequency division duration in the time domain corresponding to the near-end frequency vector is the same as the frequency division duration in the time domain corresponding to the far-end frequency vector.

[0009] Based on at most M sets of far-end frequency vectors, one set of near-end frequency vectors, and the order in which at most M sets of far-end frequency vectors enter the first-in-first-out queue, at most M cosine values ​​of the included angle in Euclidean space are calculated sequentially, and each cosine value of the included angle is weighted and averaged to obtain at most M stable cosine values.

[0010] The echo delay is calculated based on the index number of the maximum value among at most M stable cosine values ​​in the first-in-first-out queue, and the division duration in the time domain corresponding to a single far-end frequency vector or the division duration in the time domain corresponding to a near-end frequency vector.

[0011] Optionally, before the step of sequentially calculating the cosine values ​​of the included angles in at most M Euclidean spaces, the method further includes:

[0012] Extract the valid far-end frequency vectors from each group of far-end frequency vectors to obtain at most M groups of valid far-end vectors; the valid far-end frequency vectors are vectors within a specific frequency range obtained by fast Fourier transform after the far-end continuous speech data is processed by frame segmentation.

[0013] Extract the effective near-end frequency vectors from a set of near-end frequency vectors to obtain a set of effective near-end vectors; the effective near-end frequency vectors are vectors within a specific frequency range obtained by fast Fourier transform of a frame of near-end speech data after frequency division processing.

[0014] The steps for sequentially calculating the cosine values ​​of the included angles in at most M Euclidean spaces include:

[0015] Based on the order in which at most M groups of far-end effective vectors, one group of near-end effective vectors, and at most M groups of far-end frequency vectors enter the first-in-first-out queue, the cosine values ​​of the included angles in at most M Euclidean spaces are calculated sequentially.

[0016] Optionally, the specific frequency range is from 500 Hz to 3000 Hz.

[0017] Optionally, the step of performing a weighted average of the cosine values ​​of each included angle includes:

[0018] Through Cos[x+1] =β Cos[x] + (1-β)cos[x], calculate M stable cosine values ​​in sequence;

[0019] Where Cos[x] represents the x-th stable cosine value, cos[x] represents the x-th angle cosine value, β is less than 1 and greater than or equal to 0.8, and Cos[0] is 0.

[0020] Optionally, after calculating the echo delay, the method further includes:

[0021] Determine if the echo delay of the current frame is stable;

[0022] If so, skip at least one frame of far-end continuous speech data and near-end speech data and continue calculating the subsequent echo delay.

[0023] If not, calculate the echo delay based on the next frame of far-end continuous speech data and near-end speech data.

[0024] Optionally, the steps for determining whether the echo delay of the current frame is stable include:

[0025] Calculate the average of the K most recent echo delays;

[0026] The current delay convergence is obtained by averaging the average value with the absolute values ​​of the K echo delay differences;

[0027] Determine whether the difference between the current latency convergence and the echo latency of the current frame is less than a first threshold, and whether the current latency convergence is less than a second threshold; if yes, determine that the echo latency of the current frame is stable; if no, determine that the echo latency of the current frame is unstable.

[0028] The first threshold belongs to the set [3, 5], and the second threshold belongs to the set [10, 20].

[0029] Optionally, the steps of calculating at most M cosine values ​​of the included angles in Euclidean space according to at most M sets of far-end frequency vectors, one set of near-end frequency vectors, and the order in which the at most M sets of far-end frequency vectors enter the first-in-first-out queue, and then performing a weighted average on each cosine value to obtain at most M stable cosine values, include:

[0030] Skip the far-end frequency vector of group S that is closest to the entrance of the first-in-first-out queue;

[0031] Based on the far-end frequency vectors of the MS group that were not skipped, a group of near-end frequency vectors, and the order in which the far-end frequency vectors of the MS group that were not skipped entered the first-in-first-out queue, the cosine values ​​of the included angles in the Euclidean space were calculated in sequence, and a weighted average was performed on each cosine value to obtain the MS stable cosine values ​​in sequence.

[0032] The steps for calculating the echo delay, based on the index of the maximum value among at most M stable cosine values ​​in the first-in-first-out queue and the duration of a single far-end frequency vector or near-end frequency vector, include:

[0033] The echo delay is calculated based on the index number of the maximum value among the MS stable cosine values ​​in the first-in-first-out queue, and the duration of a single far-end frequency vector or near-end frequency vector.

[0034] In a second aspect, the present invention provides an echo delay estimation system, the system comprising:

[0035] The acquisition module is configured to acquire remote continuous voice data sent from a remote location via a communication network and near-end voice data acquired by the near-end voice acquisition module.

[0036] The transform module is configured to perform frame-segmentation processing on the far-end continuous speech data and perform Fast Fourier Transform on each frame of far-end continuous speech data after frequency division processing to obtain a set of far-end frequency vectors from each frame of far-end continuous speech data, and save each set of far-end frequency vectors into a first-in-first-out queue; to perform frame-segmentation processing on the near-end speech data and perform Fast Fourier Transform on each frame of near-end speech data after frame-segmentation processing to obtain a set of near-end frequency vectors; the depth of the first-in-first-out queue is M, and the frequency division duration corresponding to the near-end frequency vector in the time domain is the same as the frequency division duration corresponding to the far-end frequency vector in the time domain;

[0037] The first calculation module is configured to calculate at most M cosine values ​​of the included angle in Euclidean space according to the order in which at most M groups of far-end frequency vectors, one group of near-end frequency vectors, and at most M groups of far-end frequency vectors enter the first-in-first-out queue, and to perform weighted averaging on each included angle cosine value to obtain at most M stable cosine values.

[0038] The second calculation module is configured to calculate the echo delay based on the index number in the first-in-first-out queue corresponding to the maximum value among at most M stable cosine values, and the frequency division duration in the time domain corresponding to a single far-end frequency vector or the frequency division duration in the time domain corresponding to a near-end frequency vector.

[0039] Optionally, the system also includes:

[0040] The first extraction module is configured to extract the valid far-end frequency vectors from each group of far-end frequency vectors before executing the first calculation module, to obtain at most M groups of valid far-end vectors; the valid far-end frequency vectors are vectors within a specific frequency range obtained by fast Fourier transform after frame processing;

[0041] The second extraction module is configured to extract a set of valid near-end frequency vectors from a set of near-end frequency vectors before executing the first calculation module, to obtain a set of valid near-end vectors; the valid near-end frequency vectors are vectors within a specific frequency range obtained by fast Fourier transform in a frame of near-end speech data after frame processing;

[0042] The first calculation module is also configured to calculate the cosine values ​​of the included angles in up to M Euclidean spaces in sequence, based on the order in which up to M sets of far-end effective vectors, one set of near-end effective vectors, and up to M sets of far-end frequency vectors enter the first-in-first-out queue.

[0043] Optionally, the specific frequency range is from 500 Hz to 3000 Hz.

[0044] Optionally, the first calculation module is also configured to compute Cos[x+1] = β Cos[x] + (1-β)cos[x], calculate M stable cosine values ​​in sequence;

[0045] Where Cos[x] represents the x-th stable cosine value, cos[x] represents the x-th angle cosine value, β is less than 1 and greater than or equal to 0.8, and Cos[0] is 0.

[0046] Optionally, the system also includes:

[0047] The judgment module is configured to determine whether the echo delay of the current frame is stable after the second calculation module is executed. If it is stable, it skips at least one frame of far-end continuous speech data and near-end speech data and continues to calculate the subsequent echo delay. If it is not stable, it calculates the echo delay based on the next frame of far-end continuous speech data and near-end speech data.

[0048] Optionally, the determination module includes:

[0049] The first calculation submodule is configured to calculate the average of the K most recent echo delays;

[0050] The second judgment submodule is configured to obtain the current delay convergence by averaging the average value and the absolute value of the K echo delay differences;

[0051] The judgment submodule is configured to determine whether the difference between the current delay convergence and the echo delay of the current frame is less than a first threshold and whether the current delay convergence is less than a second threshold; if yes, determine that the echo delay of the current frame is stable; if no, determine that the echo delay of the current frame is unstable.

[0052] The first threshold belongs to the set [3, 5], and the second threshold belongs to the set [10, 20].

[0053] Optionally, the first computing module includes:

[0054] The skip submodule is configured to skip the far-end frequency vector of group S in the first-in-first-out queue that is closest to the entry of the first-in-first-out queue;

[0055] The third calculation submodule is configured to calculate the cosine values ​​of the included angles in Euclidean space sequentially based on the order in which the far-end frequency vectors of the MS group that have not been skipped, a group of near-end frequency vectors, and the order in which the far-end frequency vectors of the MS group that have not been skipped enter the first-in-first-out queue, and to perform a weighted average on each cosine value to obtain MS stable cosine values ​​in sequence.

[0056] The second calculation module is also configured to calculate the echo delay based on the index number in the first-in-first-out queue corresponding to the maximum value among the MS stable cosine values, and the duration of a single far-end frequency vector or a single near-end frequency vector.

[0057] Thirdly, the present invention provides a chip, the chip comprising:

[0058] At least one processor; and

[0059] A memory that is communicatively connected to at least one processor; wherein,

[0060] The memory stores instructions that can be executed by at least one processor, such that at least one processor can perform any of the methods described above.

[0061] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method described above.

[0062] The method and system for estimating echo delay in voice intercom provided in this invention converts continuous voice data from the far end and near-end voice data acquired by the near-end voice acquisition module into frequency domain data through Fast Fourier Transform (FFT) and stores them in a corresponding First-In-First-Out (FIFO) queue. Then, the data stored in the FIFO queue is processed by the calculation principle of the cosine value of the included angle in Euclidean space and the weighted average method to calculate the echo delay using the obtained stable cosine value, which effectively improves the accuracy and stability of echo delay estimation. Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a schematic flowchart illustrating a method for estimating echo delay in voice intercom according to an embodiment of this application;

[0065] Figure 2 This is a schematic structural diagram of an echo delay estimation system in a voice intercom according to an embodiment of this application. Detailed Implementation

[0066] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0068] When used herein, the singular forms of “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising,” “including,” or “having,” etc., specify the presence of the stated feature, whole, step, operation, component, part, or combination thereof, but do not preclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof.

[0069] In a first aspect, the present invention provides a method for estimating echo delay in voice intercom, see [link to previous section]. Figure 1 The method includes steps S101 to S104:

[0070] Step S101: Acquire remote continuous voice data sent from the remote end through the communication network and near-end voice data acquired by the near-end voice acquisition module.

[0071] In this embodiment, both the near end and the far end are cellular mobile phones; the near end voice acquisition module is the microphone at the near end.

[0072] In this process, after the continuous voice data from the far end is played through the speaker at the near end, it is picked up by the microphone at the near end and sent to the far end as part of the near-end voice data picked up by the microphone.

[0073] Step S102: Perform frame segmentation on the far-end continuous speech data and perform Fast Fourier Transform on each frame of far-end continuous speech data after frequency segmentation to obtain a set of far-end frequency vectors from each frame of far-end continuous speech data, and save each set of far-end frequency vectors into a first-in-first-out queue; perform frame segmentation on the near-end speech data and perform Fast Fourier Transform on one frame of near-end speech data after frame segmentation to obtain a set of near-end frequency vectors.

[0074] This ensures that each frame in the first-in-first-out queue stores a set of remote frequency vectors.

[0075] The depth of the first-in-first-out queue is M; the frequency division duration is a fixed value between 5ms and 10ms, but is not limited to this; the duration of the far-end frequency vector and the near-end frequency vector is the frequency division duration, and the frequency division duration in the time domain corresponding to the near-end frequency vector is the same as the frequency division duration in the time domain corresponding to the far-end frequency vector.

[0076] It should be noted that the far-end frequency vector is represented by the expression BIN_m[N], and the near-end frequency vector is represented by the expression BIN'_[N]. Here, m takes values ​​from 0 to M-1; N represents the length of the corresponding frequency vector after the frequency domain FFT transform, which is determined by the number of points in the FFT transform. The correspondence between N and the number of points in the FFT transform is shown in Table 1.

[0077] Table 1

[0078]

[0079] In this embodiment, N is 64, meaning each group of far-end frequency vectors contains 64 far-end frequency vectors, and each group of near-end frequency vectors contains 64 near-end frequency vectors; the FIFO depth M = 100, and the frequency division duration is 10ms, meaning each frame in the FIFO stores a 10ms audio frame, thus allowing 100 audio frames to be stored in one FIFO. An echo with a duration of 0.01 = 1 second can find the corresponding reference signal in the FIFO as long as the echo path does not exceed 1 second.

[0080] After frame-by-frame FFT transformation, the frequency domain speech data is saved into the corresponding FIFO. Each input and output of the FIFO indicates that a new frame of speech data has entered, corresponding to the original data in the FIFO. This means that the delay has increased by one frame time, i.e., 10ms.

[0081] Step S103: Based on at most M groups of far-end frequency vectors, one group of near-end frequency vectors, and the order in which at most M groups of far-end frequency vectors enter the first-in-first-out queue, calculate at most M cosine values ​​of the included angle in Euclidean space in sequence, and perform weighted averaging on each included angle cosine value to obtain at most M stable cosine values ​​in sequence.

[0082] It is understandable that the order in which the far-end frequency vectors of group M enter the first-in-first-out queue is represented by m in their corresponding expression. The smaller the value of m in the expression, the later the frequency vector enters the first-in-first-out queue. That is, m gradually increases as the distance of the corresponding frequency vector from the entrance of the first-in-first-out queue gradually increases.

[0083] In an optional embodiment, before the step of sequentially calculating the cosine values ​​of the included angles in at most M Euclidean spaces, the method further includes:

[0084] Extract the valid far-end frequency vectors from each set of far-end frequency vectors to obtain at most M sets of valid far-end vectors, that is, a set of far-end frequency vectors will extract a set of valid far-end vectors; extract the valid near-end frequency vectors from a set of near-end frequency vectors to obtain a set of valid near-end vectors, that is, a set of near-end frequency vectors will extract a set of valid near-end vectors.

[0085] Among them, the effective far-end frequency vector is a vector within a specific frequency range obtained by fast Fourier transform after the far-end continuous speech data is processed into frames; the effective near-end frequency vector is a vector within a specific frequency range obtained by fast Fourier transform after a frame of near-end speech data is processed into frames; the specific frequency range is 500 Hz to 3000 Hz, or it can be 500 Hz to 2500 Hz, but it is not limited to this.

[0086] The steps for sequentially calculating the cosine values ​​of the included angles of at most M Euclidean spaces include: based on the order in which at most M sets of far-end effective vectors, one set of near-end effective vectors, and at most M sets of far-end frequency vectors enter the first-in-first-out queue, sequentially calculating the cosine values ​​of the included angles of at most M Euclidean spaces.

[0087] It is understandable that the sub-band spectrum of both speech data played from near-end speakers and speech data collected by near-end microphones will be contaminated. However, research has found that the middle portion of the speech data spectrum is relatively less contaminated. This embodiment extracts the effective frequency domain sub-bands (also known as the effective frequency vector) from the far-end frequency vector and the effective frequency domain sub-bands from the near-end frequency vector, thereby ensuring the accuracy and stability of echo delay estimation while effectively reducing computational load and improving the efficiency of echo delay estimation.

[0088] In this embodiment, the effective frequency domain sub-bands in both the far-end and near-end frequency vectors range from 500 Hz to 3000 Hz. It should be noted that in this embodiment, sub-bands and vectors correspond to each other; sub-band is a physical concept, while vector is a mathematical concept, and the two can be interchanged in name. Taking a 16kHz sampling rate and 128-point FFT transformation as an example, each frame's far-end and near-end frequency vectors are defined to contain 64 frequency domain sub-bands, denoted as bin_y

[64] and bin'_y

[64] , respectively, where y takes values ​​from 0 to 63, and 64 represents the length of the frequency vector in which it is located. After the FFT transformation, the frequency domain sub-band interval in each frame's far-end and near-end frequency vectors is 125 Hz. After extraction, each frame's far-end effective vector and near-end effective vector each contain only 18 frequency domain sub-bands (effective frequency domain sub-bands). That is, each frame's far-end effective vector contains bin_6

[18] , bin_7

[18] , ..., bin_24

[18] , and each frame's near-end effective vector contains bin'_6

[18] , bin'_7

[18] , ..., bin'_24

[18] . Thus, compared to each far-end frequency vector and near-end frequency vector, each extracted far-end effective vector and near-end effective vector has 46 fewer frequency domain sub-bands, which greatly reduces the vector length for calculating the cosine of the angle in Euclidean space and effectively improves the estimation efficiency of echo delay.

[0089] Furthermore, the obtained far-end effective vector is denoted as a, and the near-end effective vector is denoted as b. The vector a corresponding to the M groups of far-end frequency vectors in the first-in-first-out queue is respectively compared with the vector b corresponding to the obtained group of near-end frequency vectors. The cosine of the angle between the far-end effective vectors corresponding to BIN_0

[64] to BIN_99

[64] and the near-end effective vectors corresponding to BIN'

[64] is calculated.

[0090] Specifically, the cosine of the angle between the far effective vector corresponding to BIN_0

[64] and the near effective vector corresponding to BIN'

[64] is calculated in Euclidean space. The cosine of the angle between the far effective vector corresponding to BIN_1

[64] and the near effective vector corresponding to BIN'

[64] is calculated in Euclidean space. This process is repeated until the cosine of the angle between the far effective vector corresponding to BIN_99

[64] and the near effective vector corresponding to BIN'

[64] is calculated in Euclidean space. In this way, 100 cosine values ​​of the angle in Euclidean space can be obtained in sequence.

[0091] The algorithm for the cosine value of the included angle in Euclidean space, cosθ, is given in Formula 1.

[0092] Formula 1

[0093] In Formula 1, where a represents vector a as described above; b represents vector b as described above; n represents the number of frequency domain subbands in vector a or vector b, and x... i Represented as the i-th frequency domain subband in vector a; y i It is represented as the i-th frequency domain subband in vector b.

[0094] In one alternative embodiment, the step of weighted averaging of each included angle cosine value includes: sequentially calculating M stable cosine values ​​using Formula 2 as follows.

[0095] Cos[x+1] =β Formula 2: Cos[x] + (1-β)cos[x]

[0096] Where Cos[x] represents the x-th stable cosine value, cos[x] represents the x-th angle cosine value, β is less than 1 and greater than or equal to 0.8, and Cos[0] is 0.

[0097] Understandably, microphone-collected speech data contains noise, sometimes mixed with nearby (near-end) speech, making the results of a single calculation highly random. It is impossible to find the accurate echo delay from only one or two calculations. Therefore, this application averages the calculated cosine values ​​of the included angle, and the averaging method will determine whether the most similar vector in the high-dimensional space can be recovered from the noise.

[0098] The echo reference signal of the network (remote end) that enters the FIFO entrance for the first time has 0 time delay. When it enters a new frame for the next time, it is delayed by one frame, such as 10ms. After this weighted average processing, the cosine value of the included angle will gradually appear at the position of the echo, and the echo reference signal can be determined.

[0099] Step S104: Calculate the echo delay based on the index number of the maximum value among at most M stable cosine values ​​in the first-in-first-out queue, and the duration of a single far-end frequency vector or a single near-end frequency vector.

[0100] It is understandable that the index number represents the distance between any frame in the first-in-first-out queue and the queue entrance or exit. In this embodiment, the index number represents the distance between any frame in the first-in-first-out queue and the queue entrance or exit. For example, if a stable cosine value is calculated from the far-end effective vector corresponding to BIN_32

[64] and the near-end effective vector corresponding to BIN'_32

[64] , then the index number corresponding to the stable cosine value is recorded as 33. As another example, if a stable cosine value is calculated from the far-end effective vector corresponding to BIN_12

[64] and the near-end effective vector corresponding to BIN'_12

[64] , then the index number corresponding to the stable cosine value is recorded as 13.

[0101] The largest stable cosine value indicates that the corresponding far-end frequency vector and near-end frequency vector are closest in high-dimensional space. That is, the echo reference signal of the communication network (far end) is found in the frame corresponding to this near-end frequency vector, i.e., the speech collected by the microphone. The delay is calculated as in Formula 3.

[0102] Echo delay = index number Formula 3 for frequency division duration

[0103] That is, the echo delay is equal to the index number of the reference signal distance from the FIFO entrance multiplied by the frame length of the frequency division.

[0104] The method provided in this embodiment sets the sub-band vectors of the DFT (Discrete Fourier Transform) frequency domain transform as a projection into a high-dimensional space. It uses Euclidean angles to determine the similarity of the projected vectors. Based on this, it performs average filtering on the calculated angles to remove noise from single calculations, ultimately obtaining stable similarity results and thus achieving the goal of calculating time delay.

[0105] In an alternative embodiment, after the step of calculating the echo delay, the method further includes:

[0106] Determine whether the echo delay of the current frame is stable; if so, skip at least one frame of far-end continuous speech data and near-end speech data and continue to calculate the subsequent echo delay; if not, calculate the echo delay based on the next frame of far-end continuous speech data and near-end speech data.

[0107] In this embodiment, by judging whether the echo delay is stable, if the current calculation result is the same as the previous calculation result and has remained the same for several frames, then frame-by-frame calculation is used. If they are different, it indicates that the delay is jittering, and frame-by-frame calculation is used. This can save computation, improve estimation efficiency, and achieve a balance between accuracy and computing power.

[0108] Specifically, the steps to determine whether the echo delay of the current frame is stable include:

[0109] Calculate the average of the K most recent echo delays; obtain the current delay convergence by averaging the absolute values ​​of the differences between the average and the K echo delays; determine whether the difference between the current delay convergence and the echo delay of the current frame is less than a first threshold and whether the current delay convergence is less than a second threshold; if yes, determine that the echo delay of the current frame is stable; if no, determine that the echo delay of the current frame is unstable.

[0110] Where K is 2^ª, a ∈ [4, 6]; the first threshold belongs to the set [3, 5], and the second threshold belongs to the set [10, 20]; the nearest K echo delays include the currently calculated echo delay and the K-1 echo delays that are adjacent to it.

[0111] Specifically, the average delay is the average of the most recently calculated K results (K is recommended to be 2^ª, a ∈ [4,6]):

[0112] Delay_Average_K = ∑1_K {Delay[i]} / K ,i∈[1,k]

[0113] The delay convergence is the average of the absolute values ​​of the differences between the most recent K results and Delay_Average_K:

[0114] Delay_Convergence_K = ∑1_K{abs (Delay[i]–Delay_Average_K)} / K

[0115] Assuming the current delay value is Delay_Current, the judgment condition is as follows:

[0116] IF((abs(Delay_Average_K-Delay_Current) < Threshold1)

[0117] &&(Delay_Convergence_K <Threshold2)){

[0118] The delay algorithm has converged.

[0119] },

[0120] Threshold1 ∈[3,5], Threshold2 ∈[10,20].

[0121] In one optional embodiment, the step of sequentially calculating at most M cosine values ​​of the included angle in Euclidean space based on at most M sets of far-end frequency vectors, one set of near-end frequency vectors, and the order in which at most M sets of far-end frequency vectors enter the first-in-first-out queue, and then performing a weighted average on each cosine value to obtain at most M stable cosine values, includes:

[0122] Skip the far-end frequency vector of group S that is closest to the entrance of the first-in-first-out queue;

[0123] Based on the far-end frequency vectors of the MS group that were not skipped, a set of near-end frequency vectors obtained, and the order in which the far-end frequency vectors of the MS group that were not skipped entered the first-in-first-out queue, the cosine values ​​of the included angles in the Euclidean space are calculated sequentially, and a weighted average is performed on each cosine value to obtain the MS stable cosine values.

[0124] It should be noted that S is determined by the type of communication network and the frame duration. Specifically, the type of communication network allows for an initial estimation of the echo delay, which is then used to determine S based on the estimated value and the frame duration. For example, the delay in cellular communication is definitely greater than 20ms; if a 10ms frame estimation algorithm is used, then S can be determined to be at least 2. By skipping the cosine calculation of the first few indices of the FIFO, the efficiency of echo delay estimation can be further improved.

[0125] The steps for calculating the echo delay, based on the index of the maximum value among at most M stable cosine values ​​in the first-in-first-out queue and the duration of a single far-end frequency vector or near-end frequency vector, include:

[0126] The echo delay is calculated based on the index number of the maximum value among the MS stable cosine values ​​in the first-in-first-out queue, and the duration of a single far-end frequency vector or a single near-end frequency vector.

[0127] The steps for extracting effective frequency domain sub-bands from up to M groups of far-end frequency vectors to obtain up to M groups of effective far-end vectors include: extracting effective frequency domain sub-bands from MS groups of far-end frequency vectors to obtain MS groups of effective far-end vectors.

[0128] The steps for sequentially calculating the cosine values ​​of the included angles of at most M groups of Euclidean spaces include: based on the effective far-end vectors of the MS groups, a set of effective near-end vectors extracted, and the order in which the far-end frequency vectors of the MS groups enter the first-in-first-out queue, sequentially calculating the cosine values ​​of the included angles of the MS Euclidean spaces.

[0129] The method provided in this embodiment utilizes the calculation principle of the cosine value of the angle in high-dimensional vector space, i.e., the angle in Euclidean space, and employs an average accumulation method to filter out noise, thereby greatly improving the accuracy and stability of delay estimation. Furthermore, when the echo changes dynamically, this method can very quickly track the jitter of the echo delay, enabling rapid convergence of the speech echo tracking processing.

[0130] Secondly, one embodiment of the present invention provides an echo delay estimation system 200 for voice intercom, see [link to documentation]. Figure 2The echo delay estimation system 200 includes:

[0131] The acquisition module 201 is configured to acquire remote continuous voice data sent from a remote location via a communication network and near-end voice data acquired by a near-end voice acquisition module.

[0132] The transformation module 202 is configured to perform frame-segmentation processing on the far-end continuous speech data and perform Fast Fourier Transform on each frame of far-end continuous speech data after frequency division processing to obtain a set of far-end frequency vectors from each frame of far-end continuous speech data, and save each set of far-end frequency vectors into a first-in-first-out queue; to perform frame-segmentation processing on the near-end speech data and perform Fast Fourier Transform on a frame of near-end speech data after frame-segmentation processing to obtain a set of near-end frequency vectors; the depth of the first-in-first-out queue is M, and the frequency division duration corresponding to the near-end frequency vector in the time domain is the same as the frequency division duration corresponding to the far-end frequency vector in the time domain;

[0133] The first calculation module 203 is configured to calculate at most M cosine values ​​of the included angle in Euclidean space according to the order in which at most M groups of far-end frequency vectors, a group of near-end frequency vectors, and at most M groups of far-end frequency vectors enter the first-in-first-out queue, and to perform weighted averaging on each included angle cosine value to obtain at most M stable cosine values ​​in sequence.

[0134] The second calculation module 204 is configured to calculate the echo delay based on the index number in the first-in-first-out queue corresponding to the maximum value among at most M stable cosine values, and the frequency division duration in the time domain corresponding to a single far-end frequency vector or the frequency division duration in the time domain corresponding to a near-end frequency vector.

[0135] In an optional embodiment, the echo delay estimation system 200 further includes:

[0136] The first extraction module is configured to extract the valid far-end frequency vectors from each group of far-end frequency vectors before executing the first calculation module, to obtain at most M groups of valid far-end vectors; the valid far-end frequency vectors are obtained by fast Fourier transform after the far-end continuous speech data is processed by frame segmentation.

[0137] The second extraction module is configured to extract the effective near-end frequency vectors from a set of near-end frequency vectors before executing the first calculation module, to obtain a set of effective near-end vectors; the effective near-end frequency vectors are obtained by performing a fast Fourier transform on a frame of near-end speech data after frame processing.

[0138] The first calculation module 203 is also configured to calculate the cosine values ​​of the included angles in the Euclidean space in sequence according to the order in which at most M groups of far-end effective vectors, one group of near-end effective vectors, and at most M groups of far-end frequency vectors enter the first-in-first-out queue.

[0139] In one alternative embodiment, the specific frequency range is 500 Hz to 3000 Hz.

[0140] In an alternative embodiment, the first calculation module 203 is further configured to compute Cos[x+1] = β Cos[x] + (1-β)cos[x], calculate M stable cosine values ​​in sequence;

[0141] Where Cos[x] represents the x-th stable cosine value, cos[x] represents the x-th angle cosine value, β is less than 1 and greater than or equal to 0.8, and Cos[0] is 0.

[0142] In an optional embodiment, the echo delay estimation system 200 further includes:

[0143] The judgment module is configured to determine whether the echo delay of the current frame is stable after the second calculation module 204 is executed. If it is stable, it skips at least one frame of far-end continuous speech data and near-end speech data and continues to calculate the subsequent echo delay. If it is not stable, it calculates the echo delay based on the next frame of far-end continuous speech data and near-end speech data.

[0144] In one optional embodiment, the determination module includes:

[0145] The first calculation submodule is configured to calculate the average of the K most recent echo delays;

[0146] The second judgment submodule is configured to obtain the current delay convergence by averaging the average value and the absolute value of the K echo delay differences;

[0147] The judgment submodule is configured to determine whether the difference between the current delay convergence and the echo delay of the current frame is less than a first threshold and whether the current delay convergence is less than a second threshold; if yes, determine that the echo delay of the current frame is stable; if no, determine that the echo delay of the current frame is unstable.

[0148] The first threshold belongs to the set [3, 5], and the second threshold belongs to the set [10, 20].

[0149] The first calculation module includes:

[0150] The skip submodule is configured to skip the far-end frequency vector of group S in the first-in-first-out queue that is closest to the entry of the first-in-first-out queue;

[0151] The third calculation submodule is configured to calculate the cosine values ​​of the included angles in Euclidean space sequentially based on the order in which the far-end frequency vectors of the MS group that have not been skipped, a group of near-end frequency vectors, and the order in which the far-end frequency vectors of the MS group that have not been skipped enter the first-in-first-out queue, and to perform a weighted average on each cosine value to obtain MS stable cosine values ​​in sequence.

[0152] The second calculation module is also configured to calculate the echo delay based on the index number in the first-in-first-out queue corresponding to the maximum value among the MS stable cosine values, and the duration of a single far-end frequency vector or a single near-end frequency vector.

[0153] The first extraction module is also configured to extract the effective frequency domain subbands from MS far-end frequency vectors to obtain MS far-end effective vectors.

[0154] The second calculation module is also configured to calculate the cosine values ​​of the included angles in MS Euclidean spaces sequentially based on the order in which MS far-end effective vectors, near-end effective vectors, and MS far-end frequency vectors enter the first-in-first-out queue.

[0155] Thirdly, one embodiment of the present invention provides a chip, the chip comprising:

[0156] At least one processor; and

[0157] A memory that is communicatively connected to at least one processor; wherein,

[0158] The memory stores instructions that can be executed by at least one processor, such that the instructions are executed by at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0159] Fourthly, one embodiment of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method as described in the first aspect.

[0160] In the description of this specification, the references to terms such as "some embodiments," "other embodiments," "ideal embodiments," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example that are included in at least one embodiment or example of this application. In this specification, the illustrative descriptions of the above terms do not necessarily refer to the same embodiments or examples.

[0161] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0162] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for estimating echo delay, characterized in that, The method includes: Acquire remote continuous voice data sent from a remote location via a communication network and near-end voice data acquired by a near-end voice acquisition module; The far-end continuous speech data is segmented into frames, and a Fast Fourier Transform is performed on each frame of the far-end continuous speech data after frequency division to obtain a set of far-end frequency vectors from each frame of the far-end continuous speech data. Each set of far-end frequency vectors is then stored in a first-in-first-out (FIFO) queue. The near-end speech data is also segmented into frames, and a Fast Fourier Transform is performed on each frame of the near-end speech data after segmentation to obtain a set of near-end frequency vectors. The depth of the FIFO queue is M, and the frequency division duration in the time domain corresponding to the near-end frequency vector is the same as the frequency division duration in the time domain corresponding to the far-end frequency vector. Based on at most M sets of far-end frequency vectors, one set of near-end frequency vectors, and the order in which at most M sets of far-end frequency vectors enter the first-in-first-out queue, at most M cosine values ​​of the included angle in Euclidean space are calculated sequentially, and a weighted average is performed on each included angle cosine value to obtain at most M stable cosine values. The echo delay is calculated based on the index number in the first-in-first-out queue corresponding to the maximum value among the at most M stable cosine values, and the frequency division duration in the time domain corresponding to a single far-end frequency vector or the frequency division duration in the time domain corresponding to a near-end frequency vector.

2. The method according to claim 1, characterized in that, Before the step of sequentially calculating the cosine values ​​of the included angles in at most M Euclidean spaces, the method further includes: Extract the valid far-end frequency vectors from each group of far-end frequency vectors to obtain at most M groups of valid far-end vectors; the valid far-end frequency vectors are vectors within a specific frequency range obtained by fast Fourier transform after the far-end continuous speech data is processed by frame segmentation. Extract the effective near-end frequency vectors from a set of near-end frequency vectors to obtain a set of effective near-end vectors; the effective near-end frequency vectors are vectors within a specific frequency range obtained by fast Fourier transform of a frame of near-end speech data after frame processing. The step of sequentially calculating the cosine values ​​of the included angles in at most M Euclidean spaces includes: Based on the order in which the at most M sets of far-end effective vectors, one set of near-end effective vectors, and the at most M far-end frequency vectors enter the first-in-first-out queue, the at most M cosine values ​​of the included angle in Euclidean space are calculated sequentially.

3. The method according to claim 2, characterized in that, The specific frequency range is from 500 Hz to 3000 Hz.

4. The method according to claim 1, characterized in that, The step of performing a weighted average of the cosine values ​​of each included angle includes: Through Cos[x+1] =β Cos[x] + (1-β)cos[x], calculate M stable cosine values ​​in sequence; Wherein, Cos[x] represents the xth stable cosine value, cos[x] represents the xth included angle cosine value, β is less than 1 and greater than or equal to 0.8, and Cos[0] is 0.

5. The method according to claim 1, characterized in that, After calculating the echo delay, the method further includes: Determine if the echo delay of the current frame is stable; If so, skip at least one frame of the far-end continuous speech data and the near-end speech data and continue to calculate the subsequent echo delay. If not, calculate the echo delay based on the far-end continuous speech data and the near-end speech data in the next frame.

6. The method according to claim 5, characterized in that, The step of determining whether the echo delay of the current frame is stable includes: Calculate the average of the K most recent echo delays; The current delay convergence is obtained by averaging the average value with the absolute values ​​of the K echo delay differences; Determine whether the difference between the current delay convergence and the echo delay of the current frame is less than a first threshold, and whether the current delay convergence is less than a second threshold; if yes, determine that the echo delay of the current frame is stable; if no, determine that the echo delay of the current frame is unstable. The first threshold belongs to the set [3, 5], and the second threshold belongs to the set [10, 20].

7. The method according to any one of claims 1 to 6, characterized in that, The step of calculating at most M cosine values ​​of the included angle in Euclidean space according to at most M sets of far-end frequency vectors, the near-end frequency vectors, and the order in which the at most M sets of far-end frequency vectors enter the first-in-first-out queue, and then performing a weighted average on each cosine value to obtain at most M stable cosine values, includes: Skip the S-group of far-end frequency vectors in the first-in-first-out queue that are closest to the entrance of the first-in-first-out queue; Based on the far-end frequency vectors of the MS group that were not skipped, the near-end frequency vectors of the MS group that were not skipped, and the order in which the far-end frequency vectors of the MS group that were not skipped entered the first-in-first-out queue, the cosine values ​​of the included angles in the Euclidean space were calculated sequentially, and a weighted average was performed on each cosine value to obtain the MS stable cosine values. The step of calculating the echo delay based on the index number in the first-in-first-out queue corresponding to the maximum value among the at most M stable cosine values, and the duration of a single far-end frequency vector or the duration of the near-end frequency vector, includes: The echo delay is calculated based on the index number of the maximum value among the MS stable cosine values ​​in the first-in-first-out queue, and the duration of a single far-end frequency vector or the duration of a near-end frequency vector.

8. An echo delay estimation system, characterized in that, The system includes: The acquisition module is configured to acquire remote continuous voice data sent from a remote location via a communication network and near-end voice data acquired by the near-end voice acquisition module. The transformation module is configured to perform frame-segmentation processing on the far-end continuous speech data and perform Fast Fourier Transform on each frame of the far-end continuous speech data after frequency division processing to obtain a set of far-end frequency vectors from each frame of the far-end continuous speech data, and save each set of far-end frequency vectors into a first-in-first-out queue; to perform frame-segmentation processing on the near-end speech data and perform Fast Fourier Transform on each frame of the near-end speech data after frame-segmentation processing to obtain a set of near-end frequency vectors; the depth of the first-in-first-out queue is M, and the frequency division duration corresponding to the near-end frequency vector in the time domain is the same as the frequency division duration corresponding to the far-end frequency vector in the time domain; The first calculation module is configured to calculate at most M cosine values ​​of the included angle in Euclidean space according to the order in which the at most M groups of the far-end frequency vectors, the group of the near-end frequency vectors, and the order in which the at most M groups of the far-end frequency vectors enter the first-in-first-out queue, and to perform a weighted average on each cosine value to obtain at most M stable cosine values. The second calculation module is configured to calculate the echo delay based on the index number in the first-in-first-out queue corresponding to the maximum value among the at most M stable cosine values, and the frequency division duration in the time domain corresponding to a single far-end frequency vector or the frequency division duration in the time domain corresponding to a near-end frequency vector.

9. The system according to claim 8, characterized in that, The system also includes: The first extraction module is configured to extract the valid far-end frequency vectors from each group of far-end frequency vectors before executing the first calculation module, to obtain at most M groups of valid far-end vectors; the valid far-end frequency vectors are vectors within a specific frequency range obtained by fast Fourier transform after the far-end continuous speech data is processed by frame segmentation. The second extraction module is configured to extract a set of valid near-end frequency vectors from a set of near-end frequency vectors before executing the first calculation module, to obtain a set of valid near-end vectors; the valid near-end frequency vectors are vectors within a specific frequency range obtained by fast Fourier transform of a frame of near-end speech data after frame processing. The first calculation module is further configured to calculate, sequentially, at most M cosine values ​​of the included angle in Euclidean space according to the order in which the at most M groups of far-end effective vectors, the near-end effective vectors, and the at most M groups of far-end frequency vectors enter the first-in-first-out queue.

10. The system according to claim 9, characterized in that, The specific frequency range is from 500 Hz to 3000 Hz.

11. The system according to claim 8, characterized in that, The first calculation module is also configured to use Cos[x+1] = β Cos[x] + (1-β)cos[x], calculate M stable cosine values ​​in sequence; Wherein, Cos[x] represents the xth stable cosine value, cos[x] represents the xth included angle cosine value, β is less than 1 and greater than or equal to 0.8, and Cos[0] is 0.

12. The system according to claim 8, characterized in that, The system also includes: The judgment module is configured to, after the second calculation module is executed, determine whether the echo delay of the current frame is stable. If so, skip at least one frame of the far-end continuous speech data and the near-end speech data and continue to calculate the subsequent echo delay. If not, calculate the echo delay based on the far-end continuous speech data and the near-end speech data of the next frame.

13. The system according to claim 12, characterized in that, The judgment module includes: The first calculation submodule is configured to calculate the average of the K most recent echo delays; The second judgment submodule is configured to obtain the current delay convergence by averaging the average value and the absolute values ​​of the K echo delay differences; The judgment submodule is configured to determine whether the difference between the current delay convergence and the echo delay of the current frame is less than a first threshold, and whether the current delay convergence is less than a second threshold; if yes, determine that the echo delay of the current frame is stable; if no, determine that the echo delay of the current frame is unstable. The first threshold belongs to the set [3, 5], and the second threshold belongs to the set [10, 20].

14. The system according to any one of claims 8 to 13, characterized in that, The first computing module includes: The skip submodule is configured to skip the S-group of far-end frequency vectors in the first-in-first-out queue that are closest to the entrance of the first-in-first-out queue; The third calculation submodule is configured to calculate the cosine values ​​of the included angles in Euclidean space sequentially based on the far-end frequency vectors of the MS group that have not been skipped, the group of near-end frequency vectors, and the order in which the far-end frequency vectors of the MS group that have not been skipped enter the first-in-first-out queue, and to perform a weighted average on each cosine value to obtain MS stable cosine values ​​sequentially. The second calculation module is further configured to calculate the echo delay based on the index number in the first-in-first-out queue corresponding to the maximum value among the MS stable cosine values, and the duration of a single far-end frequency vector or a single near-end frequency vector.

15. A chip, characterized in that, The chip includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 7.

Images