An echo cancellation method based on linear microphone array

By employing an echo cancellation method using a linear microphone array and utilizing adaptive adjustment of the steering vector and covariance matrix, effective echo suppression was achieved for external open-back headphones in scenarios with strong external speaker leakage and dual-talk, thereby improving call quality and near-end speech recognition rate.

CN123123774APending Publication Date: 2026-11-17SHENZHEN TIMEKETTLE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611305920.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-26
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In scenarios with strong external speaker leakage and simultaneous dual-talk, external open-back headphones have insufficient echo suppression, and near-end speech is easily suppressed, resulting in poor far-end call quality and decreased near-end speech recognition rate.

Method used

Echo cancellation is achieved by using a linear microphone array. By acquiring multiple time-domain microphone signals and performing time-frequency transformation, the steering vector of the target speech direction is determined. The recursive update rate of the covariance matrix is ​​adjusted, and the minimum variance distortionless response weight is calculated to separate the external echo from the near-end speech.

Benefits of technology

It suppresses the leakage direction of external speaker echo while maintaining the distortion-free response of the target speech, thereby improving the echo suppression depth and near-end speech protection capability, and improving call quality and speech recognition rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123774A_ABST
    Figure CN123123774A_ABST
Patent Text Reader

Abstract

This application discloses an echo cancellation method based on a linear microphone array. The technical solution provided in this application introduces the spatial orientation information of the linear microphone array and a recursive update rate mechanism that adaptively adjusts according to the activity level of near-end speech. This ensures that the minimum variance distortionless response weight maintains a distortionless response in the direction of the target speech and suppresses leakage in the direction of external speaker echo. Simultaneously, it prevents the target speech from entering the noise and echo model, achieving separation between external speaker echo and near-end speech. This improves the echo suppression depth and near-end speech protection capability in scenarios with strong external speaker leakage, meeting the stability requirements of echo suppression performance for open-back headphones in call and speech recognition scenarios.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of wireless earphone technology, and more particularly to an echo cancellation method based on a linear microphone array. Background Technology

[0002] Currently, open-back headphones typically rely on the acoustic echo cancellation module built into the Bluetooth chip to suppress external speaker echo. This type of module estimates the acoustic echo path based on the linear correlation between the playback reference signal and the microphone signal, then generates a cancellation signal based on the estimated echo path. This cancellation signal is then superimposed on the microphone signal to achieve echo suppression. This method can achieve a certain level of suppression at normal volumes for ordinary in-ear headphones.

[0003] However, in practical applications, the external speaker volume of open-back headphones is usually quite high, and the physical distance between the speaker and the microphone is short, resulting in a short coupling path and significantly higher external speaker echo leakage compared to in-ear headphones. When the external speaker leakage exceeds the linear suppression capability of the chip's built-in acoustic echo cancellation module, residual echoes become quite noticeable. Furthermore, in two-way communication scenarios where the speaker is continuously playing and the wearer is speaking simultaneously, the microphone signal contains both far-end echoes and near-end speech. The chip's built-in module struggles to distinguish between the two, easily suppressing near-end speech as echo, leading to false cancellation of near-end speech. This results in poor far-end call quality and a decreased near-end speech recognition rate. Summary of the Invention

[0004] This application provides an echo cancellation method based on a linear microphone array, which enables the minimum variance distortionless response weight to maintain a distortionless response in the direction of the target speech and to suppress the leakage direction of the external speaker echo. At the same time, it prevents the target speech from entering the noise and echo model, thereby achieving the separation of external speaker echo and near-end speech. This solves the technical problems of insufficient echo suppression and easy missuppression of near-end speech in external open-back headphones under strong external speaker leakage and dual-talk concurrent scenarios.

[0005] In a first aspect, embodiments of this application provide an echo cancellation method based on a linear microphone array, comprising: Acquire multiple time-domain microphone signals collected by a linear microphone array, perform time-frequency transformation on the multiple time-domain microphone signals, and obtain the frequency domain observation vector of each frequency point; Based on the array geometry parameters of the linear microphone array and the preset target speech direction, determine the steering vector corresponding to the target speech direction; Determine the state indicator value that represents the current level of proximal speech activity, and adjust the recursive update rate of the covariance matrix according to the state indicator value. The state indicator value is inversely proportional to the recursive update rate. The covariance matrix of noise and echo is recursively estimated based on the recursive update rate and frequency domain observation vector, and the minimum variance distortionless response weight is calculated based on the covariance matrix and steering vector. The frequency domain observation vectors are weighted and merged according to the minimum variance distortionless response weight to obtain the beamforming output signal. The beamforming output signal is then subjected to inverse time-frequency transformation to obtain the echo-free time domain output signal.

[0006] Further, determine the state indicator value representing the current level of proximal speech activity, including: Determine the current near-end speaking probability and the two-way speaking probability, and take the larger value between the near-end speaking probability and the two-way speaking probability as the status indication value.

[0007] Furthermore, determine the current near-end speaking probability and two-way speaking probability, including: Based on the signal energy of the current multi-channel time-domain microphone signals, the voice activity detection results, and the correlation between the playback reference signal and the multi-channel time-domain microphone signals, the near-end speaking probability and the two-way speaking probability are jointly estimated.

[0008] Furthermore, the recursive update rate of the covariance matrix is ​​adjusted based on the state indication value, including: Obtain the preset baseline recursive coefficient and adjustment coefficient; The target recursive coefficient is calculated based on the baseline recursive coefficient, the adjustment coefficient, and the state indication value. The recursive update rate of the covariance matrix is ​​determined based on the target recursive coefficient. Specifically, the target recursive coefficient increases and the recursive update rate decreases when the state indication value increases.

[0009] Furthermore, the preset baseline recursive coefficients and adjustment coefficients are obtained, including: Determine the parameter set corresponding to the current external speaker volume level, and obtain the preset baseline recursive coefficient and adjustment coefficient from the parameter set.

[0010] Furthermore, the parameter set also includes diagonal loading coefficients; after recursively estimating the covariance matrix of noise and echo based on the recursive update rate and frequency domain observation vectors, it also includes: The updated covariance matrix is ​​diagonally loaded based on the diagonal loading coefficients. The diagonal loading coefficients are added as constant terms to each of the main diagonal elements of the covariance matrix. The diagonal loading coefficients are used to correct the covariance matrix.

[0011] Furthermore, this echo cancellation method based on a linear microphone array also includes: When switching the external speaker volume level, the parameter value of the current parameter group will be smoothly transitioned to the value of the target level within a preset transition time.

[0012] Furthermore, before performing inverse time-frequency transformation on the beamforming output signal, the following steps are also included: Determine the low-frequency point in the beamforming output signal. The low-frequency point is the frequency point with a frequency lower than the low-frequency cutoff frequency. A set minimum gain limit is applied to low-frequency points. The minimum gain limit is used to limit the degree of suppression of low-frequency speech.

[0013] Furthermore, a time-frequency transformation is performed on the multiple time-domain microphone signals to obtain the frequency domain observation vector for each frequency point, including: The multi-channel time-domain microphone signals are subjected to frame-segmentation and windowing to obtain the windowed time-domain frame signals. Perform a short-time Fourier transform on the time-domain frame signal to obtain the frequency domain observation vector for each frequency point.

[0014] Furthermore, the linear microphone array includes three microphones, with adjacent microphones spaced equally apart; the target speech direction is 0°, and the corresponding speaker interference direction is 180°, with the 0° and 180° directions located on the same axis of the linear microphone array.

[0015] In a second aspect, embodiments of this application provide an electronic device, including: Memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the echo cancellation method based on a linear microphone array as described in the first aspect.

[0016] In a third aspect, embodiments of this application provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the echo cancellation method based on a linear microphone array as described in the first aspect.

[0017] In a fourth aspect, embodiments of this application provide a computer program product comprising a computer program that, when executed by a processor, implements the echo cancellation method based on a linear microphone array as described in the first aspect.

[0018] This application embodiment acquires multiple time-domain microphone signals collected by a linear microphone array, performs time-frequency transformation on the multiple time-domain microphone signals to obtain frequency-domain observation vectors for each frequency point; determines the steering vector corresponding to the target speech direction based on the array geometric parameters of the linear microphone array and the preset target speech direction; determines a state indicator value representing the current near-end speech activity level, adjusts the recursive update rate of the covariance matrix according to the state indicator value, and the state indicator value is inversely proportional to the recursive update rate; recursively estimates the covariance matrix of noise and echo based on the recursive update rate and the frequency-domain observation vector, calculates the minimum variance distortionless response weight based on the covariance matrix and the steering vector; weights and merges the frequency-domain observation vectors according to the minimum variance distortionless response weight to obtain the beamforming output signal, and performs inverse time-frequency transformation on the beamforming output signal to obtain the echo-free time-domain output signal. By employing the aforementioned technical means, and by introducing the spatial orientation information of the linear microphone array and a recursive update rate mechanism that adaptively adjusts according to the activity level of near-end speech, the minimum variance distortionless response weight maintains a distortionless response in the direction of the target speech and suppresses leakage in the direction of external speaker echo. At the same time, it prevents the target speech from entering the noise and echo model, thereby achieving the separation of external speaker echo and near-end speech. This improves the echo suppression depth and near-end speech protection capability in scenarios with strong external speaker leakage, and meets the requirements of external open-back headphones for the stability of echo suppression effect in call and speech recognition scenarios. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of an echo cancellation method based on a linear microphone array provided in an embodiment of this application; Figure 2 This is a schematic diagram of the headphone microphone setup in an embodiment of this application; Figure 3 This is a schematic diagram of the headphone speaker configuration in an embodiment of this application; Figure 4 This is a flowchart illustrating the calculation of the recursive update rate in an embodiment of this application; Figure 5 This is a flowchart of the low-frequency protection process in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0022] Example: Figure 1 A flowchart of an echo cancellation method based on a linear microphone array, according to an embodiment of this application, is provided. This echo cancellation method can be executed by a processing unit of an external open-back headset. This processing unit can be implemented using any of the following methods, such as a digital signal processor, a microcontroller, or an audio processing chip. The following description uses this processing unit as the main entity executing the echo cancellation method. (Refer to...) Figure 1 The echo cancellation method based on a linear microphone array specifically includes: S110. Acquire multiple time-domain microphone signals collected by the linear microphone array, perform time-frequency transformation on the multiple time-domain microphone signals, and obtain the frequency domain observation vector of each frequency point.

[0023] In this embodiment, a linear microphone array is installed in an external open-back headphone or other headphone device, consisting of multiple microphones arranged linearly, used to collect acoustic signals around the wearer. Multiple time-domain microphone signals refer to the time-domain signals collected by each microphone in the linear microphone array at the same sampling time, with one time-domain microphone signal corresponding to one microphone. The processing unit synchronously collects the time-domain signals of each microphone through a signal acquisition circuit to obtain multiple time-domain microphone signals. The sampling times of each time-domain microphone signal are kept consistent to facilitate subsequent joint processing of the signals.

[0024] After synchronous acquisition, the processing unit performs preprocessing on each time-domain microphone signal, including alignment, DC removal, and bandpass filtering. Alignment compensates for time deviations caused by hardware delays or placement differences between the time-domain microphone signals, ensuring that the sampling points of each signal correspond to the same physical time. DC removal removes the DC component from each signal. Bandpass filtering filters out interference components outside the speech band, preserving the frequency bands where speech energy is concentrated. The preprocessed multi-channel time-domain microphone signals are then framed and windowed according to a uniform frame length and frame shift, preparing for subsequent time-frequency transformation. For example, assuming a linear microphone array includes three microphones and acquires three time-domain microphone signals, the frequency domain observation vector after time-frequency transformation is represented as: in, , and Each corresponds to a time-domain microphone signal. For frequency point, For frame index.

[0025] Time-frequency transformation refers to the process of transforming a time-domain signal into the time-frequency domain, used to obtain the frequency domain representation of the signal in two dimensions: time frame and frequency point. The frequency domain observation vector is a column vector composed of the frequency domain values ​​of each microphone signal at each frequency point within the same time frame, and its dimension is the same as the number of microphones in a linear microphone array.

[0026] In addition, the processing unit also acquires a playback reference signal, which refers to the audio signal actually played by the speaker. This signal can be taken from the digital signal before the output of the digital-to-analog converter, or from the pulse-code modulation output of the player or audio decoder. The playback reference signal undergoes time-frequency transformation according to the same frame length, frame shift, and window function as the multi-channel time-domain microphone signals to obtain a frequency-domain playback reference signal. The frequency-domain playback reference signal and the frequency-domain observation vector are located on the same time-frequency grid and are used for subsequent estimation of near-end speaking probability and two-way speaking probability.

[0027] Optionally, the linear microphone array includes three microphones, with adjacent microphones spaced equally apart; the target speech direction is 0°, and the corresponding speaker interference direction is 180°, with the 0° and 180° directions located on the same axis of the linear microphone array.

[0028] like Figure 2As shown, the linear microphone array includes three microphones 11. The pickup holes of the three microphones 11 are arranged in a straight line along the edge of the earphone, forming a uniform linear array. Adjacent microphones are evenly spaced, with a spacing of 3 cm. The three microphones are designated as the first microphone, the second microphone, and the third microphone, with the second microphone located between the first and third microphones. The first and third microphones are symmetrically distributed about the second microphone. Figure 3 As shown, the headphone speaker 12 is positioned opposite the microphone. The microphone's pickup surface faces the wearer, with the 0° direction being the direction of the microphone's pickup hole, i.e., the direction from which the target speech originates. The speaker's output hole is located on the back side of the microphone array's pickup surface, with the 180° direction being the primary direction of sound propagation. The direction of the target speech and the direction of speaker interference are located on the same axis of the linear microphone array, and they have maximum distinguishability along the array axis.

[0029] Understandably, a linear microphone array can also include two, four, or more microphones. The more microphones there are, the higher the dimension of the frequency domain observation vector, and the greater the spatial freedom of beamforming. The 0° and 180° directions lie on the same axis of the linear microphone array, but are opposite in direction. Therefore, the relative time delays generated on the microphone array by near-end speech from the 0° direction and speaker echo from the 180° direction have opposite signs, resulting in the greatest spatial difference. The minimum variance distortionless response weight utilizes this directional difference, maintaining a distortionless response in the 0° direction and creating a zero or low gain in the 180° direction, thereby maximizing the distinction between target speech and speaker echo. The selection of microphone spacing affects the resolution of the steering vector as it changes direction. Increasing the spacing improves the distinguishability between steering vectors in different directions. However, excessive spacing can lead to spatial aliasing, meaning different directions produce the same relative time delay, reducing the uniqueness of directional discrimination. This application uses a 3-centimeter spacing to provide sufficient spatial phase difference information within the speech band while avoiding spatial aliasing, enabling beamforming to effectively distinguish sound sources in the 0° and 180° directions.

[0030] Optionally, time-frequency transformation is performed on the multiple time-domain microphone signals to obtain the frequency domain observation vector for each frequency point, including: The multi-channel time-domain microphone signals are subjected to frame-segmentation and windowing to obtain the windowed time-domain frame signals. Perform a short-time Fourier transform on the time-domain frame signal to obtain the frequency domain observation vector for each frequency point.

[0031] When performing time-frequency transformation on multiple time-domain microphone signals, each signal is first processed by frame segmentation and windowing, followed by a short-time Fourier transform. Frame segmentation and windowing involves dividing each time-domain microphone signal into multiple time-domain frames according to a preset frame length, and multiplying each time-domain frame by a preset window function to obtain the windowed time-domain frame signal. The frame length refers to the number of sampling points in each time-domain frame, and the frame shift refers to the interval between the starting sampling points of two adjacent time-domain frames. The window function is used to suppress spectral leakage at the boundaries of the time-domain frames.

[0032] Short-time Fourier transform (SFT) refers to performing a discrete Fourier transform on a windowed time-domain frame signal to obtain the frequency domain values ​​of that frame at each frequency point. The frequency domain values ​​of each signal within the same frame are arranged into a column vector to obtain the frequency domain observation vector for that frame. This frequency domain observation vector carries the spatial phase difference information between each microphone; this phase difference is related to the sound source direction and serves as the input basis for steering vector construction and beamforming processing.

[0033] S120. Determine the steering vector corresponding to the target speech direction based on the array geometry parameters of the linear microphone array and the preset target speech direction.

[0034] Furthermore, array geometry parameters refer to the parameters used to describe the positional relationships of microphones in a linear microphone array, including the number of microphones and the spacing between adjacent microphones. Target speech direction refers to the pre-calibrated direction in which the target speech arrives at the microphone array, while the corresponding echo direction is the pre-calibrated direction in which the echo from the loudspeaker reaches the microphone array. The steering vector is a vector characterizing the relative time delay and phase relationship between plane waves from a certain direction and the elements of the microphone array; it is used to describe the spatial response of the sound source in that direction on the array.

[0035] The processing unit constructs a steering vector corresponding to the target speech direction based on the array geometry parameters of the linear microphone array and the preset target speech direction. Optionally, in some embodiments, the steering vector of the echo direction can also be determined based on the echo direction, using the spatial characteristics of the echo direction to help understand the spatial distribution of the external speaker echo on the array.

[0036] Specifically, for a linear, equally spaced microphone array, the first Each microphone is relative to a reference microphone in the direction Relative delay The calculation formula is expressed as: in, Indicates the first One microphone in the direction The relative time delay on The spacing between adjacent microphones. For the speed of sound, The angle between the direction of the incoming wave and the direction of the array axis.

[0037] Based on this, direction The corresponding steering vector is: in, Indicates direction The corresponding guide vector, The imaginary unit, For frequency point, , These represent the relative time delays of the second and third microphones in the direction θ.

[0038] Based on the target speech direction being 0°, substituting θ=0° into the above steering vector formula yields the steering vector for the target speech direction. .

[0039] Since the 0° and 180° directions are located on the same axis of the linear microphone array, the steering vector of the target speech direction is opposite to the steering vector corresponding to the 180° direction. This results in the greatest discrimination along the array axis. The minimum variance distortionless response weight maintains a distortionless response in the 0° direction and forms suppression in the 180° direction, thereby distinguishing the target speech from the external echo to the greatest extent.

[0040] It should be noted that the specific angles of the target speech direction and the echo direction mentioned above are merely examples. In some embodiments, the target speech direction can be the orientation of the microphone array pickup surface, and the echo direction can be the direction of the speaker's sound outlet. This application does not limit the specific values ​​of the target speech direction and the echo direction, as long as they correspond to the main direction of the near-end speech and the main direction of the speaker's external echo, respectively.

[0041] S130. Determine the state indicator value representing the current level of near-field speech activity, and adjust the recursive update rate of the covariance matrix according to the state indicator value. The state indicator value is inversely proportional to the recursive update rate.

[0042] The status indicator value is a numerical value used to characterize the wearer's near-end speech activity level. The status indicator value increases with increasing near-end speech activity. When the wearer is speaking, or when both near and far-end speakers are emitting sound simultaneously, the status indicator value is at a higher level; when only the far-end speaker is emitting sound and the wearer is not speaking, the status indicator value is at a lower level. The recursive update rate refers to the speed at which the noise and echo covariance matrix absorbs the current frame's observation information during the recursive update process. A higher recursive update rate means the covariance matrix absorbs the current frame's observation information faster, and vice versa. The processing unit determines the status indicator value and adaptively adjusts the recursive update rate of the covariance matrix based on it. When the status indicator value increases, the recursive update rate decreases; when the status indicator value decreases, the recursive update rate increases. Therefore, during the wearer's speech or two-way speech, the recursive update rate decreases, the update of the noise and echo covariance matrix slows down, and the features of the target speech are not absorbed into the noise and echo model in large quantities, thus avoiding the target speech being falsely suppressed in subsequent beamforming; when the far-end signal is dominant, the recursive update rate increases, the covariance matrix tracks the current acoustic environment faster, and ensures the continuous suppression of external echo and environmental noise.

[0043] Optionally, a state indicator value representing the current level of proximal speech activity is determined, including: Determine the current near-end speaking probability and the two-way speaking probability, and take the larger value between the near-end speaking probability and the two-way speaking probability as the status indication value.

[0044] As an optional implementation, the status indication value is a gating factor. The gating factor is a coefficient determined based on the near-end speaking probability and the dual-talk probability, used to control the recursive update rate of the covariance matrix. The near-end speaking probability is a value representing the likelihood that the wearer is speaking, ranging from 0 to 1; the dual-talk probability is a value representing the likelihood that the near and far ends are speaking simultaneously, also ranging from 0 to 1. The gating factor is the larger of the near-end speaking probability and the dual-talk probability, denoted as: =max{P_talk(m),P_dt(m)} in, Let P_talk(m) represent the gating factor of the m-th frame, P_dt(m) represent the near-end talk probability of the m-th frame, and max{·} represent the maximum value operation.

[0045] When the wearer is not speaking and only the far-end speaker is emitting sound, both the probability of near-end speaking and the probability of dual-talk are low, resulting in a small gating factor. When the wearer is speaking but the far-end speaker is not emitting sound, the probability of near-end speaking is high, and the gating factor increases accordingly. When both the far-end and near-end speakers are emitting sound simultaneously, the probability of dual-talk is high, and the gating factor remains at a high level. Thus, the gating factor simultaneously represents both near-end speaking and dual-talk active states, providing a unified basis for adjusting the recursive update rate of the covariance matrix.

[0046] Optionally, the current near-end speaking probability and two-way speaking probability are determined, including: Based on the signal energy of the current multi-channel time-domain microphone signals, the voice activity detection results, and the correlation between the playback reference signal and the multi-channel time-domain microphone signals, the probabilities of near-end speaking and dual-talking are jointly estimated.

[0047] The probability of near-end speech is estimated jointly based on the signal energy of the current multi-channel time-domain microphone signals, the speech activity detection results, and the correlation between the playback reference signal and the multi-channel time-domain microphone signals. The signal energy of the multi-channel time-domain microphone signals refers to the energy value of the signal collected by the microphones in the linear microphone array within a preset time window. When the wearer speaks, near-end speech significantly increases the microphone signal energy, while the contribution of the speaker's external sound to the microphone signal energy is relatively limited after spatial propagation attenuation. Therefore, the microphone signal energy is an effective indicator of near-end speech. The speech activity detection results refer to the marker values ​​obtained after performing speech activity detection on the microphone signals, used to indicate whether the current frame contains speech activity. Speech activity detection refers to the process of determining whether speech activity exists within a time window. The correlation between the playback reference signal and the multi-channel time-domain microphone signals refers to the correlation coefficient between the playback reference signal and the multi-channel time-domain microphone signals, used to characterize the proportion of echo components from the playback reference signal in the microphone signal. The echo from the speaker leaks to the microphone through the acoustic propagation path from the playback reference signal; therefore, the echo component has a high correlation with the playback reference signal, while the correlation between near-end speech and the playback reference signal is very low.

[0048] The process involves normalizing the signal energy of multiple time-domain microphone signals to obtain a first energy indicator value; using the voice activity detection result as a second indicator value; obtaining a third indicator value based on the correlation between the playback reference signal and the multiple time-domain microphone signals; and finally, weighted summing of the first, second, and third energy indicator values ​​to obtain the near-end speech probability. The first energy indicator value is the ratio of the microphone signal energy to a preset energy reference value, obtained through saturation normalization. A value closer to 1 indicates higher microphone signal energy and a greater likelihood of the wearer speaking. The second indicator value is the voice activity detection marker value, with 1 indicating detected voice activity and 0 indicating no detected voice activity. The third indicator value is the difference between 1 and the correlation coefficient between the playback reference signal and the multiple time-domain microphone signals; a higher correlation coefficient indicates a higher proportion of echo components and less near-end speech components.

[0049] The dual-talk probability is estimated based on the correlation between the playback reference signal and the multiple time-domain microphone signals, as well as the speech activity detection results. In the dual-talk scenario, the multiple time-domain microphone signals contain both echo components related to the playback reference signal and near-end speech components unrelated to the playback reference signal. There is a moderate degree of correlation between the multiple time-domain microphone signals and the playback reference signal, while the multiple time-domain microphone signals also exhibit strong speech activity.

[0050] Specifically, when the voice activity detection result indicates the presence of voice activity, the probability of dual-talk is calculated based on the absolute value of the difference between the correlation coefficient of the playback reference signal and the multi-channel time-domain microphone signal and the preset dual-talk correlation reference value. The closer the correlation coefficient is to the preset dual-talk correlation reference value, the higher the probability of dual-talk.

[0051] It should be noted that the specific calculation formulas for the near-end speaking probability and the two-way speaking probability described above are merely examples. In some embodiments, the near-end speaking probability and the two-way speaking probability can also be estimated using statistical classification methods. The input features include microphone signal energy, speech activity detection results, correlation between playback reference signals and multiple time-domain microphone signals, etc. The embodiments of this application do not limit the specific estimation method of the probability.

[0052] Optionally, refer to Figure 4 Adjusting the recursive update rate of the covariance matrix based on the state indication value includes: S1301. Obtain the preset baseline recursive coefficient and adjustment coefficient; S1302. Calculate the target recursive coefficient based on the baseline recursive coefficient, adjustment coefficient, and status indication value. Determine the recursive update rate of the covariance matrix based on the target recursive coefficient. Among them, the target recursive coefficient increases and the recursive update rate decreases when the status indication value increases.

[0053] The processing unit of this application adaptively adjusts the recursive update rate of the covariance matrix based on a gating factor. As an optional implementation, a target recursive coefficient is determined based on the gating factor. The target recursive coefficient refers to the smoothing weight coefficient in the noise and echo covariance matrix recursive update formula, used to control the contribution of the current frame observation information to the covariance matrix estimate. The target recursive coefficient takes a value between 0 and 1; the larger the target recursive coefficient, the smaller the contribution of the current frame observation information to the covariance matrix estimate, and the lower the recursive update rate. The target recursive coefficient is determined according to the following formula: in, Indicates the first Frame, First Target recursive coefficients for frequency points Indicates the first Frame gating factor Indicates the baseline recursive coefficient. This represents the adjustment coefficient. For frequency point, This indicates a limiting operation, used to restrict the target recursive coefficient within a set range. Inside, Represents the minimum recursive coefficient. Represents the maximum recursive coefficient. Baseline recursive coefficient. This refers to the recursive coefficient value when the gating factor equals 0, used to determine the baseline level of the recursive coefficient. Adjustment coefficient. This refers to the increment of the recursive coefficient for each unit increase in the gating factor, used to determine the magnitude of the change in the target recursive coefficient with the gating factor. When the gating factor... When the target recursion coefficient increases, As the gating factor increases, the recursive update rate decreases; when the gating factor increases... When the target recursive coefficient is reduced, The rate of recursion updates decreases as the gating factor decreases. Therefore, the target recursive coefficients change continuously with the gating factor, achieving continuous adaptive adjustment of the covariance matrix recursive update rate and avoiding abrupt changes in the minimum variance distortion-free response weights during state transitions. It should be noted that the linear mapping used in the above formula for calculating the target recursive coefficients is merely an example. In some embodiments, the target recursive coefficients may also employ piecewise mapping, exponential mapping, or lookup table-based methods to establish a predetermined monotonic correspondence between the gating factor and the target recursive coefficients; this application does not limit this approach.

[0054] Optionally, preset baseline recursive coefficients and adjustment coefficients are obtained, including: Determine the parameter set corresponding to the current external speaker volume level, and obtain the preset baseline recursive coefficient and adjustment coefficient from the parameter set.

[0055] The external speaker volume level refers to the current external speaker volume level of an electronic device, used to characterize the loudness or weakness of the speaker's output volume. External speaker volume levels are typically divided into multiple levels, such as 1 to 16. The external speaker volume level of an over-ear headphone varies depending on the usage scenario, and the echo intensity leaked from the speaker to the microphone also varies accordingly. The processing unit obtains the current external speaker volume level by reading the volume adjustment interface, the volume register in the audio link, or the player's volume parameters. The external speaker volume level can also be estimated by the processing unit based on the energy of the playback reference signal. The processing unit determines the corresponding parameter set based on the current external speaker volume level. A parameter set is a collection of processing parameters corresponding to a given external speaker volume level, including a base recursive coefficient, an adjustment coefficient, and a diagonal loading coefficient. The processing unit divides multiple external speaker volume levels into multiple volume ranges, each corresponding to a parameter set. The processing unit determines the corresponding parameter set based on the volume range in which the current external speaker volume level is located, and obtains the base recursive coefficient and adjustment coefficient corresponding to that level from the parameter set to determine the recursive update rate.

[0056] As an optional implementation, the external speaker volume levels are divided into three levels: low, medium, and high. The low volume level corresponds to a value between 1 and 5, the medium volume level between 6 and 11, and the high volume level between 12 and 16. The specific parameter sets for each volume level are as follows: For the low volume level, the baseline recursive coefficient is 0.90, the adjustment coefficient is 0.04, and the diagonal loading coefficient is 1×10⁻⁶. -3 In the parameter group corresponding to the medium volume setting, the baseline recursive coefficient is 0.94, the adjustment coefficient is 0.05, and the diagonal loading coefficient is 2×10. -3 In the parameter group corresponding to the high volume setting, the baseline recursive coefficient is 0.97, the adjustment coefficient is 0.02, and the diagonal loading coefficient is 5×10. -3 In the three parameter groups mentioned above, the baseline recursive coefficient increases with the volume level, reducing the overall update rate of the covariance matrix at high volume levels and preventing excessive absorption of target speech features under strong echo conditions. The adjustment coefficient controls the intensity of the gating factor's adjustment of the target recursive coefficient, working in conjunction with the baseline recursive coefficient to set the range of change for the target recursive coefficient at each level. Therefore, the processing unit obtains the baseline recursive coefficient and adjustment coefficient from the corresponding parameter group based on the current speaker volume level, and determines the target recursive coefficient for the current frame by combining it with the gating factor, thus adaptively adjusting the recursive update rate according to the speaker volume.

[0057] Optionally, the echo cancellation method based on a linear microphone array further includes: When switching the external speaker volume level, the parameter value of the current parameter group will be smoothly transitioned to the value of the target level within a preset transition time.

[0058] When switching speaker volume levels, the processing unit performs a smooth transition on the parameter set, smoothly transitioning from the current level's parameter set to the target level's parameter set within a preset transition time of 100 to 200 milliseconds. The preset transition time refers to the length of time required for the parameter set to smoothly transition from the current level to the target level. Smooth transition means that within the transition time, each parameter in the parameter set continuously changes from its current value to its target value, avoiding abrupt parameter changes.

[0059] As an optional implementation, the processing unit performs linear interpolation on each parameter in the parameter set during the transition time. Let the reference recursive coefficient for the current gear be... _cur, the base recursive coefficient for the target gear is Given _tgt and a transition duration of T, the baseline recursive coefficient at time t after the transition begins is: (t)= _cur+( _tgt- _cur)·t / T. Wherein, (t) represents the baseline recursive coefficient at time t after the start of the transition. _cur represents the baseline recursive coefficient for the current gear. _tgt represents the baseline recursive coefficient for the target level, T represents the transition duration, and t represents the time elapsed since the start of the transition. The adjustment coefficients are interpolated and transitioned in the same manner. It is understood that the smooth transition of the parameter group can also be achieved using exponential interpolation, step interpolation, or filtering methods; this application does not limit this approach. Smooth transitions avoid parameter jumps caused by abrupt changes in volume levels, thereby avoiding abrupt changes in timbre, echo residual fluctuations, and abrupt changes in the minimum variance distortion-free response weights.

[0060] Optionally, the gating factor estimation in this embodiment can further incorporate wearer motion state information. The external headset can be equipped with an inertial measurement unit (IMU), which is a sensor module used to measure the acceleration and angular velocity of an object, capable of outputting the wearer's head's three-axis acceleration and three-axis angular velocity. The wearer's motion state refers to the head motion characteristics derived from the IMU output signal, including motion amplitude and activity level. When the wearer's head motion amplitude exceeds a preset threshold, it can be inferred that the wearer is engaged in speech-related activities. At this time, the estimated value of the near-speech probability is appropriately increased, allowing the gating factor to respond more quickly to the wearer starting to speak. Specifically, the processing unit calculates the root mean square value of acceleration within a preset time window from the acceleration signal output by the IMU, using it as a motion amplitude indication value. When the motion amplitude indication value exceeds a preset motion threshold, the estimated near-speech probability is multiplied by a preset gain coefficient greater than 1 to obtain a corrected near-speech probability, which is then used to calculate the gating factor. Thus, the gating factor can combine the wearer's physical motion information to further reduce the probability of near-speech being falsely suppressed.

[0061] On the other hand, the recursive update of the covariance matrix can also be set with independent update rates by frequency band grouping. The processing unit divides the frequency domain into multiple frequency bands, each containing multiple frequency points, and each frequency band uses independent recursive coefficients. A frequency band refers to an interval in the frequency domain divided according to frequency range. Since the spectrum of external echo is concentrated in the low-frequency band, the reference recursive coefficient for the low-frequency band can be set to be higher and the recursive update rate to be slower, so as to more fully protect low-frequency speech; the high-frequency band can be set to a lower reference recursive coefficient and a faster recursive update rate to accelerate the tracking of changes in environmental noise. Specifically, the processing unit divides the frequency domain into three frequency bands: low-frequency band, mid-frequency band, and high-frequency band. The recursive update rate of different frequency bands is adjusted according to their own acoustic characteristics, thereby taking into account both low-frequency speech protection and high-frequency environmental tracking.

[0062] Optionally, embodiments of this application may employ a classification model to estimate the near-end speech state. The near-end speech state classification model refers to a model used to classify and discriminate the near-end speech state in the current frame. The near-end speech state includes three categories: near-end silence, near-end single-talk, and two-talk. The near-end speech state classification model is a deep neural network model obtained through offline training based on multiple sets of training samples. The model's input is a multi-dimensional feature vector of the current frame. This multi-dimensional feature vector is composed of the microphone signal energy, speech activity detection results, the correlation between the playback reference signal and multiple time-domain microphone signals, and the energy features of each microphone signal concatenated. The dimension of the input feature vector is the same as the number of selected features. The model consists of an input layer, hidden layers, and an output layer. The input layer receives multi-dimensional feature vectors, and the number of neurons in the input layer is the same as the dimension of the input feature vectors. The hidden layer consists of multiple fully connected layers, each containing a set number of neurons. Adjacent fully connected layers are connected by a weight matrix and a bias vector. The output of each fully connected layer is processed by an activation function, which introduces non-linear mapping capabilities. The output layer outputs the probability values ​​of the current frame belonging to each near-end speech state category. The number of neurons in the output layer is the same as the number of near-end speech state categories. The output layer uses a normalized exponential function to convert the output into a probability distribution where the sum of the probabilities of each category is 1. During model training, a training dataset is constructed, in which each training sample includes a multi-dimensional feature vector and its corresponding proximal speech state category label. The multi-dimensional feature vector is input into the model, propagated forward through the input layer and each hidden layer, and the output layer outputs the predicted probability for each category. The difference between the predicted probability and the category label is calculated using the cross-entropy loss function to obtain the loss value. A stochastic gradient descent optimization algorithm is used to calculate the gradient of the weight matrix and bias vector of each layer of the model according to the loss value, and the weight matrix and bias vector are updated in the opposite direction of the gradient. These steps are repeated until the loss value converges or the preset number of training epochs is reached, resulting in a trained proximal speech state classification model. After model training is complete, during the inference phase, the multi-dimensional feature vector of the current frame is input into the trained model, and the probability value of the current frame belonging to each proximal speech state category is output, thereby determining the proximal speech probability and the double-speech probability. The above model structure is only one implementation method. This application does not limit the specific structure and training method of the proximal speech state classification model, as long as it can output the classification probability of the proximal speech state based on the input features.

[0063] S140. Based on the recursive update rate and frequency domain observation vector, recursively estimate the covariance matrix of noise and echo, and calculate the minimum variance distortionless response weight based on the covariance matrix and steering vector.

[0064] Furthermore, the covariance matrix of noise and echo refers to the matrix used to describe the statistical correlation characteristics between ambient noise and external echo at various frequencies and signals. It is an important basis for calculating the minimum variance distortionless response weight. The recursive estimation of the covariance matrix refers to the process of iteratively updating the estimated value of the covariance matrix of the previous frame using the frequency domain observation vector of the current frame, so that the covariance matrix gradually approximates the true statistical characteristics of the current acoustic environment. The processing unit recursively estimates the covariance matrix of noise and echo according to the recursive update rate.

[0065] As an optional implementation, the covariance matrices of noise and echo are updated recursively according to the following formula: in, This represents the estimated value of the noise and echo covariance matrix at the m-th frame and the f-th frequency point. This represents the estimated value of the noise and echo covariance matrix at the (m-1)th frame and the f-th frequency point. This represents the target recursive coefficient at the m-th frame and the f-th frequency point. This represents the frequency domain observation vector of the m-th frame and the f-th frequency point. express The conjugate transpose of . Target recursive coefficients The value is between 0 and 1. The larger the target recursion coefficient, the smaller the contribution of the outer product matrix of the current frame frequency domain observation vector to the covariance matrix estimate, and the lower the update rate of the covariance matrix; the smaller the target recursion coefficient, the larger the contribution of the current frame observation vector to the covariance matrix estimate, and the higher the update rate.

[0066] During near-end speech or two-way speech, the gating factor increases and the target recursive coefficient increases, weakening the contribution of the current frame observation vector to the covariance matrix. The characteristics of the target speech are not introduced into the noise and echo models, and the near-end speech is not falsely suppressed in subsequent beamforming. When the far-end signal dominates, the gating factor decreases and the target recursive coefficient decreases, causing the covariance matrix to converge more quickly to the current acoustic environment, maintaining the ability to continuously suppress external echoes and environmental noise.

[0067] Understandably, in the above recursive process, the recursive estimation of the covariance matrix is ​​performed independently at each frequency point in the frequency domain. The covariance matrix at each frequency point reflects the statistical characteristics of noise and echo at that frequency, allowing the minimum variance distortionless response weights to be calculated independently at each frequency point, adapting to the acoustic characteristics of different frequencies. After obtaining the recursively estimated covariance matrix, the processing unit calculates the minimum variance distortionless response weights based on the covariance matrix and the steering vector. The minimum variance distortionless response weights refer to a set of complex weights used to weight and combine frequency domain observation vectors to form beam output. The minimum variance distortionless response weights are calculated independently at each frequency point, and each frequency point corresponds to a set of minimum variance distortionless response weight vectors. The minimum variance distortionless response criterion is the criterion that minimizes the beamforming output power while maintaining a distortionless response in the target speech direction. Since the output power includes the power of environmental noise and external echo, minimizing the output power is equivalent to suppressing environmental noise and external echo.

[0068] Specifically, the minimum variance distortion-free response weight is calculated according to the following formula: in, This represents the minimum variance distortionless response weight vector at the m-th frame and the f-th frequency point. Represents the covariance matrix The inverse matrix, The steering vector representing the direction of the target speech. express The conjugate transpose of the above weight vectors. The weight vectors satisfy the following condition in the target speech direction. The response is 1, that is · =1, to achieve a distortion-free response in the target speech direction; at the same time, to minimize the output power and minimize the power in the non-target direction.

[0069] Since the external echo mainly comes from the 180° direction, the recursively obtained covariance matrix contains the spatial characteristics of this direction. The minimum variance distortionless response criterion, when minimizing the output power, will form a zero or low gain in the leakage direction of the external echo, thereby suppressing the external echo and making the beam distortionless in the target direction and suppressed in the external echo direction.

[0070] Optionally, the parameter set also includes diagonal loading coefficients; after recursively estimating the covariance matrix of noise and echo based on the recursive update rate and frequency domain observation vectors, it also includes: The updated covariance matrix is ​​diagonally loaded based on the diagonal loading coefficients. The diagonal loading coefficients are added as constant terms to each of the main diagonal elements of the covariance matrix. The diagonal loading coefficients are used to correct the covariance matrix.

[0071] The parameter set of this application also includes diagonal loading coefficients, which are constant coefficients superimposed on the main diagonal of the covariance matrix. These coefficients are used to improve the numerical stability of the covariance matrix inversion process and are small positive constants. After recursively estimating the noise and echo covariance matrix based on the recursive update rate and frequency domain observation vectors, the processing unit performs diagonal loading on the covariance matrix according to the diagonal loading coefficients in the parameter set. That is, it adds the product of the diagonal loading coefficients and the identity matrix to the recursively estimated covariance matrix. in, Let represent the diagonal loading coefficients, and I represent the identity matrix with the same dimensions as the covariance matrix. Diagonal loading adds the diagonal loading coefficients as constant terms to each element on the main diagonal of the covariance matrix. This superimposes a small constant on the main diagonal of the covariance matrix, improving the numerical stability of the covariance matrix inversion process and reducing the risk of ill-conditioned covariance matrix when the external echo is strong. After diagonal loading, the diagonally loaded covariance matrix replaces the undiagonally loaded covariance matrix in the matrix inversion during weight calculation, improving the condition number of the covariance matrix and preventing the minimum variance distortion-free response weights from fluctuating drastically due to matrix ill-conditionedness.

[0072] S150. The frequency domain observation vector is weighted and merged according to the minimum variance distortionless response weight to obtain the beamforming output signal. The beamforming output signal is then subjected to inverse time-frequency transformation to obtain the echo-free time domain output signal.

[0073] Frequency domain beamforming output refers to the frequency domain signal obtained by weighting and combining the frequency domain observation vectors according to the minimum variance distortionless response weights, carrying spatially filtered speech information. The processing unit weights and combines the frequency domain observation vectors of each frequency point according to the minimum variance distortionless response weights. For the m-th frame and the f-th frequency point, the frequency domain beamforming output is calculated according to the following formula: in, This represents the frequency domain beamforming output at the m-th frame and the f-th frequency point. This represents the minimum variance distortionless response weight vector at the m-th frame and the f-th frequency point. This represents the frequency domain observation vector at the m-th frame and the f-th frequency point. After performing weighted merging on all frequency points, the frequency domain beamforming output vector for the current frame can be obtained.

[0074] After obtaining the beamforming output signal, the processing unit performs an inverse time-frequency transformation on it. The inverse time-frequency transformation is the process of restoring a frequency-domain signal to a time-domain signal; it is the inverse operation of the time-frequency transformation. The echo-canceling time-domain output signal refers to the time-domain signal obtained after beamforming and external echo suppression, which can be used by subsequent voice links.

[0075] As an optional implementation, the inverse time-frequency transform employs the inverse short-time Fourier transform. An inverse short-time Fourier transform is performed on the frequency domain beamforming output of each frame to obtain the time-domain estimated signal for that frame. Then, the time-domain estimated signals of each frame are stitched together into a continuous time-domain output signal through additive reconstruction methods such as overlapping and adding or overlapping and preserving. In the resulting time-domain output signal, external echoes and environmental noise from the 180° direction are significantly suppressed, while near-end speech from the 0° direction is preserved without distortion. This time-domain output signal can be sent to a voice call link for the distant caller to listen to, to a speech recognition link for near-end speech recognition, or to a translation engine or other subsequent processing links.

[0076] Optionally, refer to Figure 5 Before performing inverse time-frequency transformation on the beamforming output signal, the process also includes: S1501. Determine the low-frequency point in the beamforming output signal. The low-frequency point is the frequency point with a frequency lower than the low-frequency cutoff frequency. S1502. Apply a set minimum gain limit to the low-frequency points. The minimum gain limit is used to limit the degree of suppression of low-frequency speech.

[0077] Before performing inverse time-frequency transformation on the beamforming output signal, the processing unit identifies the low-frequency points in the beamforming output signal and applies a minimum gain lower limit to these low-frequency points. Low-frequency points refer to frequencies no higher than the low-frequency cutoff frequency, which is the frequency boundary used to distinguish between the low-frequency band and the regular frequency band. The minimum gain lower limit is the lowest allowable gain value for the low-frequency points, used to limit the degree of suppression of low-frequency speech. Because the energy of low-frequency speech is relatively concentrated, especially the fundamental frequency and low-frequency formants, applying excessive suppression to the low-frequency band can lead to muffled speech, loss of formants, and affect speech intelligibility and recognition rate.

[0078] Specifically, the processing unit first defines a low-frequency set, which refers to the set of frequency points whose frequencies are not higher than the low-frequency cutoff frequency, denoted as F_low={f|0≤f≤f_low}, where f_low represents the low-frequency cutoff frequency. As an optional implementation, the low-frequency cutoff frequency is 1200 Hz, but other values ​​within the range of 1000 Hz to 1500 Hz can also be used. For frequency points belonging to the low-frequency set, the gain of their frequency domain beamforming output is not lower than a preset minimum gain lower limit, which ranges from 0.50 to 0.60.

[0079] For low-frequency points, the processing unit calculates the current beamforming gain, which is the amplitude ratio between the frequency-domain beamforming output and the frequency-domain observation vector. If the current beamforming gain is lower than the minimum gain lower limit, the gain is increased to the minimum gain lower limit; if the current beamforming gain is not lower than the minimum gain lower limit, the current gain remains unchanged. The processed frequency-domain beamforming output is calculated according to the following formula: Y'(f,m)=Y(f,m)·G(f,m). Where Y'(f,m) represents the processed frequency-domain beamforming output at frequency point f in frame m, Y(f,m) represents the processed frequency-domain beamforming output at frequency point f in frame m, and G(f,m) represents the gain at frequency point f in frame m. For low-frequency points, G(f,m)≥G_min,low is satisfied, where G_min,low represents the minimum gain lower limit. For frequencies higher than the low-frequency cutoff frequency, no low-frequency minimum gain lower limit is applied; instead, conventional beamforming is used to suppress gain, thus maintaining sufficient suppression of external echo and environmental noise in the mid-to-high frequency bands.

[0080] The above describes a process involving acquiring multiple time-domain microphone signals from a linear microphone array, performing time-frequency transformation on these signals to obtain frequency-domain observation vectors for each frequency point, determining steering vectors for the target speech direction based on the array geometry parameters of the linear microphone array and the preset target speech direction, determining a state indicator value representing the current near-end speech activity level, adjusting the recursive update rate of the covariance matrix based on the state indicator value (the state indicator value is inversely proportional to the recursive update rate), recursively estimating the covariance matrix of noise and echo based on the recursive update rate and the frequency-domain observation vectors, calculating minimum variance distortionless response weights based on the covariance matrix and steering vectors, weighting and merging the frequency-domain observation vectors according to the minimum variance distortionless response weights to obtain the beamforming output signal, and performing an inverse time-frequency transformation on the beamforming output signal to obtain the echo-free time-domain output signal. By employing the aforementioned technical means, and by introducing the spatial orientation information of the linear microphone array and a recursive update rate mechanism that adaptively adjusts according to the activity level of near-end speech, the minimum variance distortionless response weight maintains a distortionless response in the direction of the target speech and suppresses leakage in the direction of external speaker echo. At the same time, it prevents the target speech from entering the noise and echo model, thereby achieving the separation of external speaker echo and near-end speech. This improves the echo suppression depth and near-end speech protection capability in scenarios with strong external speaker leakage, and meets the requirements of external open-back headphones for the stability of echo suppression effect in call and speech recognition scenarios.

[0081] This application also provides an electronic device, see embodiments thereof. Figure 6The electronic device includes a processor 21, a memory 22, and a communication module 23. The electronic device may have one or more processors and one or more memories. The processor, memory, and communication module of the electronic device can be connected via a bus or other means.

[0082] Memory, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the echo cancellation method based on a linear microphone array described in any embodiment of this application. Memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the device, etc. Furthermore, memory may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0083] The communication module is used for data transmission.

[0084] The processor executes various functional applications and data processing of the device by running software programs, instructions, and modules stored in memory, thereby realizing the echo cancellation method based on the linear microphone array described above.

[0085] The electronic device provided above can be used to execute the echo cancellation method based on a linear microphone array provided in the above embodiments, and has the corresponding functions and beneficial effects.

[0086] This application embodiment also provides a storage medium containing computer-executable instructions. When executed by a computer processor, the computer-executable instructions are used to execute an echo cancellation method based on a linear microphone array. The echo cancellation method based on a linear microphone array includes: acquiring multiple time-domain microphone signals collected by a linear microphone array; performing time-frequency transformation on the multiple time-domain microphone signals to obtain frequency-domain observation vectors at each frequency point; determining a steering vector corresponding to the target speech direction based on the array geometric parameters of the linear microphone array and a preset target speech direction; determining a state indicator value representing the current near-end speech activity level; adjusting the recursive update rate of the covariance matrix according to the state indicator value, wherein the state indicator value is inversely proportional to the recursive update rate; recursively estimating the covariance matrix of noise and echo based on the recursive update rate and the frequency-domain observation vectors; calculating the minimum variance distortionless response weights based on the covariance matrix and the steering vectors; weighting and merging the frequency-domain observation vectors according to the minimum variance distortionless response weights to obtain a beamforming output signal; and performing an inverse time-frequency transformation on the beamforming output signal to obtain a time-domain output signal after echo cancellation.

[0087] Storage medium – any type of memory device or storage device. The term “storage medium” is intended to include: mounting media, such as CD-ROM, floppy disk, or magnetic tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. Storage medium may also include other types of memory or combinations thereof. Furthermore, storage medium may reside in a first computer system in which the program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet). The second computer system can provide program instructions to the first computer for execution. The term “storage medium” can include two or more storage media residing in different locations (e.g., in different computer systems connected via a network). Storage medium may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0088] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the echo cancellation method based on a linear microphone array as described above, but can also perform related operations in the echo cancellation method based on a linear microphone array provided in any embodiment of this application.

[0089] Based on the above embodiments, this application also provides a computer program product. The technical solution of this application, in essence or in other words, the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes a computer program. The computer program is used to cause a computer device, mobile terminal, or processor therein to execute all or part of the steps of the echo cancellation method based on linear microphone array of various embodiments of this application.

[0090] The storage medium, electronic device, and computer program product provided in the above embodiments can execute the echo cancellation method based on a linear microphone array provided in any embodiment of this application. For technical details not described in detail in the above embodiments, please refer to the echo cancellation method based on a linear microphone array provided in any embodiment of this application.

[0091] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. A method of echo cancellation based on a linear microphone array, characterized in that, include: Acquire multiple time-domain microphone signals collected by a linear microphone array, perform time-frequency transformation on the multiple time-domain microphone signals, and obtain the frequency domain observation vector of each frequency point; Based on the array geometry parameters of the linear microphone array and the preset target speech direction, a steering vector corresponding to the target speech direction is determined; A state indicator value representing the current level of near-field speech activity is determined, and the recursive update rate of the covariance matrix is ​​adjusted according to the state indicator value, wherein the state indicator value is inversely proportional to the recursive update rate. Based on the recursive update rate and the frequency domain observation vector, the covariance matrix of noise and echo is recursively estimated, and the minimum variance distortionless response weight is calculated based on the covariance matrix and the steering vector. The frequency domain observation vector is weighted and merged according to the minimum variance distortionless response weight to obtain the beamforming output signal. The beamforming output signal is then subjected to inverse time-frequency transformation to obtain the echo-free time domain output signal.

2. The echo cancellation method based on a linear microphone array according to claim 1, characterized in that, The determination of the state indicator value representing the current level of near-field speech activity includes: Determine the current near-end speaking probability and the two-way speaking probability, and take the larger value between the near-end speaking probability and the two-way speaking probability as the state indication value.

3. The linear microphone array based echo cancellation method according to claim 2, characterized in that, Determining the current near-end speaking probability and two-way speaking probability includes: Based on the signal energy of the current multi-channel time-domain microphone signals, the voice activity detection results, and the correlation between the playback reference signal and the multi-channel time-domain microphone signals, the near-end speaking probability and the dual-talk probability are jointly estimated.

4. The linear microphone array based echo cancellation method of claim 1, wherein, The step of adjusting the recursive update rate of the covariance matrix based on the state indication value includes: Obtain the preset baseline recursive coefficient and adjustment coefficient; The target recursive coefficient is calculated based on the baseline recursive coefficient, the adjustment coefficient, and the state indication value. The recursive update rate of the covariance matrix is ​​determined based on the target recursive coefficient. The target recursive coefficient increases and the recursive update rate decreases when the state indication value increases.

5. The linear microphone array based echo cancellation method according to claim 4, characterized in that, The process of obtaining the preset baseline recursive coefficients and adjustment coefficients includes: Determine the parameter group corresponding to the current external speaker volume level, and obtain the preset reference recursive coefficient and adjustment coefficient from the parameter group.

6. The linear microphone array based echo cancellation method of claim 5, wherein, The parameter set also includes diagonal loading coefficients; After recursively estimating the covariance matrix of noise and echo based on the recursive update rate and the frequency domain observation vector, the method further includes: The updated covariance matrix is ​​diagonally loaded according to the diagonal loading coefficients, and the diagonal loading coefficients are added as constant terms to each main diagonal element of the covariance matrix. The diagonal loading coefficients are used to correct the covariance matrix.

7. The linear microphone array based echo cancellation method of claim 5, wherein, The method further includes: When the external speaker volume level is switched, the parameter value of the current parameter group is smoothly transitioned to the value of the target level within a preset transition time.

8. The linear microphone array based echo cancellation method according to any one of claims 1-7, characterized by, Before performing the inverse time-frequency transformation on the beamforming output signal, the method further includes: Determine the low-frequency point in the beamforming output signal, wherein the low-frequency point is a frequency point with a frequency lower than the low-frequency cutoff frequency; A set minimum gain limit is applied to the low-frequency point, which is used to limit the degree of suppression of low-frequency speech.

9. The linear microphone array based echo cancellation method according to any one of claims 1-7, characterized by, The step of performing time-frequency transformation on the multi-channel time-domain microphone signals to obtain the frequency domain observation vector for each frequency point includes: The multi-channel time-domain microphone signals are subjected to frame-segmentation and windowing processing to obtain windowed time-domain frame signals. A short-time Fourier transform is performed on the time-domain frame signal to obtain the frequency-domain observation vector for each frequency point.

10. The linear microphone array based echo cancellation method according to any one of claims 1-7, characterized by, The linear microphone array includes three microphones, which are equally spaced apart. The target speech direction is 0°, and the corresponding speaker interference direction is 180°. The 0° direction and the 180° direction are located on the same axis of the linear microphone array.

11. An electronic device, comprising: include: Memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the echo cancellation method based on a linear microphone array as described in any one of claims 1-10.

12. A storage medium containing computer-executable instructions, wherein: The computer-executable instructions, when executed by a computer processor, are used to perform the echo cancellation method based on a linear microphone array as described in any one of claims 1-10.

13. A computer program product, characterised in that, The computer program product includes a computer program that, when executed by a processor, implements the echo cancellation method based on a linear microphone array as described in any one of claims 1-10.