Method for eliminating echo of intercom system

Abstract

The invention provides a method for eliminating echo of a talkback system, which comprises the following steps of: according to separated near-end speech evaluation and a plurality of recognized speaker speech segments, respectively performing residual echo suppression, constructing binary time-frequency masking and reducing residual echo spectrum caused by dynamic change of echo power; the phase information of the original mixed signal is combined with the enhanced near-end voice power spectrum, the voice waveform of each speaker is reconstructed through inverse short-time Fourier transform, and transition between voice frames is smoothed in an overlap-add mode; whether voice exists at the near end and the far end or not is judged, if the double-talk state is detected, parameters of a nonlinear echo cancellation algorithm are adjusted, and the situation that the voice at the near end is mistakenly suppressed is avoided; and carrying out transmission path modeling on the processed near-end voice signal, and evaluating propagation characteristics of the voice signal from a speaker to a microphone, including time delay and attenuation of direct sound and early reflected sound.

Description

Technical Field

[0001] The invention relates to the field of information technology, and in particular to a method for eliminating echo of an intercom system. Background Art

[0002] In a multi-channel intercom system, the separation of near-end and far-end signals is a difficult technical problem. The core of this problem is that when multiple users speak at the same time, the near-end microphone captures the voice of the local speaker and the sound played by the far-end speaker at the same time. There is a serious overlap between these two signals in the time domain and the frequency domain, making it difficult for the traditional single filtering method to effectively distinguish and separate them. Specifically, the spectral characteristics of the local speaker picked up by the near-end microphone and the sound played by the far-end speaker are very similar. They both contain the basic frequency components and harmonic structures of the human voice, making it difficult to distinguish them in the frequency domain. At the same time, the two signals are also highly overlapped in time, with no obvious time interval or demarcation point. This double overlap in the time domain and frequency domain makes it difficult for a single filtering method based on frequency features or time features to work. What's more complicated is that in actual application scenarios, factors such as the speaker's position, volume, and timbre are constantly changing, and the transmission path and reverberation characteristics of the far-end signal will also change with the environment. These dynamic changes further increase the difficulty of signal separation. Traditional methods often find it difficult to adapt to such complex and changing situations, and are prone to signal distortion or insufficient separation. Therefore, how to accurately separate near-end and far-end signals under such highly overlapping and dynamically changing conditions while maintaining the naturalness and integrity of the signals has become a key technical problem that needs to be solved in multi-channel intercom systems. Solving this problem is crucial to improving the system's call quality and user experience. Summary of the invention

[0003] The present invention provides a method for eliminating echo of an intercom system, which mainly comprises:

[0004] According to the mixed signal collected by the near-end microphone array, the speaker position is evaluated, the target speaker's voice is enhanced, the enhanced near-end voice signal is obtained, the timbre features are extracted, the speaker's timbre model is established, the reverberation of the far-end reference signal is evaluated, the room impulse response is determined, the reverberation characteristic parameters of the far-end signal are obtained, and the near-end voice signal is preliminarily processed according to the reverberation characteristic parameters and the far-end signal, and the evaluated echo component is subtracted;

[0005] The spectrum envelope of the speech signal after preliminary processing is tracked to calculate the energy distribution of each time-frequency point. According to the energy dynamic range of the far-end signal leakage echo, the speech-dominant area and the echo-dominant area are adaptively distinguished to model the near-end speech and residual echo as independent signal sources. The separation matrix is ​​evaluated by maximizing the negative entropy, and the speech signal is linearly transformed to obtain the separated near-end speech evaluation.

[0006] According to the near-end speech evaluation after separation, the multiple identified speaker speech segments are subjected to residual echo suppression respectively, and a binary time-frequency mask is constructed to reduce the residual echo spectrum caused by the dynamic change of the echo power;

[0007] The phase information of the original mixed signal is combined with the enhanced near-end speech power spectrum to reconstruct the speech waveform of each speaker through inverse short-time Fourier transform, and the overlap-add method is used to smooth the transition between speech frames.

[0008] Post-process the reconstructed near-end speech signal, smooth the signal through adaptive Wiener filtering, dynamically adjust the frequency response of the filter according to the evaluated background noise power spectrum, and suppress the music noise in two-way communication;

[0009] Determine whether there is voice at both the near-end and far-end. If a double-talk state is detected, adjust the parameters of the nonlinear echo cancellation algorithm to prevent the near-end voice from being incorrectly suppressed.

[0010] Model the transmission path of the processed near-end speech signal to evaluate the propagation characteristics of the speech signal from the speaker to the microphone, including the delay and attenuation of the direct sound and early reflection sound;

[0011] According to the transmission path model, the near-end speech signal is dedeverberated to suppress the reverberation effect caused by room reflection, thereby obtaining a near-end speech signal with improved clarity, and the noise reduction process is performed to obtain a clear speech signal with improved signal-to-noise ratio.

[0012] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0013] The present invention provides a method for eliminating echo of an intercom system. By processing a mixed signal, the position of a speaker can be accurately evaluated, thereby enhancing the speech signal of a target speaker and improving the intelligibility and quality of the speech. By evaluating the impulse response of a room and the reverberation characteristics of a far-end signal, the echo component can be effectively subtracted from the near-end speech signal, thereby reducing the echo problem in two-way communication. By adopting spectrum envelope tracking and independent signal source modeling, the residual echo can be further removed through an optimized separation matrix to improve the purity of the speech. By using phase information and an enhanced speech power spectrum, a clear speech waveform is reconstructed through an inverse short-time Fourier transform, thereby ensuring a natural transition between speech frames and enhancing the coherence of the speech. The adaptive Wiener filtering technology can dynamically adjust the frequency response, effectively suppress background noise and music noise in two-way communication, and improve the clarity of the speech signal. When a two-way talk state is detected, the echo elimination algorithm parameters can be intelligently adjusted to prevent the near-end speech from being mistakenly suppressed, thereby maintaining the integrity of the speech of both parties.

[0014] Evaluate the propagation characteristics of speech signals, and suppress the reverberation effect caused by room reflections through dereverberation processing to further improve the clarity of speech signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The present invention is a flow chart of a method for eliminating echo of an intercom system.

[0016] Figure 2 A schematic diagram of a method for eliminating echo of an intercom system according to the present invention.

[0017] Figure 3 It is another schematic diagram of a method for eliminating echo of an intercom system according to the present invention. DETAILED DESCRIPTION

[0018] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] like Figure 1-3 In this embodiment, a method for eliminating echo of an intercom system may specifically include:

[0020] S101. According to the mixed signal collected by the near-end microphone array, the speaker position is evaluated, the target speaker's voice is enhanced, the enhanced near-end voice signal is obtained, the timbre features are extracted, a speaker timbre model is established, the reverberation of the far-end reference signal is evaluated, the room impulse response is determined, the reverberation characteristic parameters of the far-end signal are obtained, and the near-end voice signal is preliminarily processed according to the reverberation characteristic parameters and the far-end signal, and the evaluated echo component is subtracted.

[0021] According to the mixed signal collected by the near-end microphone array, the speaker position is evaluated by the beamforming algorithm, and the target speaker's voice is enhanced by the beamforming technology to obtain the enhanced near-end voice signal. The timbre characteristics of the enhanced near-end voice signal are extracted by using the Mel frequency cepstrum coefficient, and a Gaussian mixture model is established as the speaker timbre model. The far-end reference signal is subjected to time-frequency analysis, the reverberation time and early decay rate are calculated, the room impulse response is determined, and the reverberation characteristic parameters of the far-end signal are obtained. According to the reverberation characteristic parameters and the far-end signal, the near-end voice signal is subjected to envelope modulation filtering, the signal amplitude is adjusted to match the reverberation characteristics of the far-end signal, and the evaluated echo component is subtracted to obtain the near-end voice signal after preliminary processing. The near-end voice signal after preliminary processing is subjected to echo elimination by using an adaptive filter, the filter coefficient is updated by using the minimum mean square error criterion, and the double echo is suppressed by nonlinear processing to obtain the final enhanced near-end voice signal. According to the speaker timbre model, the timbre of the final enhanced near-end voice signal is verified to determine whether the signal enhancement process introduces obvious timbre distortion.

[0022] Specifically, the mixed signal is collected according to the near-end microphone array, the speaker position is evaluated by the beamforming algorithm, the target speaker's voice is enhanced by the beamforming technology, the enhanced near-end voice signal is obtained, the timbre features are extracted by the Mel-frequency cepstrum coefficients, and a Gaussian mixture model is established as the speaker's timbre model. The far-end reference signal is subjected to time-frequency analysis, the reverberation time and early decay rate are calculated, the room impulse response is determined, the reverberation characteristic parameters of the far-end signal are obtained, and the near-end voice signal is envelope modulated and filtered according to the reverberation characteristic parameters and the far-end signal. The signal amplitude is adjusted to match the reverberation characteristics of the far-end signal, and the evaluated echo component is subtracted. The voice activity detection algorithm is used to distinguish the voice segment and the non-voice segment in the near-end voice signal, and the cepstrum subtraction method is applied to the voice segment to eliminate the residual echo. Specifically, the cepstrum of the voice signal is calculated, the evaluated echo cepstrum is subtracted, and then the signal is reconstructed. The non-voice segment is subjected to noise suppression to obtain the near-end voice signal after preliminary processing. The near-end speech signal after preliminary processing is further echo-cancelled by using an adaptive filter. The filter coefficients are updated using the minimum mean square error criterion. Double echoes are suppressed by nonlinear processing. Specifically, the residual echo is processed by half-wave rectification or full-wave rectification to obtain the final enhanced near-end speech signal. According to the speaker timbre model established earlier, the timbre of the final enhanced near-end speech signal is verified to ensure that no obvious timbre distortion is introduced during the signal enhancement process. The near-end microphone array collects mixed signals. Eight omnidirectional microphones are used to form a circular array with a spacing of 4 cm. The beamforming algorithm uses the delay and sum method to calculate the energy of the sound source in different directions and determine the direction with the maximum energy as the speaker's position. The beamforming technology uses the minimum variance distortionless response algorithm, sets the gain value to 1, the suppression coefficient to 0.1, and iterates 50 times to obtain the enhanced near-end speech signal. When extracting timbre features, the signal is pre-emphasized with a coefficient of 0.97, a frame length of 25 milliseconds, and a frame shift of 10 milliseconds. A Hamming window is applied to calculate the 13th-order Mel-frequency cepstrum coefficients. The Gaussian mixture model contains 32 mixture components and is trained for 100 iterations using the expectation maximization algorithm. The far-end reference signal is subjected to short-time Fourier transform with a window length of 512 points and an overlap rate of 50%. The energy decay curve is calculated to obtain a reverberation time T60 of 0.5 seconds and an early decay rate of 10 dB. The envelope modulation filter uses an exponential decay function with a decay rate that matches T60. The voice activity detection uses an energy threshold method with a threshold set to -20 dB. In the cepstrum subtraction method, the logarithmic amplitude spectrum of a 25 millisecond frame length is calculated, and the cepstrum coefficients are obtained by discrete cosine transform. The signal is reconstructed by inverse transform after subtracting the evaluated echo cepstrum. Noise suppression uses spectral subtraction with an over-subtraction factor of 1.5. The adaptive filter uses the normalized minimum mean square error algorithm with a filter length of 1024 and a step size of 0.1. Nonlinear processing uses half-wave rectification with a threshold of -40 dB.The timbre verification calculates the Mel-frequency cepstral coefficients before and after processing, and measures them by cosine similarity. The threshold is set to 0.85, and the timbre is considered to be well maintained if it is higher than this value.

[0023] S102. Perform spectral envelope tracking on the speech signal after preliminary processing, calculate the energy distribution of each time-frequency point, adaptively distinguish the speech-dominant area and the echo-dominant area according to the energy dynamic range of the far-end signal leakage echo, model the near-end speech and residual echo as independent signal sources, and perform linear transformation on the speech signal by maximizing the negative entropy evaluation separation matrix to obtain the separated near-end speech evaluation.

[0024] The speech signal after preliminary processing is subjected to short-time Fourier transform to obtain a time-frequency spectrum of the speech signal; according to the time-frequency spectrum, the spectrum envelope is tracked by using the cepstrum analysis method, the energy distribution of each time-frequency point is calculated, and an energy distribution matrix is ​​obtained; according to the energy dynamic range of the far-end signal leakage echo, the threshold is dynamically adjusted based on the sliding average value of the far-end signal energy, the area above the threshold in the energy distribution matrix is ​​marked as the speech-dominated area, and the area below the threshold is marked as the echo-dominated area, and a time-frequency masking map is generated; the time-frequency masking map information is applied to the independent component analysis process, the signal is divided into two sources of speech and echo according to the time-frequency masking map, and a mixing matrix is ​​constructed; the near-end speech and residual echo are modeled as independent signal sources, the separation matrix is ​​evaluated by using the independent component analysis algorithm, and the separation matrix parameters are optimized by maximizing the negative entropy criterion; the speech signal is linearly transformed, the evaluated separation matrix is ​​multiplied with the initial mixed signal, and the separated signal is reconstructed by using a Wiener filter in combination with the time-frequency masking map to obtain a separated near-end speech evaluation signal.

[0025] Specifically, the speech signal after preliminary processing is subjected to short-time Fourier transform to obtain a time-frequency spectrum, and the spectrum envelope is tracked by cepstrum analysis to calculate the energy distribution of each time-frequency point and obtain an energy distribution matrix. According to the energy dynamic range of the far-end signal leakage echo, the threshold is dynamically adjusted based on the sliding average of the far-end signal energy, and the area above the threshold in the energy distribution matrix is ​​marked as the speech-dominated area, and the area below the threshold is marked as the echo-dominated area, and a time-frequency masking map is generated. The time-frequency masking map information is applied to the independent component analysis process, and the signal is divided into two sources, speech and echo, according to the time-frequency masking map, and a 2x2 mixing matrix is ​​constructed. The near-end speech and residual echo are modeled as independent signal sources, and the separation matrix is ​​evaluated by the independent component analysis algorithm, and the separation matrix parameters are optimized by maximizing the negative entropy criterion. The speech signal is linearly transformed, the evaluated separation matrix is ​​multiplied with the initial mixed signal, and the separated signal is reconstructed by using the Wiener filter combined with the time-frequency masking map to obtain the separated near-end speech evaluation signal. The speech signal after preliminary processing is subjected to short-time Fourier transform, using a Hamming window with a window length of 512 points and an overlap rate of 50% to obtain a time-frequency spectrum. The cepstrum analysis method is used to track the spectrum envelope. The logarithmic amplitude spectrum is first calculated, and then Fourier transform is performed. The first 13 coefficients are taken as cepstrum coefficients, and a smooth spectrum envelope is obtained by inverse transform. The energy distribution of each time-frequency point is calculated to obtain an energy distribution matrix. According to the energy dynamic range of the far-end signal leakage echo, the threshold is dynamically adjusted based on the sliding average of the far-end signal energy. The average energy is calculated using a 200ms sliding window, and the initial threshold is set to -20dB. The threshold is updated every frame. The area above the threshold in the energy distribution matrix is ​​marked as the speech-dominated area, and the area below the threshold is marked as the echo-dominated area, and a binary time-frequency masking map is generated. The time-frequency masking map information is applied to the independent component analysis process. According to the time-frequency masking map, the signal is divided into two sources, speech and echo, and a 2x2 mixing matrix is ​​constructed. The near-end speech and residual echo are modeled as independent signal sources. The separation matrix is ​​evaluated using the Fast I CA algorithm. The separation matrix parameters are optimized by maximizing the negative entropy criterion. The maximum number of iterations is set to 100 and the convergence threshold is set to 1e-6. The speech signal is linearly transformed, the evaluated separation matrix is ​​multiplied with the initial mixed signal, and the separated signal is reconstructed using the Wiener filter combined with the time-frequency masking map. The noise evaluation window length of the Wiener filter is set to 20 frames, and the smoothing factor is set to 0.98 to obtain the separated near-end speech evaluation signal.

[0026] S103, according to the separated near-end speech evaluation, the identified multiple speaker speech segments are subjected to residual echo suppression respectively, a binary time-frequency mask is constructed, and a residual echo spectrum caused by dynamic changes in echo power is reduced.

[0027] A Gaussian mixture model based on Mel-frequency cepstral coefficients is used to process speech signals in segments, and speech segments of multiple speakers are identified, wherein the speech segments contain speaker identity and time range information; residual echo suppression is performed according to the speaker identity information of the speech segments, and the echo component is subtracted from the power spectrum density of the near-end speech signal to obtain the speech signal after echo suppression; short-time energy and zero-crossing rate are calculated for the speech signal after echo suppression, and the speech signal is normalized using the long-term average spectrum of the speech signal, and an adaptive threshold is set to determine the speech activity area; if the speech activity area is determined, a binary time-frequency masking matrix is ​​constructed, and the speech activity area is marked as 1 and the non-speech activity area is marked as 0; a Wiener filter is used in combination with the binary time-frequency masking matrix to perform spectrum enhancement, and the residual echo spectrum caused by the dynamic change of the echo power is reduced to obtain an enhanced near-end speech signal; the parameters of the Wiener filter are adjusted by monitoring the short-time change of the echo power to further optimize the enhanced near-end speech signal.

[0028] Specifically, according to the evaluation of the near-end speech after separation, the Gaussian mixture model based on the Mel-frequency cepstral coefficient is used to segment the speech signal, identify the speech segments of multiple speakers, and mark the time range and speaker identity of each segment. At the same time, the overlapping speech is processed and the different speakers are distinguished by the sound source separation algorithm. For each identified speech segment, the adaptive filter and spectral subtraction are combined to suppress the residual echo. By evaluating the power spectrum density of the echo signal, the echo component is subtracted from the power spectrum density of the near-end speech signal. The adaptive filter parameters are adjusted according to the speaker identity information. According to the speech segment after residual echo suppression, the short-time energy and zero-crossing rate are calculated, and the long-term average spectrum of the speech signal is used for normalization. The adaptive threshold is set, the speech activity area is determined, and a binary time-frequency masking matrix is ​​constructed. The speech activity area is marked as 1 and the non-speech activity area is marked as 0. The constructed binary time-frequency masking matrix is ​​used to perform spectral weighting on the speech signal after residual echo suppression. The Wiener filter combined with binary time-frequency masking is used for spectral enhancement to reduce the residual echo spectrum caused by the dynamic change of echo power. The filter parameters are adjusted by monitoring the short-term change of echo power to obtain the final enhanced near-end speech signal. According to the separated near-end speech evaluation, the 25 millisecond frame length and 10 millisecond frame shift are used for frame processing to extract the 13-dimensional Mel-frequency cepstral coefficient features. The Gaussian mixture model of 32 Gaussian mixture components is used to perform speaker recognition on the speech signal, identify the speech segments of multiple speakers, and mark the time range and speaker identity of each segment. For overlapping speech, the independent component analysis algorithm is used for sound source separation, and the maximum number of iterations is set to 100 and the convergence threshold is 1e-6. For each identified speech segment, the adaptive filter and spectral subtraction are combined to perform residual echo suppression. The adaptive filter uses the normalized minimum mean square error algorithm, the filter length is 1024, the step size is 0.1, and the spectral subtraction over-reduction factor is set to 1.5. According to the speaker identity information, the initial coefficients of the adaptive filter are adjusted. The short-time energy and zero-crossing rate of the speech segment are calculated, the frame length is 20 milliseconds, the overlap rate is 50%, and the long-term average spectrum of the speech signal of the 500 millisecond window is used for normalization. The initial value of the energy adaptive threshold is set to -20dB, the zero-crossing rate threshold is 0.3, and a binary time-frequency masking matrix is ​​constructed. Using the binary time-frequency masking matrix, the Wiener filter is used for spectrum enhancement, the noise power evaluation window length is 20 frames, and the smoothing factor is 0.98. By monitoring the short-term changes in the echo power, the Wiener filter parameters are updated every 50 milliseconds to reduce the residual echo spectrum caused by the dynamic changes in the echo power, and finally an enhanced near-end speech signal is obtained.

[0029] S104, using the phase information of the original mixed signal in combination with the enhanced near-end speech power spectrum, reconstructing the speech waveform of each speaker through inverse short-time Fourier transform, and smoothing the transition between speech frames by overlapping and adding.

[0030] The original mixed signal is subjected to short-time Fourier transform to obtain phase information of each time-frequency point; an independent component analysis algorithm is applied to the enhanced near-end speech signal according to the phase information to obtain a speech power spectrum of multiple speakers; the original signal phase information is combined with the speech power spectrum, and a complete time-frequency domain representation is constructed by a time-frequency masking method; an inverse short-time Fourier transform is performed on the time-frequency domain representation to obtain a time domain signal; an overlap-add method is used to perform inter-frame smoothing on the time domain signal, and adjacent frames are weightedly superimposed by setting an overlap rate and using a Hanning window function; if the time domain signal contains speech of multiple speakers, weighted superposition is performed according to the energy ratio of each speaker to obtain a final reconstructed complete speech waveform.

[0031] Specifically, the original mixed signal is subjected to short-time Fourier transform to extract the phase information of each time-frequency point. Meanwhile, the enhanced near-end speech signal is subjected to independent component analysis algorithm to separate the multi-speaker speech and obtain the speech power spectrum of each speaker. The extracted original signal phase information is combined with the separated speech power spectrum, and the time-frequency masking method is used to construct a complete time-frequency domain representation for each speaker. The combined time-frequency domain representation is subjected to inverse short-time Fourier transform to convert the signal from the frequency domain back to the time domain. The dynamic time warping algorithm is used to handle the time alignment problem of the speech of different speakers, and the preliminary reconstructed speech waveform of each speaker is obtained. The overlap-add method is used to perform inter-frame smoothing on the preliminary reconstructed speech waveform, and the overlap rate is set to 50%. The adjacent frames are weighted superimposed using the Hanning window function to eliminate the frame boundary effect. The reconstructed speech of multiple speakers is finally synthesized, and weighted superposition is performed according to the energy ratio of each speaker to obtain the final reconstructed complete speech waveform. The original mixed signal is subjected to short-time Fourier transform, using 25 millisecond frame length, 10 millisecond frame shift, Hamming window, and 512-point FFT to extract the phase information of each time-frequency point. The Fast ICA algorithm is applied to the enhanced near-end speech signal for multi-speaker speech separation, with the maximum number of iterations set to 100 and the convergence threshold set to 1e-6. Assuming there are three speakers, the speech power spectrum of each speaker is obtained. The original signal phase information is combined with the separated speech power spectrum, and the binary time-frequency masking method is used. The energy threshold is set to -40dB to construct a complete time-frequency domain representation for each speaker. A 512-point I FFT is performed on the combined time-frequency domain representation to convert the signal from the frequency domain back to the time domain. The dynamic time warping algorithm is used to handle the time alignment problem of the speech of different speakers. The window size is set to 100 milliseconds and the step size is 10 milliseconds to obtain the preliminary reconstructed speech waveform of each speaker. The overlap-add method is used to smooth the initial reconstructed speech waveform between frames, and the overlap rate is set to 50%. The 32-millisecond long Hanning window function is used to weightedly superimpose adjacent frames to eliminate the frame boundary effect. The reconstructed speech of multiple speakers is finally synthesized, the short-time energy of each speaker's speech is calculated, and the average value of the 200-millisecond window is taken as the weight for weighted superposition to obtain the final reconstructed complete speech waveform.

[0032] S105, post-processing the reconstructed near-end speech signal, smoothing the signal through adaptive Wiener filtering, dynamically adjusting the frequency response of the filter according to the evaluated background noise power spectrum, and suppressing music noise in two-way communication.

[0033] The reconstructed near-end speech signal is subjected to short-time Fourier transform to obtain a time-frequency representation of the signal; based on the time-frequency representation, the power spectrum of the background noise is evaluated by using a recursive averaging method to obtain an initial noise power spectrum evaluation value; a priori signal-to-noise ratio is adaptively updated by a decision-oriented method, and the priori signal-to-noise ratio is used to calculate the frequency response of the Wiener filter; the Wiener filter frequency response is applied to the amplitude spectrum of the original signal, and the time domain signal is reconstructed by an inverse short-time Fourier transform to obtain a near-end speech signal after preliminary smoothing processing; music noise detection is performed on the signal after preliminary processing, and the stationarity of each frequency band is calculated by using a spectral entropy method. If the spectral entropy value is less than a preset threshold, it is determined that music noise exists; if it is determined that music noise exists, the corresponding frequency band is selectively suppressed by adjusting the gain curve of the Wiener filter; the Wiener filter coefficient is dynamically updated by monitoring the change of the short-time signal-to-noise ratio to implement an adaptive process, and the near-end speech signal after suppressing the music noise is obtained.

[0034] Specifically, the reconstructed near-end speech signal is subjected to short-time Fourier transform to obtain the time-frequency representation of the signal, and the power spectrum of the background noise is evaluated by the recursive averaging method to obtain the initial noise power spectrum evaluation value. According to the evaluated background noise power spectrum, the signal-to-noise ratio of each time-frequency point is calculated, and the decision-oriented method is used to adaptively update the prior signal-to-noise ratio. The smoothing factor is set to 0.98, and the frequency response of the Wiener filter is calculated according to the updated signal-to-noise ratio. The calculated Wiener filter frequency response is applied to the amplitude spectrum of the original signal, and the phase information of the original signal is retained to obtain the enhanced speech spectrum. The time domain signal is reconstructed by inverse short-time Fourier transform to obtain the near-end speech signal after preliminary smoothing. Music noise detection is performed on the signal after preliminary processing, and the spectral entropy method is used to calculate the stationarity of each frequency band. The spectral entropy threshold is set to 0.6. If music noise is detected, the corresponding frequency band is selectively suppressed by adjusting the gain curve of the Wiener filter. The parameters of the Wiener filter are dynamically adjusted, and the filter coefficients are updated by monitoring the changes in the short-time signal-to-noise ratio to achieve the adaptive process, and finally the near-end speech signal after suppressing the music noise is obtained. The reconstructed near-end speech signal is subjected to short-time Fourier transform, using 25 millisecond frame length, 10 millisecond frame shift, Hamming window, and 512-point FFT to obtain the time-frequency representation of the signal. The power spectrum of the background noise is evaluated using the recursive averaging method, the smoothing factor is set to 0.95, and the noise evaluation is updated every 50 frames. The posterior signal-to-noise ratio of each time-frequency point is calculated, and the decision-oriented method is used to adaptively update the prior signal-to-noise ratio, the smoothing factor is set to 0.98, and the frequency response of the Wiener filter is calculated. The Wiener filter frequency response is applied to the amplitude spectrum of the original signal, the original phase is retained, and the time domain signal is reconstructed by 512-point I FFT. The processed signal is subjected to music noise detection, and the spectral entropy is calculated using a 32 millisecond window, and the spectral entropy threshold is set to 0.6. If music noise is detected, the Wiener filter gain is reduced by 3dB in the corresponding frequency band. The parameters of the Wiener filter are dynamically adjusted, and the short-time signal-to-noise ratio is calculated every 100 milliseconds. If the signal-to-noise ratio changes by more than 5dB, the filter coefficient is updated and the smoothing factor is adjusted to 0.9. Finally, the enhanced speech signal is over l ap-add synthesized to obtain the near-end speech signal after suppressing music noise. The delay of the entire processing process is controlled within 50 milliseconds, which is suitable for real-time two-way communication scenarios.

[0035] S106: Determine whether there is voice at both the near end and the far end at the same time. If a double talk state is detected, adjust the parameters of the nonlinear echo cancellation algorithm to prevent the near end voice from being erroneously suppressed.

[0036] The short-time energy of the near-end and far-end signals is calculated, and the threshold is dynamically adjusted according to the long-term average energy of the signal by using an adaptive threshold technology to determine whether there is voice activity at the near-end and far-end at the same time, so as to obtain a preliminary double-talk detection result. According to the preliminary double-talk detection result, the correlation coefficient of the near-end signal and the far-end signal is calculated by a cross-correlation analysis method. If the correlation coefficient exceeds a preset threshold, the double-talk state is further confirmed. According to the double-talk state, the suppression factor in the nonlinear echo cancellation algorithm is adjusted, and the adjustment range of the suppression factor is set to a preset interval to avoid the near-end voice being incorrectly suppressed. The spectrum envelope similarity of the near-end and far-end signals is compared by spectrum analysis, the spectrum distance is calculated by using the logarithmic spectrum distortion metric, and the reliability of the double-talk state is comprehensively determined by a weighted fusion method in combination with time domain and frequency domain features. The weighted fusion double-talk detection result after the weighted fusion is used to dynamically update the parameters of the nonlinear echo cancellation algorithm, including the step size of the adaptive filter and the gain control of the nonlinear processing. The step size of the adaptive filter and the gain control of the nonlinear processing are adjusted in real time according to the double-talk detection result to improve the effect of echo cancellation.

[0037] Specifically, the short-time energy of the near-end and far-end signals is calculated, and the adaptive threshold technology is used to dynamically adjust the threshold according to the long-term average energy of the signal to determine whether there is voice activity at the near-end and far-end at the same time, and obtain preliminary double-talk detection results. The cross-correlation analysis method is used, and a 20-ms time window is used to calculate the correlation coefficient of the near-end signal and the far-end signal. If the correlation coefficient exceeds 0.7, the double-talk state is further confirmed, and the processing delay is reduced by parallel calculation. According to the double-talk detection results, the suppression factor in the nonlinear echo cancellation algorithm is adjusted. In the double-talk state, an exponential decay function is used to reduce the suppression strength. The adjustment range of the suppression factor is set between 0.5 and 1 to avoid the near-end speech being incorrectly suppressed. The spectrum envelope similarity of the near-end and far-end signals is compared through spectrum analysis, and the spectrum distance is calculated using the logarithmic spectrum distortion metric. Combined with the time domain and frequency domain features, the weighted fusion method is used to comprehensively judge the reliability of the double-talk state, and the weight is dynamically adjusted according to the stability of each feature. The fused double-talk detection results are used to dynamically update the parameters of the nonlinear echo cancellation algorithm, including the step size of the adaptive filter and the gain control of the nonlinear processing. The short-time energy of the near-end and far-end signals is calculated, and the Hamming window with a frame length of 25 milliseconds and a frame shift of 10 milliseconds is used for framing, and the logarithmic energy of each frame is calculated. Adaptive threshold technology is used, and the initial threshold is set to -40dB. The threshold is updated every 50 milliseconds based on the average energy of the past 1 second, and the update coefficient is 0.9. If the energy of the near-end and far-end signals exceeds their respective thresholds at the same time, it is preliminarily determined to be a double-talk state. The cross-correlation analysis method is used, and a 20-millisecond rectangular window is used to calculate the normalized cross-correlation coefficient of the near-end signal and the far-end signal. If the maximum correlation coefficient exceeds 0.7, the double-talk state is further confirmed. Through parallel computing, the cross-correlation results are updated every 10 milliseconds to control the processing delay within 30 milliseconds. According to the double-talk detection results, the suppression factor in the nonlinear echo cancellation algorithm is adjusted, and an exponential decay function is used.

[0038] α(n)=α_min+(α_max-α_min)*exp(-n / τ), where α_min=0.5, α_max=1, τ=50, and n is the number of frames of double talk. Calculate the logarithmic spectral distortion metric LSD of the near-end and far-end signals, use 512-point FFT, take the first 257 frequency points, and set the LSD threshold to 5dB. Use the weighted fusion method, with the weight distribution of short-time energy 0.3, cross-correlation 0.4, and spectral distortion 0.3. If the fusion score exceeds 0.6, it is judged as double talk. Dynamically adjust the parameters of the nonlinear echo cancellation algorithm according to the fusion results, adjust the adaptive filter step size between 0.1 and 0.5, and the nonlinear processing gain control range is -6dB to 0dB. The total delay of the entire processing process is controlled within 50 milliseconds to meet the needs of real-time communication.

[0039] S107. Perform transmission path modeling on the processed near-end speech signal to evaluate the propagation characteristics of the speech signal from the speaker to the microphone, including the delay and attenuation of the direct sound and the early reflected sound.

[0040] Performing a short-time Fourier transform on the processed near-end speech signal to obtain a time-frequency representation of the signal, and obtaining Mel-frequency cepstral coefficients by calculating the envelope and phase information of the signal; using a minimum mean square error algorithm to separate the near-end speech signal from direct sound based on the Mel-frequency cepstral coefficients, and evaluating the time delay and attenuation coefficient of the direct sound by minimizing the mean square error function; decomposing the speech signal into multiple frequency bands using a uniformly distributed filter group, performing early reflection sound evaluation on the multiple frequency bands respectively, and identifying the main reflection path by peak detection and K-means clustering analysis; constructing a multipath propagation model based on convolution based on the time delay and attenuation coefficient of the direct sound and the main reflection path, and calculating the frequency response and phase response; calculating the mean square error and correlation coefficient of the multipath propagation model by comparing with a known room impulse response, and judging whether the mean square error exceeds a preset threshold; if the mean square error exceeds the preset threshold, returning to direct sound separation, and re-evaluating parameters until the accuracy requirement is met or the maximum number of iterations is reached.

[0041] Specifically, the processed near-end speech signal is subjected to short-time Fourier transform to obtain the time-frequency representation of the signal. By calculating the envelope and phase information of the signal, the Mel-frequency cepstrum coefficients are extracted as acoustic features for subsequent transmission path modeling and direct sound identification. The minimum mean square error algorithm is used to separate the direct sound of the near-end speech signal. The delay and attenuation coefficient of the direct sound are evaluated by minimizing the mean square error function to obtain a preliminary transmission path model, which provides a reference for the early reflection sound evaluation. The speech signal is decomposed into multiple frequency bands using a uniformly distributed filter group. The early reflection sound is evaluated for each frequency band. The main reflection path is identified by peak detection and K-means clustering analysis. The number of cluster centers is set to 3, corresponding to the three main reflection paths. According to the evaluated direct sound and early reflection sound parameters, a multipath propagation model based on convolution is constructed, the frequency response and phase response are calculated, and a complete transmission path model is obtained, including the delay and attenuation characteristics. By comparing with the known room impulse response, the mean square error and correlation coefficient are calculated to verify the accuracy of the transmission path model. If the error exceeds the preset threshold, the second step is returned to re-evaluate the parameters until the accuracy requirement is met or the maximum number of iterations is reached. The processed near-end speech signal is subjected to short-time Fourier transform, and the time-frequency representation of the signal is obtained using a 25 millisecond frame length, a 10 millisecond frame shift, a Hamming window, and a 512-point FFT. The 13th-order Mel-frequency cepstral coefficients are calculated as acoustic features for direct sound identification. The minimum mean square error algorithm is applied for direct sound separation, with the initial step size set to 0.01, adaptively adjusted to minimize the mean square error function, and iterated 200 times to evaluate the direct sound delay and attenuation coefficient. The signal is decomposed into multiple frequency bands using an 8-channel uniformly distributed filter bank with a frequency range of 0-8kHz. Peak detection is performed on each frequency band, with the threshold set to 1.5 times the average energy, and K-means clustering (K=3) is applied to identify the main reflection path. Construct a convolution model to represent multipath propagation. The time delays of the direct sound and the three main reflection paths are set to 0ms, 5ms, 10ms, and 15ms, respectively, with corresponding attenuations of 1, 0.8, 0.6, and 0.4. Calculate the 512-point frequency response and phase response to obtain the transmission path model. Compare with the pre-measured room impulse response, calculate the mean square error and correlation coefficient, set the mean square error threshold to -20dB, and the correlation coefficient threshold to 0.9. If the accuracy requirements are not met, return to the direct sound separation step for re-evaluation, with a maximum of 5 iterations. The delay of the entire processing process is controlled within 100 milliseconds, which is suitable for real-time speech enhancement systems.

[0042] S108, performing dereverberation processing on the near-end speech signal according to the transmission path model to suppress the reverberation effect caused by room reflection to obtain a near-end speech signal with improved clarity, and performing noise reduction processing to obtain a clear speech signal with improved signal-to-noise ratio.

[0043] An inverse filter is constructed according to the transmission path model, and the inverse filter uses the minimum mean square error criterion to determine the filter parameters; a time domain convolution operation is performed on the near-end speech signal to obtain a speech signal after preliminary dereverberation processing; a short-time Fourier transform is performed on the dereverberation speech signal to obtain a frequency domain representation of the speech signal; a minimum statistic method is used to evaluate the noise power spectrum, and a spectral subtraction method is applied in the frequency domain to obtain a speech power spectrum after noise reduction; a Wiener filter is used to enhance the speech signal after noise reduction, and the Wiener filter uses a decision-oriented method to evaluate the prior signal-to-noise ratio; the filter parameters are dynamically adjusted according to the evaluated signal-to-noise ratio to obtain an enhanced speech signal; an inverse short-time Fourier transform is performed on the enhanced speech signal to restore the time domain signal; an overlap-addition method is used to smooth the inter-frame transition to obtain a final clear speech signal, and the overlap-addition method uses an overlap rate of 50% between the frames; a cepstrum distance is used to calculate the degree of improvement in speech clarity of the clear speech signal; and a segmented signal-to-noise ratio calculation method is used to evaluate the improvement in the signal-to-noise ratio of the clear speech signal.

[0044] Specifically, according to the transmission path model, the minimum mean square error criterion is used to construct an inverse filter, and a time domain convolution operation is performed on the near-end speech signal to suppress the reverberation effect caused by the room reflection, and the speech signal after preliminary dereverberation processing is obtained. The dereverberated speech signal is subjected to short-time Fourier transform, and the noise power spectrum is evaluated by the minimum statistics method. The spectrum subtraction method is applied in the frequency domain to subtract the noise component from the speech power spectrum to achieve frequency domain noise reduction processing. The denoised speech signal is further enhanced by using the Wiener filter, and the prior signal-to-noise ratio is evaluated by the decision-oriented method. The filter parameters are dynamically adjusted according to the evaluated signal-to-noise ratio to improve the signal-to-noise ratio of the speech. The enhanced speech signal is subjected to an inverse short-time Fourier transform to restore the time domain signal, and the inter-frame transition is smoothed by the overlap-addition method with a 50% overlap rate to obtain the final clear speech signal. The degree of improvement in speech clarity is calculated by using the cepstrum distance, and the improvement in the signal-to-noise ratio is evaluated by using the segmented signal-to-noise ratio calculation method. The speech signals before and after processing are compared to quantify the enhancement effect. According to the transmission path model, the inverse filter is constructed using the minimum mean square error criterion. The filter order is set to 1024, the learning rate is 0.01, and the parameters are optimized by iteration 200 times. The near-end speech signal is subjected to time domain convolution operation with a frame length of 25ms and a frame shift of 10ms to suppress the reverberation effect caused by room reflection. The dedeverberated signal is subjected to a 512-point short-time Fourier transform, and the noise power spectrum is evaluated using the minimum statistics method. The search window length is set to 1.5 seconds. The spectral subtraction method is applied, and the over-subtraction factor is set to 1.2 to achieve frequency domain noise reduction. The Wiener filter is used for further enhancement, and the decision-oriented method is used to evaluate the prior signal-to-noise ratio. The smoothing factor α is set to 0.98, and the filter parameters are dynamically adjusted according to the evaluated signal-to-noise ratio. The enhanced signal is subjected to a 512-point inverse short-time Fourier transform, and a Hanning window with a 50% overlap rate is used for overlap-addition to smooth the transition between frames. The cepstral distance before and after processing is calculated, and the 13th-order Mel-frequency cepstral coefficients are taken to evaluate the degree of clarity improvement. The segmented signal-to-noise ratio is calculated using a 20ms non-overlapping window, and the average is taken to evaluate the overall signal-to-noise ratio improvement. The entire processing delay is controlled within 50ms, which is suitable for real-time voice communication systems. Finally, a voice signal with improved clarity and signal-to-noise ratio is obtained, and the enhancement effect is quantified.

[0045] The above is only a preferred embodiment of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and supplements without departing from the principle of the present invention. These improvements and supplements should also be regarded as the scope of protection of the present invention.

Claims

1. A method for eliminating echo in an intercom system, characterized in that: The method comprises: According to the mixed signal collected by the near-end microphone array, the speaker position is evaluated, the target speaker's voice is enhanced, the enhanced near-end voice signal is obtained, the timbre features are extracted, the speaker's timbre model is established, the reverberation of the far-end reference signal is evaluated, the room impulse response is determined, the reverberation characteristic parameters of the far-end signal are obtained, and the near-end voice signal is preliminarily processed according to the reverberation characteristic parameters and the far-end signal, and the evaluated echo component is subtracted; The spectrum envelope of the speech signal after preliminary processing is tracked to calculate the energy distribution of each time-frequency point. According to the energy dynamic range of the far-end signal leakage echo, the speech-dominant area and the echo-dominant area are adaptively distinguished to model the near-end speech and residual echo as independent signal sources. The separation matrix is ​​evaluated by maximizing the negative entropy, and the speech signal is linearly transformed to obtain the separated near-end speech evaluation. According to the near-end speech evaluation after separation, the multiple identified speaker speech segments are subjected to residual echo suppression respectively, and a binary time-frequency mask is constructed to reduce the residual echo spectrum caused by the dynamic change of the echo power; The phase information of the original mixed signal is combined with the enhanced near-end speech power spectrum to reconstruct the speech waveform of each speaker through inverse short-time Fourier transform, and the overlap-add method is used to smooth the transition between speech frames. Post-process the reconstructed near-end speech signal, smooth the signal through adaptive Wiener filtering, dynamically adjust the frequency response of the filter according to the evaluated background noise power spectrum, and suppress the music noise in two-way communication; Determine whether there is voice at both the near-end and far-end. If a double-talk state is detected, adjust the parameters of the nonlinear echo cancellation algorithm to prevent the near-end voice from being incorrectly suppressed. Model the transmission path of the processed near-end speech signal to evaluate the propagation characteristics of the speech signal from the speaker to the microphone, including the delay and attenuation of the direct sound and early reflection sound; According to the transmission path model, the near-end speech signal is dedeverberated to suppress the reverberation effect caused by room reflection, thereby obtaining a near-end speech signal with improved clarity, and the noise reduction process is performed to obtain a clear speech signal with improved signal-to-noise ratio.

2. The method according to claim 1, wherein: The method includes: evaluating the speaker position according to the mixed signal collected by the near-end microphone array, enhancing the target speaker's voice, obtaining the enhanced near-end voice signal, extracting the timbre characteristics, establishing the speaker's timbre model, performing reverberation evaluation on the far-end reference signal, judging the room impulse response, obtaining the reverberation characteristic parameters of the far-end signal, performing preliminary processing on the near-end voice signal according to the reverberation characteristic parameters and the far-end signal, and subtracting the evaluated echo component, including: Based on the mixed signal collected by the near-end microphone array, the speaker position is evaluated using a beamforming algorithm, and the target speaker's voice is enhanced through beamforming technology to obtain an enhanced near-end voice signal; Mel-frequency cepstral coefficients are used to extract the timbre characteristics of the enhanced near-end speech signal, and a Gaussian mixture model is established as a speaker timbre model; Perform time-frequency analysis on the far-end reference signal, calculate the reverberation time and early decay rate, determine the room impulse response, and obtain the reverberation characteristic parameters of the far-end signal; According to the reverberation characteristic parameter and the far-end signal, envelope modulation filtering is performed on the near-end speech signal, the signal amplitude is adjusted to match the reverberation characteristic of the far-end signal, and the evaluated echo component is subtracted to obtain a preliminarily processed near-end speech signal; Using an adaptive filter to perform echo cancellation on the preliminarily processed near-end speech signal, using a minimum mean square error criterion to update the filter coefficients, suppressing double echoes through nonlinear processing, and obtaining a final enhanced near-end speech signal; According to the speaker's timbre model, the timbre of the final enhanced near-end speech signal is verified to determine whether the signal enhancement process introduces obvious timbre distortion.

3. The method according to claim 1, wherein: The method includes performing spectral envelope tracking on the preliminarily processed speech signal, calculating the energy distribution of each time-frequency point, adaptively distinguishing the speech-dominant area and the echo-dominant area according to the energy dynamic range of the far-end signal leakage echo, modeling the near-end speech and the residual echo as independent signal sources, and performing linear transformation on the speech signal by maximizing the negative entropy evaluation separation matrix to obtain the separated near-end speech evaluation, including: Performing short-time Fourier transform on the preliminarily processed speech signal to obtain a time-frequency spectrum diagram of the speech signal; According to the time-frequency spectrum diagram, the spectrum envelope is tracked by using the cepstrum analysis method, and the energy distribution of each time-frequency point is calculated to obtain an energy distribution matrix; According to the energy dynamic range of the far-end signal leakage echo, dynamically adjust the threshold based on the sliding average of the far-end signal energy, mark the area above the threshold in the energy distribution matrix as the speech-dominated area, and mark the area below the threshold as the echo-dominated area, and generate a time-frequency masking map; Applying the time-frequency masking map information to an independent component analysis process, dividing the signal into two sources, speech and echo, according to the time-frequency masking map, and constructing a mixing matrix; Modeling near-end speech and residual echo as independent signal sources, evaluating a separation matrix using an independent component analysis algorithm, and optimizing the separation matrix parameters by maximizing a negative entropy criterion; The speech signal is linearly transformed, the estimated separation matrix is ​​multiplied by the initial mixed signal, and the separated signal is reconstructed using a Wiener filter in combination with the time-frequency masking graph to obtain a separated near-end speech evaluation signal.

4. The method according to claim 1, wherein: The method of performing residual echo suppression on the multiple speaker speech segments identified according to the separated near-end speech evaluation, constructing a binary time-frequency mask, and reducing the residual echo spectrum caused by the dynamic change of the echo power includes: A Gaussian mixture model based on Mel-frequency cepstral coefficients is used to segment the speech signal to identify speech segments of multiple speakers, wherein the speech segments contain speaker identity and time range information; According to the speaker identity information of the speech segment, residual echo suppression is performed in combination with an adaptive filter and a spectral subtraction method, and the echo component is subtracted from the power spectral density of the near-end speech signal to obtain a speech signal after echo suppression; For the speech signal after echo suppression, short-time energy and zero-crossing rate are calculated, long-term average spectrum of the speech signal is used for normalization, an adaptive threshold is set, and a speech activity area is determined; If a speech activity area is determined, a binary time-frequency masking matrix is ​​constructed, marking the speech activity area as 1 and the non-speech activity area as 0; The Wiener filter is combined with the binary time-frequency masking matrix to perform spectrum enhancement, reduce the residual echo spectrum caused by the dynamic change of echo power, and obtain an enhanced near-end speech signal; The enhanced near-end speech signal is further optimized by adjusting the parameters of the Wiener filter by monitoring the short-term change of the echo power.

5. The method according to claim 1, wherein: The method utilizes the phase information of the original mixed signal, combines the enhanced near-end speech power spectrum, reconstructs the speech waveform of each speaker through inverse short-time Fourier transform, and smoothes the transition between speech frames by overlapping and adding, including: Performing short-time Fourier transform on the original mixed signal to obtain phase information of each time-frequency point; Applying an independent component analysis algorithm to the enhanced near-end speech signal according to the phase information to obtain a speech power spectrum of multiple speakers; Combining the original signal phase information with the speech power spectrum, and constructing a complete time-frequency domain representation through a time-frequency masking method; Performing an inverse short-time Fourier transform on the time-frequency domain representation to obtain a time-domain signal; The overlap-add method is used to perform inter-frame smoothing processing on the time domain signal, and adjacent frames are weightedly superimposed by setting the overlap rate and using the Hanning window function; If the time domain signal contains multi-speaker speech, weighted superposition is performed according to the energy ratio of each speaker to obtain a final reconstructed complete speech waveform.

6. The method according to claim 1, wherein: The reconstructed near-end speech signal is post-processed, the signal is smoothed by adaptive Wiener filtering, and the frequency response of the filter is dynamically adjusted according to the evaluated background noise power spectrum to suppress music noise in two-way communication, including: Performing short-time Fourier transform on the reconstructed near-end speech signal to obtain a time-frequency representation of the signal; According to the time-frequency representation, the power spectrum of the background noise is evaluated by using a recursive averaging method to obtain an initial noise power spectrum evaluation value; Adopting a decision-oriented method to adaptively update a priori signal-to-noise ratio, wherein the priori signal-to-noise ratio is used to calculate the frequency response of the Wiener filter; Applying the Wiener filter frequency response to the amplitude spectrum of the original signal, reconstructing the time domain signal by inverse short-time Fourier transform, and obtaining a near-end speech signal after preliminary smoothing; Performing music noise detection on the signal after the preliminary processing, calculating the stationarity of each frequency band by using the spectral entropy method, and determining that music noise exists if the spectral entropy value is less than a preset threshold; If it is determined that music noise exists, the corresponding frequency band is selectively suppressed by adjusting the gain curve of the Wiener filter; By monitoring the short-time signal-to-noise ratio change, the Wiener filter coefficient is dynamically updated to realize the adaptive process, and the near-end speech signal after suppressing the music noise is obtained.

7. The method according to claim 1, wherein: The determining whether there are voices at both the near-end and the far-end at the same time, and if a double-talk state is detected, adjusting the parameters of the nonlinear echo cancellation algorithm to avoid the near-end voice being erroneously suppressed, includes: The short-term energy of the near-end and far-end signals is calculated, and the adaptive threshold technology is used to dynamically adjust the threshold according to the long-term average energy of the signal to determine whether there is voice activity at the near-end and far-end at the same time, and obtain preliminary double-talk detection results; According to the preliminary double-talk detection result, a correlation coefficient between a near-end signal and a far-end signal is calculated by a cross-correlation analysis method, and if the correlation coefficient exceeds a preset threshold, the double-talk state is further confirmed; According to the double-talk state, adjusting the suppression factor in the nonlinear echo cancellation algorithm, wherein the adjustment range of the suppression factor is set to a preset interval to prevent the near-end speech from being erroneously suppressed; By comparing the spectral envelope similarity of near-end and far-end signals through spectrum analysis, the spectrum distance is calculated using logarithmic spectrum distortion metric, and the reliability of the dual-talk state is comprehensively judged by combining time domain and frequency domain features and using weighted fusion method. The weighted fusion double talk detection result is used to dynamically update the parameters of the nonlinear echo cancellation algorithm, including the step size of the adaptive filter and the gain control of the nonlinear processing; The step size of the adaptive filter and the gain control of the nonlinear processing are adjusted in real time according to the double talk detection result, thereby improving the effect of echo cancellation.

8. The method according to claim 1, wherein: The transmission path modeling of the processed near-end speech signal is performed to evaluate the propagation characteristics of the speech signal from the speaker to the microphone, including the delay and attenuation of the direct sound and the early reflected sound, including: Performing short-time Fourier transform on the processed near-end speech signal to obtain a time-frequency representation of the signal, and obtaining Mel-frequency cepstrum coefficients by calculating the envelope and phase information of the signal; According to the Mel-frequency cepstral coefficients, a minimum mean square error algorithm is used to separate the direct sound from the near-end speech signal, and a time delay and an attenuation coefficient of the direct sound are evaluated by minimizing the mean square error function; Decomposing the speech signal into multiple frequency bands by using a uniformly distributed filter bank, performing early reflection sound evaluation on the multiple frequency bands respectively, and identifying the main reflection path by peak detection and K-means clustering analysis; According to the time delay and attenuation coefficient of the direct sound and the main reflection path, a convolution-based multipath propagation model is constructed to calculate the frequency response and phase response; By comparing with a known room impulse response, the mean square error and correlation coefficient of the multipath propagation model are calculated, and it is determined whether the mean square error exceeds a preset threshold; If the mean square error exceeds a preset threshold, the process returns to direct sound separation and re-evaluates the parameters until the accuracy requirement is met or the maximum number of iterations is reached.

9. The method according to claim 1, wherein: The method of performing dereverberation processing on the near-end speech signal according to the transmission path model to suppress the reverberation effect caused by room reflection to obtain a near-end speech signal with improved clarity, and performing noise reduction processing to obtain a clear speech signal with improved signal-to-noise ratio includes: Constructing an inverse filter according to the transmission path model, wherein the inverse filter uses a minimum mean square error criterion to determine filter parameters; Perform time domain convolution operation on the near-end speech signal to obtain a speech signal after preliminary dereverberation processing; Performing short-time Fourier transform on the dedeverberated speech signal to obtain a frequency domain representation of the speech signal; The minimum statistics method is used to evaluate the noise power spectrum, and the spectrum subtraction method is applied in the frequency domain to obtain the speech power spectrum after noise reduction; The denoised speech signal is enhanced by using a Wiener filter, wherein the Wiener filter uses a decision-oriented method to evaluate a priori signal-to-noise ratio; Dynamically adjusting filter parameters according to the evaluated signal-to-noise ratio to obtain an enhanced speech signal; Performing an inverse short-time Fourier transform on the enhanced speech signal to restore the time domain signal; The final clear speech signal is obtained by smoothing the transition between frames through the overlap-add method, wherein the overlap rate between the frames is 50%; Calculate the degree of improvement of speech clarity of the clear speech signal using the cepstral distance; A segmented signal-to-noise ratio calculation method is used to evaluate the improvement of the signal-to-noise ratio of the clear speech signal.

Images