Multi-channel echo suppression and false interruption control methods, devices and storage media

Abstract

This invention discloses a multi-channel echo suppression and false interruption control method, apparatus, and storage medium, aiming to improve the accuracy of echo suppression and speech recognition precision in voice interaction systems. The method includes acquiring the decoded TTS reference stream, the main channel acquisition signal, and the auxiliary microphone acquisition signal; constructing an echo spatial fingerprint based on the relationship between the main channel signal and the TTS reference stream and the auxiliary microphone signal; and determining near-end start protection information. Then, through weighted fusion, gated fusion, or normalized mapping processing, an echo interpretability is generated, and echo path updates, echo suppression, or interruption release control are performed based on the echo interpretability. This invention effectively improves the accuracy of echo suppression, avoids confusion between echo and user speech, improves the response speed and accuracy of user voice interaction, and further enhances speech recognition precision, thus improving the user's interactive experience.

Description

Technical Field

[0001] This invention relates to the field of speech processing and echo suppression technology, specifically to a multi-channel echo suppression and false interruption control method, device, and storage medium. Background Technology

[0002] With the rapid development of voice interaction technology, applications based on speech recognition, synthesis, and echo cancellation have gradually become core components of many smart devices. However, despite the progress made in speech recognition and echo cancellation, some challenges remain in practical applications, especially in handling echo and speech interruption issues in multi-channel microphone configurations.

[0003] Traditional echo cancellation techniques typically rely on a single microphone signal or simple noise suppression algorithms, offering limited effectiveness against echoes, especially in complex environments such as multi-channel audio input and near-end speech risk control. Existing echo cancellation methods still suffer from the following problems: First, in traditional echo suppression systems, echoes are often difficult to distinguish from near-end speech (i.e., the user's real-time speech), which makes it difficult for the system to accurately distinguish between the user's speech and the echo, thus affecting the accuracy of speech recognition and the user experience.

[0004] Secondly, most existing echo suppression methods are based on single-microphone signal processing, failing to fully utilize the spatial information in a multi-channel microphone configuration. Traditional techniques have failed to improve the accuracy of echo suppression by fully leveraging the spatial consistency and correlation of multi-channel acquired signals.

[0005] Secondly, voice interruptions often occur when users interact with the system. In existing technologies, when users attempt to interrupt the system, the update of the echo path is often affected, leading to misjudgment or delay, and the echo and false interruption phenomena cannot be effectively controlled.

[0006] Finally, existing echo suppression algorithms often fail to operate stably in complex environments, especially in extreme scenarios (such as high noise and rapidly changing speech input), where the echo suppression effect is poor and may even result in echo enhancement or distortion. Summary of the Invention

[0007] The purpose of this invention is to provide a multi-channel echo suppression and false interruption control method, apparatus and storage medium to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides a multi-channel echo suppression and false interruption control method, applied to a voice interaction system including a speaker, a main channel microphone, and at least one auxiliary microphone, the method comprising: Acquire the decoded TTS reference stream, the main channel acquisition signal, and the auxiliary microphone acquisition signal; Reference echo characterization information is determined based on the decoded TTS reference stream and the main channel acquisition signal. An echo space fingerprint is constructed and updated based on the auxiliary microphone acquisition signal and the main channel acquisition signal. Proximal initiation protection information is determined based on the change relationship between the main channel acquisition signal and the decoded TTS reference stream. The echo interpretation degree is determined based on the reference echo characterization information, the echo spatial fingerprint, and the near-end initiation protection information; Based on the echo interpretation degree, at least one of echo path update, echo suppression, and interruption release control is controlled.

[0009] Preferably, determining the reference echo characterization information based on the decoded TTS reference stream and the main channel acquisition signal includes: The decoded TTS reference stream is time-delay aligned with the main channel acquisition signal; Generate a main channel linear echo estimate based on the aligned decoded TTS reference stream; The reference echo characterization information is determined based on the correspondence between the main channel acquired signal and the main channel linear echo estimation.

[0010] Preferably, the step of aligning the decoded TTS reference stream with the main channel acquisition signal by time delay includes: For each candidate delay in the candidate delay set, calculate the correlation metric between the decoded TTS reference stream and the main channel acquisition signal; Candidate delays that satisfy preset conditions for the correlation metric are determined as target delays, and the decoded TTS reference stream is aligned based on the target delays.

[0011] Preferably, the echo spatial fingerprint is feature information characterizing the spatial distribution relationship of the speaker echo between the auxiliary microphone and the main channel microphone; The construction and updating of the echo space fingerprint includes: The current spatial observation is generated based on the amplitude relationship, phase relationship, or complex ratio relationship between the auxiliary microphone acquisition signal and the main channel acquisition signal, and the echo spatial fingerprint is recursively updated based on historical spatial observations.

[0012] Preferably, the recursive update of the echo spatial fingerprint includes: The echo space fingerprint is updated when the high echo confidence condition and the low near-end risk condition are met at the current moment. If there is a risk of near-end speech at the current moment, freeze the update of the echo space fingerprint.

[0013] Preferably, the near-end start protection information is used to characterize the degree of synchronization between the energy change of the main channel acquired signal and the energy change of the decoded TTS reference stream; When the energy rise of the main channel acquisition signal meets the first preset condition and the energy rise of the decoded TTS reference stream does not meet the second preset condition, the near-end start protection information indicates an increase in near-end voice risk.

[0014] Preferably, determining the echo interpretation degree based on the reference echo characterization information, the echo spatial fingerprint, and the near-end initiation protection information includes: The reference echo characterization information, the echo spatial fingerprint, and the near-end start protection information are subjected to weighted fusion, gated fusion, or normalized mapping to obtain the echo interpretability. The echo spatial fingerprint is used to enhance echo dominance determination, and the near-end start protection information is used to suppress near-end speech from being misidentified as echo.

[0015] Preferably, the control based on the echo interpretation degree includes: When the echo interpretation satisfies the echo dominance condition, the echo path is allowed to be updated and / or the echo suppression is enhanced; When the echo interpretation degree meets the near-end speech risk condition, freeze the echo path update and / or reduce the echo suppression; The interruption confidence level is determined based on the frequency point level, sub-band level, or frame level echo interpretation level, and interruption release control is performed based on the interruption confidence level.

[0016] In a second aspect, the present invention provides a multi-channel echo suppression and false interruption control device for implementing the method described in any one of the first aspects, comprising: The acquisition module is used to acquire the decoded TTS reference stream, the main channel acquisition signal, and the auxiliary microphone acquisition signal; The information determination module is used to determine reference echo characterization information based on the decoded TTS reference stream and the main channel acquisition signal, construct and update the echo space fingerprint based on the auxiliary microphone acquisition signal and the main channel acquisition signal, and determine near-end start protection information based on the change relationship between the main channel acquisition signal and the decoded TTS reference stream. An interpretation degree determination module is used to determine the echo interpretation degree based on the reference echo characterization information, the echo spatial fingerprint, and the near-end start protection information; The control module is used to control at least one of echo path updating, echo suppression, and interruption release control based on the echo interpretability.

[0017] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any one of the first aspects.

[0018] Compared with the prior art, the beneficial effects of the present invention are: First, by combining spatial consistency information of multi-channel microphone signals and echo spatial fingerprints, this invention can effectively distinguish between echoes and near-end speech, improving the accuracy of echo suppression. Second, by utilizing near-end start protection information, this invention can prevent misjudgment as an echo when the user begins speaking, thereby avoiding unintended interruptions and improving the system's response accuracy. Furthermore, this invention enables the system to dynamically adjust echo suppression strength, echo path updates, and interruption release control, enhancing the system's adaptability and making it suitable for various complex voice interaction scenarios.

[0019] This invention can also effectively reduce the consumption of system resources and improve processing efficiency; at the same time, the system of this invention can improve the accuracy of speech recognition and avoid echo or noise interference, thereby improving the user interaction experience. Attached Figure Description

[0020] Figure 1 This is a schematic diagram illustrating an application environment according to an exemplary embodiment; Figure 2 This is a flowchart illustrating a multi-channel echo suppression and false interruption control method according to an exemplary embodiment; Figure 3 This is a decoded TTS reference flow diagram illustrated according to an exemplary embodiment; Figure 4 This is a device frame diagram illustrating an exemplary embodiment; Figure 5 This is a diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] To facilitate a clear understanding of the technical solution of this invention, the key terms involved in this embodiment are first defined and explained: 1. Decoded TTS (Text To Speech) reference stream: This refers to the original reference signal output by the text-to-speech module after audio decoding. It serves as a reference for echo suppression and is consistent with the content played by the speaker.

[0023] 2. Main channel acquisition signal: refers to the mixed signal acquired by the main microphone, which includes echo, noise and near-field voice of the user.

[0024] 3. Auxiliary microphone signal acquisition: refers to the signal acquired synchronously by the auxiliary microphone, which is used to construct the spatial distribution characteristics of the echo.

[0025] 4. Reference echo characterization information: used to characterize the degree of linear matching between the acquired signal and the TTS reference echo.

[0026] 5. Echo spatial fingerprint: It is a stable spatial distribution feature of echoes among multiple microphones, used to distinguish echoes from near-end speech.

[0027] 6. Proximal start protection information: Used to identify the user's first words and prevent the initial speech from being misinterpreted as an echo.

[0028] 7. Echo Interpretation: This invention uses a unified control quantity to characterize the degree to which a signal can be interpreted echoically.

[0029] 8. Echo Path Update: The process of adaptively adjusting the echo cancellation filter.

[0030] 9. Interruption and Release Control: Control logic to determine whether users are allowed to interrupt TTS broadcasts.

[0031] 10. Near-end speech: Real speech signals emitted by the user in real time that need to be recognized by the system.

[0032] Please refer to Figure 1, which is a schematic diagram of an application environment according to an exemplary embodiment. As shown in Figure 1, the application environment may include a voice interaction terminal 10, an audio processing system 20, a voice recognition system 30, and a parameter management system 40. The various parts interact and work collaboratively with each other through a communication network.

[0033] The voice interaction terminal 10 is used to perform front-end execution of voice broadcasting, multi-channel sound acquisition, and user voice interaction, and is the hardware carrier of the entire echo suppression and false interruption control scheme. The voice interaction terminal 10 includes at least a speaker 11, a main channel microphone 12, at least one auxiliary microphone 13, and a TTS playback link 15. The speaker 11 is used to play TTS synthesized audio; the main channel microphone 12 and the auxiliary microphone 13 form a multi-channel acquisition array, used to simultaneously acquire speaker echo, ambient noise, and near-field user speech; the TTS playback link 15 is used to output the audio to be played to the speaker 11 and synchronously output the decoded TTS reference stream to the audio processing system 20, providing a standard reference signal for subsequent echo suppression.

[0034] The audio processing system 20 is the core processing unit of this invention, used to perform multi-channel signal parsing, time delay alignment, linear echo estimation, echo spatial fingerprint construction, near-end start protection detection, echo interpretability calculation, as well as echo path update control, echo suppression intensity control, and interruption release control. The audio processing system 20 receives multi-channel acquired signals and decoded TTS reference streams from the voice interaction terminal 10, completes the entire algorithm processing, and outputs the clean near-end speech signal to the speech recognition system 30.

[0035] The speech recognition system 30 is used to receive the near-end speech signal optimized by the audio processing system 20, perform subsequent processing such as speech recognition (ASR) and semantic understanding, convert the user's speech into executable instructions, and complete the complete voice interaction loop.

[0036] The parameter management system 40 is used to maintain and issue control parameters, including TTS status information, echo interpretation weight coefficients, threshold tables, state switching parameters, filter update thresholds, interruption hysteresis parameters, etc. The TTS status information includes at least one or more of the following: volume level, speech rate level, and broadcast style level, which is used to support the audio processing system 20 in realizing state-conditional adaptive control.

[0037] External data sources (not shown separately in Figure 1, but can be integrated into the parameter management system 40 or deployed independently) are used to provide external information related to the audio environment and device status, such as device posture, ambient noise level, pickup distance, network status, etc., which can further assist the audio processing system 20 in optimizing control strategies.

[0038] In actual operation, the voice interaction terminal 10, audio processing system 20, voice recognition system 30 and parameter management system 40 can be connected through one or more combinations of local area network (LAN), wide area network (WAN), Internet, 4G / 5G mobile data network and dedicated communication link to achieve low latency and high reliability transmission of decoded TTS reference stream, multi-channel acquisition signals, control parameters, status information and recognition results.

[0039] In one optional embodiment, the voice interaction terminal 10, audio processing system 20, voice recognition system 30, and parameter management system 40 can be integrated into the same hardware device, and data interaction can be achieved through system bus, shared memory, and inter-process communication. In another optional embodiment, the above system can be deployed in a distributed manner, and the terminal side and the cloud / edge side can complete data transmission and collaborative computing through the network to adapt to the needs of different computing power and deployment scenarios.

[0040] Through the collaborative work of various systems, this solution can accurately distinguish between speaker echo and near-end speech during real-time TTS broadcasting, dynamically adjust the echo suppression strength and interruption strategy, effectively reduce the false interruption rate, protect near-end speech, and improve recognition accuracy.

[0041] It should be noted that Figure 1 shows only one application environment of the multi-channel echo suppression and false interruption control method provided in this disclosure. In other embodiments, modules can be added or removed according to hardware configuration and business scenarios, or cloud deployment, edge computing, embedded single-machine deployment and other architectures can be adopted. This disclosure does not limit this.

[0042] It should be noted that the user information (including but not limited to user voice data, device usage status, user interaction behavior, etc.) and data (including but not limited to audio signals, channel status, operating parameters, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with relevant laws and regulations and privacy protection norms.

[0043] Please see Figure 2 , Figure 2 This can be a flowchart of a multi-channel echo suppression and false interruption control method provided in an embodiment of the present invention. The method includes the following steps: Step S201: Acquire the decoded TTS reference stream, the main channel acquisition signal, and the auxiliary microphone acquisition signal.

[0044] In the embodiments of this specification, the decoded TTS reference stream can be the reference audio signal output by the TTS playback link in the voice interaction system and the audio decoding completed, which is used as a reference signal for echo suppression. This signal does not contain echo, noise, or near-end speech interference.

[0045] The main channel acquisition signal can be a time-domain audio signal acquired in real time by the main channel microphone. It typically includes echo components generated by the speaker, ambient noise components, and near-end speech components emitted by the user in real time. The auxiliary microphone acquisition signal can be a time-domain audio signal acquired synchronously by at least one auxiliary microphone. It is used to form a multi-channel spatial observation with the main channel acquisition signal, providing support for echo spatial feature extraction and near-end speech discrimination.

[0046] In one possible implementation, the decoded TTS reference stream can be directly obtained from the TTS playback link built into the voice interaction terminal. The TTS playback link can simultaneously output the decoded reference stream to the audio processing system while outputting the audio to be played to the speaker, ensuring that the reference stream and the actual playback signal are consistent in content and timing.

[0047] In one alternative implementation, the main channel microphone and the auxiliary microphone can adopt a synchronous sampling mode to synchronously acquire sound field signals at a fixed sampling rate, thereby avoiding sampling offset or phase mismatch between multi-channel signals and improving the accuracy of subsequent echo spatial fingerprint construction and spatial consistency judgment.

[0048] In another possible implementation, the main channel acquisition signal and the auxiliary microphone acquisition signal can be acquired in real time through an audio interface or data bus. After receiving the acquisition signal, the audio processing system can preprocess the signal, including DC offset cancellation, pre-filtering, gain calibration, etc., to improve the signal quality and ensure the stability of subsequent processing.

[0049] In one alternative implementation, the decoded TTS reference stream, main channel acquisition signal, and auxiliary microphone acquisition signal can be framed, aligned, and buffered using the buffer unit inside the audio processing system to ensure that the three signals are strictly aligned on the time axis, providing a reliable data foundation for subsequent time delay estimation, linear echo estimation, and echo interpretation calculation.

[0050] In one possible implementation, while acquiring the above signals, TTS status information and voice activity detection results from the acquisition end can also be acquired simultaneously. The TTS status information may include volume level, speech rate level, and broadcast style level, etc. The voice activity detection results are used to identify whether there is voice activity in the currently acquired signal, so as to further improve the accuracy of echo judgment and interruption control.

[0051] In practical applications, by accurately acquiring the decoded TTS reference stream, the main channel acquisition signal, and the auxiliary microphone acquisition signal, a complete input foundation can be provided for echo suppression, spatial fingerprint construction, and near-end voice protection, ensuring that echo discrimination is more stable and reliable, effectively reducing the probability of false interruption, and improving the response accuracy and user experience of the voice interaction system.

[0052] Step S203: Determine reference echo characterization information based on the decoded TTS reference stream and the main channel acquisition signal; construct and update the echo space fingerprint based on the auxiliary microphone acquisition signal and the main channel acquisition signal; and determine near-end start protection information based on the change relationship between the main channel acquisition signal and the decoded TTS reference stream.

[0053] In the embodiments described in this specification, the reference echo characterization information is used to characterize the degree to which the main channel acquired signal can be explained by the linear echo model corresponding to the TTS reference stream, and is an important basis for determining whether the current signal is echo-dominated.

[0054] Echo spatial fingerprinting is used to characterize the spatial distribution characteristics of loudspeaker echoes between the main channel microphone and the auxiliary microphone, including amplitude relationship, phase relationship and complex ratio relationship, which can effectively distinguish echoes from near-end speech under multi-channel conditions.

[0055] Proximal initiation protection information is used to characterize the degree of synchronization between the energy changes of the main channel acquisition signal and the energy changes of the TTS reference stream. When the energy of the main channel suddenly increases but the reference stream does not rise synchronously, the risk of the user's near-end speech initiation can be identified, avoiding misjudging the user's opening speech as an echo.

[0056] In one possible implementation, determining the reference echo characterization information may include time-delay alignment of the decoded TTS reference stream with the main channel acquisition signal, generating a linear echo estimate based on the aligned reference stream, and obtaining the reference echo characterization information according to the degree of matching between the acquisition signal and the linear echo estimate.

[0057] In one alternative implementation, the echo spatial fingerprint can be constructed by real-time observation based on the complex ratio of the signals acquired by the auxiliary microphone and the main channel, and recursively updated under the conditions of high echo reliability and low near-end risk. When there is near-end voice risk, the update is frozen to avoid user voice contamination of the spatial fingerprint.

[0058] In another possible implementation, the near-end initiation protection information can be determined by calculating the rise in energy of the main channel signal and the rise in energy of the reference stream. When the rise in energy of the main channel meets the preset conditions but the rise in energy of the reference stream does not, it is determined that the near-end voice risk has increased and protection is initiated.

[0059] In one alternative implementation, reference echo characterization information, echo spatial fingerprint, and near-end start protection information can be calculated in parallel after each frame of audio data arrives to ensure the real-time and continuous processing of the system.

[0060] In practical applications, by simultaneously determining the reference echo characterization information, echo spatial fingerprint, and near-end initiation protection information, the signal attributes can be comprehensively described from three dimensions: linear matching degree, spatial distribution characteristics, and initiation energy mutation, providing solid support for subsequent high-precision echo discrimination.

[0061] In another possible implementation, reference echo characterization information that can characterize whether the current observation can be interpreted by the reference echo can be extracted from the decoded TTS reference stream and the main channel acquisition signal; an echo spatial fingerprint that can characterize the spatial distribution characteristics of the loudspeaker echo can be constructed by using the spatial relationship between the auxiliary microphone and the main channel; in addition, the change relationship between the main channel acquisition signal and the decoded TTS reference stream can be compared to determine the near-end initiation protection information in order to identify the user's actual opening risk.

[0062] In this embodiment, determining the reference echo characterization information includes: performing time delay alignment between the decoded TTS reference stream and the main channel acquired signal; generating a main channel linear echo estimate based on the aligned decoded TTS reference stream; and determining the reference echo characterization information according to the correspondence between the main channel acquired signal and the main channel linear echo estimate. Further, time delay alignment can be achieved by calculating a correlation metric for each candidate time delay in the candidate time delay set and determining the target time delay.

[0063] In another possible implementation, the echo space fingerprint can be constructed online based on the amplitude, phase, or complex ratio relationship between the auxiliary microphone signal and the main channel signal, and updated using a recursive method with a forgetting factor. Furthermore, the echo space fingerprint is only updated when the current moment meets the conditions of high echo reliability and low near-end risk; when there is a high probability of near-end speech at the current moment, the echo space fingerprint is frozen to avoid incorrectly writing real user speech into the echo template.

[0064] In another possible implementation, near-end start protection information is used to characterize the synchronization between the energy changes of the main channel acquired signal and the energy changes of the decoded TTS reference stream. When the main channel energy suddenly rises while the reference stream energy does not rise significantly in sync, it is more likely that the user is actually speaking, rather than an echo enhancement caused by changes in TTS playback intensity. Therefore, this information is used to cancel out subsequent echo detection.

[0065] Step S205: Determine the echo interpretation degree based on the reference echo characterization information, the echo spatial fingerprint, and the near-end start protection information.

[0066] In the embodiments of this specification, echo interpretability is an intermediate control quantity uniformly used to determine the degree to which the current signal is interpreted by the speaker echo. The value range can be normalized to the interval [0,1]. The higher the echo interpretability, the closer the current signal is to pure echo, and the lower the value, the higher the risk of near-end speech.

[0067] The echo interpretation degree is obtained by fusing reference echo characterization information, echo spatial fingerprint, and near-end start protection information, which can make up for the shortcomings of single feature judgment and improve stability in complex scenarios.

[0068] In one possible implementation, a weighted fusion method can be used to calculate the echo interpretation degree, with weights set for reference echo characterization, spatial consistency, and near-end protection, and the results can be limited to a reasonable range using the clip function to avoid numerical anomalies.

[0069] In one alternative implementation, echo spatial fingerprinting can enhance the confidence of echo-dominant scenarios, and near-end start protection information can suppress the misjudgment of user-initiated speech as echo. The combination of the two can significantly reduce the misjudgment rate.

[0070] In another possible implementation, gated fusion or normalized mapping can be used to generate echo interpretability to adapt to the system requirements of different hardware platforms, different sampling rates, and different microphone configurations.

[0071] In one alternative implementation, echo interpretation can be calculated at the frequency point level, sub-band level, or frame level. Fine-level calculation can improve the suppression and control accuracy, while coarse-level calculation can reduce system resource consumption.

[0072] In practical applications, by fusing multiple features to obtain a unified echo interpretation degree, the conflict problem caused by independent judgment of multiple modules such as AEC, residual suppression, and interruption control in traditional schemes can be solved, and full-link collaborative control can be achieved.

[0073] Please see Figure 3 This step is used to generate a unified intermediate control value. Echo interpretability is used to characterize the degree to which the current signal is interpreted by the speaker echo: the higher the echo interpretability, the more likely the current signal is dominated by the speaker echo; the lower the echo interpretability, the more likely there is a near-end speech risk.

[0074] In another possible implementation, the reference echo characterization information, spatial consistency information, and near-end initiation protection information can be weighted, gated, or normalized to obtain the echo interpretability. The echo interpretability is defined as follows: Xi(t,k)=clip(λ1·Sr(t,k)+λ2·Ss(t,k)-λ3·So(t),0,1) Where Xi(t,k) represents the echo interpretability at the k-th frequency point in the t-th frame, Sr(t,k) represents the reference interpretability, Ss(t,k) represents the spatial consistency, So(t) represents the near-end start protection information, λ1, λ2, and λ3 represent the corresponding weights, and clip(·) represents truncating the input to the [0,1] interval.

[0075] Step S207: Based on the echo interpretation degree, control at least one of echo path update, echo suppression, and interruption release control.

[0076] In the embodiments described in this specification, the control logic based on echo interpretation is the core component for achieving high-precision echo suppression and low false interruption rate, and can dynamically adjust the system behavior according to real-time signal attributes.

[0077] When the echo interpretation meets the echo dominance condition, the system can allow echo path updates and enhance echo suppression strength to eliminate loudspeaker echoes to the greatest extent possible.

[0078] When the echo interpretation meets the near-end speech risk conditions, the system can freeze the echo path update and reduce the echo suppression strength to avoid near-end speech being over-suppressed and distorted.

[0079] In one possible implementation, the interruption confidence can be calculated based on the frequency point level, sub-band level, or frame level echo interpretation, and the interruption release control can be completed in combination with a dual threshold hysteresis mechanism to avoid frequent state jitter.

[0080] In one alternative implementation, the echo path update control can be linked with the near-end start protection information, allowing filter updates only when the echo interpretation meets the standard and the near-end protection is not triggered, thereby improving the stability of dual-talk scenarios.

[0081] In another possible implementation, the echo suppression intensity can be adaptively adjusted according to the echo interpretation level. The higher the echo interpretation level, the stronger the suppression, and the lower the echo interpretation level, the weaker the suppression, thus achieving a smooth transition in suppression intensity.

[0082] In one alternative implementation, the interruption release control can be combined with the voice activity detection results at the acquisition end, and the interruption can only be released when voice activity is detected and the echo interpretation is low, thereby reducing the probability of false triggering.

[0083] In practical applications, by uniformly controlling echo path updates, echo suppression, and interruption release through echo interpretability, the robustness of voice interaction systems can be significantly improved, the false interruption rate can be reduced, the quality of near-end speech can be protected, and the overall user interaction experience can be enhanced.

[0084] In another possible implementation, if the echo interpretability satisfies the echo dominance condition and the near-end start protection information is not triggered, the echo path of the corresponding frequency point or sub-band is allowed to continue updating; if the echo interpretability does not satisfy the echo dominance condition, the update is frozen to avoid writing real near-end speech into the echo path model.

[0085] In another possible implementation, the corresponding echo suppression strength can be determined based on the echo resolving power of the current frequency point, subband, or frame. Echo suppression is enhanced when the echo resolving power is high and weakened when the echo resolving power is low, in order to protect near-end speech. Furthermore, state-conditional control can be implemented by combining TTS state information to prevent fixed thresholds from failing under high volume, high speech rate, or specific broadcast styles.

[0086] In another possible implementation, the frequency-level echo interpretation can be further aggregated to the frame level to generate a unified release confidence required for interruption release control, and an interruption confidence can be generated by combining the voice activity detection results from the acquisition end. When the interruption confidence meets the entry condition and lasts for no less than a first preset number of frames, the interruption state is allowed. When the system is already in the interruption state, it will only exit the interruption state when the interruption confidence meets the exit condition and lasts for no less than a second preset number of frames, thereby reducing state jitter by using a dual threshold hysteresis mechanism.

[0087] Through the above four steps, this invention extends the traditional single echo cancellation front-end into an "echo discrimination and unified control front-end" for real voice interaction services: When the system is in an echo-dominant state, the reference interpretability is high, the spatial consistency is high, the near-end initiation protection is not triggered, the echo interpretability increases, and the system allows the echo path to be updated and the suppression is enhanced; when the user actually speaks, the main channel energy rises rapidly, the reference flow does not rise synchronously, the current multi-channel relationship deviates from the echo spatial fingerprint, the echo interpretability decreases, the system freezes the update, weakens the suppression, and increases the probability of interruption and release.

[0088] Please see Figure 3 Embodiments of the present invention provide an apparatus for implementing the above-described method. The apparatus may include an acquisition module 100, an information determination module 200, an interpretation level determination module 300, and a control module 400.

[0089] The acquisition module 100 is used to acquire the decoded TTS reference stream, the main channel acquisition signal, and the auxiliary microphone acquisition signal. In engineering implementation, this module may include a reference stream access unit, a main channel acquisition unit, an auxiliary microphone acquisition unit, and a buffer management unit, used to complete the reception, buffering, and time-frequency framing of multi-source audio streams.

[0090] The information determination module 200 is used to determine reference echo characterization information based on the decoded TTS reference stream and the main channel acquisition signal, construct and update the echo space fingerprint based on the auxiliary microphone acquisition signal and the main channel acquisition signal, and determine the near-end start protection information based on the change relationship between the main channel acquisition signal and the decoded TTS reference stream.

[0091] The interpretation degree determination module 300 is used to determine the echo interpretation degree based on the reference echo characterization information, echo space fingerprint and near-end start protection information.

[0092] The control module 400 is used to control at least one of echo path update, echo suppression, and interruption release control based on echo interpretability. This module may include an echo path update control submodule, an echo suppression control submodule, and an interruption state machine submodule. The echo path update control submodule allows path updates under echo-dominant conditions and freezes updates under near-end speech risk conditions; the echo suppression control submodule adjusts the suppression intensity based on echo interpretability; and the interruption state machine submodule switches between interruption release states based on interruption confidence, entry threshold, exit threshold, and duration frame threshold.

[0093] In one possible implementation, the aforementioned device can be deployed in a layered manner: the acquisition module is located in the input access layer, the information determination module is located in the representation construction layer, the interpretability determination module is located in the fusion decision layer, and the control module is located in the policy execution layer; intermediate results can be passed between the layers through shared memory, a circular buffer, a message queue, or a unified data bus. This facilitates both local deployment and distributed implementation on edge devices or server sides.

[0094] The present invention can also correspond to an electronic device, which includes a processor and a storage module. The storage module may be a volatile memory, a non-volatile memory, or a combination of both, for storing program instructions, model parameters, threshold parameters, state transition parameters, and echo space fingerprint historical data that can be executed by the processor.

[0095] Please see Figure 4 , Figure 4 A schematic diagram of the mechanism of an electronic device 20 that can implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of control devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0096] Electronic device 20 includes at least one processor 21 and a memory, such as read-only memory (ROM) 22 and random access memory (RAM) 23, communicatively connected to at least one processor 21. The memory stores computer programs executable by at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 22 or loaded from storage unit 28 into the RAM 13. The RAM 23 may also store various programs and data required for the operation of electronic device 20. The processor 21, ROM 22, and RAM 23 are interconnected via bus 24. Input / output (I / O) interface 25 is also connected to bus 24.

[0097] Multiple components in electronic device 20 are connected to I / O interface 25, including: input unit 26, such as keyboard, mouse, etc.; output unit 27, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 29, such as network card, modem, wireless transceiver, etc. Communication unit 29 allows electronic device 20 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0098] Processor 21 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 21 performs the various methods and processes described above.

[0099] In some embodiments, the methods described above can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 28. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 20 via ROM 22 and / or communication unit 29. When the computer program is loaded into RAM 23 and executed by processor 21, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, processor 21 can be configured to perform the methods described above by any other suitable means (e.g., by means of firmware).

[0100] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0101] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0102] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0103] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0104] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0105] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0106] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-channel echo suppression and false interruption control method, characterized in that, The method, applied to a voice interaction system including a speaker, a main channel microphone, and at least one auxiliary microphone, comprises: Acquire the decoded TTS reference stream, the main channel acquisition signal, and the auxiliary microphone acquisition signal; Reference echo characterization information is determined based on the decoded TTS reference stream and the main channel acquisition signal. An echo space fingerprint is constructed and updated based on the auxiliary microphone acquisition signal and the main channel acquisition signal. Proximal initiation protection information is determined based on the change relationship between the main channel acquisition signal and the decoded TTS reference stream. The echo interpretation degree is determined based on the reference echo characterization information, the echo spatial fingerprint, and the near-end initiation protection information; Based on the echo interpretation degree, at least one of echo path update, echo suppression, and interruption release control is controlled.

2. The method according to claim 1, characterized in that, The determination of reference echo characterization information based on the decoded TTS reference stream and the main channel acquired signal includes: The decoded TTS reference stream is time-delay aligned with the main channel acquisition signal; Generate a main channel linear echo estimate based on the aligned decoded TTS reference stream; The reference echo characterization information is determined based on the correspondence between the main channel acquired signal and the main channel linear echo estimation.

3. The method according to claim 2, characterized in that, The step of aligning the decoded TTS reference stream with the main channel acquisition signal by time delay includes: For each candidate delay in the candidate delay set, calculate the correlation metric between the decoded TTS reference stream and the main channel acquisition signal; Candidate delays that satisfy preset conditions for the correlation metric are determined as target delays, and the decoded TTS reference stream is aligned based on the target delays.

4. The method according to claim 1, characterized in that, The echo spatial fingerprint is characteristic information that represents the spatial distribution relationship of the speaker echo between the auxiliary microphone and the main channel microphone; The construction and updating of the echo space fingerprint includes: The current spatial observation is generated based on the amplitude relationship, phase relationship, or complex ratio relationship between the auxiliary microphone acquisition signal and the main channel acquisition signal, and the echo spatial fingerprint is recursively updated based on historical spatial observations.

5. The method according to claim 4, characterized in that, The recursive update of the echo space fingerprint includes: The echo space fingerprint is updated when the high echo confidence condition and the low near-end risk condition are met at the current moment. If there is a risk of near-end speech at the current moment, freeze the update of the echo space fingerprint.

6. The method according to claim 1, characterized in that, The near-end start protection information is used to characterize the degree of synchronization between the energy change of the main channel acquired signal and the energy change of the decoded TTS reference stream; When the energy rise of the main channel acquisition signal meets the first preset condition and the energy rise of the decoded TTS reference stream does not meet the second preset condition, the near-end start protection information indicates an increase in near-end voice risk.

7. The method according to claim 1, characterized in that, The step of determining the echo interpretation degree based on the reference echo characterization information, the echo spatial fingerprint, and the near-end initiation protection information includes: The reference echo characterization information, the echo spatial fingerprint, and the near-end start protection information are subjected to weighted fusion, gated fusion, or normalized mapping to obtain the echo interpretability. The echo spatial fingerprint is used to enhance echo dominance determination, and the near-end start protection information is used to suppress near-end speech from being misidentified as echo.

8. The method according to claim 1, characterized in that, The control based on echo interpretation includes: When the echo interpretation satisfies the echo dominance condition, the echo path is allowed to be updated and / or the echo suppression is enhanced; When the echo interpretation degree meets the near-end speech risk condition, freeze the echo path update and / or reduce the echo suppression; The interruption confidence level is determined based on the frequency point level, sub-band level, or frame level echo interpretation level, and interruption release control is performed based on the interruption confidence level.

9. A multi-channel echo suppression and false interruption control device, characterized in that, include: The acquisition module is used to acquire the decoded TTS reference stream, the main channel acquisition signal, and the auxiliary microphone acquisition signal; The information determination module is used to determine reference echo characterization information based on the decoded TTS reference stream and the main channel acquisition signal, construct and update the echo space fingerprint based on the auxiliary microphone acquisition signal and the main channel acquisition signal, and determine near-end start protection information based on the change relationship between the main channel acquisition signal and the decoded TTS reference stream. An interpretation degree determination module is used to determine the echo interpretation degree based on the reference echo characterization information, the echo spatial fingerprint, and the near-end start protection information; The control module is used to control at least one of echo path updating, echo suppression, and interruption release control based on the echo interpretability.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images