A method, device, equipment, and medium for interrupting dialogue in human-computer interaction
By eliminating echoes through differential isolation technology and a lightweight inverse sound field model, combined with feature extraction and micro neural network verification, the accuracy and low latency issues of user interruption of TTS voice broadcast under extremely low resource conditions are solved, achieving efficient and low-power human-computer interaction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- MALANSHAN AUDIO & VIDEO LABORATORY
- Filing Date
- 2026-04-21
- Publication Date
- 2026-07-14
AI Technical Summary
With extremely limited resources on the low-end side, users cannot interrupt TTS voice broadcasts in a timely, accurate and low-latency manner. This is especially true on low-power IoT devices or older chips, where the user's voice signal is easily interfered with by the sound being played, resulting in interruption failure or excessive latency.
Echoes are eliminated using differential isolation techniques, and dialogue interruption events are detected with extremely low computational resources using feature extractors and neural networks, including the construction of inverse sound field models and micro neural networks for rapid verification.
It achieves efficient and accurate detection of user interruption intent under extremely low resource conditions, ensuring high signal-to-noise ratio and low power consumption in human-computer interaction, avoiding reliance on complex wake word models, and improving interruption success rate and energy efficiency.
Smart Images
Figure CN122392519A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of human-computer interaction, and in particular to a method, apparatus, device, and medium for interrupting dialogue in human-computer interaction. Background Technology
[0002] Currently, users cannot interrupt TTS (Text-to-Speech) voice broadcasts in a timely, accurate, and low-latency manner. In traditional intelligent interaction architectures, voice interruption typically relies on cloud-based or resource-rich edge models for wake-word detection or general voice activity detection. However, on terminals with extremely low resource constraints, such as low-power IoT devices or older chips, the strict limitations of computing power, memory, and power consumption make it impossible to maintain a continuously running and complex acoustic model. Especially when the device is playing TTS content at high volume, the user's interrupting voice signal is easily interfered with by its own playback sound, i.e., acoustic echo and self-interference, leading to interruption failure or excessively high latency.
[0003] In conclusion, how to detect user interruption intentions in real time under extremely low-end resource conditions is a problem that urgently needs to be solved. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for interrupting dialogue in human-computer interaction, capable of real-time detection of user interruption intentions even with extremely low-end resources. The specific solution is as follows: Firstly, this application provides a method for interrupting dialogue in human-computer interaction, including: The initial TTS playback content generated by the target AI model is obtained, and the initial TTS playback content is played in the target environment through a preset speaker. The initial TTS playback content is then sent to a preset signal processing channel to be determined as the reference TTS playback content. The target sound signal of the target environment is acquired through the target device; wherein, the target sound signal includes any one or more of the echo of the initial TTS playback content played through the preset speaker and the voice content of the user terminal; In the preset signal processing channel, differential isolation technology is used to remove the echo of the initial TTS playback content in the target audio signal based on the reference TTS playback content, so as to generate processed audio content. The target indicators are extracted from the processed speech content by a preset feature extractor, and the existence of a target dialogue interruption event is determined based on the target indicators. If so, a hard interrupt operation is performed to interrupt the current playback progress of the initial TTS playback content of the target AI model, and the target dialogue interruption event is verified using a preset neural network. Based on the verification result, the voice content of the user terminal is obtained to complete the dialogue interruption in human-computer interaction.
[0005] Optionally, the step of using differential isolation technology in the preset signal processing channel to remove the echo of the initial TTS playback content in the target audio signal based on the reference TTS playback content to generate processed audio content includes: In the preset signal processing channel, the target inverse sound field model is determined based on the acoustic transfer function of the preset loudspeaker and the real-time acoustic environment parameters; The target inverse sound field model is used to perform acoustic analysis on the reference TTS playback content in order to identify the target acoustic components of the reference TTS playback content; Based on the target acoustic components, a canceling sound wave with the opposite echo phase to the initial TTS playback content is generated. The echo of the initial TTS playback content in the target audio signal is canceled out by the canceling sound wave through differential isolation operation to generate processed speech content.
[0006] Optionally, the step of extracting target metrics from the processed speech content using a preset feature extractor includes: A first time window is constructed based on preset time division rules; The target metrics of the acoustic energy of the processed speech content within each of the first time windows are extracted using a preset feature extractor.
[0007] Optionally, the target indicators include acoustic energy indicators, signal fundamental frequency indicators, and target high-frequency entropy indicators.
[0008] Optionally, the step of extracting the target metrics of the acoustic energy of the processed speech content within each of the first time windows using a preset feature extractor includes: Determine the current time window to be processed from the first time window; The relative increase of the acoustic energy average between the current time window and a consecutive preset number of first time windows is obtained, and an instantaneous steep increase greater than a preset increase threshold is identified based on the relative increase, which is used as the acoustic energy index. Extract the fundamental frequency trajectory of the speech signal within the current time window, identify the variation characteristics of the fundamental frequency based on the fundamental frequency trajectory, and determine the signal fundamental frequency index based on the variation characteristics; Determine the target entropy value of the frequency band that meets the preset high-frequency conditions within the current time window, and generate a target high-frequency band entropy value index to distinguish high-frequency information from environmental white noise by using a preset entropy value threshold and the target entropy value.
[0009] Optionally, determining whether a target dialogue interruption event exists based on the target metric includes: Determine the preset threshold values corresponding to each of the target indicators; Determine whether the acoustic energy index, the signal fundamental frequency index, and the target high-frequency entropy index are greater than the preset index threshold within the same first time window; If so, the user's voice is identified, and a corresponding target dialogue interruption event is generated.
[0010] Optionally, the step of verifying the target dialogue interruption event using a preset neural network and obtaining the user's voice content based on the verification result includes: Using the triggering time of the target dialogue interruption event as the time anchor point, a second time window with a preset time period is extracted before and after the time of the target dialogue interruption event; Obtain the processed speech content corresponding to the second time window, and extract the acoustic feature sequence from the processed speech content; An end-to-end predefined binary classification inference operation is performed on the acoustic feature sequence using a predefined neural network to determine whether the processed speech content is non-human residual noise; the non-human residual noise includes any one or more of TTS residual echo, stable environmental noise and sudden non-human interference noise. If not, the target dialogue interruption event is deemed valid, and the system switches to full voice input mode to collect the voice content from the user terminal. If so, the target dialogue interruption event is determined to be invalid, and the steps of obtaining the initial TTS playback content generated by the target AI model, playing the initial TTS playback content in the target environment through a preset speaker, and sending the initial TTS playback content to a preset signal processing channel to determine it as the reference TTS playback content are continued.
[0011] Secondly, this application provides a dialogue interruption device for human-computer interaction, comprising: The playback content determination module is used to obtain the initial TTS playback content generated by the target AI model, play the initial TTS playback content in the target environment through a preset speaker, and send the initial TTS playback content to a preset signal processing channel to determine it as reference TTS playback content. The signal acquisition module is used to acquire the target sound signal of the target environment through the target device; wherein, the target sound signal includes any one or more of the echo of the initial TTS playback content played through the preset speaker and the voice content of the user terminal; The content generation module is used to use differential isolation technology in the preset signal processing channel to remove the echo of the initial TTS playback content in the target audio signal based on the reference TTS playback content, so as to generate processed audio content. The event judgment module is used to extract target indicators from the processed speech content through a preset feature extractor, and to determine whether there is a target dialogue interruption event based on the target indicators. The dialogue interruption module is used to perform a hard interruption operation if the condition is met, thereby interrupting the current playback progress of the initial TTS playback content of the target AI model, and to verify the target dialogue interruption event using a preset neural network. Based on the verification result, the user's voice content is obtained to complete the dialogue interruption in human-computer interaction.
[0012] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the dialogue interruption method for human-computer interaction as described above.
[0013] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned human-computer interaction dialogue interruption method.
[0014] In summary, this application first obtains the initial TTS playback content generated by the target AI model, plays the initial TTS playback content in the target environment through a preset speaker, and sends the initial TTS playback content to a preset signal processing channel to determine it as reference TTS playback content; then, it obtains the target sound signal of the target environment through the target device; wherein, the target sound signal includes any one or more of the echo of the initial TTS playback content played through the preset speaker and the user's voice content; in the preset signal processing channel, it uses differential isolation technology based on the reference TTS playback content to remove the echo of the initial TTS playback content in the target sound signal to generate processed voice content; it uses a preset feature extractor to extract target indicators from the processed voice content, and determines whether there is a target dialogue interruption event based on the target indicators; if so, it performs a hard interruption operation to interrupt the current playback progress of the initial TTS playback content of the target AI model, and uses a preset neural network to verify the target dialogue interruption event, and obtains the user's voice content based on the verification result to complete the dialogue interruption in human-computer interaction. As described above, this application first acquires the initial TTS content generated by the target AI model, plays it through a speaker, and simultaneously sends it to the signal processing channel as a reference signal. Then, it acquires ambient sound signals through the target device, including the echo of the played content and the user's voice. Subsequently, differential isolation technology is used in the signal processing channel to eliminate the echo with the reference signal, resulting in processed speech content. Next, a feature extractor extracts target metrics from the speech content to determine if a dialogue interruption event has occurred. If so, the current TTS playback is interrupted, and a neural network is used to verify the interruption event. Finally, the user's voice is acquired based on the verification result to complete the interruption interaction. In this way, by utilizing complete knowledge of its own TTS content, a minimalist and lightweight inverse sound field model is constructed. Through differential comparison between the digital domain reference signal and the microphone input, TTS echoes are efficiently eliminated with extremely low computational overhead, providing high signal-to-noise ratio audio for subsequent capture. Simultaneously, in ultra-low power mode, potential interruption events are generated by monitoring the joint synchronous bursts of ultra-lightweight human voice feature metrics such as instantaneous steep increases and aperiodic changes in fundamental frequency, avoiding reliance on complex wake-word models. Upon triggering of a potential event, a hard interrupt is executed, and TTS is stopped. A miniature neural network with deep quantization and pruning is activated to perform binary classification verification on extremely short audio clips, quickly identifying high-probability human intentions. If verification is successful, the interruption is formally initiated; otherwise, TTS playback resumes, ensuring high accuracy and high energy efficiency. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0016] Figure 1 This application discloses a flowchart of a method for interrupting dialogue in human-computer interaction. Figure 2 A flowchart of a specific human-computer interaction dialogue interruption method disclosed in this application; Figure 3 This is a schematic diagram of a dialogue interruption device for human-computer interaction disclosed in this application; Figure 4 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation
[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0018] Currently, users cannot interrupt TTS (Text-to-Speech) voice broadcasts in a timely, accurate, and low-latency manner. In traditional intelligent interaction architectures, voice interruption typically relies on cloud-based or resource-rich edge models for wake-word detection or general voice activity detection. However, on terminals with extremely low resource constraints, such as low-power IoT devices or older chips, the strict limitations of computing power, memory, and power consumption make it impossible to maintain a continuously running and complex acoustic model. Especially when the device is playing TTS content at high volume, the user's interruption voice signal is easily interfered with by its own playback sound, i.e., acoustic echo and self-interference, leading to interruption failure or excessively high latency. To solve the above technical problems, this application discloses a method, apparatus, device, and medium for interrupting dialogue in human-computer interaction, which can detect the user's interruption intention in real time even with extremely low edge resources.
[0019] See Figure 1 As shown in the figure, an embodiment of the present invention discloses a method for interrupting dialogue in human-computer interaction, including: Step S11: Obtain the initial TTS playback content generated by the target AI model, play the initial TTS playback content in the target environment through a preset speaker, and send the initial TTS playback content to a preset signal processing channel to determine it as reference TTS playback content.
[0020] In this embodiment, during the dialogue between the AI model and the user, the TTS audio signal being played in the digital domain, i.e. the initial TTS playback content, is simultaneously sent to two different channels: one is sent to the speaker for physical playback, and the other is sent to a dedicated digital signal processing channel as the ideal reference TTS playback content.
[0021] Step S12: Obtain the target sound signal of the target environment through the target device; wherein, the target sound signal includes any one or more of the echo of the initial TTS playback content played through the preset speaker and the voice content of the user terminal.
[0022] In this embodiment, relying on the microphone array configured on the target device, real-time sound information within the target environment is collected from all directions to obtain the mixed audio signal in the current environment, i.e., the target sound signal. The target sound signal is a comprehensive sound data formed by the superposition of multiple sound sources, mainly containing two types of key sound components. One type is the TTS audio echo signal generated by the sound propagating and reflecting in the target environment space after the target device plays the initial TTS playback content through its own preset speaker. The other type is the voice content signal emitted in real time by the user terminal in the target environment.
[0023] Step S13: In the preset signal processing channel, differential isolation technology is used to remove the echo of the initial TTS playback content in the target audio signal based on the reference TTS playback content, so as to generate processed audio content.
[0024] In this embodiment, after acquiring the target sound signal, in the preset signal processing channel, a target inverse sound field model is determined based on the acoustic transfer function of the preset loudspeaker and real-time acoustic environment parameters. The target inverse sound field model is then used to perform acoustic analysis on the reference TTS playback content to identify the target acoustic components of the reference TTS playback content. Based on the target acoustic components, a canceling sound wave with an echo phase opposite to that of the initial TTS playback content is generated. Differential isolation is then used to cancel the echo of the initial TTS playback content in the target sound signal using the canceling sound wave, thereby generating processed speech content. Specifically, since the acoustic path of the TTS playback content is from the loudspeaker to the microphone, the TTS playback content and its acoustic path are relatively fixed and predictable. The algorithm in the digital signal processing channel does not need to perform complex and continuous adaptive filter coefficient updates; instead, it runs a very simple, lightweight inverse sound field model. This model, through differential isolation technology, can identify and eliminate the main linear acoustic components generated by the known TTS signal along a fixed path, generating processed speech content. In this way, with extremely low computational resource consumption, most of the self-interference noise can be efficiently canceled out from the mixed audio signal input by the microphone, significantly improving the signal-to-noise ratio of the residual audio stream and creating a clean and ideal acoustic environment for subsequent user voice capture. This is the key to ensuring successful interruption and avoiding false triggering.
[0025] Step S14: Extract target indicators from the processed speech content using a preset feature extractor, and determine whether there is a target dialogue interruption event based on the target indicators.
[0026] In this embodiment, a first time window is constructed based on a preset time division rule; target indicators of the acoustic energy of the processed speech content within each first time window are extracted using a preset feature extractor. Target indicators include acoustic energy indicators, signal fundamental frequency indicators, and target high-frequency entropy indicators. Specifically, extremely short time windows are created on the time axis, for example, 200 millisecond time windows. A lightweight feature extractor running on a microcontroller or coprocessor is used to obtain target indicators within each time window. It is important to note that these target indicators focus on monitoring those that consume very little resources but are highly indicative of bursts of human vocal activity. Target indicators include, but are not limited to, instantaneous steep increases in acoustic energy, indicating the sudden onset of new, non-TTS sound source activity (i.e., acoustic energy indicators); rapid, non-periodic changes in signal fundamental frequency, a core characteristic of human vocal cord vibration (i.e., signal fundamental frequency indicators); and entropy values in specific high-frequency bands above 4kHz, which effectively distinguish high-frequency information from structured, articulated sounds from random environmental white noise (i.e., target high-frequency entropy indicators).
[0027] Furthermore, to determine the various target indicators, the current time window to be processed can be determined from the first time window; the relative increase in acoustic energy mean between the current time window and a consecutive preset number of first time windows can be obtained, and instantaneous steep increases greater than a preset increase threshold can be identified based on the relative increase as the acoustic energy indicator; the fundamental frequency trajectory of the speech signal within the current time window can be extracted, and the variation characteristics of the fundamental frequency can be identified based on the fundamental frequency trajectory, and the signal fundamental frequency indicator can be determined based on the variation characteristics; the target entropy value of the frequency band that meets the preset high-frequency conditions within the current time window can be determined, and a target high-frequency entropy value indicator for distinguishing high-frequency information from environmental white noise can be generated by using a preset entropy value threshold and the target entropy value. Specifically, firstly, the current time window to be processed needs to be accurately defined within the preset first time window range, and the time boundary and coverage of the current time window need to be clarified. Subsequently, the acoustic energy mean of the current time window needs to be obtained, and the acoustic energy mean of a consecutive preset number of first time windows can be calculated. Based on the data results of the two, the relative increase between them can be solved. Next, the relative increase is analyzed and identified, and instantaneous steep increases with values exceeding a preset increase threshold are selected and designated as acoustic energy indicators. Based on this, the speech signal within the current time window is processed to extract its complete fundamental frequency trajectory, focusing on the specific changes in fundamental frequency over time, identifying and analyzing the characteristics of fundamental frequency variation, including the amplitude and trend of variation. Based on the identified variation characteristics, a signal fundamental frequency indicator that can characterize the fundamental frequency properties of the speech signal is extracted and determined. Finally, specific frequency bands within the current time window that meet preset high-frequency conditions are defined, and the target entropy value corresponding to this frequency band is calculated. Then, using a preset entropy threshold as a reference standard, combined with the calculated target entropy value, a target high-frequency entropy value indicator is generated to distinguish target high-frequency information from environmental white noise.
[0028] Next, a multi-indicator joint judgment mechanism is used to determine the preset indicator thresholds corresponding to each of the target indicators. It is then determined whether the acoustic energy indicator, the signal fundamental frequency indicator, and the target high-frequency entropy indicator are greater than the preset indicator thresholds within the same first time window. If so, the user's speech is identified, and a corresponding target dialogue interruption event is generated. Specifically, after extracting each target indicator, a multi-indicator joint judgment mechanism is used to determine the corresponding preset indicator thresholds for the acoustic energy indicator, the signal fundamental frequency indicator, and the target high-frequency entropy indicator, thus constructing a complete indicator judgment standard. Subsequently, using a single first time window as the analysis unit, the acoustic energy indicator, the signal fundamental frequency indicator, and the target high-frequency entropy indicator calculated within the window are compared one by one with their respective preset indicator thresholds to determine whether each indicator value is greater than the corresponding preset indicator threshold. When all three types of indicators within the time window meet the judgment condition of being greater than the corresponding preset indicator threshold, based on the joint judgment conclusion, it is confirmed that a valid speech signal generated by the user has been detected, and a target dialogue interruption event matching it is generated according to preset rules.
[0029] Step S15: If yes, then perform a hard interrupt operation to interrupt the current playback progress of the initial TTS playback content of the target AI model, and use a preset neural network to verify the target dialogue interruption event. Based on the verification result, obtain the voice content of the user terminal to complete the dialogue interruption in human-computer interaction.
[0030] In this embodiment, once a potential interruption event is triggered, a hard interrupt operation will be executed immediately to prevent the user's subsequent voice signal from being interfered with again, and a burst of instantaneous high-power computing state will be entered to perform fast intent verification using a Micro-NN (Micro-Neural Network) with extremely optimized resources. The process involves using the triggering time of the target dialogue interruption event as a time anchor point, and extracting a second time window with a preset time period before and after the time of the target dialogue interruption event. The processed speech content corresponding to the second time window is then acquired, and acoustic feature sequences are extracted from the processed speech content. A preset neural network is used to perform an end-to-end preset binary classification inference operation on the acoustic feature sequences to determine whether the processed speech content is non-human voice residual noise. The non-human voice residual noise includes any one or more of TTS residual echo, stable environmental noise, and sudden non-human voice interference noise. If not, the target dialogue interruption event is determined to be valid, and the system switches to full voice input mode to collect the speech content from the user terminal. If yes, the target dialogue interruption event is determined to be invalid, and the process continues to acquire the initial TTS playback content generated by the target AI model, play the initial TTS playback content in the target environment through a preset speaker, and send the initial TTS playback content to a preset signal processing channel to determine it as reference TTS playback content. Specifically, the trigger moment of the target dialogue interruption event is used as a fixed time anchor. Before and after this trigger moment, extremely short audio segments of a preset duration are extracted as a second time window, for example, a 500-millisecond audio segment before and after the trigger moment. This serves as the core data for intent verification. Subsequently, the audio signal within this second time window is preprocessed to obtain processed speech content. Then, a continuous acoustic feature sequence is extracted from the processed speech content as input data for the Micro-NN. Next, a preset micro-neural network is used to perform an end-to-end preset binary classification inference operation on the extracted acoustic feature sequence. The core objective of this classification inference is to clearly distinguish between two situations: "high-probability human voice intent" and "non-human voice residual noise." Non-human voice residual noise specifically includes echoes remaining after TTS playback interruption, continuous stable environmental noise in the target environment, and sudden non-human voice interference noise. Because this binary classification task has a single objective and the input audio data is extremely short, the resource-optimized Micro-NN can complete the inference calculation in a very short time, for example, the inference time can be controlled within 100 milliseconds.If the inference result of the micro neural network is "no," meaning that the processed speech content is determined to be valid human speech rather than non-human residual noise, then the target dialogue interruption event is confirmed to be valid, and the system immediately switches to full speech input mode to continuously collect subsequent speech content sent by the user. If the inference result is "yes," meaning that the processed speech content is determined to be non-human residual noise, then the target dialogue interruption event is confirmed to be invalid, and the system immediately resumes the previous operation process, continuing to execute the steps of obtaining the initial TTS playback content generated by the target AI model, playing the initial TTS playback content in the target environment through a preset speaker, and sending it to a preset signal processing channel to be determined as the reference TTS playback content.
[0031] As described above, this embodiment first acquires the initial TTS content generated by the target AI model, plays it through a speaker, and simultaneously sends it to the signal processing channel as a reference signal. Then, it acquires ambient sound signals through the target device, including the echo of the played content and the user's voice. Subsequently, differential isolation technology is used in the signal processing channel to eliminate the echo with the reference signal, resulting in processed speech content. Next, a feature extractor extracts the target metrics of the speech content to determine if a dialogue interruption event has occurred. If so, the current TTS playback is interrupted, and a neural network is used to verify the interruption event. Finally, the user's voice is acquired based on the verification result, completing the interruption interaction. In this way, by utilizing complete knowledge of its own TTS content, a minimalist and lightweight inverse sound field model is constructed. Through differential comparison between the digital domain reference signal and the microphone input, TTS echoes are efficiently eliminated with extremely low computational overhead, providing high signal-to-noise ratio audio for subsequent capture. Simultaneously, in ultra-low power mode, potential interruption events are generated by monitoring the joint synchronous bursts of ultra-lightweight human voice feature metrics such as instantaneous steep increases and aperiodic changes in fundamental frequency, avoiding reliance on complex wake-word models. Upon triggering of a potential event, a hard interrupt is executed, and TTS is stopped. A miniature neural network with deep quantization and pruning is activated to perform binary classification verification on extremely short audio clips, quickly identifying high-probability human intentions. If verification is successful, the interruption is formally initiated; otherwise, TTS playback resumes, ensuring high accuracy and high energy efficiency.
[0032] Based on the previous embodiment, this application discloses a method for interrupting dialogue in human-computer interaction, which can detect the user's interruption intention in real time even with extremely low-end resources. Next, taking the human-computer interaction of a smart speaker in a home setting as an example, it will address issues such as... Figure 2 The methods for interrupting dialogue in human-computer interaction are explained in detail.
[0033] When a user requests a "family recipe recommendation this week" from the smart speaker, the target AI model generates an initial TTS (Text-to-Speech) message containing the daily recipe. The smart speaker then plays this initial TTS message clearly in the home environment through its preset speaker. At the same time, it sends this initial TTS message to the preset signal processing channel inside the speaker to determine it as the reference TTS message for subsequent signal processing, preparing for subsequent echo cancellation.
[0034] During TTS content playback, the smart speaker uses its built-in microphone array to acquire target sound signals from the home environment in real time. The acquired sound signals may include two types: one is the TTS echo formed by the reflection of the speaker playing TTS content on the walls and furniture in the room, and the other is the voice content suddenly requested by the user during TTS playback, such as "change a few light recipes", or it may be a mixture of the two sound signals.
[0035] Subsequently, in the preset signal processing channel, differential isolation technology is used to accurately identify and remove TTS echoes contained in the target audio signal based on the previously determined reference TTS playback content, and filter out irrelevant interference, thereby generating processed audio content containing only the user's voice.
[0036] Next, the preset feature extractor inside the smart speaker will analyze the processed voice content, extract target indicators such as acoustic energy and fundamental frequency, and then use these target indicators to determine whether there is a user-initiated interruption event in the target dialogue.
[0037] If a dialogue interruption event is detected, a hard interruption operation is immediately executed, pausing the current TTS playback progress of the smart speaker and ceasing playback of the remaining recipe content. Simultaneously, a preset micro neural network is invoked to quickly verify the dialogue interruption event. Once it is confirmed that the interruption is a valid voice request from the user and not environmental interference, the system switches to full voice input mode to continuously collect subsequent voice content from the user, such as the user stating "I want a light vegetable recipe." The system then adjusts its response content according to the user's needs, ultimately completing the dialogue interruption process in human-computer interaction and achieving smooth interaction between the user and the smart speaker.
[0038] See Figure 3 As shown, this embodiment of the invention discloses a dialogue interruption device for human-computer interaction, comprising: The playback content determination module 11 is used to obtain the initial TTS playback content generated by the target AI model, play the initial TTS playback content in the target environment through a preset speaker, and send the initial TTS playback content to a preset signal processing channel to determine it as reference TTS playback content. The signal acquisition module 12 is used to acquire the target sound signal of the target environment through the target device; wherein, the target sound signal includes any one or more of the echo of the initial TTS playback content played through the preset speaker and the voice content of the user terminal; The content generation module 13 is used to use differential isolation technology in the preset signal processing channel to remove the echo of the initial TTS playback content in the target sound signal based on the reference TTS playback content, so as to generate processed speech content. Event judgment module 14 is used to extract target indicators from the processed speech content through a preset feature extractor, and to determine whether there is a target dialogue interruption event based on the target indicators; The dialogue interruption module 15 is used to perform a hard interruption operation if the condition is met, so as to interrupt the current playback progress of the initial TTS playback content of the target AI model, and to verify the target dialogue interruption event using a preset neural network, and to obtain the voice content of the user terminal based on the verification result, so as to complete the dialogue interruption in human-computer interaction.
[0039] As described above, this application first acquires the initial TTS content generated by the target AI model, plays it through a speaker, and simultaneously sends it to the signal processing channel as a reference signal. Then, it acquires ambient sound signals through the target device, including the echo of the played content and the user's voice. Subsequently, differential isolation technology is used in the signal processing channel to eliminate the echo with the reference signal, resulting in processed speech content. Next, a feature extractor extracts target metrics from the speech content to determine if a dialogue interruption event has occurred. If so, the current TTS playback is interrupted, and a neural network is used to verify the interruption event. Finally, the user's voice is acquired based on the verification result to complete the interruption interaction. In this way, by utilizing complete knowledge of its own TTS content, a minimalist and lightweight inverse sound field model is constructed. Through differential comparison between the digital domain reference signal and the microphone input, TTS echoes are efficiently eliminated with extremely low computational overhead, providing high signal-to-noise ratio audio for subsequent capture. Simultaneously, in ultra-low power mode, potential interruption events are generated by monitoring the joint synchronous bursts of ultra-lightweight human voice feature metrics such as instantaneous steep increases and aperiodic changes in fundamental frequency, avoiding reliance on complex wake-word models. Upon triggering of a potential event, a hard interrupt is executed, and TTS is stopped. A miniature neural network with deep quantization and pruning is activated to perform binary classification verification on extremely short audio clips, quickly identifying high-probability human intentions. If verification is successful, the interruption is formally initiated; otherwise, TTS playback resumes, ensuring high accuracy and high energy efficiency.
[0040] In some specific implementations, the content generation module 13 may specifically include: The model determination unit is used to determine the target inverse sound field model in the preset signal processing channel based on the acoustic transfer function of the preset loudspeaker and the real-time acoustic environment parameters. The component identification unit is used to perform acoustic analysis on the reference TTS playback content using the target inverse sound field model in order to identify the target acoustic components of the reference TTS playback content. The content generation unit is used to generate a canceling sound wave with the opposite echo phase to the initial TTS playback content based on the target acoustic components, and to cancel the echo of the initial TTS playback content in the target sound signal according to the canceling sound wave through differential isolation operation, so as to generate processed speech content.
[0041] In some specific implementations, the event determination module 14 may specifically include: A window building unit is used to build a first time window based on a preset time division rule; The indicator extraction unit is used to extract target indicators of the acoustic energy indicators of the processed speech content within each first time window using a preset feature extractor.
[0042] In some specific implementations, the target indicators include acoustic energy indicators, signal fundamental frequency indicators, and target high-frequency entropy values.
[0043] In some specific implementations, the indicator extraction unit may specifically include: A window determination subunit is used to determine the current time window to be processed from the first time window; The index acquisition subunit is used to acquire the relative increase of the acoustic energy average of the current time window and the first time window consecutively preset number of times, and to identify instantaneous steep increases greater than a preset increase threshold based on the relative increase, so as to use the acoustic energy index. The index determination subunit is used to extract the fundamental frequency trajectory of the speech signal within the current time window, identify the variation characteristics of the fundamental frequency based on the fundamental frequency trajectory, and determine the signal fundamental frequency index based on the variation characteristics. The index generation subunit is used to determine the target entropy value of the frequency band that meets the preset high-frequency conditions within the current time window, and to generate a target high-frequency band entropy value index to distinguish high-frequency information from environmental white noise by using a preset entropy value threshold and the target entropy value.
[0044] In some specific implementations, the event determination module 14 may specifically include: A threshold determination unit is used to determine the preset threshold values corresponding to each of the target indicators; The indicator judgment unit is used to determine whether the acoustic energy indicator, the signal fundamental frequency indicator, and the target high-frequency band entropy value indicator are greater than the preset indicator threshold within the same first time window; The event generation unit is used to determine and identify the user's voice if the event is so, and generate a corresponding target dialogue interruption event.
[0045] In some specific implementations, the dialogue interruption module 15 may specifically include: The window capture unit is used to capture a second time window with a preset time period before and after the time of the target dialogue interruption event, using the trigger time of the target dialogue interruption event as the time anchor point. The feature sequence extraction unit is used to obtain the processed speech content corresponding to the second time window and extract the acoustic feature sequence from the processed speech content. The content judgment unit is used to perform an end-to-end preset binary classification reasoning operation on the acoustic feature sequence using a preset neural network to determine whether the processed speech content is non-human residual noise; the non-human residual noise includes any one or more of TTS residual echo, stable environmental noise and sudden non-human interference noise. The first content determination unit is used to determine that the target dialogue interruption event is valid if not, and switch to the full voice input mode to collect the voice content of the user terminal. The second content determination unit is used to determine that the target dialogue interruption event is invalid if the event is invalid, and to continue to execute the steps of obtaining the initial TTS playback content generated by the target AI model, playing the initial TTS playback content in the target environment through a preset speaker, and sending the initial TTS playback content to a preset signal processing channel to determine it as the reference TTS playback content.
[0046] Furthermore, embodiments of this application also disclose an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.
[0047] Figure 4 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the human-computer interaction dialogue interruption method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0048] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0049] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.
[0050] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the human-computer interaction dialogue interruption method disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.
[0051] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned human-computer interaction dialogue interruption method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.
[0052] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0053] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0054] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0055] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0056] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for interrupting dialogue in human-computer interaction, characterized in that, include: The initial TTS playback content generated by the target AI model is obtained, and the initial TTS playback content is played in the target environment through a preset speaker. The initial TTS playback content is then sent to a preset signal processing channel to be determined as the reference TTS playback content. The target sound signal of the target environment is acquired through the target device; wherein, the target sound signal includes any one or more of the echo of the initial TTS playback content played through the preset speaker and the voice content of the user terminal; In the preset signal processing channel, differential isolation technology is used to remove the echo of the initial TTS playback content in the target audio signal based on the reference TTS playback content, so as to generate processed audio content. The target indicators are extracted from the processed speech content by a preset feature extractor, and the existence of a target dialogue interruption event is determined based on the target indicators. If so, a hard interrupt operation is performed to interrupt the current playback progress of the initial TTS playback content of the target AI model, and the target dialogue interruption event is verified using a preset neural network. Based on the verification result, the voice content of the user terminal is obtained to complete the dialogue interruption in human-computer interaction.
2. The method for interrupting dialogue in human-computer interaction according to claim 1, characterized in that, The step of using differential isolation technology in the preset signal processing channel to remove the echo of the initial TTS playback content from the target audio signal based on the reference TTS playback content, in order to generate processed audio content, includes: In the preset signal processing channel, the target inverse sound field model is determined based on the acoustic transfer function of the preset loudspeaker and the real-time acoustic environment parameters; The target inverse sound field model is used to perform acoustic analysis on the reference TTS playback content in order to identify the target acoustic components of the reference TTS playback content; Based on the target acoustic components, a canceling sound wave with the opposite echo phase to the initial TTS playback content is generated. The echo of the initial TTS playback content in the target audio signal is canceled out by the canceling sound wave through differential isolation operation to generate processed speech content.
3. The method for interrupting dialogue in human-computer interaction according to any one of claims 1 to 2, characterized in that, The step of extracting target metrics from the processed speech content using a preset feature extractor includes: A first time window is constructed based on preset time division rules; The target metrics of the acoustic energy of the processed speech content within each of the first time windows are extracted using a preset feature extractor.
4. The method for interrupting dialogue in human-computer interaction according to claim 3, characterized in that, The target indicators include acoustic energy indicators, signal fundamental frequency indicators, and target high-frequency entropy indicators.
5. The method for interrupting dialogue in human-computer interaction according to claim 4, characterized in that, The step of extracting the target metrics of the acoustic energy index of the processed speech content within each of the first time windows using a preset feature extractor includes: Determine the current time window to be processed from the first time window; The relative increase of the acoustic energy average between the current time window and a consecutive preset number of first time windows is obtained, and an instantaneous steep increase greater than a preset increase threshold is identified based on the relative increase, which is used as the acoustic energy index. Extract the fundamental frequency trajectory of the speech signal within the current time window, identify the variation characteristics of the fundamental frequency based on the fundamental frequency trajectory, and determine the signal fundamental frequency index based on the variation characteristics; Determine the target entropy value of the frequency band that meets the preset high-frequency conditions within the current time window, and generate a target high-frequency band entropy value index to distinguish high-frequency information from environmental white noise by using a preset entropy value threshold and the target entropy value.
6. The method for interrupting dialogue in human-computer interaction according to claim 5, characterized in that, The determination of whether a target dialogue interruption event exists based on the target metric includes: Determine the preset threshold values corresponding to each of the target indicators; Determine whether the acoustic energy index, the signal fundamental frequency index, and the target high-frequency entropy index are greater than the preset index threshold within the same first time window; If so, the user's voice is identified, and a corresponding target dialogue interruption event is generated.
7. The method for interrupting dialogue in human-computer interaction according to claim 1, characterized in that, The step of verifying the target dialogue interruption event using a preset neural network and obtaining the user's voice content based on the verification result includes: Using the triggering time of the target dialogue interruption event as the time anchor point, a second time window with a preset time period is extracted before and after the time of the target dialogue interruption event; Obtain the processed speech content corresponding to the second time window, and extract the acoustic feature sequence from the processed speech content; An end-to-end predefined binary classification inference operation is performed on the acoustic feature sequence using a predefined neural network to determine whether the processed speech content is non-human residual noise; the non-human residual noise includes any one or more of TTS residual echo, stable environmental noise and sudden non-human interference noise. If not, the target dialogue interruption event is deemed valid, and the system switches to full voice input mode to collect the voice content from the user terminal. If so, the target dialogue interruption event is determined to be invalid, and the steps of obtaining the initial TTS playback content generated by the target AI model, playing the initial TTS playback content in the target environment through a preset speaker, and sending the initial TTS playback content to a preset signal processing channel to determine it as the reference TTS playback content are continued.
8. A dialogue interruption device for human-computer interaction, characterized in that, include: The playback content determination module is used to obtain the initial TTS playback content generated by the target AI model, play the initial TTS playback content in the target environment through a preset speaker, and send the initial TTS playback content to a preset signal processing channel to determine it as reference TTS playback content. The signal acquisition module is used to acquire the target sound signal of the target environment through the target device; wherein, the target sound signal includes any one or more of the echo of the initial TTS playback content played through the preset speaker and the voice content of the user terminal; The content generation module is used to use differential isolation technology in the preset signal processing channel to remove the echo of the initial TTS playback content in the target audio signal based on the reference TTS playback content, so as to generate processed audio content. The event judgment module is used to extract target indicators from the processed speech content through a preset feature extractor, and to determine whether there is a target dialogue interruption event based on the target indicators. The dialogue interruption module is used to perform a hard interruption operation if the condition is met, thereby interrupting the current playback progress of the initial TTS playback content of the target AI model, and to verify the target dialogue interruption event using a preset neural network. Based on the verification result, the user's voice content is obtained to complete the dialogue interruption in human-computer interaction.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the human-computer interaction dialogue interruption method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the human-computer interaction dialogue interruption method as described in any one of claims 1 to 7.