Voice interruption detection method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides a voice interruption detection method and device, electronic equipment and a medium, and the method comprises the steps: collecting environment sound in the audio playing process of voice interaction equipment, generating a residual signal based on the audio and the environment sound, carrying out the voice activity detection of the residual signal, and carrying out the voice interruption detection. The method comprises the following steps: acquiring a voice probability representing the possibility that the voice comprises the user voice, further determining a confidence coefficient score of effective interruption of the user voice by combining the audio, a residual signal and the voice probability, deciding to continue playing or interrupt playing the audio according to the confidence coefficient score, and sending the environment sound to a server when the confidence coefficient score is higher than a first preset threshold value. And receiving and executing a control instruction returned by the server so as to finally determine an audio playing state. According to the invention, the mistaken interruption rate of the voice interaction device during audio playing can be obviously reduced, and timely and reliable response to an effective interruption instruction of a user is ensured, so that the accuracy and fluency of interaction are improved in a complex acoustic environment.

Description

Technical Field

[0001] This invention relates to the field of voice interaction technology, and in particular to a voice interruption detection method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] With the rapid development of artificial intelligence and human-computer interaction technologies, voice interaction devices (such as smart speakers, in-vehicle voice assistants, and smart home control terminals) have been widely used in daily life and work. In these devices, users typically interact with the system through voice commands, and the device often needs to play prompts, music, navigation information, or other audio content during the response process. However, while the device is playing audio, users may want to interrupt the current playback at any time to issue a new voice command, which places a demand on the system's "voice interruption" capability.

[0003] Current voice interaction devices typically use fixed thresholds or single voice activity detection strategies to determine if a user has interrupted the audio playback. However, in practical applications, the sound played by the device itself may be picked up again by the microphone through air transmission, creating echo interference that is difficult to completely eliminate. At the same time, background human voices, television sounds, or other sudden noises in the environment can also be easily misjudged as valid voice input, causing the device to stop playback when it should not be interrupted, or to fail to respond in time when the user clearly issues a command, thus affecting the consistency and smoothness of the overall interactive experience. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a speech interruption detection method, apparatus, electronic device and readable storage medium that overcomes or at least partially solves the above problems.

[0005] In a first aspect, embodiments of the present invention provide a voice interruption detection method, applied to a voice interaction device, the method comprising: During audio playback, ambient sound is captured. Determine the residual signals of the audio and the ambient sound; Speech activity detection is performed on the residual signal to obtain a speech probability, which represents the likelihood that the residual signal contains the user's speech. A confidence score is determined based on the audio, the residual signal, and the speech probability, wherein the confidence score represents the probability that the user's speech constitutes a valid interruption; The audio can be played either continue or interrupted based on the confidence score. If the confidence score is higher than a first preset threshold, the ambient sound is sent to the server; The system receives control commands sent by the server based on the ambient sound and executes the control commands to play the audio or interrupt the playback of the audio.

[0006] Optionally, determining the residual signals of the audio and the ambient sound includes: The ambient sound collected is subjected to echo cancellation processing based on the audio to obtain the residual signal.

[0007] Optionally, before performing voice activity detection on the residual signal, the method further includes: Calculate the residual energy ratio, which is the ratio of the energy of the residual signal to the energy of the audio signal; The current acoustic scene is determined based on the residual energy ratio; When the current acoustic scene is a mixed sound scene or a pure human voice scene, the residual signal is used for speech activity detection.

[0008] Optionally, determining the confidence score based on the audio, the residual signal, and the speech probability includes: Obtain the signal-to-noise ratio and speech duration of the current environment; The residual energy ratio, the speech probability, the signal-to-noise ratio, and the speech duration are normalized. The confidence score is obtained by weighting and summing the normalized parameters according to preset weights.

[0009] Optionally, the step of continuing to play the audio or interrupting playback of the audio based on the confidence score includes: If the confidence score is lower than the second preset threshold, the audio continues to play; if the second preset threshold is higher than the first preset threshold. If the confidence score is greater than or equal to the second preset threshold, the playback of the audio is interrupted.

[0010] Optionally, receiving control commands sent by the server based on the ambient sound, and executing the control commands to play the audio or interrupt the playback of the audio, includes: If the audio continues to play, and a control command instructing the playback of the audio is received, then the playback of the audio is interrupted; if the control command instructing the playback of the audio is received, then the playback of the audio continues. If the audio playback is interrupted, and a control command instructing the interruption of the audio playback is received, then the interruption of the audio playback is maintained; if the control command instructing the playback of the audio playback is received, then the audio playback is resumed.

[0011] Optionally, it also includes: Record the confidence score corresponding to the environmental sound sent to the server each time and the control command sent by the server to build a historical interaction database; The preset weights are updated periodically using the historical interaction database.

[0012] Secondly, embodiments of the present invention provide a voice interruption detection device, applied to a voice interaction device, the device comprising: An ambient sound acquisition module is used to acquire ambient sound during audio playback. A residual signal determination module is used to determine the residual signals of the audio and the ambient sound; A voice activity detection module is used to perform voice activity detection on the residual signal to obtain a voice probability, wherein the voice probability represents the possibility that the residual signal contains the user's voice. A confidence determination module is used to determine a confidence score based on the audio, the residual signal, and the speech probability, wherein the confidence score represents the probability that the user's speech constitutes a valid interruption; The local decision module is used to continue playing the audio or interrupt the playback of the audio based on the confidence score; An ambient sound transmission module is used to transmit the ambient sound to a server when the confidence score is higher than a first preset threshold. The instruction execution module is used to receive control instructions sent by the server based on the ambient sound, and execute the control instructions to play the audio or interrupt the playback of the audio.

[0013] Thirdly, embodiments of the present invention provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the voice interruption detection method as described in the first aspect.

[0014] Fourthly, embodiments of the present invention provide a readable storage medium storing a program or instructions, which, when executed by a processor, implement the steps of the voice interruption detection method as described in the first aspect.

[0015] The embodiments of the present invention have the following advantages: By acquiring ambient sound during audio playback; determining the residual signal between the audio and the ambient sound; performing voice activity detection on the residual signal to obtain a voice probability characterizing the possibility that it contains user voice; determining a confidence score characterizing the possibility that the user voice constitutes a valid interruption based on the audio, the residual signal, and the voice probability; continuing to play the audio or interrupting playback based on the confidence score; and when the confidence score is higher than a first preset threshold, sending the ambient sound to the server, receiving and executing control commands returned by the server to finally determine the audio playback state, the embodiments of the present invention can significantly reduce the false interruption rate of voice interaction devices when playing audio, while ensuring timely and reliable response to valid interruption commands from users, thereby improving the accuracy and fluency of interaction in complex acoustic environments. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the steps of a speech interruption detection method provided in an embodiment of the present invention; Figure 2 This is a flowchart of another speech interruption detection method provided in an embodiment of the present invention; Figure 3 This is a structural block diagram of a voice interruption detection device provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and are not used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0020] With the increasing popularity of smart voice interaction devices, users expect to interact with these devices in real-time and smoothly via natural voice commands. In voice interaction scenarios, devices typically need to play audio feedback (such as music, voice responses, etc.) while continuously monitoring ambient sounds to respond promptly to user interruptions. However, in practical applications, accurately distinguishing between ambient noise and valid user interruptions, and avoiding interaction interruptions or response delays due to false detections, remains a challenging technical problem.

[0021] Traditional speech interruption detection methods typically rely on simple energy threshold detection or fixed-pattern speech activity detection (VAD). While these methods may perform well in quiet environments, they are prone to false positives in complex acoustic environments (such as background music, multi-person conversations, and sudden noise). For example, audio played by the device might be incorrectly identified as external speech, leading to unnecessary interruptions; or environmental noise might be mistaken for user commands, triggering incorrect responses. Furthermore, setting the detection sensitivity too high, while reducing missed detections, may increase the probability of false interruptions; conversely, setting it too low may prevent valid user commands from being recognized in a timely manner.

[0022] This invention significantly reduces the false interruption rate of voice interaction devices during audio playback by acquiring ambient sound during playback; determining the residual signal between the audio and the ambient sound; performing voice activity detection on the residual signal to obtain a voice probability characterizing the possibility that it contains user voice; determining a confidence score characterizing the possibility that the user voice constitutes a valid interruption based on the audio, the residual signal, and the voice probability; continuing to play the audio or interrupting playback based on the confidence score; and sending the ambient sound to the server when the confidence score is higher than a first preset threshold, receiving and executing control commands returned by the server to finally determine the audio playback status. This approach ensures timely and reliable response to valid user interruption commands, thereby improving the accuracy and fluency of interaction in complex acoustic environments.

[0023] Figure 1 This is a flowchart of the steps of a speech interruption detection method provided in an embodiment of the present invention.

[0024] like Figure 1 As shown, the method, applied to voice interaction devices, may specifically include the following steps: Step 101: During audio playback, ambient sound is captured; In this embodiment of the invention, the voice interaction device can be any intelligent terminal or system that has audio playback, sound acquisition, signal processing, and network communication capabilities, and can interact with users via voice, such as a smart speaker, in-vehicle voice assistant, smart TV, or home appliances with voice control functions. The method of this invention is applied to voice interaction devices to accurately determine whether the user has issued a valid interruption voice while the device is playing audio content, and dynamically adjust the audio playback behavior accordingly.

[0025] In the method of this invention embodiment, the voice interaction device can continuously play audio content (such as music, voice replies, prompts, etc.) through an audio output module such as a speaker, while simultaneously collecting ambient sounds using one or more built-in microphones. This ambient sound is a mixed signal actually received by the device's microphones, which may include audio components played by the device itself, user voice commands, background noise, and other environmental sounds. This ensures that the device can maintain real-time monitoring capabilities of the user's potential interactive intentions while providing audio output, laying the foundation for uninterrupted voice interaction.

[0026] Step 102: Determine the residual signals of the audio and the ambient sound; In the method of this embodiment of the invention, the voice interaction device processes the acquired ambient sound and the currently playing audio signal to calculate the residual signal between the two. The purpose is to suppress or eliminate interference components such as echoes or self-noise generated by the device's own audio playback from the mixed ambient sound. This is because the ambient sound signal acquired by the microphone is a mixture of user speech, ambient noise, and echoes from the device's audio playback; directly analyzing the ambient sound signal may affect the accuracy of subsequent voice activity detection. By calculating the residual signal, external sound source components, including user speech, can be highlighted, creating conditions for accurate subsequent detection.

[0027] Step 103: Perform speech activity detection on the residual signal to obtain the speech probability, wherein the speech probability represents the possibility that the residual signal contains the user's speech; Speech activity detection is a signal processing technique whose core task is to automatically determine whether a given audio signal contains valid speech components (usually specifically human voice) within a specific time period, rather than only containing noise, silence, or other non-speech sounds. In this invention, the speech activity detection module is applied to the residual signal.

[0028] In this embodiment of the invention, the speech probability obtained by speech activity detection of the calculated residual signal quantifies the possibility that user speech exists in the ambient sound at the current moment, providing an important probabilistic basis for the voice interaction device to transition from speech detection to understanding intent (whether it is an effective interruption).

[0029] Step 104: Determine a confidence score based on the audio, the residual signal, and the speech probability. The confidence score represents the probability that the user's speech constitutes a valid interruption. In practical applications, voice probability alone may not be sufficient to reliably determine a "valid interruption." For example, a high voice probability might stem from a brief cough, voices on a television, or indistinct conversations in the distance, none of which constitute a valid interruption command for the current device. Therefore, a comprehensive evaluation considering more context is necessary.

[0030] In this embodiment of the invention, the voice interaction device comprehensively analyzes information from three dimensions: the currently playing audio signal, the residual signal, and the speech probability, and calculates a comprehensive confidence score using a preset algorithm or model. The confidence score can more accurately quantify the probability that the detected user speech constitutes a valid interruption command in the current scenario, rather than simply the probability of speech presence.

[0031] Step 105: Continue playing the audio or stop playing the audio based on the confidence score; Voice-interactive devices can make a preliminary, rapid playback control decision locally based on confidence scores to enable an immediate response to user interruption intentions and provide crucial buffering and preprocessing for subsequent, more precise cloud processing.

[0032] In one embodiment, the device has a preset decision threshold. The decision threshold can be preset or adaptively adjusted to divide the device into two regions: "suspected invalid / low-risk interruption" and "suspected valid / high-risk interruption".

[0033] When the confidence score is lower than or equal to the decision threshold, the device determines that the current user's voice is unlikely to constitute a valid interruption. This may be due to the low probability of the voice itself, or the fact that although there is voice, it is judged to be irrelevant dialogue or background noise based on the context. To avoid unnecessary interaction interruptions and a decline in user experience due to misjudgment, the device may choose not to interrupt the current audio playback process and maintain the status quo; when the confidence score is higher than the decision threshold, the device determines that there is a high probability of a valid interruption intention. In order to provide the user with immediate feedback, such as stopping the music to indicate "I am listening," and to create a quieter acoustic environment for the reception of possible voice commands, eliminating the continuous interference of its own audio on subsequent microphone acquisition, the device may immediately or interrupt the playback of the current audio according to a predetermined fade-out curve.

[0034] Step 106: If the confidence score is higher than the first preset threshold, the ambient sound is sent to the server; In the method of this embodiment of the invention, when the confidence score exceeds the first preset threshold, regardless of the local decision on whether to interrupt playback in step 105, the device can initiate an interaction process with the cloud server to obtain the final instruction decision.

[0035] By setting a first preset threshold, not all detected voice activity needs to be uploaded to the cloud. The device filters out a large number of low-confidence noise or irrelevant voice events, avoiding unnecessary network transmission and server computing resource consumption, thus improving the overall system efficiency. The division of labor between local and cloud servers is clearly defined: the device is responsible for real-time, low-latency signal preprocessing, initial detection, and rapid response; the cloud server is responsible for speech recognition, natural language understanding, and command parsing of the filtered high-value candidate audio. Through this collaborative architecture of local coarse screening and cloud-based fine judgment, the timeliness of response is guaranteed while ensuring the accuracy of the final command execution.

[0036] Step 107: Receive the control command sent by the server based on the ambient sound, and execute the control command to play the audio or interrupt the playback of the audio.

[0037] In this embodiment of the invention, the device can receive a final control command issued by the server, which is generated after complete speech recognition and semantic understanding based on the uploaded ambient sound, and execute the command to achieve final control over the audio playback status.

[0038] In some embodiments, executing the control command serves as the system's final response to the user's interruption intention and may confirm or correct the initial local decision: if the cloud determines it to be a valid command, the device will perform the corresponding operation even if playback has not been interrupted locally; if the cloud determines it to be invalid, the device will resume playback according to the command even if playback has been interrupted locally.

[0039] This invention significantly reduces the false interruption rate of voice interaction devices during audio playback by acquiring ambient sound during playback; determining the residual signal between the audio and the ambient sound; performing voice activity detection on the residual signal to obtain a voice probability characterizing the possibility that it contains user voice; determining a confidence score characterizing the possibility that the user voice constitutes a valid interruption based on the audio, the residual signal, and the voice probability; continuing to play the audio or interrupting playback based on the confidence score; and sending the ambient sound to the server when the confidence score is higher than a first preset threshold, receiving and executing control commands returned by the server to finally determine the audio playback status. This approach ensures timely and reliable response to valid user interruption commands, thereby improving the accuracy and fluency of interaction in complex acoustic environments.

[0040] Figure 2 This is a flowchart of another speech interruption detection method provided in an embodiment of the present invention.

[0041] like Figure 2 As shown, the method, applied to voice interaction devices, may specifically include the following steps: Step 201: During audio playback, ambient sound is captured; In some embodiments, a voice interaction device can continuously pick up sound signals from its environment, i.e., ambient sounds, using one or more microphones while outputting audio, such as system prompts, music, news broadcasts, or other responsive voices, through its speakers. Ambient sounds are a mixture of signals from all sound sources in a space and typically contain the following components: (1) Voice commands that the user may issue; (2) Background noise (such as air conditioner noise, street noise, etc.); (3) The acoustic echo formed by the audio played by the device itself being transmitted through the air and then received by the microphone.

[0042] Since the device is in a full-duplex state of playback and monitoring, ambient sound is inevitably contaminated by the self-playing audio. Therefore, this mixed signal can be processed to extract effective user voice information.

[0043] Step 202: Perform echo cancellation processing on the collected ambient sound based on the audio to obtain the residual signal.

[0044] In this embodiment of the invention, the voice interaction device can use the currently playing audio signal as a reference signal to perform echo cancellation processing on the collected ambient sound. Since the device collects ambient sound through a microphone while playing audio, the sound emitted by the speaker will be recaptured by the microphone via airflow, forming an acoustic echo. This echo can severely interfere with the detection of the user's actual speech. The purpose of echo cancellation processing is to estimate and subtract the echo component caused by the playing audio from the ambient sound, thereby obtaining an output signal dominated by the non-playback source sound, i.e., a residual signal.

[0045] Specifically, the ambient sound captured by the microphone is modeled as a superposition of user speech, echo, and background noise, where the echo can be considered as the convolution result of a reference signal through an unknown acoustic path (i.e., the transfer function from the speaker to the microphone). The echo cancellation processing module can use an adaptive filtering algorithm (such as the normalized minimum mean square error algorithm) to estimate this acoustic path in real time and generate the corresponding echo estimation signal. The residual signal is obtained by subtracting this echo estimation signal from the original ambient sound. Residual signal = ambient sound Echo estimation signal Ideally, the residual signal has significantly reduced interference from the device's own playback content, mainly preserving the user's voice and background noise.

[0046] In some embodiments, due to the physical propagation delay (typically 2–10 milliseconds) of the audio emitted by the speaker traveling through the air to the microphone, misalignment between the audio signal considered as the reference signal and the actual echo in time can severely affect echo cancellation. To address this, a generalized cross-correlation-phase transform algorithm can be employed to calculate the time delay between the reference signal and the microphone signal, and accordingly time-align the reference signal to ensure precise temporal synchronization with the echo components in the ambient sound. The time delay estimate can also be updated periodically (e.g., every 500 milliseconds) to adapt to the effects of device movement, changes in user location, or dynamic adjustments to the acoustic environment.

[0047] In other embodiments, a dual-talk detection mechanism can be integrated. When it is determined that a user is speaking (i.e., there is both valid speech and echo), the coefficient update of the adaptive filter is paused to prevent the user's speech from being mistakenly included in the echo path model, leading to distortion or cancellation. Furthermore, residual echoes and steady-state noise can be further suppressed by applying frequency-domain post-filtering to the residual signal, resulting in a cleaner signal for subsequent speech activity detection.

[0048] Step 203: Perform speech activity detection on the residual signal to obtain the speech probability, wherein the speech probability represents the possibility that the residual signal contains the user's speech; In the method of this invention embodiment, the voice interaction device can perform voice activity detection on the residual signal to determine whether there are valid human voice components, and output a voice probability between 0 and 1. Speech probability can be used to quantify the likelihood that a residual signal contains user speech: the higher the value, the higher the confidence in the presence of user speech; conversely, it is more likely to be noise or invalid signal.

[0049] To further improve the accuracy and scene adaptability of voice activity detection, in some embodiments, before performing voice activity detection on the residual signal, the method further includes: Sub-step S11: Calculate the residual energy ratio, whereby the residual energy ratio is the ratio of the energy of the residual signal to the energy of the audio signal; Calculate the energy of the residual signal in the current frame or the current time period respectively. With the energy of playing audio signals And obtain the residual energy ratio The residual energy ratio reflects the relative intensity of the ambient sound signal relative to the playback volume. It is an important indicator for judging the current acoustic state. The larger the ratio, the more significant the non-echo components contained in the residual, and the more likely there is user voice or other external sound sources.

[0050] Sub-step S12: Determine the current acoustic scene based on the residual energy ratio; In one embodiment, based on a preset residual energy ratio threshold range, the current acoustic scene can be divided into three categories: Pure echo scenario: When RER<0.05, it indicates that the residual energy is extremely low, the echo cancellation process has effectively suppressed the echo, and there is no significant external speech, so it can be determined that there is no user interruption intention; Mixed audio scenarios: When 0.05 ≤ RER < 0.3, the residual energy is moderate and may contain weak user voice, background voice or residual interference. This is a critical scenario with high uncertainty and is prone to false interruptions. In pure vocal scenarios: when RER ≥ 0.3, the residual energy is significantly higher than the playing audio, indicating that the user is speaking and is very likely to constitute an effective interruption.

[0051] In other embodiments, to enhance stability, the scene classification results of multiple consecutive frames can be smoothed, for example, by majority voting or state preservation mechanisms, to prevent scene misjudgment caused by instantaneous noise or energy fluctuations.

[0052] Sub-step S13: In the case that the current acoustic scene is a mixed sound scene or a pure human voice scene, perform voice activity detection on the residual signal.

[0053] The speech activity detection module is activated only when the current acoustic scenario is a mixed-tone scenario or a pure human voice scenario to analyze the residual signal and output the speech probability. In the case of a pure echo scenario, speech activity detection is skipped or the speech probability is set to near zero to avoid unnecessary calculations and false triggers.

[0054] Traditional solutions typically perform only simple noise reduction and coarse-grained speech activity detection on the device side, then upload the raw or pre-processed audio to the cloud for unified evaluation. However, due to significant differences in microphone performance, structural layout, and acoustic characteristics among devices from different manufacturers, the cloud struggles to accurately model the residual signal quality for specific terminals, especially in ambiguous situations like mixed-tone scenarios, where a lack of local context can easily lead to misjudgments. This solution, by first achieving scene awareness on the device side and then performing conditional speech activity detection, improves the ability to identify uncertain scenes, significantly reduces the false interruption rate, and thus achieves a more natural and reliable voice interaction experience.

[0055] Step 204: Determine a confidence score based on the audio, the residual signal, and the speech probability. The confidence score represents the probability that the user's speech constitutes a valid interruption. In the method of this invention embodiment, the voice interaction device can comprehensively assess whether the user truly intends to interrupt audio playback based on the played audio content, the residual signal obtained after echo cancellation processing, and the speech probability output by voice activity detection, thereby generating a confidence score. The confidence score is used to characterize the probability that the speech contained in the residual signal is a valid user interruption (rather than noise, other people speaking, or system false detection).

[0056] In some embodiments, step 204 specifically includes the following sub-steps: Sub-step S21: Obtain the signal-to-noise ratio and speech duration of the current environment; In some embodiments, two key contextual features can be extracted from the residual signal: one is the signal-to-noise ratio (SNR) of the current acoustic environment, which measures the clarity of the user's speech relative to the background noise; Second, speech duration, which is the length of time during which speech is continuously detected. Real user interruptions usually have a certain energy stability and duration (e.g., more than 200–300 ms), while transient interferences (e.g., coughing, door closing sounds) are often short in duration. Therefore, speech duration can effectively help distinguish between valid speech and occasional noise.

[0057] Sub-step S22 involves normalizing the residual energy ratio, the speech probability, the signal-to-noise ratio, and the speech duration. The numerical ranges and physical meanings of residual energy ratio, speech probability, signal-to-noise ratio, and speech duration are different. To facilitate unified fusion, a preset mapping function (such as linear scaling or piecewise normalization) can be used to transform each parameter to the [0, 1] interval, ensuring that each feature has comparability and balanced contribution during the fusion process.

[0058] Sub-step S23: The normalized parameters are weighted and summed according to preset weights to obtain the confidence score.

[0059] Each normalized feature can be assigned a corresponding weight. In one example, speech probability and residual energy ratio have higher weights, reflecting their strong correlation with interruption intent; signal-to-noise ratio and duration are used as auxiliary criteria, with relatively lower weights. The final confidence score is then calculated using a weighted summation method. + + +

[0060] Each of these terms represents the residual energy ratio, speech probability, signal-to-noise ratio, and duration, respectively. , , , These are non-negative weighting coefficients, and their sum can be normalized to 1.

[0061] Step 205: Continue playing the audio or stop playing the audio based on the confidence score; In this embodiment of the invention, the voice interaction device can make a local real-time decision on whether to interrupt the current audio playback based on a determined confidence score. This decision directly affects the fluency and timeliness of user interaction, requiring a balance between avoiding accidental interruptions and ensuring effective command responses.

[0062] In some embodiments, step 205 specifically includes the following sub-steps: Sub-step S31: If the confidence score is lower than the second preset threshold, continue playing the audio; the second preset threshold is higher than the first preset threshold. When the confidence score does not reach the second preset threshold, it can be determined that the currently detected voice signal is insufficient to constitute a highly reliable and effective interruption, and may be weak voice, background voice, or residual interference. In this case, the device maintains the normal playback of the current audio without interrupting the output to ensure the continuity of the main task (such as music playback, navigation broadcast, etc.). The second preset threshold is set higher than the first preset threshold used to trigger the upload, forming a two-level judgment mechanism of low-threshold upload and high-threshold interruption. In one example, it may also include suspected interruptions with medium confidence (i.e., confidence scores higher than the first preset threshold but lower than the second preset threshold). In this case, although playback is not interrupted, it will trigger the uploading of ambient sound to the server for secondary cloud confirmation, reflecting the strategy of "conservative response and background verification". This ensures that only highly credible interruptions will immediately stop playback, while suspected interruptions with medium confidence will be further confirmed by the cloud server.

[0063] Sub-step S32: If the confidence score is greater than or equal to the second preset threshold, the playback of the audio is interrupted.

[0064] When the confidence score reaches or exceeds the second preset threshold, it can be assumed that the user is highly likely to be issuing a valid interruption command. Therefore, the current audio playback is immediately paused or terminated, releasing the voice channel to prepare for receiving complete voice commands or executing control commands returned by the server. This operation significantly improves the interaction response speed in high-confidence scenarios, allowing for an initial response without waiting for cloud feedback, thus optimizing the user experience.

[0065] This solution establishes a device-driven hierarchical decision-making mechanism by setting a first preset threshold (for triggering uploads) and a second preset threshold (for triggering interruptions): low-confidence events are filtered, medium-confidence events are uploaded without interruption, and high-confidence events receive a rapid local response and are processed in collaboration with the cloud server. Compared to existing technologies that lack end-side prediction and rely solely on simple audio pickup followed by full upload, this solution effectively reduces invalid network transmissions and avoids response delays caused by cloud round-trip latency. Especially under poor network conditions, it ensures immediate processing of high-priority interruptions, significantly optimizing the naturalness and reliability of human-computer voice interaction.

[0066] Step 206: If the confidence score is higher than the first preset threshold, the ambient sound is sent to the server; In this embodiment of the invention, when the confidence score is higher than the first preset threshold, it can be considered that there is a certain degree of possibility of user voice in the current environment. Although it may not reach the high confidence level of immediately interrupting playback (i.e., it has not reached the higher second preset threshold), it still has the value of further analysis, and thus triggers the upload process.

[0067] In some embodiments, the collected raw ambient sound can be used as the data to be uploaded. To improve transmission efficiency and reduce bandwidth usage, selected audio segments can also be compressed and encoded, and sent to a remote server via a network protocol. This invention does not limit the specific compression encoding method or network protocol selection, and can be chosen according to actual needs.

[0068] In other embodiments, the uploaded data packet may carry key metadata, which may include, but is not limited to, contextual information such as timestamps, current confidence scores, acoustic scene types, and current audio playback progress.

[0069] Step 207: Receive the control command sent by the server based on the ambient sound, and execute the control command to play the audio or interrupt the playback of the audio.

[0070] Voice interaction devices can receive control commands from a server. These commands are the final decision generated by the server after performing speech recognition and semantic understanding based on the ambient sound uploaded by the device. The device can correct or confirm the current audio playback status according to the control commands. Control commands can explicitly indicate whether to "interrupt audio playback" or "play audio." The device must perform the corresponding operation based on its current actual playback status to ensure consistency between local behavior and cloud-based semantic judgment.

[0071] In some embodiments, step 207 specifically includes the following sub-steps: Sub-step S41: If, while continuing to play the audio, a control command instructing to interrupt the playback of the audio is received, then the playback of the audio is interrupted; if, a control command instructing to play the audio is received, then the playback of the audio continues. In cases where audio playback continues due to a confidence score below the second preset threshold (e.g., within a medium confidence range, employing a conservative strategy), if the server accurately identifies and confirms a valid user instruction and returns an interrupt playback instruction, the device can immediately stop the current audio output. If the server determines there is no valid interruption intention and returns an audio playback instruction, the device can maintain the current playback state, avoiding unnecessary interruptions and ensuring the continuity of the main task.

[0072] In sub-step S42, if the playback of the audio is interrupted and a control command instructing the playback of the audio to be interrupted is received, the playback of the audio is kept interrupted; if the control command instructing the playback of the audio is received, the playback of the audio is resumed.

[0073] In the case of an active interruption of playback due to a confidence score reaching or exceeding a second preset threshold (aggressive strategy), if the server confirms the interruption is reasonable (e.g., successfully recognizing a valid voice command), it returns an interruption playback command, and the device can continue in the interrupted state and enter the subsequent command processing flow; if the server determines it is a false trigger (e.g., no valid voice was detected or the recognition result does not constitute an interruption intention) and returns an audio playback command, the device can roll back and resume the previously interrupted audio playback (e.g., resume audio content from the breakpoint), thereby correcting the erroneous operation and ensuring the continuity and accuracy of the user experience.

[0074] In some embodiments, the method further includes: recording the confidence score corresponding to the ambient sound sent to the server each time and the control command sent by the server, and constructing a historical interaction database; The preset weights are updated periodically using the historical interaction database.

[0075] In some embodiments, during the execution of the above steps, the voice interaction device may also simultaneously record key interaction data, including the confidence score corresponding to the ambient sound uploaded to the server, the final control command returned by the server (such as interrupting or resuming playback), and optional contextual information (such as acoustic scene type, voice duration, signal-to-noise ratio, etc.). This data is stored locally or in the cloud in a time-series format to form a historical interaction database. This historical interaction database essentially reflects the consistency or deviation between the on-device prediction results and the final cloud-based judgment, providing a basis for system self-optimization.

[0076] Based on the aforementioned historical interaction database, the preset weights used to calculate the confidence score in step 204 can be calibrated or updated periodically (e.g., daily, weekly, or when the device is idle).

[0077] Specifically, by analyzing the statistical correlation between different features (such as residual energy ratio, speech probability, signal-to-noise ratio, and speech duration) and the final control commands in a large number of samples, machine learning methods (such as logistic regression, gradient boosting, or online learning algorithms) can be used to re-optimize the weight allocation of each feature. This makes the calculation of confidence scores more closely reflect real user behavior and actual acoustic environments. As the number of times a user uses the device increases, its judgment of what constitutes a valid interruption can continuously align with the user's personal habits, thereby continuously improving the accuracy of interruption detection, reducing false triggers and missed triggers, and further enhancing the naturalness and intelligence of voice interaction.

[0078] This invention significantly reduces the false interruption rate of voice interaction devices during audio playback by acquiring ambient sound during playback; determining the residual signal between the audio and the ambient sound; performing voice activity detection on the residual signal to obtain a voice probability characterizing the possibility that it contains user voice; determining a confidence score characterizing the possibility that the user voice constitutes a valid interruption based on the audio, the residual signal, and the voice probability; continuing to play the audio or interrupting playback based on the confidence score; and sending the ambient sound to the server when the confidence score is higher than a first preset threshold, receiving and executing control commands returned by the server to finally determine the audio playback status. This approach ensures timely and reliable response to valid user interruption commands, thereby improving the accuracy and fluency of interaction in complex acoustic environments.

[0079] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0080] It should be noted that the voice interruption detection method provided in this embodiment of the invention can be executed by a voice interruption detection device, or a control module within the voice interruption detection device for executing the voice interruption detection method. This embodiment of the invention uses the execution of the voice interruption detection method by a voice interruption detection device as an example to illustrate the voice interruption detection method provided in this embodiment of the invention.

[0081] Figure 3 This is a structural block diagram of a voice interruption detection device provided in an embodiment of the present invention.

[0082] like Figure 3 As shown in the figure, the voice interruption detection device provided in this embodiment of the invention is applied to a voice interaction device and may specifically include the following modules: The ambient sound acquisition module 301 is used to acquire ambient sound during audio playback. The residual signal determination module 302 is used to determine the residual signals of the audio and the ambient sound; The voice activity detection module 303 is used to perform voice activity detection on the residual signal to obtain a voice probability, wherein the voice probability represents the possibility that the residual signal contains the user's voice. The confidence determination module 304 is used to determine a confidence score based on the audio, the residual signal, and the speech probability, wherein the confidence score represents the probability that the user's speech constitutes a valid interruption; Local decision module 305 is used to continue playing the audio or interrupt the playback of the audio based on the confidence score; An ambient sound sending module 306 is used to send the ambient sound to a server when the confidence score is higher than a first preset threshold. The instruction execution module 307 is used to receive control instructions sent by the server according to the ambient sound, and execute the control instructions to play the audio or interrupt the playback of the audio.

[0083] This invention significantly reduces the false interruption rate of voice interaction devices during audio playback by acquiring ambient sound during playback; determining the residual signal between the audio and the ambient sound; performing voice activity detection on the residual signal to obtain a voice probability characterizing the possibility that it contains user voice; determining a confidence score characterizing the possibility that the user voice constitutes a valid interruption based on the audio, the residual signal, and the voice probability; continuing to play the audio or interrupting playback based on the confidence score; and sending the ambient sound to the server when the confidence score is higher than a first preset threshold, receiving and executing control commands returned by the server to finally determine the audio playback status. This approach ensures timely and reliable response to valid user interruption commands, thereby improving the accuracy and fluency of interaction in complex acoustic environments.

[0084] As the apparatus embodiment is basically similar to the method embodiment, it is described in a relatively simple manner. For relevant details, please refer to the description of the method embodiment.

[0085] This invention also provides an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described speech interruption detection method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0086] It should be noted that the electronic devices in the embodiments of the present invention include the mobile electronic devices and non-mobile electronic devices described above.

[0087] This invention also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described speech interruption detection method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0088] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0089] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0090] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0091] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0092] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0094] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0095] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0096] The above provides a detailed description of the speech interruption detection method, apparatus, electronic device, and computer-readable storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for detecting speech interruptions, characterized in that, Applied to voice interaction devices, the method includes: During audio playback, ambient sound is captured. Determine the residual signals of the audio and the ambient sound; Speech activity detection is performed on the residual signal to obtain a speech probability, which represents the likelihood that the residual signal contains the user's speech. A confidence score is determined based on the audio, the residual signal, and the speech probability, wherein the confidence score represents the probability that the user's speech constitutes a valid interruption; The audio can be played either continue or interrupted based on the confidence score. If the confidence score is higher than a first preset threshold, the ambient sound is sent to the server; The system receives control commands sent by the server based on the ambient sound and executes the control commands to play the audio or interrupt the playback of the audio.

2. The speech interruption detection method according to claim 1, characterized in that, Determining the residual signals of the audio and the ambient sound includes: The ambient sound collected is subjected to echo cancellation processing based on the audio to obtain the residual signal.

3. The speech interruption detection method according to claim 1, characterized in that, Before performing voice activity detection on the residual signal, the method further includes: Calculate the residual energy ratio, which is the ratio of the energy of the residual signal to the energy of the audio signal; The current acoustic scene is determined based on the residual energy ratio; When the current acoustic scene is a mixed sound scene or a pure human voice scene, the residual signal is used for speech activity detection.

4. The speech interruption detection method according to claim 3, characterized in that, The step of determining the confidence score based on the audio, the residual signal, and the speech probability includes: Obtain the signal-to-noise ratio and speech duration of the current environment; The residual energy ratio, the speech probability, the signal-to-noise ratio, and the speech duration are normalized. The confidence score is obtained by weighting and summing the normalized parameters according to preset weights.

5. The speech interruption detection method according to claim 1, characterized in that, The step of continuing to play the audio or interrupting playback of the audio based on the confidence score includes: If the confidence score is lower than the second preset threshold, the audio continues to play; if the second preset threshold is higher than the first preset threshold. If the confidence score is greater than or equal to the second preset threshold, the playback of the audio is interrupted.

6. The speech interruption detection method according to claim 1, characterized in that, The step of receiving control commands sent by the server based on the ambient sound, and executing the control commands to play the audio or interrupt the playback of the audio, includes: If the audio continues to play, and a control command instructing the playback of the audio is received, then the playback of the audio is interrupted; if the control command instructing the playback of the audio is received, then the playback of the audio continues. If the audio playback is interrupted, and a control command instructing the interruption of the audio playback is received, then the interruption of the audio playback is maintained; if the control command instructing the playback of the audio playback is received, then the audio playback is resumed.

7. The speech interruption detection method according to claim 4, characterized in that, Also includes: Record the confidence score corresponding to the environmental sound sent to the server each time and the control command sent by the server to build a historical interaction database; The preset weights are updated periodically using the historical interaction database.

8. A voice interruption detection device, characterized in that, The device is used in voice interaction devices and includes: An ambient sound acquisition module is used to acquire ambient sound during audio playback. A residual signal determination module is used to determine the residual signals of the audio and the ambient sound; A voice activity detection module is used to perform voice activity detection on the residual signal to obtain a voice probability, wherein the voice probability represents the possibility that the residual signal contains the user's voice. A confidence determination module is used to determine a confidence score based on the audio, the residual signal, and the speech probability, wherein the confidence score represents the probability that the user's speech constitutes a valid interruption; The local decision module is used to continue playing the audio or interrupt the playback of the audio based on the confidence score; An ambient sound transmission module is used to transmit the ambient sound to a server when the confidence score is higher than a first preset threshold. The instruction execution module is used to receive control instructions sent by the server based on the ambient sound, and execute the control instructions to play the audio or interrupt the playback of the audio.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the voice interruption detection method as described in claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the speech interruption detection method as described in claims 1-7.

Images