Outbound call real-time interruption processing method and device, computer equipment and storage medium

Abstract

The invention discloses an outbound call real-time interruption processing method and device, computer equipment and a storage medium. The method comprises the following steps: when a preset verbal skill is played, monitoring and converting an audio stream into a text stream in real time through VAD and ASR services; inputting the processed data into a streaming interruption model for node configuration analysis; judging whether interruption is supported or not; if yes, when the confidence coefficient of the streaming interruption model exceeds a threshold value, stopping playing the preset verbal skill and switching to a waiting / recognition mode until the VAD confirms that the speaking of the user is finished; if not, monitoring the audio stream without interrupting the playing until the VAD confirms that the speaking of the user is ended; obtaining the whole dialogue content of the round; inputting the data into a whole-section hanging-up model and a high-risk model for posterior judgment; and performing corresponding processing, and temporarily storing the whole dialogue content of the round. By means of the method, high-precision intention recognition can be guaranteed, meanwhile, low-delay response speed can be provided, and unnecessary false triggering conditions are effectively reduced.

Description

Technical Field

[0001] This invention relates to human-computer interaction methods, and more specifically to methods, apparatus, computer equipment, and storage media for handling real-time interruption of outbound calls. Background Technology

[0002] In outbound call robot systems, the traditional approach is to recognize and analyze the user's voice input only after the robot has played a pre-set script or waited for a period of silence. This mechanism typically involves buffering the user's voice first, and then sending the audio data to the system's intent recognition module for processing after the robot has finished playing the current utterance.

[0003] However, this workflow reveals several technical challenges in practical applications: when a user attempts to interrupt or interrupt the robot's speech, if the system cannot detect this behavior in time and make corresponding adjustments, it will continue to play the pre-set script. This not only affects the user experience but may also reduce the accuracy of subsequent intent recognition processes due to missing key information. In some special cases, such as voicemail, voice assistant interaction, or extension prompts, the system may fail to recognize accurately, resulting in ineffective use of call resources or even mistakenly triggering unrelated business logic branches. This situation can significantly reduce outbound call efficiency and may lead to confusion in business processing logic.

[0004] Therefore, it is necessary to design a new method to monitor the user's voice input in real time during the outbound call robot's playback, accurately distinguish between user interruptions and accidental hang-ups, and execute corresponding policy actions based on different dialogue nodes. This ensures high-precision intent recognition while providing low-latency response speed, effectively reducing unnecessary false triggers, and thus improving the overall system's interactive experience and operational efficiency. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, apparatus, computer equipment and storage medium for real-time interruption processing of outbound calls.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for handling real-time interruption of outbound calls, comprising:

[0007] When playing preset scripts, the audio stream is monitored and converted into a text stream in real time through VAD and ASR services;

[0008] The processed text stream is then input into the streaming interruption model for node configuration analysis to obtain the analysis results.

[0009] Determine whether the analysis results support interruption;

[0010] If the analysis result supports interruption, then when the confidence level of the streaming interruption model exceeds the threshold, the playback of the preset script will stop and the system will switch to wait / recognition mode until the VAD confirms that the user's speech has ended.

[0011] If the analysis results do not support interruption, the audio stream will be monitored but playback will not be interrupted until VAD confirms that the user's speech has ended.

[0012] Get the entire conversation content for this round;

[0013] The entire dialogue content of this round is input into the whole hang-up model and the high-risk model for posterior judgment to obtain the posterior judgment result;

[0014] The corresponding processing is performed based on the posterior judgment result, and the entire dialogue content of this round is temporarily stored.

[0015] The further technical solution is as follows: when playing the preset script, the audio stream is monitored and converted into a text stream in real time through VAD and ASR services, including:

[0016] When playing the preset script, the VAD service is used to monitor the audio stream in real time to determine the start and end times of the user's speech;

[0017] The audio signal is converted into a text stream using the ASR service. When a new streaming result is received, the newly obtained text segment is appended to the text stream to update the text stream.

[0018] The further technical solution is as follows: the streaming interruption model is obtained by using a text stream with whether it is interrupted as a sample to train a text classification model, and N-gram features are introduced to capture local word order information during the text stream processing.

[0019] The further technical solution is as follows: the whole-segment hang-up model is obtained by using a text stream with labels indicating whether the text has been hung up as a sample to train a text classification model, and N-gram features are introduced during the text stream processing to capture local word order information.

[0020] The further technical solution is as follows: the high-risk model is obtained by using a text stream with labels indicating whether specified key content exists in the user's language as samples to train a text classification model, and N-gram features are introduced to capture local word order information during the text stream processing.

[0021] The further technical solution is as follows: the processed text stream is input into the streaming interruption model for node configuration analysis to obtain analysis results, including:

[0022] A vocabulary is constructed based on ASR-transcribed text, and the text stream is segmented into words and N-gram features. The words and N-gram features are then converted into integer IDs through index mapping to form the input sequence.

[0023] The input sequence is fed into the streaming interruption model for node configuration analysis to obtain the analysis results.

[0024] The further technical solution is as follows: the input sequence is input into the streaming interruption model for node configuration analysis to obtain analysis results, including:

[0025] In the input layer of the streaming interruption model, the input sequence is transformed into a corresponding vector representation using a predefined embedding matrix;

[0026] The hidden layer of the streaming interruption model processes all vectors provided by the input layer through a global averaging operation, adds them together and calculates the average to generate a fixed-length sentence vector, wherein the sentence vector includes semantic information of words and local order features;

[0027] The output layer of the streaming interruption model receives sentence vectors generated by the hidden layer. After linear transformation and processing by the Softmax function, a probability distribution for each category is generated. When the predicted probability of a specified category exceeds a set threshold, the analysis result is determined to support the intention to interrupt.

[0028] The present invention also provides a real-time outbound call interruption processing device, comprising:

[0029] The conversion unit is used to monitor and convert the audio stream into a text stream in real time through VAD and ASR services when playing preset scripts;

[0030] The node analysis unit is used to process the text stream and input it into the streaming interruption model for node configuration analysis to obtain analysis results.

[0031] A judgment unit is used to determine whether the analysis result supports interruption;

[0032] The node interruption processing unit is used to stop playing the preset script and switch to the waiting / recognition mode when the confidence of the streaming interruption model exceeds the threshold if the analysis result supports interruption, until the VAD confirms that the user's speech has ended.

[0033] A continuous monitoring unit is used to monitor the audio stream without interrupting playback if the analysis result does not support interruption, until the VAD confirms that the user's speech has ended.

[0034] The whole content acquisition unit is used to acquire the entire dialogue content in this round;

[0035] The posterior decision unit is used to input the entire dialogue content of this round into the whole hang-up model and the high-risk model for posterior decision to obtain the posterior decision result.

[0036] The temporary storage unit is used to perform corresponding processing based on the posterior judgment result and to temporarily store the entire dialogue content of this round.

[0037] The present invention also provides a computer device, the computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the above-described method.

[0038] The present invention also provides a storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0039] The advantages of this invention compared to existing technologies are as follows: This invention utilizes VAD and ASR services to monitor and convert the audio stream into a text stream in real time during the playback of a preset script. The text stream is then input into a streaming interruption model for node configuration analysis to determine whether interruption is supported. If interruption is supported and the confidence level exceeds a threshold, playback is immediately stopped and the system switches to a wait / recognition mode until the user finishes speaking. If interruption is not supported, only the audio stream is monitored without interrupting playback. Next, the entire dialogue content is acquired and subjected to posterior judgment using a high-precision hang-up model and a high-risk model. Based on the results, appropriate processing is performed, and the dialogue content is temporarily stored. This method enables real-time monitoring of user voice input during outbound call robot playback, accurate differentiation between interruption and accidental hang-up behaviors, and strategic actions based on different dialogue nodes. This ensures the accuracy of intent recognition and the efficiency of response speed, reduces false triggering, and improves the system's interactive experience and operational efficiency.

[0040] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram illustrating an application scenario of the outbound call real-time interruption processing method provided in an embodiment of the present invention;

[0043] Figure 2 A flowchart illustrating the outbound call real-time interruption processing method provided in an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the fastText architecture provided in an embodiment of the present invention;

[0045] Figure 4 This is a schematic block diagram of an outbound call real-time interruption processing device provided in an embodiment of the present invention;

[0046] Figure 5 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0049] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0050] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0051] Please see Figure 1 and Figure 2 , Figure 1 This is a schematic diagram illustrating an application scenario of the outbound call real-time interruption processing method provided in an embodiment of the present invention. Figure 2This is a flowchart illustrating the real-time outbound call interruption processing method provided in this embodiment of the invention. This method is applied to a server. During the playback of a preset script, the user's voice input is monitored in real-time via VAD and ASR services and converted into a text stream. A streaming interruption model is used to analyze the text stream to determine whether interruption is supported. Based on the confidence level, it is decided whether to interrupt the current script playback and switch to a waiting / recognition mode. Simultaneously, after the dialogue ends, a post-hoc judgment is performed using a whole-segment hang-up model and a high-risk model to accurately distinguish between the user's interruption intent and accidental hang-up behavior, executing corresponding policy actions according to different nodes. This method not only ensures high-precision intent recognition but also guarantees low-latency response, effectively reducing the false trigger rate and improving the system's interactive experience and operational efficiency. The entire process integrates steps such as vocabulary construction, N-gram feature extraction, and text classification model training, strengthening the ability to capture local word order information, enabling the system to more accurately understand and respond to user commands. Outbound call robots detect and respond to user interruptions in real time during calls, and are an online processing method that identifies special and high-risk business situations. This aims to improve the response speed and interaction accuracy of outbound call systems, while reducing false triggers and resource waste.

[0052] Specifically, Voice Activity Detection (VAD) is a technique used to determine the presence of speech signals in an audio stream, helping to pinpoint the start and end times of user speech segments.

[0053] Automatic speech recognition (ASR): a technology that converts speech signals into text, which is divided into two modes: streaming recognition and whole-segment recognition.

[0054] Natural Language Understanding (NLU): A technology that performs semantic understanding and intent recognition on user speech or text.

[0055] Text-to-speech (TTS): A technology that synthesizes text content into speech output.

[0056] Dialogue Manager (DM): A module that determines the next system behavior based on NLU output and preset rules.

[0057] Node: A single step or action in the robot's dialogue process, divided into "interruptible nodes" and "non-interruptible nodes".

[0058] Segment A / Segment B: These refer to the portion of the user's speech that falls within the robot's playback phase and the portion of the speech that remains unplayed after playback ends, respectively.

[0059] Lightweight text classification model: A model for quickly identifying short texts, mainly used for real-time interruption, hanging up, and preliminary assessment of high business risks.

[0060] Figure 2This is a flowchart illustrating the outbound call real-time interruption handling method provided in an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110 to S180.

[0061] S110. When playing preset scripts, the audio stream is monitored and converted into a text stream in real time through VAD and ASR services.

[0062] In this embodiment, the text stream refers to a continuous text record generated in real time by converting and accumulating continuous audio signals through the ASR service during the user's speech. It is a dynamically updated process, a sequence of text data that grows continuously as the user speaks.

[0063] Step S110 describes how, during the outbound call robot's playback of a preset script, the user's audio input is monitored in real time via VAD (Voice Activity Detection) and ASR (Automatic Speech Recognition) services and converted into a text stream. This process is a key step in achieving rapid response to user intent.

[0064] In one embodiment, step S110 described above may include steps S111 to S112.

[0065] S111. When playing the preset script, use the VAD service to monitor the audio stream in real time to determine the start and end times of the user's speech.

[0066] While playing the preset script, the system runs a VAD service to monitor the audio stream in real time. Its main task is to accurately identify the moments when the user begins and stops speaking. This is to accurately capture a complete round of the user's speech (segment A), even while the robot is playing the script. VAD technology effectively distinguishes background noise from actual human voice, ensuring that subsequent processing is only triggered when the user actually begins speaking.

[0067] S112. Use the ASR service to convert the audio signal into a text stream. When a new streaming result is received, append the newly obtained text segment to the text stream to update the text stream.

[0068] Once VAD confirms that the user has started speaking, the ASR service begins real-time transcription of the audio signal. ASR converts the user's voice into text format and continuously updates the text stream as the user continues speaking. Whenever ASR generates a new streaming recognition result (i.e., a new text segment), that segment is added to the existing text stream, forming a continuous text record. During this process, the system not only needs to efficiently process the audio data but also ensure the real-time updating of the text stream to provide the latest analysis material to subsequent modules (such as interruption models and hang-up models).

[0069] The core of this part lies in combining VAD and ASR technologies to extract valuable information from the user's voice input as quickly as possible without affecting the user experience, and then making corresponding decisions or actions based on this information. Effective execution of this step is crucial for improving system response speed, reducing misjudgments, and enhancing the overall interactive experience.

[0070] S120. The processed text stream is input into the streaming interruption model for node configuration analysis to obtain the analysis results.

[0071] In this embodiment, the analysis result refers to the streaming interruption model processing and classifying the text stream transcribed by ASR to determine whether the user has the intention to interrupt, and confirming the intention to interrupt when the predicted probability exceeds a set threshold.

[0072] The streaming interruption model is obtained by using a text stream labeled with whether it is interrupted as a sample to train a text classification model, and N-gram features are introduced during the text stream processing to capture local word order information.

[0073] The streaming interruption model is a lightweight text classification model specifically designed for real-time identification of whether a user intends to interrupt. It is trained on text data labeled with interruption intent (typically from historical call logs and manually annotated). To improve the model's understanding of short texts, particularly in capturing local word order information, N-gram features are introduced during training.

[0074] The input first needs to undergo data preprocessing, converting the text transcribed from ASR into a numerical sequence that the model can read, and then extracting features, as follows:

[0075] Vocabulary construction: Scan the training corpus and count word frequencies. Retain words with frequencies above a set threshold to form the basic vocabulary. Each word is assigned a unique integer index (ranging from 0 to V-1).

[0076] Word segmentation and N-gram generation: Based on a vocabulary, the text transcribed from ASR is segmented into words, and N-gram features (such as Bi-gram, with a sliding window size of 2) are generated through a sliding window to capture local word order information. Index mapping: Based on a pre-built vocabulary, the segmented word sequences and N-gram fragments are mapped to unique integer indices (IDs).

[0077] Output: Generate a discrete sequence of integer indices. , as input to the neural network.

[0078] Model architecture such as Figure 3 As shown, it mainly consists of three layers: an input layer, a hidden layer, and an output layer.

[0079] Input layer:

[0080] Input: Receive the preprocessed discrete index sequence X, which contains both the IDs of the original words and the calculated N-gram feature IDs.

[0081] Embedding matrix: The weight parameters corresponding to the input layer are a matrix A with dimensions (V+B). D. Where V is the vocabulary size, B is the number of N-gram features, and D is the dimension of the embedding vector. Each row of the matrix corresponds to a vector representation of a feature.

[0082] The input layer does not perform mathematical operations; instead, it acts as a lookup table. Based on the input index sequence X, it directly extracts the corresponding M dense vectors from matrix A. Then pass it to the hidden layer.

[0083] Hidden layer:

[0084] Computational logic: The hidden layer performs a global average operation. It sums all vectors extracted from the input layer (including word vectors and N-gram vectors) element-wise and then calculates the average. Described as follows: , where H is a real vector of dimension D.

[0085] This layer compresses a variable-length text feature sequence into a fixed-length sentence vector. By superimposing word vectors with N-gram vectors, the sentence vector not only contains the semantics of words, but also implicitly contains the local sequence features of the text, which is crucial for recognizing specific phrase patterns (such as "wait a moment") in "streaming interruptions".

[0086] Output layer:

[0087] Linear transformation: The sentence vector H output from the hidden layer is input into a fully connected layer. This layer has a weight matrix W (dimension D). K, where K is the number of categories. Calculate the logits: .

[0088] Probability normalization: The Softmax function is used to calculate the probability of belonging to each category. .

[0089] Output: Outputs a probability distribution vector. If the predicted probability exceeds a preset threshold, the system determines it as a specific intent (such as "triggering interruption") and issues a control command.

[0090] This streaming interruption model is trained using supervised learning, with data consisting of manually annotated historical call texts and "text-category label" pairs.

[0091] In one embodiment, step S120 described above may include steps S121 to S122.

[0092] S121. Construct a vocabulary based on ASR-transcribed text, and segment the text stream into words and N-gram features. Convert the words and N-gram features into integer IDs through index mapping to form an input sequence.

[0093] In this embodiment, the input sequence refers to multiple consecutive subsequences into which the text stream obtained by ASR (Automatic Speech Recognition) is divided in chronological order, and each subsequence contains the user's voice content within a certain time window.

[0094] First, all available training corpora are scanned, and the frequency of each word is counted. Words with frequencies exceeding a certain threshold are retained, thus creating a basic vocabulary. Each word is assigned a unique integer index.

[0095] Next, this basic vocabulary is used to segment the text stream obtained from ASR, and N-gram features (e.g., Bi-gram) are generated using a sliding window technique to capture local order information in the text.

[0096] Then, based on the pre-built vocabulary, the segmented word sequence and the generated N-gram fragments are mapped to unique integer IDs to form the final input sequence.

[0097] S122. Input the input sequence into the streaming interruption model for node configuration analysis to obtain the analysis results.

[0098] In one embodiment, step S122 described above may include steps S1221 to S1223.

[0099] S1221. In the input layer of the streaming interruption model, the input sequence is transformed into a corresponding vector representation using a predefined embedding matrix.

[0100] The input layer does not perform complex mathematical operations; instead, it acts as a lookup table. It extracts the corresponding vector representation from a predefined embedding matrix based on the integer IDs in the input sequence.

[0101] S1222. The hidden layer of the streaming interruption model processes all vectors provided by the input layer through a global averaging operation, adds them together and calculates the average value to generate a fixed-length sentence vector, wherein the sentence vector includes semantic information of words and local sequence features.

[0102] The hidden layer performs a global averaging operation, which involves summing all vectors provided by the input layer (including original word vectors and N-gram vectors) element by element and averaging them to generate a fixed-length sentence vector. This step not only captures the semantic information of the words but also implicitly includes the local sequence features of the text.

[0103] S1223. The output layer of the streaming interruption model receives the sentence vector generated by the hidden layer. After linear transformation and processing by the Softmax function, the probability distribution of each category is generated. When the predicted probability of a specified category exceeds a set threshold, the analysis result is determined to support the intention to interrupt.

[0104] The output layer is responsible for transforming the sentence vectors generated by the hidden layer into category logits through a linear transformation, then normalizing them using the Softmax function, and finally outputting the probability distribution of each category. If the predicted probability of a certain category exceeds a preset confidence threshold, the system will determine that the text stream expresses an intention to support interruption and trigger the corresponding action accordingly.

[0105] Through the above steps, the streaming interruption model can detect and respond to user interruptions during the playback phase with a latency of seconds or less, while ensuring the accuracy of the judgment and the flexibility of the system.

[0106] S130. Determine whether the analysis result supports interruption.

[0107] This step primarily determines whether user interruption is allowed during the current playback stage based on the dialogue node configuration. If the node is configured to "support interruption," the system will enable a streaming interruption model to monitor user input in real time; otherwise, no interruption processing will be performed, and only the audio stream will be monitored and cached.

[0108] S140. If the analysis result supports interruption, then when the confidence level of the streaming interruption model exceeds the threshold, stop playing the preset script and switch to the waiting / recognition mode until the VAD confirms that the user's speech has ended, and then execute step S160.

[0109] Once the streaming interruption model determines the user's interruption intent based on the received ASR text fragments, and the confidence level exceeds a preset threshold, the system will immediately interrupt the current speech playback. After playback stops, the system enters a wait / recognition mode, preparing to receive and process further statements from the user. Voice activity detection (VAD) technology is used to determine when the user's speech ends, thus ensuring that the system does not prematurely return to the original playback flow or jump to other operations.

[0110] S150. If the analysis result does not support interruption, the audio stream is monitored but playback is not interrupted until VAD confirms that the user's speech has ended.

[0111] Even if a user interrupts, if the node is configured to "not support interruption," the system will ignore these interruptions and continue playing the original content. Although no interruption is performed, the system will still monitor user speech through VAD and ASR services and cache it for later processing.

[0112] S160, Obtain the entire dialogue content for this round.

[0113] Regardless of whether an interruption occurs, the system collects and organizes the content of the complete conversation after a round of user speech ends. This includes segment A during the playback phase and segment B (if any) during the waiting phase.

[0114] S170. Input the entire dialogue content of this round into the whole hanging-up model and the high-risk model for posterior judgment to obtain the posterior judgment result.

[0115] In this embodiment, the collected complete dialogue text is input into the complete hang-up model and the high-risk model for further analysis. The so-called "posterior judgment" here refers to making a more accurate decision based on more comprehensive information (i.e., the entire dialogue content rather than immediate fragments), such as whether the call should be hung up or whether there is a high business risk.

[0116] S180. Perform corresponding processing based on the posterior judgment result, and temporarily store the entire dialogue content of this round.

[0117] Based on the posterior judgment result obtained in the previous step, the system will execute the corresponding action. For example:

[0118] If the entire hang-up model deems it necessary to terminate the call, it will trigger the hang-up process.

[0119] If a high-risk model detects a potential problem, it may initiate risk management procedures such as using reassuring language or transferring the user to a human agent.

[0120] If none of the above models detect the corresponding hang-up or risk, then after NLU processing, the normal process will proceed.

[0121] In addition, the content of this round of dialogue will be temporarily saved for future review or as training data to improve model performance.

[0122] Through the above process, this embodiment not only improves the response speed and accuracy to emergencies during outbound calls, but also enhances the system's flexibility and configurability, enabling the needs of different business scenarios to be effectively met.

[0123] The aforementioned whole-segment hang-up model is obtained by using a text stream labeled with whether it has been hung up as a sample to train a text classification model. In the process of processing the text stream, N-gram features are introduced to capture local word order information.

[0124] The high-risk model is obtained by training a text classification model using processed text streams labeled with whether specified key content exists in the user's language. In the process of processing the text stream, N-gram features are introduced to capture local word order information.

[0125] In this embodiment, to improve the outbound call robot's ability to detect user interruptions, abnormal hang-ups, and high business risks, three core models are designed: a streaming interruption model, a whole-segment hang-up model, and a whole-segment high business risk model. Furthermore, Natural Language Understanding (NLU) is used as an external service to supplement or cover the judgment results based on these lightweight models.

[0126] The streaming interruption model (real-time) receives intermediate text output (which can be the text itself, text plus confidence scores, or speech features, etc.) from streaming Automatic Speech Recognition (ASR) and determines whether it contains an interruption intent. Its main function is to quickly respond to user interruptions during playback and trigger the termination of the current playback action at supported interruption points. This ensures the system can react within seconds or even less, providing a smoother user experience.

[0127] The whole-segment hang-up model (posterior) differs from the streaming interruption model. It takes the entire ASR text as input to evaluate whether the call needs to be actively ended. For example, when a voice assistant prompt, a message prompt, or a clear rejection statement is detected, the model will decide whether to terminate the current call. This posterior analysis method relies on the entire dialogue context rather than immediate fragments, thus more accurately capturing situations where hanging up is necessary.

[0128] The high-risk business process model (posterior) focuses on identifying complaints, abusive language, or other high-risk content requiring special handling in user language. Upon detection of such content, it triggers a corresponding downgrade process or transfers the conversation to a human agent for processing. Similarly, this model uses the entire ASR text as input to ensure that decisions are based on comprehensive information.

[0129] In addition to the three models mentioned above, Natural Language Understanding (NLU) is also integrated as a means of calling external services. NLU can perform deep semantic analysis of a user's speech or text to identify specific user questions or intentions. This not only enhances the accuracy of the model but can also correct initial judgments made by lightweight models when necessary. For example, when a streaming model fails to reach the required confidence level but the entire text indicates a clear intention, NLU can be used to correct the action.

[0130] Considering real-time requirements, all three models—the streaming interruption model, the full-segment hang-up model, and the full-segment high-risk model—adopt the fastText architecture. FastText is a lightweight and efficient text classification model, particularly suitable for handling high-concurrency text streams. Its advantages lie in its extremely low computational latency and its ability to capture local word order information by introducing N-gram features, ensuring classification accuracy and efficiency. This approach makes it possible to achieve fast and accurate decision-making in outbound call robots while maintaining low resource consumption. The structures of these three models are identical; therefore, for the specific structures of the full-segment hang-up model and the high-risk model, please refer to the description of the streaming interruption model above, which will not be repeated here.

[0131] In summary, by combining streaming interruption models, whole-segment hang-up models, whole-segment high-business-risk models, and NLU services, and by using the fastText architecture to optimize performance, this invention effectively improves the ability of outbound call robots to handle complex interactive scenarios.

[0132] In this embodiment, to coordinate the action outputs of different models and ensure that the system can respond to user interactions efficiently and accurately, a decision-making mechanism based on priority rules is designed. The following is the specific process by which each model outputs actions according to its priority:

[0133] Streaming interruption model: When this model detects user intent at a node that supports interruption and reaches a preset confidence threshold, it will immediately stop the currently playing content and switch to a process of waiting for the user to speak. This operation has the highest priority, ensuring an immediate response to the user's interruption.

[0134] Whole-session hang-up model: If the whole-session hang-up model analyzes the entire conversation and determines that the call needs to be ended, then regardless of whether the current state is an interruptible node, the corresponding hang-up action will be triggered. This rule has the second highest priority, ensuring that the call can be terminated quickly and decisively when necessary.

[0135] The entire high-risk business model: Once it identifies that a user's language contains complaints, insults, or other high-risk content, the model will initiate a special processing procedure, such as providing reassuring messages or transferring the call to a human customer service representative. Depending on the specific strategy, it will decide whether to further execute a hang-up operation. This is the third level of priority, providing necessary intervention measures for potentially high-risk situations.

[0136] NLU Supplementation and Correction: Natural Language Understanding (NLU), as an external service, is used to supplement or correct the initial judgments of the aforementioned lightweight model. Especially when the streaming model fails to reach the required confidence level but the entire text clearly indicates intent, NLU can be used to adjust the course of action, ensuring more accurate decision-making. Furthermore, NLU is responsible for parsing the user's specific questions or intents, guiding the Dialogue Manager (DM) to select appropriate nodes and subsequent processes.

[0137] Dynamic adjustment and recording: Priority and confidence thresholds can be flexibly configured according to product strategy and security requirements. Simultaneously, all decision-making processes are recorded in detail during implementation, forming a log for subsequent backtracking analysis and model iteration optimization.

[0138] Through this multi-level, priority-based decision-making mechanism, combined with real-time monitoring and dynamic adjustment strategies, this embodiment effectively improves the outbound call robot's ability to handle complex interactive scenarios, while also enhancing the system's adaptability and reliability.

[0139] In addition, the system continuously calls the Voice Activity Detection (VAD) and Automatic Speech Recognition (ASR) services during playback. VAD detects the start and end times of user speech, while ASR converts the speech signal in the audio stream into a text stream. The results of both are returned in real time in a streaming manner and saved by the system for subsequent processing. Whenever a new streaming result is obtained, the system adds the text generated by ASR to the previously accumulated text and sends the newly added or accumulated text fragments to the streaming interruption model for analysis and judgment.

[0140] Support for interrupted nodes: If the current node is configured to "support interruption," and the confidence level given by the streaming interruption model based on the user's input exceeds a preset threshold, the system will immediately stop the current speech playback, pause TTS, and begin treating subsequent received audio as part of the user's speech. Simultaneously, it will record the trigger reason as "user interruption" and enter a wait / recognition mode until the VAD determines the end of this round of speech.

[0141] Nodes not supported for interruption: For nodes configured as "not supported for interruption," although VAD and streaming ASR are still executed to reset the mute timer and monitoring, the current playback will not be interrupted. Once VAD confirms that the user has finished speaking, the entire text segment A will be submitted to the whole-segment hang-up and high-risk models for posterior judgment. The text segment A will be cached and may be concatenated with segment B later for deeper recognition (including NLU, whole-segment hang-up model, and high-business-risk model) and corresponding processing.

[0142] Special handling during playback: If the end of segment A is detected during playback and the entire segment hang-up model is hit, the system can start the hang-up process immediately after playback ends (e.g., when an automatic voice response or extension prompt tone is detected) to save resources, according to the strategy.

[0143] When the robot finishes playing and enters a waiting state, any subsequent voice messages are considered as segment B. After segment B ends, the system will merge it with the previously cached segment A (if it exists), and send the merged complete text to the NLU, the whole-segment disconnection model, and the high-business-risk model for further judgment and processing.

[0144] If there is only segment A cache and no segment B (i.e., the user only speaks during the playback phase), the product strategy can determine whether to perform additional NLU processing on segment A or simply rely on the results of previous and subsequent decisions.

[0145] The output of NLU determines the subsequent business process, such as transferring to the corresponding node, playing an explanation, or transferring to human customer service.

[0146] To prevent false muting due to large intervals in the streaming ASR segments, the system resets the mute timer each time streaming ASR output is detected. This ensures that even if the user continues speaking, brief pauses will not be mistaken for silence. The duration of the mute timer and related thresholds can be configured according to product requirements.

[0147] For example, the robot plays a prompt message, during which the user interrupts to ask a question.

[0148] VAD detects the audio signal and activates streaming ASR. As the accumulated text volume increases, the streaming interruption model eventually identifies it as an interruption request. If the node allows interruption, the system immediately stops playback and switches to waiting mode. After the entire ASR is completed, the text is analyzed by NLU to determine the user's intent, and the Dialogue Manager (DM) navigates to the relevant knowledge base node to answer the question. If streaming analysis does not detect an interruption intent, but the entire ASR result, after NLU parsing, shows "complaint" or "abuse" content, a high-risk response procedure is triggered, and the case is downgraded according to the set strategy or transferred to human customer service.

[0149] The unique feature of this embodiment lies in its combination of streaming and whole-segment recognition methods, the application of a real-time lightweight classification model, and a node-level policy control mechanism, thereby achieving efficient real-time response capabilities, accurate recognition results, and flexible configurability for business applications. Furthermore, this design also considers ease of engineering implementation and efficient resource utilization, making it particularly suitable for deployment in edge computing environments.

[0150] Specifically, while the robot is playing the script, streaming speech recognition (ASR) and voice activity detection (VAD) are performed simultaneously. A lightweight classification model is used to analyze the short-term streaming text in real time to determine whether it is an interruption.

[0151] When a user's speech ends (the start and end times are determined by VAD), the entire speech is processed using high-precision ASR, and the entire recognition result is used for hang-up determination, high business risk assessment (such as complaints, insults, do not disturb), and deep semantic understanding (NLU).

[0152] Based on the configuration of the dialogue nodes (whether they support interruption or not), dynamic actions are taken based on the streaming judgment results: for nodes that support interruption, playback can be stopped immediately; for nodes that do not support interruption, only the speech is monitored and cached, and then processed uniformly after playback ends.

[0153] Priority determination and action execution strategies were designed to ensure that conflicts are handled according to established rules (e.g., emergency hang-up takes precedence over normal intent response, streaming interruption determination takes precedence over non-real-time intent determination, etc.).

[0154] Introduce a silent timer reset rule: reset the silent timer whenever a new streaming ASR intermediate result is received to prevent it from being mistakenly judged as silent due to short intervals.

[0155] By running VAD, streaming ASR, and a lightweight text classification model (interruption model) in parallel, the system detects user interruptions and quickly determines whether an interruption intent exists with a latency of seconds or less. After a user's turn of speech ends, a full-length high-precision ASR is executed for subsequent hang-up and high-business-risk model determinations, and as input to NLU for more accurate intent recognition. Based on node configuration (supporting / not supporting interruption), policy restrictions are implemented on streaming determination actions to ensure system flexibility and adaptability. Clear priority rules are designed to resolve conflict determination issues and optimize the decision-making process. A mute timer reset mechanism is introduced to avoid erroneously triggering mute prompts due to large intervals during continuous user speech.

[0156] The aforementioned real-time outbound call interruption handling method utilizes VAD and ASR services to monitor and convert the audio stream into a text stream during the playback of a preset script. The text stream is then input into a streaming interruption model for node configuration analysis to determine whether interruption is supported. If interruption is supported and the confidence level exceeds a threshold, playback is immediately stopped and the system switches to a wait / recognition mode until the user finishes speaking. If interruption is not supported, only the audio stream is monitored without interrupting playback. Next, the entire dialogue content is acquired and subjected to a post-hoc judgment using a high-precision hang-up model and a high-risk model. Based on the result, appropriate processing is performed, and the dialogue content is temporarily stored. This method enables real-time monitoring of user voice input during outbound call robot playback, accurately distinguishes between interruptions and accidental hang-ups, and takes strategic actions based on different dialogue nodes. This ensures the accuracy of intent recognition and the efficiency of response speed, reduces false triggers, and improves the system's interactive experience and operational efficiency.

[0157] Figure 4 This is a schematic block diagram of an outbound call real-time interruption processing device 300 provided in an embodiment of the present invention. Figure 4 As shown, corresponding to the above-described outbound call real-time interruption processing method, the present invention also provides an outbound call real-time interruption processing device 300. This outbound call real-time interruption processing device 300 includes a unit for executing the above-described outbound call real-time interruption processing method, and the device can be configured in a server. Specifically, please refer to... Figure 4 The outbound call real-time interruption processing device 300 includes a conversion unit 301, a node analysis unit 302, a judgment unit 303, a node interruption processing unit 304, a continuous monitoring unit 305, a whole content acquisition unit 306, a post-judgment unit 307, and a temporary storage unit 308.

[0158] The conversion unit 301 is used to monitor and convert the audio stream into a text stream in real time through VAD and ASR services when playing a preset script; the node analysis unit 302 is used to process the text stream and input it into the streaming interruption model for node configuration analysis to obtain analysis results; the judgment unit 303 is used to determine whether the analysis result supports interruption; the node interruption processing unit 304 is used to stop playing the preset script and switch to a waiting / recognition mode when the confidence level of the streaming interruption model exceeds a threshold if the analysis result supports interruption, until VAD is used. D confirms that the user's speech has ended; Continuous monitoring unit 305 is used to monitor the audio stream but not interrupt playback if the analysis result does not support interruption, until VAD confirms that the user's speech has ended; Full content acquisition unit 306 is used to acquire the full dialogue content of this round; Posterior judgment unit 307 is used to input the full dialogue content of this round into the full hang-up model and the high-risk model for posterior judgment to obtain the posterior judgment result; Temporary storage unit 308 is used to perform corresponding processing according to the posterior judgment result and temporarily store the full dialogue content of this round.

[0159] In one embodiment, the conversion unit 301 includes:

[0160] The real-time monitoring subunit is used to monitor the audio stream in real time using the VAD service when playing preset scripts, so as to determine the start and end times of the user's speech; the text stream conversion subunit is used to convert the audio signal into a text stream using the ASR service, and when a new streaming result is received, the newly obtained text segment is appended to the text stream to update the text stream.

[0161] In one embodiment, the node analysis unit 302 includes:

[0162] The preprocessing subunit is used to construct a vocabulary based on the text transcribed by ASR, and to segment the text stream into words and N-gram features. The words and N-gram features are converted into integer IDs through index mapping to form an input sequence. The node configuration analysis subunit is used to perform node configuration analysis based on the input sequence input into the streaming interruption model to obtain the analysis results.

[0163] In one embodiment, the node configuration analysis subunit includes:

[0164] The transformation module is used to transform the input sequence into a corresponding vector representation in the input layer of the streaming interruption model using a predefined embedding matrix. The sentence vector generation module is used to process all vectors provided by the input layer through a global averaging operation by the hidden layer of the streaming interruption model, sum them, and calculate the average to generate a fixed-length sentence vector, wherein the sentence vector includes semantic information of words and local order features. The classification module is used to receive the sentence vectors generated by the hidden layer by the output layer of the streaming interruption model, process them through linear transformation and the Softmax function, and generate the probability distribution of each category. When the predicted probability of a specified category exceeds a set threshold, the analysis result is determined to support the intention to interrupt.

[0165] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned outbound call real-time interruption processing device 300 and each unit can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0166] The aforementioned outbound call real-time interruption processing device 300 can be implemented as a computer program, which can, for example... Figure 5 It runs on the computer device shown.

[0167] Please see Figure 5 , Figure 5 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0168] See Figure 5 The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0169] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a real-time outbound call interruption processing method.

[0170] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0171] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a real-time outbound call interruption processing method.

[0172] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0173] The processor 502 is used to run the computer program 5032 stored in the memory to implement all the steps of the outbound call real-time interruption processing method.

[0174] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0175] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0176] Therefore, the present invention also provides a storage medium. This storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when executed by a processor, the computer program causes the processor to perform all the steps of the outbound call real-time interruption processing method.

[0177] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0178] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0179] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0180] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0181] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0182] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for handling real-time interruptions in outbound calls, characterized in that, include: When playing preset scripts, the audio stream is monitored and converted into a text stream in real time through VAD and ASR services; The processed text stream is then input into the streaming interruption model for node configuration analysis to obtain the analysis results. Determine whether the analysis results support interruption; If the analysis result supports interruption, then when the confidence level of the streaming interruption model exceeds the threshold, the playback of the preset script will stop and the system will switch to wait / recognition mode until the VAD confirms that the user's speech has ended. If the analysis results do not support interruption, the audio stream will be monitored but playback will not be interrupted until VAD confirms that the user's speech has ended. Get the entire conversation content for this round; The entire dialogue content of this round is input into the whole hang-up model and the high-risk model for posterior judgment to obtain the posterior judgment result; The corresponding processing is performed based on the posterior judgment result, and the entire dialogue content of this round is temporarily stored.

2. The outbound call real-time interruption processing method according to claim 1, characterized in that, The process of real-time monitoring and conversion of the audio stream into a text stream via VAD and ASR services during the playback of preset scripts includes: When playing the preset script, the VAD service is used to monitor the audio stream in real time to determine the start and end times of the user's speech; The audio signal is converted into a text stream using the ASR service. When a new streaming result is received, the newly obtained text segment is appended to the text stream to update the text stream.

3. The outbound call real-time interruption processing method according to claim 1, characterized in that, The streaming interruption model is obtained by using a text stream labeled with whether it is interrupted as a sample to train a text classification model, and N-gram features are introduced during the text stream processing to capture local word order information.

4. The outbound call real-time interruption processing method according to claim 1, characterized in that, The entire hanging-up model is obtained by using a text stream labeled with whether it has been hung up as a sample to train a text classification model. In the process of processing the text stream, N-gram features are introduced to capture local word order information.

5. The outbound call real-time interruption processing method according to claim 1, characterized in that, The high-risk model is obtained by training a text classification model using a processed text stream with tags indicating whether specified key content exists in the user's language as samples, and N-gram features are introduced during the text stream processing to capture local word order information.

6. The outbound call real-time interruption processing method according to claim 3, characterized in that, The text stream is processed and then input into a streaming interruption model for node configuration analysis to obtain analysis results, including: A vocabulary is constructed based on ASR-transcribed text, and the text stream is segmented into words and N-gram features. The words and N-gram features are then converted into integer IDs through index mapping to form the input sequence. The input sequence is fed into the streaming interruption model for node configuration analysis to obtain the analysis results.

7. The outbound call real-time interruption processing method according to claim 6, characterized in that, The step of inputting the input sequence into the streaming interruption model for node configuration analysis to obtain analysis results includes: In the input layer of the streaming interruption model, the input sequence is transformed into a corresponding vector representation using a predefined embedding matrix; The hidden layer of the streaming interruption model processes all vectors provided by the input layer through a global averaging operation, adds them together and calculates the average to generate a fixed-length sentence vector, wherein the sentence vector includes semantic information of words and local order features; The output layer of the streaming interruption model receives sentence vectors generated by the hidden layer. After linear transformation and processing by the Softmax function, a probability distribution for each category is generated. When the predicted probability of a specified category exceeds a set threshold, the analysis result is determined to support the intention to interrupt.

8. A real-time outbound call interruption processing device, characterized in that, include: The conversion unit is used to monitor and convert the audio stream into a text stream in real time through VAD and ASR services when playing preset scripts; The node analysis unit is used to process the text stream and input it into the streaming interruption model for node configuration analysis to obtain analysis results. A judgment unit is used to determine whether the analysis result supports interruption; The node interruption processing unit is used to stop playing the preset script and switch to the waiting / recognition mode when the confidence of the streaming interruption model exceeds the threshold if the analysis result supports interruption, until the VAD confirms that the user's speech has ended. A continuous monitoring unit is used to monitor the audio stream without interrupting playback if the analysis result does not support interruption, until the VAD confirms that the user's speech has ended. The whole content acquisition unit is used to acquire the entire dialogue content in this round; The posterior decision unit is used to input the entire dialogue content of this round into the whole hang-up model and the high-risk model for posterior decision to obtain the posterior decision result. The temporary storage unit is used to perform corresponding processing based on the posterior judgment result and to temporarily store the entire dialogue content of this round.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images