Information processing method, device, storage medium and program product of a smartphone system

Abstract

Embodiments of the present application provide an information processing method and device of a smart phone system, a storage medium and a program product. In the embodiments of the present application, the recognized text information and its accumulated character number and meta information such as a silence event are input into three parallel channels of an emotion perception fast response channel, a local intention understanding pre-communication channel and a complete intention understanding service response channel; when the accumulated character number reaches different thresholds, emotion analysis and local fused intention recognition are triggered respectively, and corresponding response information is generated for response; when a silence event exceeding a set time length occurs, complete intention recognition is performed based on all the text information, and a target service system is called through a service routing engine to complete service processing and generate a response. The scheme realizes progressive response of the emotion layer, the semantic layer and the service layer, effectively eliminates call interaction delay and blank waiting period, and improves the real-time performance, continuity and user experience of telephone voice interaction.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an information processing method, device, storage medium and program product for an intelligent telephone system. Background Technology

[0002] With the development of artificial intelligence technology, voice-interactive telephone systems are widely used in customer service, marketing outbound calls, intelligent customer service and other business scenarios, providing users with 24 / 7 automated voice interaction services.

[0003] In existing technologies, AI (Artificial Intelligence) telephone systems typically employ a sequential processing architecture based on ASR (Automatic Speech Recognition), NLU (Natural Language Understanding), DM (Dialogue Management), and TTS (Text-to-Speech) modules. Specifically, after the user's complete voice input, the system sequentially executes processing steps such as speech recognition, semantic understanding, response generation, and speech synthesis, ultimately providing the generated voice feedback to the user.

[0004] However, in practical applications, due to the strict sequential dependencies between the processing modules, the response cannot be generated until the voice input is completely finished and all the preceding processing is completed. This results in a long overall response time and a delay. During telephone interaction, this delay manifests as a noticeable blank waiting period. Users frequently perceive the delay during telephone interactions, which seriously affects the naturalness and continuity of the interaction, leading to a poor user call experience. Summary of the Invention

[0005] Embodiments of this application provide an information processing method, device, storage medium, and program product for a smart telephone system, which eliminates the interaction delay and blank waiting period caused by the serial processing architecture, improves the naturalness and coherence of voice interaction, and enhances the user's call experience.

[0006] This application provides an information processing method for an intelligent telephone system. The system includes a streaming speech recognition engine and three parallel channels: a fast response channel based on an emotion perception model, a pre-communication channel based on a partial intent understanding model, and a business response channel based on a complete intent understanding model. The method includes: during a user's call with the intelligent telephone system via a terminal, using the streaming speech recognition engine to convert the user's speech into text information in real time, and inputting the text information and its metadata into the three channels in parallel; the metadata includes the cumulative number of characters in the text information and a description of the silent event when a silent event occurs; when the cumulative number of characters reaches a first threshold, the fast response channel is triggered, and the emotion perception model is used to analyze the user's emotional state based on the current text information, combined with an emotion response strategy library. The system outputs a first response to achieve a rapid emotional response. When the cumulative number of characters reaches a second threshold greater than the first threshold, a pre-communication channel is triggered. Using a local intent understanding model, based on the current text information and the dialogue history memory, local intent recognition and historical intent association are performed simultaneously to obtain a local fusion intent. Based on this local fusion intent, a second response is output to achieve semantic interactive feedback. When a silence event occurs with a duration greater than a third threshold, a business response channel is triggered. Using a complete intent understanding model, complete intent recognition is performed on all text information. The business routing engine then calls the target business system that matches the complete intent to perform the corresponding business operation. Based on the execution result of this business operation, a third response is output to achieve a complete business-level response.

[0007] This application also provides an electronic device, including a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, the processor is able to implement the various steps in the information processing method of the intelligent telephone system provided in this application.

[0008] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the various steps in the information processing method of the intelligent telephone system provided in this application.

[0009] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, enable the processor to implement the various steps in the information processing method of the intelligent telephone system provided in this application.

[0010] In this embodiment, a streaming speech recognition engine is introduced into the intelligent telephone system, and three parallel channels are constructed, including an emotion-aware rapid response channel, a partial intent-understanding pre-communication channel, and a complete intent-understanding business response channel. Real-time recognized text information, its cumulative character count, and metadata such as silent events are input into each channel in parallel. When the cumulative character count reaches different thresholds, emotion analysis, partial fusion intent recognition, and corresponding response information generation are triggered respectively. When a silent event exceeding a set duration occurs, complete intent recognition is performed based on all text information, and the target business system is called through the business routing engine to complete business processing and generate a business response. This solution, through a multi-channel parallel and layered triggering real-time processing mechanism, achieves progressive responses at the emotion, semantic, and business layers, effectively eliminating call interaction delays and blank waiting periods, and improving the real-time performance, continuity, and user experience of telephone voice interaction. Attached Figure Description

[0011] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1a A flowchart illustrating an information processing method for a smart telephone system provided in an exemplary embodiment of this application; Figure 1b A schematic diagram of the model architecture of an information processing method for a smart telephone system provided in an exemplary embodiment of this application; Figure 2 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] It should be noted that, in the cases involving user information in the embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to language models or large models) comply with relevant laws and standards.

[0014] To address the technical issues of existing AI telephony systems employing a serial processing architecture of ASR, NLU, dialogue management, and TTS, which requires a response to be generated only after the user's voice input has finished and all processing is complete, resulting in significant interaction delays, blank waiting periods during calls, and a poor user experience, this application proposes a solution. This involves introducing a streaming speech recognition engine into the intelligent telephony system and constructing three parallel channels: an emotion-aware rapid response channel, a partial intent-understanding pre-communication channel, and a complete intent-understanding business response channel. Real-time recognized text information, along with its accumulated character count and silent events, are input into each channel in parallel. When the accumulated character count reaches different thresholds, emotion analysis, partial fusion intent recognition, and corresponding response information generation are triggered. When a silent event exceeding a set duration is detected, complete intent recognition is performed based on all text information, and the target business system is invoked through a business routing engine to complete business processing and generate a business response. This solution, through a multi-channel parallel and layered triggering real-time processing mechanism, achieves progressive responses at the emotion, semantic, and business layers, effectively eliminating call interaction delays and blank waiting periods, and improving the real-time performance, continuity, and user experience of telephony voice interaction.

[0015] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0016] Figure 1a This is a schematic flowchart illustrating an information processing method for a smart telephone system provided as an exemplary embodiment of this application. The method can be based on... Figure 1b The schematic diagram shown below illustrates the model architecture of the information processing method for a smart telephone system provided in an exemplary embodiment of this application. Figure 1b As shown, the model architecture includes: a dynamic threshold prediction model, an emotion perception model, a partial intent understanding model, a complete intent understanding model, and an AI speech generation model. Figure 1a As shown, the method includes: S101. During a call between a user and a smart phone system via a terminal, the user's speech is converted into text information in real time using a streaming speech recognition engine, and the text information and its metadata are input into three channels in parallel. The metadata includes the cumulative number of characters in the text information and the description information of the silent event when a silent event occurs. S102. When the cumulative number of characters reaches the first threshold, the fast response channel is triggered. The emotion perception model is used to analyze the user's emotional state based on the current text information, and the first response information is output in combination with the emotion response strategy library to achieve a fast response at the emotion level. S103. When the cumulative number of characters reaches a second threshold greater than the first threshold, the pre-communication channel is triggered. The local intent understanding model is used to simultaneously perform local intent recognition and historical intent association based on the current text information and the dialogue history memory bank to obtain the local fusion intent. Based on the local fusion intent, the second response information is output to achieve semantic-level interactive feedback. S104. When a silence event occurs with a duration exceeding the third threshold, the business response channel is triggered. The complete intent understanding model is used to perform complete intent recognition on all text information. The business routing engine is then used to call the target business system that matches the complete intent to perform the corresponding business operation. Based on the execution result of the business operation, the third response information is output to achieve a complete response at the business level.

[0017] In this embodiment, the intelligent telephone system is a human-computer voice interaction system that is combined with a large model. It can simulate human conversation behavior, conduct real-time and coherent dialogue with users through voice, and automatically perform tasks such as information inquiry, business processing, problem answering, and outbound marketing for business systems. It can be widely used in customer service, business consultation, appointment reminders, satisfaction follow-up and other scenarios. That is, it can automatically complete the understanding, analysis and response to the user's voice during the telephone call, and realize natural interaction with the user.

[0018] Users can communicate with the smartphone system via their terminals. Users establish a two-way voice communication connection with the smartphone system deployed on the server using their terminals, enabling real-time voice interaction between humans and machines. Optionally, users can dial or answer phone calls via their terminals. The terminals transmit the user's voice signal to the smartphone system through communication networks such as the Public Switched Telephone Network (PSTN), mobile communication networks (such as 4G / 5G), or VoIP. Alternatively, users can directly access the smartphone system via an IP (Internet Protocol) network. The terminals can establish an audio and video communication connection with the server via the Internet, and voice data is transmitted in the form of data packets over the IP network, without passing through traditional circuit-switched networks.

[0019] Alternatively, users can also initiate voice interaction requests through the service page of the app (mobile application). In this scenario, after logging into the app, users can click on "Online Customer Service," "Voice Service," or similar function entries, and the system will automatically establish an internet-based VoIP connection in the background. Voice data is transmitted to the smartphone system via mobile data network or Wi-Fi network to achieve real-time voice communication.

[0020] In this embodiment, the server refers to the computing resource entity that deploys the core processing capabilities of the intelligent telephone system. It is responsible for carrying key components such as the streaming speech recognition engine, three parallel channels, emotion perception model, intent understanding model, and business routing engine, and completing the entire process of speech recognition, intent analysis, response generation, and business invocation. This embodiment does not limit the specific physical form of the server. Any computing platform capable of stably running the functional modules of the aforementioned intelligent telephone system and establishing bidirectional voice data interaction with the terminal through a communication network can be considered a server. For example, the server can be a virtual server cluster deployed on a public or private cloud, a physical server in a self-built enterprise data center, a distributed edge computing node or a containerized microservice architecture, or even an elastic computing resource pool in a hybrid cloud environment.

[0021] A terminal is a communication device or software carrier on the user side used to initiate, receive, and carry voice calls. It is used to collect (through a microphone), play (through a speaker), and interact with the communication network protocol. In this application embodiment, there is no limitation on the specific type, operating system, hardware form, or communication protocol of the terminal. Any user-side access point that can establish a real-time two-way communication connection with the server via voice can be considered a terminal. For example, a terminal can be a traditional landline telephone (such as a landline phone), a mobile communication device (such as a smartphone or feature phone), a soft terminal based on Internet protocols (such as the voice call module of instant messaging software on a personal computer or mobile device), or an Internet of Things device with voice interaction capabilities (such as a smart speaker, an in-vehicle communication system, or a smartwatch that supports calling functions), and other devices or software carriers that can realize voice input and output.

[0022] In the embodiments of this application, the dynamic threshold prediction model, emotion perception model, local intent understanding model, complete intent understanding model, and AI speech generation model in the model architecture can all be implemented using different types of intelligent algorithm models. Specifically, these models can employ traditional neural network models, such as dedicated models built based on convolutional neural networks, recurrent neural networks, long short-term memory networks, or Transformer structures, or they can adopt large-scale model architectures with larger parameter sizes and stronger generalization capabilities. Preferably, the above models can be implemented using large-scale models, such as large language models based on pre-training mechanisms, to improve adaptability to complex semantics, multi-turn dialogues, and multi-scene changes.

[0023] Among them, emotion perception models, partial intent understanding models, and complete intent understanding models can all be built on large language models. Large language models, through pre-training on large-scale corpora, possess strong semantic understanding, context modeling, and reasoning capabilities, and can maintain high recognition accuracy even when the speech-to-text is incomplete or the context is complex.

[0024] In one optional embodiment, the emotion perception model, the partial intent understanding model, and the complete intent understanding model can be built on the same large language model. Functional differentiation can be achieved through different task instructions, prompt word templates, or parameter fine-tuning strategies, thereby sharing the underlying semantic representation capabilities and reducing model deployment costs.

[0025] In another optional embodiment, the three models—emotion perception model, local intent understanding model, and complete intent understanding model—can also adopt different large language models or be fine-tuned separately to optimize performance for different tasks such as emotion state analysis, local intent recognition, and complete business understanding, thereby improving the professionalism and accuracy of their respective processing channels.

[0026] Alternatively, the AI ​​speech generation model can be implemented using a deep learning model based on text-to-speech (TTS), such as a large-scale text-to-speech (TTS) model based on an end-to-end neural network structure. This type of model can be pre-trained on large-scale speech data and possesses capabilities for natural speech synthesis, emotional tone control, and multi-speaker style transfer, thereby making the generated speech more natural, fluent, and human-like.

[0027] In a preferred embodiment, the AI ​​speech generation model can also adopt a large model architecture to achieve higher quality speech synthesis results and support dynamic adjustment of speech tone, speed and expression style according to emotional state, thereby further improving the overall interactive experience.

[0028] In this embodiment, the smart phone system includes a streaming speech recognition engine and three parallel channels. The streaming speech recognition engine is a speech recognition module that supports real-time speech-to-text conversion. During a call between a user and the smart phone system via a terminal, it continuously frames the speech signal, converting the user's speech into text information in real time, and then inputting the text information and its metadata into the three parallel channels.

[0029] Unlike traditional ASR (Automatic Speech Recognition) which requires waiting for the user to finish speaking before recognizing the entire sentence, streaming speech recognition engines can continuously output intermediate recognition results (i.e., current text information) with millisecond-level granularity during user voice input. This is accompanied by metadata such as the cumulative character count of the output text and descriptions of silence events. In other words, streaming speech recognition engines can generate intermediate recognition results without waiting for the user to finish speaking, achieving low-latency speech-to-text output. This provides real-time, continuous text information support for subsequent multi-channel parallel processing, significantly reducing the overall system response time.

[0030] Meta-information, which is an auxiliary data set output synchronously with the main text information, describes the text generation process, state characteristics, and contextual environment. It does not constitute the semantic content of the user's expression, but provides crucial timing, state, and behavioral clues for subsequent processing modules, supporting the system in making more accurate and timely response decisions. Meta-information includes at least the cumulative number of characters in the text information and descriptions of silent events. Optionally, meta-information can also be expanded to include speech confidence, timestamps, audio energy values, and speech rate characteristics. These meta-information elements collectively constitute a multi-dimensional characterization of the user's voice input behavior, enabling the system to shift from a passive waiting to an active prediction-based interaction paradigm.

[0031] The cumulative character count of the text information is the total number of characters (including Chinese characters, letters, numbers, punctuation, and spaces) in all recognized text information output by the streaming speech recognition engine since the user begins voice input in the current dialogue round. This value continuously and dynamically accumulates as the user's voice input progresses, used to quantify the user's speaking progress. For example, when the user says "I want," the streaming engine outputs the text "I want," with a cumulative character count of 2; if the user continues with "Check the bill," the engine appends "Check the bill," and the cumulative character count is updated to 7; if the user subsequently adds "last month's," the cumulative character count further increases to 11.

[0032] A silence event is a continuous period of time during which a user's voice input is interrupted during a voice interaction with a smartphone system. The silence event description information describes the user's silent behavior that generates the silence event during the voice interaction process. This description information includes the time of the silence event, the duration of the silence, and the frequency of the silence event.

[0033] The timing of a silence event refers to the point in time during a call when the user's voice signal is detected to transition from an active to a silent state. The timing of a silence event can correspond to a specific point in the current conversation turn or a timestamp in the voice stream, used to identify when the user becomes silent. For example, it could be the 3rd second after the start of the call or 200 milliseconds after the end of the previous voice segment. The duration of a silence event refers to the length of time between the detection of a silent state and the re-detection of an active state, used to measure the duration of the silence. For example, if a user pauses for 1.8 seconds after saying "I want to check my bill," the silence duration is 1800ms. The frequency of silence events refers to the number of times a silence event is detected within a conversation turn, used to describe the density of pauses in the user's expression. For example, if a user pauses multiple times in a single speech ("Um...I want to...check...my bill"), three silence events can be recorded for that conversation turn. A higher frequency of silence may reflect hesitation, semantic uncertainty, or emotional fluctuations in the user's expression, while a lower frequency usually indicates fluent expression.

[0034] In this embodiment, the three parallel channels include a fast response channel based on an emotion perception model, a pre-communication channel based on a partial intent understanding model, and a business response channel based on a complete intent understanding model. These three parallel channels operate simultaneously based on the text information and metadata output by the streaming speech recognition engine, processing user input from the emotion layer, semantic layer, and business layer, respectively. The business response channel based on the complete intent understanding model requires acquiring complete text information and performing complex intent recognition and business routing, resulting in a relatively long processing time. In contrast, the fast response channel based on the emotion perception model and the pre-communication channel based on the partial intent understanding model can trigger and output responses in advance during the user's voice input process, providing continuous feedback to the user during the waiting period of the business response channel. This forms a progressive, multi-layered response mechanism, effectively covering the business processing waiting time and reducing the perceived blank waiting period for the user.

[0035] The fast response channel, based on an emotion-aware model, is used to quickly assess the user's emotions in the early stages of voice input. When the cumulative number of characters output by the streaming speech recognition engine reaches a first threshold, the fast response channel is triggered. The emotion-aware model analyzes the user's emotional state based on the currently recognized text information and generates a corresponding first response based on a pre-configured emotion response strategy library. The first threshold is the cumulative character count threshold used to trigger the fast response channel, representing the minimum text length at which the user's voice input can be used for preliminary emotion assessment. In this way, the system can provide reassuring, guiding, or confirmatory voice feedback, enhancing the naturalness and friendliness of the interaction.

[0036] The emotion perception model is a lightweight emotion analysis model deployed in a fast response channel, built upon a large language model. It can identify and classify a user's current emotional state in real time based on partial text information generated during the user's voice input (i.e., intermediate recognition results). The large language model is a natural language processing model based on deep neural networks, pre-trained on massive text corpora, possessing powerful contextual understanding and semantic representation capabilities. Through a combination of pre-training and fine-tuning, this large language model can understand semantic content while capturing implicit emotional tendencies, attitude expressions, and tone changes in the text, thereby outputting the user's current emotional state and generating the first response information corresponding to the emotional state by combining it with an emotion response strategy library.

[0037] It should be noted that the specific structural form of the large language model in this application embodiment is not limited. It can be an autoregressive language model based on Transformer, a multi-task fine-tuning model, or a dialogue-specific language model trained with domain adaptation, which can realize real-time emotion recognition and response generation for partial text input.

[0038] In this context, a user's emotional state refers to the psychological and emotional tendencies or attitudes expressed through language content, expression methods, and voice characteristics during voice interaction. It reflects the user's current subjective feelings and willingness to interact. This application does not limit the specific type of emotional state. For example, emotional states can be positive emotions (including satisfaction, pleasure, and anticipation), negative emotions (including anger, anxiety, and disappointment), neutral or complex emotions (including confusion, hesitation, and calmness), or even multi-level and multi-dimensional emotional representations specific to business scenarios (including complaint emotions, consultation emotions, and emergency help emotions).

[0039] An emotion response strategy library refers to a pre-configured set of response strategies that map different emotional states to corresponding response phrases and interactive behaviors. It provides standardized, scenario-based response guidelines for the output of the emotion perception model, i.e., emotional states, ensuring that the system can quickly generate appropriate, compliant, and emotionally intelligent feedback content based on the user's emotional state. This emotion response strategy library can be stored in the form of a data table, knowledge graph, or rule engine; no specific limitation is made here. Each strategy record can contain an emotional state and a response phrase template. The emotional state and response phrase template determine the expression style, and combined with the soothing objective, text is generated to obtain the first response information, achieving rapid emotional-level response.

[0040] For example, when the "anger" emotional state is detected, the emotion perception model can match a soothing dialogue template from the emotion response strategy library, such as "We understand how you feel, we'll handle it for you right away," with a gentle and calming style. When the "satisfaction" emotion is detected, a positive reinforcement dialogue template, "I'm glad to help you!", can be matched, with a positive style. Based on the flexible configuration of the emotion response strategy library, the system can complete the closed loop from emotion recognition to emotional response in milliseconds, effectively alleviating users' negative emotions, enhancing positive experiences, and laying a good emotional foundation for subsequent business interactions.

[0041] The pre-communication channel, based on a local intent understanding model, is used to interact with the user in advance at the semantic level. The pre-communication channel is triggered when the accumulated character count reaches a second threshold greater than the first threshold. The local intent understanding model, based on currently recognized partial text information and combined with a dialogue history memory, identifies the user's current local intent and fuses it with historical intents to obtain a fused local intent. Based on this, a second response is generated to confirm, follow up, or provide guided feedback to the user. The second threshold refers to the cumulative character count threshold used to trigger the pre-communication channel; its value is strictly greater than the first threshold, representing the text length stage where the user's voice input has accumulated sufficient semantic information for local intent recognition and contextual association.

[0042] Among them, the local intent understanding model can be a lightweight semantic understanding model deployed in the pre-communication channel, built on a large language model. Unlike traditional intent recognition models that perform overall analysis after the user has finished expressing their complete statement, the local intent understanding model performs real-time semantic inference based on the currently identified partial text content, even before the user has completed their input. The local intent understanding model combines natural language understanding technologies, such as word vector representation, sequence modeling, or incremental semantic parsing mechanisms, to analyze keywords, semantic structure, and contextual clues in the current segment, predicting the user's intent direction. It can form staged intent judgments even with incomplete sentences, thus supporting the system in generating guiding feedback in advance.

[0043] The dialogue history memory is a structured data collection used to store and manage users' historical dialogue information, including long-term interaction records across sessions, such as historical intents. Historical intents include not only users' business needs but also user preferences, behavioral patterns, and confirmed desired outcomes. The dialogue history memory enables the system to quickly retrieve relevant historical intents during intent recognition. By associating, matching, and fusing current local intents with historical intents, the system can enhance the accuracy of intent judgment, avoid redundant queries, and achieve a more personalized interactive experience.

[0044] Through a collaborative mechanism between the local intent understanding model and the dialogue history memory, the system can associate and fuse currently identified text with historical intents to form a locally fused intent. Based on this locally fused intent, the system can generate a second response, which can be used to confirm the user's true needs, ask follow-up questions, or provide guided feedback, achieving semantic-level interactive feedback. This mechanism enables the system to guide users to express more explicit needs, further eliminating call interaction delays and blank waiting periods, thereby improving interaction efficiency and user experience.

[0045] For example, in a scenario implementing a bank's intelligent telephone system, a user says, "Can I…" At this point, the streaming speech recognition engine outputs a portion of the text in real time before the user finishes speaking, continuously counting the accumulated characters. When the accumulated character count reaches a second threshold (e.g., the second threshold is 10), the pre-communication channel is triggered. At this point, the local intent understanding model, based on the current partial text information, determines that the user's intent is related to "billing processing," but it is not yet clear whether it is a specific intent such as "checking bills," "paying off," "installment payments," or "credit limit adjustment." The local intent understanding model further combines the dialogue history memory to associate historical intents. For example, the dialogue history memory records that the user inquired about "billing installment rules" in a previous call and recently clicked on the "installment application" page within the application. Based on this historical intent, the system performs a fusion inference on the current local intent, obtaining a local fusion intent of "the user is inquiring whether bill installment payments are possible." Further, a second response is generated. For example, "Okay, you want to know if last month's bill can be paid in installments, right? I can help you check." By employing the above methods, users' perceived waiting time and pauses during calls are reduced, making the interaction process more natural and coherent, and improving call efficiency and user experience.

[0046] Among them, the business response channel based on the complete intent understanding model is used to realize the complete processing at the business level. When a silence event with a silence duration exceeding the third threshold is detected, it indicates that the user has completed the expression of the current round, and this channel is triggered.

[0047] In this embodiment, a silence event with a duration exceeding a third threshold is a valid silence event determined when the system detects a cessation of the user's voice signal during user voice input, and the duration of this cessation exceeds a preset third threshold. This indicates that the user has completed their current round of expression, rather than engaging in brief thinking or pausing within a sentence. In other words, when the silence duration exceeds the third threshold, the system can determine that the user has finished speaking and can proceed to the full intent processing stage.

[0048] The third threshold is a time threshold parameter used to distinguish between "short pauses during expression" and "the end of expression." This third threshold can be preset in advance based on the actual business scenario, the user's average speaking speed, and the interaction rhythm. For example, the third threshold can be set to a duration of 3 seconds or 4 seconds; there is no specific limitation. When the silence duration is less than the third threshold, the system assumes that the user may continue to express themselves and does not trigger the business response channel. When the silence duration is greater than the third threshold, the system determines that the current round has ended, thereby triggering the business response channel based on the complete intent understanding model. This mechanism avoids prematurely interrupting the user's expression while ensuring that the complete business processing flow is initiated at the appropriate time.

[0049] The complete intent understanding model is a natural language understanding model built on a large language model and deployed in the business response channel, capable of deep semantic parsing. Unlike the partial intent understanding model, the complete intent understanding model can perform overall semantic analysis based on all identified text information after the user's expression ends. The complete intent understanding model can identify the user's business intent, i.e., the complete intent.

[0050] After identifying the complete intent, the system uses the business routing engine to invoke the corresponding target business system to perform the appropriate operation. The business routing engine is a scheduling and mapping component that distributes requests to the appropriate business systems based on the identified complete intent. This engine achieves scheduling by maintaining a "intent-business system" mapping table. Once the complete intent is identified, the business routing engine selects the corresponding target business system according to the rules in the mapping table.

[0051] In this context, the business system refers to the backend system or service module that actually carries out specific business functions. For example, a business system could be a customer information query system, an order management system, a payment processing system, a logistics query system, or an after-sales service system. The target business system, on the other hand, is the specific business system that matches the identified complete intent in the current dialogue scenario. For instance, when the complete intent is "check account balance," the target business system could be the account management system; when the complete intent is "modify delivery address," the target business system could be the order management system.

[0052] The specific functional actions actually performed in the target business system are called business operations. For example, a business operation could be querying data, creating an order, updating information, submitting an application, or canceling a service. The execution result is the data or processing status returned by the target business system after completing the corresponding business operation. For example, the execution structure could be query result data, success or failure status codes, error messages, etc.

[0053] The final voice feedback content generated by the system based on the execution results of business operations is called the third response information. The third response information contains a complete explanation of the business processing results and is a response at the business level addressing the user's complete business needs. For example, the third response information may be a data result announcement, confirmation of successful operation, or a prompt indicating the reason for failure, used to provide the user with a clear and accurate business response.

[0054] For example, in a scenario implementing a bank's intelligent telephone system, a user says, "I want to check my credit card statement amount for this month." The system gradually acquires text information through streaming recognition during the voice input process. When the user finishes speaking and there is a silence event lasting longer than a third threshold (e.g., 3 seconds), the system determines that the current round has ended and triggers the business response channel. The complete intent understanding model identifies the complete intent as "check credit card statement amount" and extracts the "time range = this month." Based on this complete intent, the business routing engine distributes the request to the credit card statement query system (the target business system). The target business system executes the "statement amount query" business operation and returns the execution result, such as "This month's statement amount is RMB 3580." The intelligent telephone system generates a third response based on this execution result, such as: "Your credit card statement amount for this month is RMB 3580. Do you need any other information?".

[0055] It should be noted that the first, second, and third thresholds are set in ascending order of value, satisfying the relationship that the first threshold is less than the second threshold, and the second threshold is less than the third threshold. The first and second thresholds are character count thresholds triggered based on the cumulative character count, with the second threshold being greater than the first threshold. This ensures that the triggering of the pre-communication channel is later than the rapid response channel, thus forming a gradual response from the emotional level to the semantic level. The third threshold is a time threshold set based on the silence duration, corresponding to the judgment standard that the user's expression is basically complete. It is numerically greater than the broadcast duration-related threshold used for insertion judgment, and is used to trigger the complete intent recognition and business response channel. By setting the first threshold < the second threshold < the third threshold, the system triggers step by step in the order of emotional response, semantic feedback, and business processing during the user's expression process, achieving phased and hierarchical response control. This ensures both timely response and avoids premature entry into the complete business processing flow, thereby improving the overall continuity and stability of the interaction.

[0056] In this embodiment, through the collaborative work of the above-mentioned streaming speech recognition engine and three parallel channels, the intelligent telephone system can realize a phased and progressive interaction from emotional response and semantic feedback to business processing, effectively reducing the blank waiting period for users during telephone interaction and improving the real-time performance, continuity and user experience of the overall telephone voice interaction.

[0057] In an optional embodiment, the first threshold and the second threshold are dynamically determined, and the method further includes: calculating the user's real-time speech rate based on the number of characters in the text information within at least one unit of time; predicting the background noise level of the call environment based on the spectral characteristics of the user's speech; inputting the real-time speech rate and the background noise level into a dynamic threshold prediction model to perform real-time prediction of the character number threshold, and outputting the first threshold and the second threshold.

[0058] The first and second thresholds are dynamically determined rather than fixed preset values ​​because different users have significantly different speaking habits, speaking speeds, expression styles, and call environments. Using a fixed number of characters as the trigger standard might trigger too late when the speaking speed is fast, affecting real-time response; conversely, it might trigger too early when the speaking speed is slow or there is high noise, leading to insufficient semantic understanding and misjudgment. Therefore, by dynamically adjusting the character number threshold based on the user's real-time speaking speed and the noise level of the call environment, a more reasonable triggering timing can be achieved in different scenarios, balancing response speed and comprehension accuracy, thereby improving the overall interactive experience.

[0059] The character growth per unit time refers to the number of new text characters output by the streaming speech recognition engine within a preset time window during continuous speech recognition. For example, within a certain time window (e.g., 1 second), the system counts the number of newly recognized characters to reflect the user's expression density during that time period. At least one unit time can be a single time window (e.g., 1 second) or multiple consecutive time windows (e.g., a sliding time window of two or three consecutive seconds). Using multiple consecutive time windows for statistics helps smooth out short-term fluctuations, reduces errors caused by recognition jitter or momentary pauses, and thus improves the stability and accuracy of speech rate prediction.

[0060] The user's real-time speech rate is the speed at which the user outputs language per unit of time during the current call phase, which can be expressed in "characters / second" or "words / second". For example, if 6 new characters are recognized within 1 second, the real-time speech rate is 6 characters / second; if 6 and 8 new characters are recognized in two consecutive seconds, respectively, the average (7 characters / second) can be taken as the current real-time speech rate. Based on this, the system can dynamically grasp the user's current expression rhythm. When the user's speech rate is fast, it indicates that their expressive intent is formed quickly, and the first and second thresholds can be appropriately increased to obtain more complete semantics; when the speech rate is slow, the first and second thresholds can be appropriately decreased to avoid excessive response delay.

[0061] The spectral characteristics of user speech are frequency distribution information obtained through frequency domain analysis of the speech signal, including energy distribution, spectral flatness, and signal-to-noise ratio characteristics in different frequency bands. Spectral characteristics reflect the mixing of the speech signal with background noise and are an important basis for analyzing the acoustic conditions of the call environment. The spectral characteristics of user speech can be extracted from the speech signal using Fast Fourier Transform or Mel-frequency cepstral coefficients.

[0062] The background noise level of the call environment refers to the intensity or complexity of environmental interference sounds other than the user's valid speech signal. For example, background noise can be stable low-frequency air conditioning noise, continuous traffic noise, conversations in multi-person environments, wind noise, and electrical noise. The background noise level can be represented by the SNR (signal-noise ratio). The system can estimate the background noise level by analyzing the spectral characteristics of silent or low-energy speech segments. For example, extracting the audio spectrum during a short pause by the user; if the measured SNR is 5dB, it can be classified as a high-noise environment; if the SNR is above 20dB, it can be classified as a low-noise environment. Depending on the noise level, the system can appropriately increase the character count threshold using a dynamic threshold prediction model to wait for more semantic information, or decrease the threshold to avoid the accumulation of recognition errors.

[0063] The dynamic threshold prediction model can take into account real-time speech rate features, background noise level features, and optional historical interaction features, and outputs the optimal first and second thresholds for the current scenario. Trained on a large amount of historical call data, the dynamic threshold prediction model learns the distribution patterns of optimal trigger points under different speech rates and noise conditions, thus achieving adaptive threshold adjustment. It comprehensively models multi-dimensional environmental factors, avoiding reliance on manual experience to set fixed parameters. This allows the system to maintain stable and efficient triggering effects under different users, devices, and environments, further improving the accuracy and robustness of the progressive response mechanism.

[0064] like Figure 1bThe diagram illustrates an example of the internal structure of a dynamic threshold prediction model. In this example, the dynamic threshold prediction model includes a feature input layer A, a feature fusion layer B, a threshold prediction layer C, and a threshold output layer D. Based on the internal structure of the dynamic threshold prediction model, a specific implementation method is provided that inputs real-time speech rate and background noise level into the dynamic threshold prediction model to perform real-time prediction of the character number threshold and output a first threshold and a second threshold. This implementation includes: inputting real-time speech rate and background noise level into the feature input layer A; standardizing and extracting features from the real-time speech rate and background noise level to obtain speech rate features and noise features; inputting the speech rate features and noise features into the feature fusion layer B for joint modeling to extract fusion features reflecting semantic density and recognition stability; inputting the fusion features into the threshold prediction layer C; generating a character number threshold adjustment parameter based on the relationship between the fusion features and preset speech rate thresholds and preset noise thresholds; and inputting the character number threshold adjustment parameter into the threshold output layer D to adjust a preset baseline character number threshold and output the first threshold and the second threshold.

[0065] The feature input layer A is the front-end processing module of the dynamic threshold prediction model. It standardizes and extracts features from the raw input data, namely real-time speech rate and background noise level. Feature input layer A receives two input parameters: real-time speech rate and background noise level. It standardizes these parameters, for example, by using normalization to map the speech rate to the 0-1 range and linearly scaling the noise level according to a preset range to eliminate the influence of units on the numerical value. Further, feature extraction is performed to form structured speech rate and noise features. Speech rate features may include instantaneous speech rate, rate of change of speech rate, and amplitude of speech rate fluctuations; noise features may include current noise intensity, noise change trend, and short-term average energy. These features characterize the speaking rhythm and the degree of environmental interference, providing basic data for subsequent modeling.

[0066] After obtaining speech rate and noise features, feature fusion layer B jointly models the multi-source features, namely speech rate and noise features. Feature fusion layer B is essentially a multi-feature association analysis module, which can be implemented using neural network structures (such as fully connected layers or attention mechanism layers) or statistical modeling methods to explore the coupling relationship between speech rate and noise features. Joint modeling no longer considers the independent effects of speech rate or noise separately, but simultaneously considers their interaction effects. For example, when the speech rate is fast and the noise level is high, the stability of the recognition system may decrease significantly. Through joint modeling, fused features are extracted. Fusion features are high-dimensional representations that comprehensively reflect the complexity of the current speech input and can be used to characterize semantic density and recognition stability. Semantic density refers to the concentration of effective semantic information per unit time; for example, the faster the speech rate and the fewer the pauses, the higher the semantic density. Recognition stability refers to the reliability of the speech recognition results; for example, under low noise and stable speech rate conditions, the recognition results fluctuate less and have higher stability, while under high noise and drastic speech rate changes, the recognition results are prone to errors and have lower stability. By comprehensively analyzing speech rate and noise, fusion features can indirectly reflect whether the current user's voice input is information-dense and unstable, or information-moderate and stable.

[0067] Based on this, the threshold prediction layer C can generate a character count threshold adjustment parameter according to the fused features. The threshold prediction layer C can adopt a regression model or a neural network output layer structure, and can map the fused features to specific adjustment coefficients or offsets. The character count threshold adjustment parameter is a scalar coefficient, a control variable used to increase or decrease the baseline character count threshold; for example, it can be represented as a proportional coefficient α or an increment ΔN, which is not limited here. The value range of the character count threshold adjustment parameter is generally (0.5, 1.5), used to linearly scale the preset baseline character count threshold: α < 1 indicates a harsh environment requiring a lower threshold to respond earlier, while α > 1 indicates a favorable environment allowing for a higher threshold to wait for more complete semantics.

[0068] The preset speech rate threshold is an environmental quality reference standard configured during system initialization, representing "standard interaction conditions." For example, setting 4 Chinese characters per second as the standard speech rate is the preset speech rate threshold. The preset noise threshold is an environmental noise boundary value set by the system; for example, setting 60dB as the acceptable noise upper limit is the preset noise threshold. The relationship between the fusion feature and the preset speech rate threshold and preset noise threshold can be understood as follows: when the real-time speech rate corresponding to the fusion feature exceeds the preset speech rate threshold, or the background noise level exceeds the preset noise threshold, the character number threshold adjustment parameter tends to decrease the character number threshold (first threshold and second threshold); when the real-time speech rate is low and the noise is low, and the system is in a stable state, the character number threshold adjustment parameter tends to increase the character number threshold. The relationship between the fused features and these two preset thresholds can be represented as a bias metric: when the fused features indicate that the real-time speech rate is significantly higher than the preset speech rate threshold (e.g., 6 Chinese characters / second > 4 Chinese characters / second) and the noise is significantly higher than the preset noise threshold (e.g., 70dB > 50dB), the dynamic threshold prediction model determines that the current environment is "semantically dense but recognition unstable," and the response threshold needs to be lowered to avoid interaction interruption due to recognition delay. In this case, the output α = 0.8. Conversely, when the real-time speech rate is lower than the preset threshold (e.g., 3 Chinese characters / second < 4 Chinese characters / second) and the noise is lower than the preset threshold (e.g., 40dB < 50dB), the dynamic threshold prediction model determines that the environment is "semantically sparse but recognition stable," and the threshold can be appropriately increased to obtain more complete semantics. In this case, the output α = 1.2. This mapping relationship can be learned through training data or implemented through the design of a rule function; there are no limitations on this.

[0069] Furthermore, the threshold output layer D can dynamically adjust the preset baseline character number threshold based on the character number threshold adjustment parameter output by the threshold prediction layer C, and output the first threshold and the second threshold. The threshold output layer D can be understood as the final decision layer, which can transform the abstract adjustment parameter into a practically usable character number threshold. The preset baseline character number threshold is a standard value set by default in the system, for example, setting the number of characters processed at one time to 50 characters in an ideal environment. The adjustment logic can be as follows: if the character number threshold adjustment parameter is a proportional coefficient α, then the first threshold = baseline character number threshold × α1, and the second threshold = baseline character number threshold × α2; if the character number threshold adjustment parameter is an increment ΔN, then the first threshold = baseline threshold + ΔN1, and the second threshold = baseline threshold + ΔN2. When the semantic density is high or the recognition stability is low, the dynamic threshold prediction model will appropriately reduce the character number threshold to avoid information loss or the spread of recognition errors; while when the environment is stable, the character number threshold will be increased to improve processing efficiency.

[0070] For example, suppose the system presets a first baseline threshold of 4 characters, a second baseline threshold of 6 characters, a preset speech rate threshold of 4 Chinese characters per second, and a preset noise threshold of 50dB. When a user is making a call in a noisy subway environment, and the real-time detected speech rate is 7 Chinese characters per second with a background noise of 75dB, the feature input layer A normalizes the speech rate to 0.85 and extracts the "drastic speech rate change" feature, normalizes the noise to 0.92 and extracts the "consistently high noise" feature; the feature fusion layer B uses an attention mechanism to jointly model and outputs a fused feature vector, determining that the current semantic density is high (the user expresses themselves quickly) but the recognition stability is low (high noise interference); the threshold prediction layer C calculates the adjustment parameter α=0.75 based on the deviation between the fused feature and the preset threshold; the threshold output layer D adjusts the first baseline threshold 4 to 3 characters (4×0.75=3), and the second baseline threshold 6 to 4.5 characters and rounds up to 5 characters (6×0.75=4.5→5), ensuring that the second threshold 5 > the first threshold 3.

[0071] Conversely, in a quiet office environment, with a real-time speech rate of 2.5 Chinese characters per second and a noise level of 35 dB, α = 1.3 was calculated using the same process. The output thresholds were 5 characters (4 × 1.3 = 5.2 → 5) and 8 characters (6 × 1.3 = 7.8 → 8). This dynamic adjustment mechanism enables the system to adapt to different interactive environments and achieve an intelligent balance between response timeliness and recognition accuracy.

[0072] like Figure 1bThe image shows an example of the internal implementation structure of an emotion perception model. In this example, the emotion perception model includes: a feature encoding layer E, an emotion feature extraction layer F, an emotion classification layer G, and a first response generation layer H. Based on the internal implementation structure of the emotion perception model, a specific implementation method is provided that utilizes the emotion perception model to analyze the user's emotional state based on current text information and outputs first response information in conjunction with an emotion response strategy library to achieve rapid response at the emotion level. This includes: inputting the current text information into the feature encoding layer E for encoding processing at three levels: intonation, speech rate, and semantics, resulting in a word vector sequence including intonation vector, speech rate vector, and semantic vector; inputting the word vector sequence into the emotion feature extraction layer F, extracting emotion intensity features reflecting the fluctuations in the user's tone based on the intonation vector, extracting features reflecting the pace of the user's speech based on the speech rate vector, and extracting semantic emotion features reflecting the emotional tendency of the text based on the semantic vector, thus processing the three levels of... Emotional features are fused to obtain an emotional context feature vector. This emotional context feature vector is then input into the emotion classification layer G, where the similarity between the emotional context feature vector and multiple emotional state category vectors learned during model training is calculated. A target emotional state category vector whose similarity to the emotional context feature vector meets a set condition is selected, and the emotional state expressed by the target emotional state category vector is taken as the user's emotional state. The emotional state is then input into the first response generation layer H, which, combined with preset emotional response rules, queries the emotional response strategy library for a response strategy that matches the emotional state. Based on the expression style and soothing goal defined by the response strategy, text is generated to obtain the first response information corresponding to the emotional state.

[0073] Among them, the feature encoding layer E is the input representation layer of the emotion perception model, which can transform the current text information input from multiple perceptual dimensions into a structured numerical vector sequence, providing a multi-angle input basis for subsequent emotion feature extraction.

[0074] In this embodiment, the feature encoding layer E needs to perform encoding processing at three levels: intonation, speech rate, and semantics. Intonation encoding can capture the implicit tonal fluctuations in the text. By analyzing punctuation and rhetorical features such as exclamation marks, question marks, interjections (e.g., "ah," "ne"), and repeated characters (e.g., "too slow!!"), it infers the user's potential speech intonation patterns to compensate for the acoustic information missing in pure text interaction, enabling the system to hear the tone from the text. Speech rate encoding is similar to the speech rate feature mentioned above and will not be repeated here. Speech rate encoding can indirectly reflect the speed of the user's speech and is used to identify emotional states strongly related to speech rate, such as anxiety and impatience. Semantic encoding uses a pre-trained word vector model (e.g., Word2Vec, BERT) to map each word into a high-dimensional semantic vector, capturing the emotional polarity and contextual meaning of the words themselves.

[0075] It should be noted that after encoding, each word or character corresponds to a vector representation with multiple dimensions, thus forming a word vector sequence. The word vector sequence of intonation vectors is a sequence of vectors that adds a vector reflecting the pronunciation tone features to each word in the text information; the word vector sequence of speech rate vectors is a sequence of word vectors that adds a time feature vector related to the pronunciation rhythm to each word; and the word vector sequence of semantic vectors is a sequence of semantic embeddings in the traditional sense, used to represent the emotional meaning and contextual relationship of words.

[0076] After feature encoding is completed, the emotion feature extraction layer F further fuses the word vector sequences of the aforementioned intonation vector, speech rate vector, and semantic vector. In this embodiment, the specific structure of the emotion feature extraction layer is not limited. For example, the emotion feature extraction layer F can employ a convolutional neural network, a recurrent neural network, or an attention mechanism structure to extract higher-order representations related to emotion from the lower-level encoded features. Specifically, based on intonation vector sequences, F extracts emotional intensity features through one-dimensional convolution or attention mechanisms to reflect the amplitude of pitch fluctuations when a user speaks. For example, it detects the peak region corresponding to consecutive high-weight punctuation marks (such as "!!!") to quantify the explicit intensity of the user's emotions; high intensity corresponds to anger / excitement, and low intensity corresponds to calmness. Based on speech rate vector sequences, F can calculate the rate of change of character density within a sliding window to extract agility features to reflect the speed of speech. For example, it can identify the impatience represented by short-term high-density output (such as "faster, faster, faster") or the hesitation represented by long pauses and low-density output. Based on semantic vector sequences, F can use LSTM (Long Short-Term Memory) or Transformer encoders to extract semantic sentiment features, identify sentiment keywords (such as "annoyed" and "satisfied") and their contextual modifications (such as the negation weakening in "not very satisfied") to determine the sentiment tendency of the text content itself.

[0077] After extracting the three types of features mentioned above, the emotion feature extraction layer further fuses them to obtain the emotion context feature vector. The emotion context feature vector is a high-dimensional representation that can comprehensively express the overall emotional state of the current speech input content. It not only contains the emotional information of individual words, but also the continuous trend of contextual changes, thus more accurately depicting the user's current emotion and providing a basis for discrimination for the emotion classification layer G.

[0078] Furthermore, the emotion classification layer G classifies the emotion context feature vector. Essentially, the emotion classification layer G is an emotion recognition classifier that has learned multiple emotion state category vectors during model training, such as positive emotions, negative emotions, neutral or complex emotions, etc. The similarity between the emotion context feature vector and the multiple emotion state category vectors is calculated using vector similarity to measure the degree of matching between the currently input emotion context feature vector and each emotion category. In this embodiment, the specific method of similarity calculation is not limited. For example, similarity calculation can be cosine similarity or dot product calculation. The calculation method can be: Similarity = Current emotion vector × Emotion category vector / (Product of the magnitudes of the two). The category whose similarity meets the set conditions is selected as the target emotion state category vector.

[0079] The set conditions can be either maximum similarity or similarity greater than a preset similarity threshold, without any limitation. The emotional state expressed by the target emotional state category vector is the user's current emotion that is finally identified, such as "anxiety" or "dissatisfaction".

[0080] After obtaining the user's emotional state, the first response generation layer H generates the first response information based on the recognition result, i.e., the emotional state. The first response generation layer H is an emotion response decision module. It can combine preset emotion response rules, query the emotion response strategy library for response strategies matching the emotional state, and transform the emotional state into specific emotional reassurance or guidance statements based on the expression style and soothing goals defined by the response strategy. The preset emotion response rules are the response logic rules pre-set by the system for different emotional states. For example: when the emotion is "anxiety," a soothing tone should be used and a clear commitment should be given; when the emotion is "doubt," a brief confirmation and explanation should be given; when the emotion is "anger," understanding and apology should be expressed first.

[0081] In the emotion response strategy library, each strategy record can include an emotional state and a response template. The emotion perception model, based on the identified emotional state, searches the library for strategy records with the same emotional label; if multiple templates exist, the optimal template is selected based on the current business scenario or context. The emotional state and the response template jointly determine the expression style. Since the target of reassurance is the user, the generated text prioritizes a gentle tone, positive commitment, and clear action statements. If a template-based generation method is used, variables in the template (such as business name and processing time) are replaced with current business information to obtain the initial response information.

[0082] In the above embodiments, through three - layer encoding of intonation, speech rate, and semantics, combined with emotion feature extraction and vector similarity classification, the rapid recognition of the user's emotional state is achieved, and corresponding emotional responses are generated through the policy library, making the emotion recognition more accurate. Moreover, the regularized emotion response policy library ensures the consistency of the response style and the soothing effect, significantly improving the user's subjective experience.

[0083] For example, when the user says "Too slow!!" (the cumulative number of characters is 6, reaching the first threshold) during a call, the streaming speech recognition engine inputs this text information and meta - information into the emotion perception model. The feature encoding layer E performs triple encoding: at the intonation level, recognizing the double exclamation marks generates a high - weight intonation vector sequence [0.2, 0.3, 0.9, 0.9]; at the speech rate level, based on the timestamp, the character density is calculated to be 8 characters per second, generating a high - value speech rate vector sequence [0.85, 0.85, 0.85, 0.85]; at the semantic level, mapping "too", "slow", and "了" into a BERT word vector sequence. The emotion feature extraction layer F extracts respectively: the intonation vector sequence obtains an emotion intensity feature of 0.9 (high intensity) through max - pooling; the speech rate vector sequence obtains a rapid - slow feature of 0.85 (urgent) through mean calculation; the semantic vector sequence obtains a semantic emotion feature [-0.6, 0.2] (negative emotion) through LSTM encoding. The three are fused to generate an emotion context feature vector v = [0.9, 0.85, -0.6, 0.2, …]. The emotion classification layer G calculates the cosine similarity between v and the vectors of each category. Among them, the similarity with the "angry" category vector reaches 0.86 (meeting the set condition: the similarity is higher than the 0.7 threshold), and it is determined that the user's emotional state is "angry". The first response generation layer H matches according to the "angry" state in the emotion response policy library: emotion state = "angry", response speech template = "We fully understand your [XX] mood, and we will handle it for you immediately", expression style = low speech rate (3.2 characters per second)+steady intonation, soothing target = relieve anxiety. The system fills in the implied object "response speed" mentioned by the user into the template, generating the first response message "We fully understand your mood regarding the response speed, and we will handle it for you immediately".

[0084] Such as Figure 1bThe diagram illustrates an example of the internal implementation structure of a local intent understanding model. In this example, the local intent understanding model includes: a first semantic encoding layer I, a local intent recognition layer J, an intent fusion layer K, and a second response generation layer L. Based on the internal implementation structure of the local intent understanding model, a specific implementation method is provided that utilizes the local intent understanding model to simultaneously perform local intent recognition and historical intent association based on current text information and a dialogue history memory, to obtain a locally fused intent, and outputs a second response information based on the locally fused intent, thereby achieving semantic-level interactive feedback. The method includes: inputting the current text information into the first semantic encoding layer I for local semantic encoding to obtain a local semantic feature vector; inputting the local semantic feature vector into the local intent recognition layer J, decoding the local semantic feature vector to obtain a local intent; inputting the local intent into the intent fusion layer K, extracting historical intents related to the user's speech from the dialogue history memory based on the local intent, and fusing the local intent and historical intent using a cross-self-attention mechanism to obtain a locally fused intent; and inputting the locally fused intent into the second response generation layer L, generating text from the locally fused intent to obtain the second response information.

[0085] The first semantic encoding layer I is the input semantic representation layer of the local intent understanding model. It is used to perform local semantic encoding on the current text information and extract semantic features that can be used for subsequent intent recognition. Local semantic encoding is a real-time semantic vectorization representation of the currently recognized text information. Since the user may not have fully expressed their needs at this stage, the encoding process emphasizes "incremental understanding" and "context-sensitive modeling". The first semantic encoding layer can use a pre-trained language model (such as BERT, Transformer encoder, etc.) to embed the current text segment to obtain local semantic feature vectors.

[0086] Local semantic feature vectors are high-dimensional vector representations of the semantic meaning of the current text information. They not only contain keyword information but also encode the grammatical relationships between words, contextual information, and potential intent cues. For example, when the current text information is "I want to check my bill...", even if the sentence is not finished, the local semantic feature vector can already reflect the semantic direction of "query-related needs," providing a foundation for subsequent intent recognition.

[0087] After obtaining the local semantic feature vector, the local intent recognition layer J decodes it. Essentially, the local intent recognition layer J is an intent classification or sequence labeling module that maps abstract semantic feature vectors to specific intent labels. The decoding process of local intent involves a classifier or decoding network that transforms the semantic feature vector into a probability distribution of one or more candidate intent categories, thereby predicting the stage-specific intent corresponding to the current text information, i.e., the local intent.

[0088] Next, the intent fusion layer K is used to associate and fuse the current local intent with historical intents. Historical intents related to the user's voice refer to those extracted from the dialogue history memory. In the actual extraction process, the current local intent can be used as a query vector to perform similarity matching in the dialogue history memory. For example, vector retrieval or keyword matching can be used to filter historical intents whose semantic similarity reaches the historical intent similarity threshold. For instance, when the current local intent is "query bill," the system can retrieve the user's history of repeatedly querying "last month's bill" over the past three months, which contains the historical intent.

[0089] During the fusion process, a cross-attention mechanism can be used to fuse local intents and historical intents. The cross-attention mechanism is an attention calculation method in neural networks. It uses the vector of the current local intent as the query vector and the vectors of historical intents as the key and value, calculates the relevance weights between the two, and then integrates the historical intents based on these weights. Through this mechanism, the local intent understanding model can automatically focus on information from historical intents most relevant to the current semantics, thereby generating a fused semantic representation, i.e., the locally fused intent.

[0090] The result of fusion is a partial fusion intent. A partial fusion intent is an enhanced intent representation that integrates the current content with the historical context, making it more accurate and stable than a single partial intent. For example, if a user says, "I want to check that fee again," the current statement alone might be somewhat ambiguous. However, considering the historical records, if the user recently queried "broadband fees," then after fusion, it can be determined as "querying broadband fee bills." The purpose of obtaining partial fusion intents is to improve the accuracy of semantic understanding and provide a basis for generating more targeted feedback in the next step.

[0091] Furthermore, the second response generation layer L, based on the local fusion intent, can generate second response information. The second response generation layer L can be a template generation module or a language model-based generation module, converting the abstract local fusion intent into natural language feedback. Text generation of the local fusion intent involves generating confirmatory, guiding, or probing statements based on the identified local fusion intent. For example, when the local fusion intent is "to check your broadband bill," the following can be generated: "Do you want to check your broadband bill for this month?"

[0092] For example, in the first round of a call, a user says, "My phone number is 1385678." After the system completes full intent recognition, it stores {phone number: 1385678} in the dialogue history memory. In the second round, the user says, "I want to check last month's data" (cumulative character count 22, reaching the second threshold of 20). The streaming speech recognition engine inputs this text information into the local intent understanding model. The first semantic encoding layer I uses lightweight BERT to encode the segment, outputting a local semantic feature vector. Here, "check" activates the query intent dimension, and "last month" activates the time constraint dimension. Based on this, the local intent recognition layer J can decode the local intent {action: query, time: last month, object: to be filled, confidence: 0.72}. The intent fusion layer K retrieves the dialogue history memory based on "Time: Last Month", matches the first-round stored {Mobile Number: 1385678}, and calculates the association weight (0.81) between "Last Month" and "1385678" through a cross-self-attention mechanism. It then fuses these elements to generate a partially fused intent: {Action: Query, Time: Last Month, Object: Phone Bill [Speculation], Mobile Number: 1385678, Confidence: 0.85} (the phone bill is reasonably inferred based on the user's historical behavior of querying phone bills in the last 5 times). The second response generation layer L, based on the association weight of 0.85, uses an affirmative tone to generate the second response: "Okay, we are querying the phone bill for 1385678 from last month, please wait."

[0093] In one optional embodiment, outputting the first response information includes: calling an AI speech generation model to convert the first response information into first speech broadcast information, adding the first speech broadcast information to a speech output queue, and predicting the first broadcast duration required to broadcast the first speech broadcast information; if a first silence event with a silence duration greater than or equal to the first broadcast duration occurs after the cumulative character count reaches a first threshold but before reaching a second threshold, then the first speech broadcast information is broadcast within the silence time corresponding to the first silence event; if no silence event occurs after the cumulative character count reaches the first threshold but before reaching the second threshold, or if no first silence event with a silence duration greater than or equal to the first broadcast duration occurs, then the broadcast of the first speech broadcast information is delayed until another silence event with a silence duration greater than or equal to the first broadcast duration occurs before a silence event with a silence duration greater than a third threshold occurs.

[0094] In this application embodiment, the AI ​​speech generation model refers to an artificial intelligence synthesis module that converts text-based response information into natural and fluent speech signals, thereby realizing the conversion from text to speech and enabling the system output to have rhythm, intonation, and emotional expressiveness close to that of a real person, thereby improving the naturalness and affinity of human-computer interaction.

[0095] In this application, the specific form of the AI ​​speech generation model is not limited. Any model architecture that can achieve real-time text-to-speech synthesis and support playback duration prediction is applicable. For example, the AI ​​speech generation model can be an end-to-end TTS model based on deep neural networks, achieving highly natural speech output through Mel spectrum generation and vocoder synthesis; it can also be a real-time TTS engine based on streaming generation, supporting simultaneous generation and playback to reduce end-to-end latency; or it can be a multimodal generation system combining a large language model and speech synthesis, capable of dynamically adjusting speech rate and intonation parameters according to emotional state to generate emotional speech that matches the user's emotional state. All of the above models can be flexibly selected according to the computing power constraints and real-time requirements of the deployment environment.

[0096] The first voice broadcast message is voice data generated by the AI ​​voice generation model based on the first response message. It can be an audio stream or an audio segment, and is used to play it to the user during a telephone call. After being converted into the first voice broadcast message, the first voice broadcast message is added to the voice output queue.

[0097] The voice output queue is an internal data structure used by the system to manage voice segments to be played. It can queue and schedule multiple voice tasks to avoid overlapping playback or interruption of existing playback content. Through the voice output queue, the system can rationally arrange the order of responses to broadcast voice information.

[0098] The first playback duration is the estimated time required for the first voice broadcast to begin and end, measured in seconds. This duration can be predicted within the AI ​​speech generation model. Specific implementation methods include: estimating the overall speech duration during text input based on the number of text characters, speech rate parameters, and punctuation / pause rules; or directly calculating the total duration after the speech waveform is generated based on the audio frame count and sampling rate. For example, if the generated audio length is 32,000 frames and the sampling rate is 16kHz, the playback duration is 2 seconds. This first playback duration is used to subsequently determine if there is sufficient silence time for playback.

[0099] The first silence event refers to a silence event detected after the cumulative character count reaches the first threshold but before reaching the second threshold, where the silence duration is greater than or equal to the first broadcast duration. This is because at this stage, the system has completed rapid emotional response preparation, but the user has not yet entered a deeper semantic expression stage. If a sufficiently long pause occurs at this time, it indicates that there is a usable time window to insert the first voice broadcast information without interrupting the user's expression. Figure 1bAs shown, when the first silence event occurs, the first voice message is broadcast during the corresponding silence period. The system uses this silence period as a time window to provide emotional feedback without interrupting the user's speech, thereby improving the naturalness of the interaction.

[0100] If no silence event occurs after the cumulative character count reaches the first threshold but before reaching the second threshold, or if silence occurs but its duration is less than the first broadcast duration, it indicates that the user's expression is relatively coherent or the pauses are short, making it unsuitable to insert a complete emotional broadcast. Forcing playback at this point would overlap with or interrupt the user's speech, negatively impacting the user experience. Therefore, the system chooses to delay the broadcast of the first voice message and continue monitoring subsequent silence events. This delay prioritizes the smoothness of the dialogue between the user and the system, avoiding voice conflicts. Broadcasting is only completed within the silence window when other silence events occur with a duration greater than or equal to the first broadcast duration. If a silence event with a duration greater than the third threshold has already occurred before this, it indicates that the user has finished their current expression, and the system will switch to the full intent understanding and business response channel.

[0101] For example, suppose the first threshold is 20 characters, the second threshold is 50 characters, and the third threshold is 3 seconds. A user says, "Why can't it be processed? I've been waiting for ages..." When 25 characters have been recognized, the system generates the first response: "Don't worry, I'm speeding up the processing for you." The AI ​​speech generation model generates a 1.5-second first voice message and predicts its duration to be 1.5 seconds. If the user pauses for 2 seconds (greater than 1.5 seconds), this pause constitutes the first silence event, and the system plays the first voice message within that 2-second silence. If the user pauses for only 0.5 seconds and then continues speaking, the first playback duration requirement is not met, and the system delays playback. If a subsequent pause of more than 2 seconds occurs but has not yet exceeded the 3-second expression end threshold, playback occurs during that pause. This mechanism neither interrupts the user's subsequent expression nor fails to fill the interaction gaps, allowing the user to perceive that the system is listening attentively and empathizing, significantly improving the naturalness of the interaction and ensuring that the user does not experience a sense of disconnect due to system silence.

[0102] In one optional embodiment, outputting the second response information includes: calling an AI speech generation model to convert the second response information into second speech broadcast information, adding the second speech broadcast information to the speech output queue, and predicting the second broadcast duration required to broadcast the second speech broadcast information; if the speech output queue only includes the second speech broadcast information, then if a second silence event with a silence duration greater than or equal to the second broadcast duration occurs before the silence duration exceeds a third threshold, then the second speech broadcast information is broadcast within the silence time corresponding to the second silence event.

[0103] The second voice broadcast information is voice data that can be directly played, converted from the text-based second response information using an AI voice generation model. It is used to play this data to the user during a phone call. After being converted into second voice broadcast information, it is added to the voice output queue.

[0104] The second broadcast duration is the estimated time, in seconds, required for the second voice broadcast information to play from start to finish. The method for predicting the second broadcast duration is the same as the method for predicting the first broadcast duration in the above embodiments, and will not be repeated here.

[0105] The second silence event is a silence detected during the user's speech, and this silence meets the condition that "the silence duration is greater than or equal to the second broadcast duration." In other words, it is not an arbitrary silence, but a silence window that is "long enough to completely broadcast the second voice broadcast information." The silence time corresponding to the second silence event refers to the time interval during which the user does not speak. The system chooses to insert the broadcast within this time interval to achieve the effect of not interrupting the user while reducing blank waiting time.

[0106] like Figure 1b As shown, when the voice output queue contains only the second voice broadcast information, if the second silence event occurs before the silence duration exceeds the third threshold, the system will broadcast the second voice broadcast information. The third threshold corresponds to the key silence criterion for triggering the business response channel, usually indicating that "the user has finished this round of expression." Before reaching the third threshold, the user's silence is more likely to be a pause in the sentence or a pause in thinking. If a silence window is long enough to broadcast the second voice broadcast information (i.e., the second silence event), the system can broadcast the second voice broadcast information in that silence window, using "early semantic feedback" to cover the waiting time and reduce the silence perceived by the user.

[0107] Meanwhile, since only the second voice broadcast information is in the voice output queue, the broadcast will not compete with or preempt other response information (such as the first or third response information), avoiding output chaos caused by the superposition of multiple voice segments. Furthermore, in order to ensure that the broadcast can be completed within the same silent window, the silence duration must be greater than or equal to the second broadcast duration to reduce the risk of interruption or overlap caused by the user suddenly continuing to speak during the broadcast, thereby improving the naturalness of the call.

[0108] For example, the system generates a second response message in the pre-communication channel: "Let me confirm, are you trying to check the order status?". This is converted to a second voice broadcast message via TTS, with a predicted broadcast duration of 1.8 seconds. During the call, the user says, "My order… (pause)," where the pause is detected as a silence event with a duration of 2.2 seconds. Since the silence duration has not yet exceeded the third threshold (e.g., 3.0 seconds), this 2.2-second silence satisfies the condition of "silence duration ≥ second broadcast duration," constituting a second silence event. The system broadcasts the second voice message within this 2.2-second silence window, thus reducing waiting time.

[0109] In an optional embodiment, if the voice output queue includes both first voice broadcast information and second voice broadcast information, then as follows: Figure 1b If, before the silence duration exceeds the third threshold, a third silence event occurs with a silence duration greater than or equal to the sum of the first and second broadcast durations, then the first and second voice broadcast information are broadcast sequentially within the silence time corresponding to the third silence event. If the silence duration of the third silence event is less than the sum of the first and second broadcast durations, then the semantic coherence score between the first and second voice broadcast information is calculated. If the semantic coherence score is greater than or equal to a preset coherence threshold, the first and second voice broadcast information are semantically fused to obtain fused voice broadcast information with a broadcast duration less than the silence duration of the third silence event. Then, the fused voice broadcast information is broadcast within the silence time corresponding to the third silence event.

[0110] Specifically, when the voice output queue simultaneously includes both the first and second voice broadcast messages, the system is configured as follows: before a silence duration exceeds a third threshold, if a third silence event occurs with a silence duration greater than or equal to the sum of the first and second broadcast durations, the first and second voice broadcast messages will be broadcast sequentially within that silence period. The first voice broadcast message is used for emotional soothing or confirmation, while the second voice broadcast message is used for semantic confirmation, follow-up questions, or guidance. The two have a natural progressive relationship in their interaction logic; soothing emotions before advancing semantic communication aligns better with human conversation habits. Furthermore, sequential broadcasting only occurs when the silence duration is sufficient to cover the complete broadcast of both voice broadcast messages, preventing users from suddenly interrupting, causing overlap, or disruption, thereby improving the naturalness and continuity of the telephone interaction.

[0111] If the silence duration of the third silence event is less than the sum of the first and second broadcast durations, the current silence window is insufficient to fully broadcast both audio segments. If sequential broadcasting is still forced, the user may continue speaking before the broadcast is completed, resulting in audio overlap or information loss, thus affecting the interactive experience. Therefore, in this case, the system further introduces a semantic coherence scoring mechanism to evaluate the semantic relationship between the first and second audio broadcast information.

[0112] The semantic coherence score is a metric used to measure whether the first and second voice broadcast information can be naturally connected in semantic expression and whether there is logical consistency. The semantic coherence score can be calculated using a semantic similarity model or a sentence vector encoding model. This application does not limit the method of calculating the semantic coherence score. For example, the first and second response information corresponding to the first and second voice broadcast information can be encoded into semantic vectors respectively, and the cosine similarity between the semantic vectors of the first and second response information can be calculated; or a coherence discrimination model trained on the system can output probability values. A higher semantic coherence score indicates that the two voice contents can be more easily integrated into a coherent expression. For example, a reassuring statement + a confirmation question has high coherence, while a reassuring statement + an irrelevant business prompt has low coherence.

[0113] like Figure 1b As shown, when the silence duration of the third silence event is less than the sum of the first and second broadcast durations, and the semantic coherence score is greater than or equal to the preset coherence threshold, the fused voice broadcast information is broadcast within the silence time corresponding to the third silence event.

[0114] Specifically, if the semantic coherence score is greater than or equal to a preset coherence threshold, the first and second voice broadcast information are semantically fused to obtain fused voice broadcast information with a broadcast duration shorter than the silence duration of the third silence event. The preset coherence threshold is a pre-set judgment standard used to determine whether semantic fusion is allowed. For example, the preset coherence threshold can be set to 0.75 or 0.8, indicating that when the coherence score reaches this value, the system considers the first and second voice broadcast information to be merged into a single natural expression.

[0115] Semantic fusion involves compressing redundant expressions, removing duplicate sentences, and merging similar semantics in the text content corresponding to the first and second voice broadcast information. This ensures that the broadcast duration of the fused voice broadcast information generated by the AI ​​voice synthesis model is less than the silence duration of the third silence event, thereby guaranteeing that the fused voice broadcast information can be broadcast completely within the silence time corresponding to the third silence event and avoiding interruptions in the broadcast.

[0116] For example, suppose the first voice broadcast message is "I understand your anxiety, please don't worry," with a duration of 1.2 seconds; the second voice broadcast message is "Let me confirm first, do you want to check the order status?", with a duration of 1.6 seconds, for a total duration of 2.8 seconds. At this point, the system detects a third silent event with a silence duration of 2.0 seconds, which is less than 2.8 seconds, indicating that the silent window is insufficient to fully play both voice broadcast messages. The system calculates the semantic coherence score of the two voice broadcast messages, obtaining 0.85, and the preset coherence threshold is 0.8, therefore the fusion condition is met. The system semantically fuses the response information corresponding to the two voice broadcast messages to generate a fused voice broadcast message: "I understand you are in a hurry. Let me confirm for you first. Do you want to check the order status?" After fusion, the broadcast duration is shortened to 1.9 seconds, which is less than 2.0 seconds. Therefore, the system broadcasts the fused voice broadcast message during the silence time corresponding to the third silence event, achieving the technical effect of not interrupting the user and effectively eliminating call interaction delays and blank waiting periods.

[0117] In an optional embodiment, if the semantic coherence score is less than a preset coherence threshold, the relationship between the silence duration of the third silence event and the first and second broadcast durations is determined. If the silence duration of the third silence event is greater than the second broadcast duration, the second voice broadcast information is broadcast within the silence time corresponding to the third silence event. If the silence duration of the third silence event is greater than or equal to the first broadcast duration but less than the second broadcast duration, the first voice broadcast information is broadcast within the silence time corresponding to the third silence event.

[0118] like Figure 1b As shown, when the silence duration of the third silence event is less than the sum of the first and second broadcast durations, and the semantic coherence score is less than the preset coherence threshold, the first or second voice broadcast information is broadcast within the silence time corresponding to the third silence event.

[0119] Specifically, when the semantic coherence score is less than a preset coherence threshold, the user's current speech is semantically disjointed, incomplete, or logically disjointed. In such cases, the system needs to insert appropriate voice feedback to guide or reassure the user, effectively eliminating call delays and waiting periods. The system determines the relationship between the silence duration of the third silence event and the durations of the first and second broadcasts to clarify which voice message can be inserted within the currently available silence time and which message to broadcast, thus avoiding interrupting the user or causing speech overlap.

[0120] If the silence duration of the third silence event is greater than the second broadcast duration, it means that the current silence window is sufficient to broadcast the second voice broadcast information completely, and the second voice broadcast information is selected to be broadcast first. If the silence duration is greater than or equal to the first broadcast duration but less than the second broadcast duration, it means that the current silence window is insufficient to broadcast the complete second voice broadcast information, but can accommodate the shorter first voice broadcast information, so the first voice broadcast information is selected to be broadcast.

[0121] Since the second voice prompt is a semantic-level pre-emptive communication feedback, typically containing confirmation, follow-up questions, or guidance, it directly drives the dialogue towards a clear need, thus having a more direct impact on improving interaction efficiency. The first voice prompt, on the other hand, is a rapid emotional response; while it helps improve user experience, its contribution to understanding specific business needs is relatively weak. Therefore, the second voice prompt should be prioritized, with the first prompt only used as a fallback if the first fails to meet the requirements. This strategy of prioritizing semantic guidance and then using emotional reassurance when time is limited improves the utilization efficiency of the silent window, allowing the system to output more valuable feedback within the limited silent time.

[0122] For example, during a customer service call, a user abruptly says, "I'm so annoyed! I want to change my plan!" (5 characters reach the first threshold, 12 characters reach the second threshold). The emotion perception model identifies the "annoyed" emotion and generates the first response, "We understand your annoyance" (first broadcast duration 900ms); the local intent understanding model identifies the local fusion intent of "plan change" and generates the second response, "Checking the list of available plans for you" (second broadcast duration 1300ms). The user pauses for 1.1 seconds after saying "change plan" (third silence event duration 1100ms). The silence duration of the third silence event is less than the sum of the first and second broadcast durations. The system calculates the semantic coherence score of the two responses: because the "annoyed" emotion and the "check plan" action lack logical connection (the user is expressing dissatisfaction rather than inquiring), and the emotional response is more passive and the semantic response is more proactive, the score is only 0.45 (<0.7 preset coherence threshold), and it is determined that they cannot be fused. The system performs a duration comparison: 1100ms < 1300ms (second broadcast duration), but 1100ms ≥ 900ms (first broadcast duration). According to the priority rules, the system first checks whether the second voice broadcast message can be broadcast (1100ms < 1300ms, not allowed), then checks the first voice broadcast message (1100ms ≥ 900ms, allowed), and finally broadcasts the first voice broadcast message "We understand your frustration" within the silence time corresponding to the 1.1-second third silence event.

[0123] In an optional embodiment, for the first, second, or third silence event, the silence duration of the first, second, or third silence event is predicted based on the user's speech prosody decay characteristics before the occurrence of the first, second, or third silence event, the semantic boundary characteristics of the current text information, and the user's historical silence behavior patterns.

[0124] To more accurately determine whether a user's silence is a pause within a sentence or the end of an expression, the system predicts the potential duration of silence at the initial stage of the first, second, or third silence event. This prediction is based on three types of information: speech prosody decay features, semantic boundary features of the current text information, and the user's historical silence behavior patterns. These are explained below: Speech prosodic decay features are the acoustical weakening trends of a user's speech over a period of time before a silence occurs. Speech prosody includes information such as pitch (fundamental frequency F0), energy (volume), speech rate, rhythm, and stress distribution. When a user is about to finish a sentence or a unit of expression, phenomena such as a drop in pitch, a decrease in volume, a slowdown in speech rate, or a lengthening of the final sound will occur—a phenomenon known as "decay." Therefore, speech prosodic decay features can include: the rate of change in pitch at the end (such as the slope of a gradual decrease in F0), the magnitude of energy decay at the end, the degree of slowing in speech rate, and the degree of reduction in stress intensity. For example, if a significant drop in pitch and a gradual decrease in energy are detected within 200 milliseconds before silence, it can be identified as "sentence-end decay"; if the pitch remains high and the energy does not decrease significantly, it is more likely to be a "thinking pause." Speech prosodic decay features can help determine whether a user's expression is nearing completion.

[0125] The semantic boundary features of the current text information are boundary information obtained by analyzing the sentence structure and semantic completeness based on speech recognition to text conversion. Semantic boundaries can be manifested as periods, question marks, pause words (such as "um," "then"), grammatical structure closure (such as subject-verb-object completeness), and completion of intent expression. The system can identify whether semantic unit closure exists through natural language processing methods, such as whether a complete declarative or interrogative sentence is formed, or whether obvious modal ending words appear. If the current text has formed a semantically complete unit before the silence occurs, such as "My orders haven't been updated.", then the semantic boundary is strong; if the text is "My orders have been...", then the semantics are not closed, and the boundary is weak. Semantic boundary features can determine whether the current silence is located at the end of a semantic paragraph.

[0126] User historical silence behavior patterns are statistical regularities of a user's silent behavior during speech in past calls or the preceding part of the current conversation. For example, does the user habitually pause frequently in short sentences (e.g., 0.5 seconds every 2-3 seconds), typically exhibit longer silences at the end of a speech (e.g., more than 2 seconds), or show signs of thoughtful pauses? The system can model the distribution of a user's past silence duration to create a personalized silence probability model, such as establishing statistical characteristics like the mean, variance, and distribution range of silence duration. User historical silence behavior patterns can refine single-instance silence predictions, making predictions more consistent with individual expression habits.

[0127] For example, in a conversation between a user and an intelligent telephone system, the user says, "I want to ask when this order will be shipped." A silence follows "shipped." The system detects a drop in pitch, a decrease in energy, and a complete semantic closure. Historical data indicates that the user typically pauses for about 2.5 seconds at the end of a sentence. Therefore, the predicted silence duration is about 2-3 seconds, classifying it as an expression-ending silence (corresponding to the third silence event). However, if the user says, "I want to ask if this order… (pause) has been shipped," then the silence at "order…" does not show a significant drop in pitch, the semantic closure is not complete, and historical data shows that their pauses within sentences are typically less than 1 second. Therefore, the predicted silence duration is about 0.6 seconds, classifying it as a sentence-interval pause (corresponding to the first or second silence event).

[0128] Optionally, the specific implementation of predicting the silence duration based on the above three types of features is as follows: Construct a lightweight multimodal temporal regression model, which includes a feature encoding subnetwork, a temporal fusion subnetwork, and a regression output subnetwork.

[0129] The feature encoding subnetwork processes three types of inputs in parallel: a one-dimensional convolutional layer is used to extract the attenuation slope and curvature of the speech prosody attenuation features; an embedding layer is used to map the semantic boundary features into a 16-dimensional vector; and a lookup table mechanism is used to obtain the prior distribution parameters under the current speech rate condition for the user's historical silent behavior pattern.

[0130] The temporal fusion subnetwork uses gated recurrent units to process feature sequences from the past 2 seconds in 100-millisecond steps. It dynamically weights the three types of features using an attention mechanism and then fuses them based on these weights to obtain the fused temporal features. Specifically, the dynamic weighting logic is as follows: when the absolute value of the prosodic decay slope is greater than a threshold θ1, the acoustic feature weight is increased to 0.6; when the semantic boundary score is greater than θ2, the linguistic feature weight is increased to 0.5; and when the historical pattern standard deviation σ is less than θ3, the personalized feature weight is increased to 0.7.

[0131] The regression output subnetwork inputs the fused temporal features into a two-layer fully connected network, outputting a point estimate of the silence duration and its confidence interval. The multimodal temporal regression model runs in a streaming manner during user speech. When prosodic decay is detected (energy decay rate > 0.4 for 3 consecutive frames), it outputs an initial prediction and dynamically updates the prediction every 100 milliseconds with new feature input. 300 milliseconds before the actual occurrence of the silence event, it outputs the predicted silence duration of the first, second, or third silence event for broadcast scheduling.

[0132] Predicting silence duration using the methods described above can improve the accuracy of judging the completion of a user's expression, reducing the likelihood of prematurely interrupting due to misjudgment that the user has finished speaking. Simultaneously, it can improve the naturalness of the voice interaction rhythm, making the system's broadcast timing closer to human conversation habits. Through the fusion prediction of these multi-dimensional features, misjudgments caused by relying on a single time threshold can be avoided, thereby achieving more intelligent and refined dialogue control in real-time interactive intelligent telephone systems.

[0133] like Figure 1b The image shows an example of the internal implementation structure of a complete intent understanding model. In this example, the complete intent understanding model includes: a second semantic encoding layer M, a complete intent recognition layer N, a business routing layer O, and a third response generation layer P. Based on the internal implementation structure of the complete intent understanding model, a specific implementation method is provided that utilizes the complete intent understanding model to perform complete intent recognition on all text information, calls the target business system matching the complete intent through the business routing engine to perform corresponding business operations, and outputs third response information based on the execution result of the business operation to achieve a complete response at the business level. The method includes: inputting all text information into the second semantic encoding layer M, performing keyword extraction, sentence segmentation, paragraph division, and central semantic summarization to obtain multiple keywords, multiple sentences, at least one text paragraph, and central semantics; vectorizing the multiple keywords, multiple sentences, at least one text paragraph, and central semantics to obtain multiple word vectors, multiple sentence vectors, at least one text paragraph vector, and central semantic vector; and vectorizing the multiple word vectors, multiple sentences, at least one text paragraph, and central semantics to obtain multiple word vectors, multiple sentence vectors, at least one text paragraph vector, and central semantic vector. Sentence vectors, at least one text segment vector, and a central semantic vector are input into the complete intent recognition layer N, where intent recognition is performed in parallel across keyword, sentence, text segment, and central semantic dimensions. The intents at the keyword dimension and sentence dimension are fused in a first layer to obtain a first fused intent. This first fused intent is then fused in a second layer with the intents at the text segment dimension to obtain a second fused intent. Finally, this second fused intent is fused in a third layer with the intents at the central semantic dimension to obtain the complete intent. The complete intent is then input into the business routing layer O, which matches the target business system based on the complete intent and invokes the target business system through the business routing engine to execute the corresponding business operations, obtaining the execution result. The execution result is then input into the third response generation layer P, which generates a third response based on the user's complete business requirements.

[0134] The second semantic encoding layer M is the semantic structure parsing module of the complete intent understanding model. It can perform structured semantic decomposition and abstract expression of all user text information, providing hierarchical input for subsequent multi-dimensional intent recognition. All text information refers to the complete expression content formed by the user in this round of dialogue, such as continuous text integrated after speech recognition.

[0135] In the second semantic encoding layer M, keyword extraction, sentence segmentation, paragraph division, and central semantic summarization are performed. Keyword extraction identifies words from the entire text that represent the core business meaning. These include nouns, verbs, or business entity words, used to capture key triggers of intent. Sentence segmentation divides the text into multiple sentence units based on punctuation, pauses, or grammatical structure for local semantic analysis. Paragraph division divides the text into at least one semantic paragraph based on semantic coherence or topic changes, reflecting the logical hierarchy of user expression. Central semantic summarization performs a semantic summary or extracts the core intent from the entire text, forming a semantic representation that summarizes the overall requirements.

[0136] For example, the complete text message could be: "My order from last month hasn't been shipped yet, and I want to apply for a refund. Can you help me check the progress?" Keywords could include: "last month," "order," "not shipped," "refund," and "progress." The sentence could be segmented into: "My order from last month hasn't been shipped yet," "I want to apply for a refund now," and "Can you help me check the progress?" The entire text could be considered a single business paragraph, and the central meaning could be summarized as: "Check order status and apply for a refund."

[0137] After structural decomposition, the second semantic encoding layer M can vectorize multiple keywords, multiple sentences, at least one text paragraph, and the central semantic element. Vectorization converts all textual information from natural language into a computable numerical representation, enabling the model to perform similarity matching and classification through vector operations. Multiple keywords are encoded by the embedding model to form multiple word vectors. Each sentence can be encoded at the sentence level to form a sentence vector; text paragraphs can be encoded at the paragraph level to obtain text paragraph vectors; and the central semantic element can be encoded at the semantic summary level to obtain a central semantic vector. These vectors express the semantic information of the text at different granularities: word vectors can emphasize local triggering information, sentence vectors can emphasize semantic relationships, text paragraph vectors can reflect the logical structure, and the central semantic vector can reflect the overall business objective.

[0138] Furthermore, the complete intent recognition layer N can perform parallel intent recognition on the aforementioned multi-layer vectors. The complete intent recognition layer N can employ a multi-channel neural network structure, predicting intent categories across four dimensions: keywords, sentences, text segments, and central semantics. At the keyword level, it can identify explicit business intents such as "refund," "query," and "complaint." At the sentence level, it can determine whether a sentence expresses a semantic intent such as "problem statement," "operation request," or "status confirmation." At the text segment level, it can identify whether the user's current paragraph belongs to a "after-sales processing scenario" or an "order query scenario." At the central semantics level, it can determine the overall intent of all text information, such as "order not shipped leading to a refund request and progress query."

[0139] For example, in the example above: the intent at the keyword dimension could be a refund request; the intent at the sentence dimension could be query status + initiate a refund; the intent at the text segment dimension could be after-sales issue handling; and the intent at the central semantic dimension could be initiating a refund process for unshipped orders.

[0140] After obtaining the intents from each dimension, a complete intent is obtained through layered fusion. The first layer of fusion combines the intents at the keyword dimension with those at the sentence dimension to eliminate keyword ambiguity and enhance contextual constraints, resulting in the first fused intent. For example, combining "refund" with "query progress" determines it as "refund after query" rather than simply a refund.

[0141] The second layer of fusion combines the first fusion intent with the intent at the text segment level to obtain the second fusion intent, which is used to confirm the business scenario affiliation, such as confirming that it belongs to the "after-sales refund scenario".

[0142] The third layer of fusion combines the second fusion intent with the intent on the central semantic dimension, using the central semantic as a global constraint to obtain the final complete intent. The complete intent can be represented as a structured result, such as: "Business type = after-sales refund; Trigger reason = not shipped; Attached requirement = query processing progress".

[0143] The layered fusion approach can gradually eliminate ambiguity from the local to the global level, avoiding misjudgment based solely on keywords and avoiding ignoring key trigger words based solely on global semantics, thereby improving the accuracy and robustness of complete intent recognition.

[0144] After obtaining the complete intent, the complete intent is input into the business routing layer O. The role of the business routing layer O is to determine the corresponding target business system based on the complete intent, and to execute the corresponding business operation by calling the interface of the target business system through the business routing engine. The specific implementation method of this part has been described in detail in the previous embodiments and will not be repeated here.

[0145] After the business system returns the execution result, the result is input into the third response generation layer P. The third response generation layer P is a response generation module oriented towards business results, capable of generating complete and accurate third response information based on the execution result of the business operation. Based on the above-mentioned multi-dimensional parallel recognition and hierarchical fusion of keywords, sentences, text segments, and central semantics, the accuracy and robustness of complete intent recognition are improved. Combined with a business routing engine to achieve automatic business invocation, this multi-dimensional fusion structure can enhance intent recognition capabilities in complex semantic scenarios, improve business matching accuracy, and reduce the risk of misrouting.

[0146] For example, if the system execution result is the query result "The order has not yet been shipped, but it meets the refund conditions, and the refund will arrive within 3 business days," then the third response generation layer P organizes the language according to this execution result to generate a complete reply, such as: "Your order has not yet been shipped. We have submitted a refund application for you, which is expected to be returned to the original payment method within 3 business days. Please check your account." Through the structure of the above complete intent understanding model, a closed-loop processing can be achieved from natural language to business execution to accurate response, which can meet the complete response requirements at the business level.

[0147] like Figure 1b As shown, the third response information is output, including: calling the AI ​​voice generation model to convert the third response information into third voice broadcast information, adding the third voice broadcast information to the voice output queue, and directly broadcasting the third voice broadcast information.

[0148] Among them, the third response information is generated based on the business execution result and converted into voice output in real time to realize a complete business closed loop. This enables the system to form a fully automated processing flow from voice input to business execution and then to voice feedback, thereby improving processing efficiency and automation level.

[0149] In summary, by combining response information generation with voice output queue management, silence detection mechanisms, and broadcast duration prediction, and rationally broadcasting voice information under the condition of silence events, not only is abrupt interruption of the user's expression process avoided, but the system can also make full use of the user's silence time to achieve pre-emptive feedback, thereby shortening the user's perceived waiting time and improving the continuity of interaction and the timeliness of response. At the same time, by matching and controlling the broadcast duration with the silence duration, the system reduces the risk of voice overlap and interruption, improves the naturalness and stability of the call process, and enables the system to achieve a smoother, more efficient, and more humanized voice interaction experience while ensuring accurate understanding.

[0150] It should be noted that in the above embodiments, a large language model refers to a model whose parameters meet the set parameter quantity requirements, and the parameter quantity requirements are not limited. Different definitions may exist in different scenarios and domains. For example, in some scenarios or domains, a large language model refers to a model with a parameter scale reaching tens of billions, hundreds of billions, or even trillions; this definition of the model's parameter quantity is merely an example.

[0151] The detailed implementation methods and beneficial effects of each step in this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0152] Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0153] Figure 2 This is a schematic diagram of an electronic device structure provided for an exemplary embodiment of this application. For example... Figure 2 As shown, the electronic device includes: a memory 24 and a processor 25.

[0154] Memory 24 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, metadata, first response information, second response information, and third response information, etc.

[0155] The memory 24 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0156] Processor 25, coupled to memory 24, executes computer programs in memory 24 for: The system includes a streaming speech recognition engine and three parallel channels: a fast response channel based on an emotion perception model, a pre-communication channel based on a partial intent understanding model, and a business response channel based on a complete intent understanding model; during a user's call with the smartphone system via a terminal, the streaming speech recognition engine converts the user's speech into text information in real time, and inputs the text information and its metadata into the three channels in parallel; the metadata includes the cumulative number of characters in the text information and a description of the silence event; when the cumulative number of characters reaches a first threshold, the fast response channel is triggered, and the emotion perception model analyzes the user's emotional state based on the current text information, combined with the emotion response... The response strategy library outputs the first response information to achieve a rapid response at the emotional level. When the cumulative number of characters reaches a second threshold greater than the first threshold, the pre-communication channel is triggered. Using the local intent understanding model, based on the current text information and the dialogue history memory library, local intent recognition and historical intent association are performed simultaneously to obtain the local fusion intent. Based on this local fusion intent, the second response information is output to achieve semantic-level interactive feedback. When a silence event occurs with a silence duration greater than a third threshold, the business response channel is triggered. Using the complete intent understanding model, complete intent recognition is performed on all text information. The business routing engine calls the target business system that matches the complete intent to perform the corresponding business operation. Based on the execution result of the business operation, the third response information is output to achieve a complete response at the business level.

[0157] In an optional embodiment, the first threshold and the second threshold are dynamically determined. The processor 25 calculates the user's real-time speech rate based on the number of characters in the text information within at least one unit of time; predicts the background noise level of the call environment based on the spectral characteristics of the user's speech; inputs the real-time speech rate and the background noise level into the dynamic threshold prediction model to perform real-time prediction of the character number threshold, and outputs the first threshold and the second threshold.

[0158] In one optional embodiment, the dynamic threshold prediction model includes a feature input layer, a feature fusion layer, a threshold prediction layer, and a threshold output layer. The processor 25 inputs real-time speech rate and background noise level into the dynamic threshold prediction model to perform real-time prediction of the character number threshold and outputs a first threshold and a second threshold. This includes: inputting real-time speech rate and background noise level into the feature input layer, performing standardization processing and feature extraction on the real-time speech rate and background noise level to obtain speech rate features and noise features; inputting speech rate features and noise features into the feature fusion layer for joint modeling to extract fusion features reflecting semantic density and recognition stability; inputting the fusion features into the threshold prediction layer, generating a character number threshold adjustment parameter based on the relationship between the fusion features and a preset speech rate threshold and a preset noise threshold; and inputting the character number threshold adjustment parameter into the threshold output layer to adjust a preset baseline character number threshold and output the first threshold and the second threshold.

[0159] In one optional embodiment, the emotion perception model includes a feature encoding layer, an emotion feature extraction layer, an emotion classification layer, and a first response generation layer. The processor 25 utilizes the emotion perception model to analyze the user's emotional state based on current text information and outputs first response information in conjunction with an emotion response strategy library to achieve rapid response at the emotion level. This includes: inputting the current text information into the feature encoding layer for encoding processing at three levels: tone, speech rate, and semantics, resulting in a word vector sequence including tone vector, speech rate vector, and semantic vector; inputting the word vector sequence into the emotion feature extraction layer, extracting emotion intensity features reflecting the user's tone fluctuations based on the tone vector, extracting features reflecting the user's speaking rhythm based on the speech rate vector, and extracting features reflecting the text's emotional tendency based on the semantic vector. Semantic sentiment features are fused from three levels of emotional features to obtain an emotional context feature vector. This emotional context feature vector is then input into the emotion classification layer, where the similarity between the emotional context feature vector and multiple emotional state category vectors learned during model training is calculated. A target emotional state category vector whose similarity to the emotional context feature vector meets a set condition is selected, and the emotional state expressed by the target emotional state category vector is taken as the user's emotional state. The emotional state is then input into the first response generation layer, which, combined with preset emotional response rules, queries the emotional response strategy library for a response strategy that matches the emotional state. Based on the expression style and soothing goal defined by the response strategy, text is generated to obtain the first response information corresponding to the emotional state.

[0160] In an optional embodiment, the local intent understanding model includes a first semantic encoding layer, a local intent recognition layer, an intent fusion layer, and a second response generation layer. The processor 25 utilizes the local intent understanding model to simultaneously perform local intent recognition and historical intent association based on current text information and a dialogue history memory, obtaining a locally fused intent. Based on this locally fused intent, it outputs second response information to achieve semantic-level interactive feedback. This includes: inputting current text information into the first semantic encoding layer for local semantic encoding to obtain a local semantic feature vector; inputting the local semantic feature vector into the local intent recognition layer for local intent decoding to obtain a local intent; inputting the local intent into the intent fusion layer, extracting historical intents related to the user's speech from the dialogue history memory based on the local intent, and fusing the local intent and historical intent using a cross-self-attention mechanism to obtain a locally fused intent; and inputting the locally fused intent into the second response generation layer for text generation to obtain the second response information.

[0161] In an optional embodiment, the processor 25 outputs the first response information by: calling an AI speech generation model to convert the first response information into first speech broadcast information, adding the first speech broadcast information to a speech output queue, and predicting the first broadcast duration required to broadcast the first speech broadcast information; if a first silence event with a silence duration greater than or equal to the first broadcast duration occurs after the cumulative character count reaches a first threshold but before reaching a second threshold, the first speech broadcast information is broadcast within the silence time corresponding to the first silence event; if no silence event occurs after the cumulative character count reaches the first threshold but before reaching the second threshold, or if no first silence event with a silence duration greater than or equal to the first broadcast duration occurs, the broadcast of the first speech broadcast information is delayed until another silence event with a silence duration greater than or equal to the first broadcast duration occurs before a silence event with a silence duration greater than a third threshold occurs.

[0162] In one optional embodiment, the processor 25 outputs second response information, including: calling an AI speech generation model to convert the second response information into second speech broadcast information, adding the second speech broadcast information to a speech output queue, and predicting the second broadcast duration required to broadcast the second speech broadcast information; if the speech output queue only includes the second speech broadcast information, then if a second silence event occurs before the silence duration exceeds a third threshold, and the silence duration is greater than or equal to the second broadcast duration, then the second speech broadcast information is broadcast within the silence time corresponding to the second silence event.

[0163] In an optional embodiment, if the voice output queue includes both first voice broadcast information and second voice broadcast information, then before a silence duration greater than a third threshold occurs, if a third silence event occurs with a silence duration greater than or equal to the sum of the first and second broadcast durations, then the processor 25 will sequentially broadcast the first and second voice broadcast information within the silence time corresponding to the third silence event. If the silence duration of the third silence event is less than the sum of the first and second broadcast durations, then the semantic coherence score between the first and second voice broadcast information is calculated. If the semantic coherence score is greater than or equal to a preset coherence threshold, the first and second voice broadcast information are semantically fused to obtain fused voice broadcast information with a broadcast duration less than the silence duration of the third silence event. Then, the fused voice broadcast information will be broadcast within the silence time corresponding to the third silence event.

[0164] In an optional embodiment, if the semantic coherence score is less than a preset coherence threshold, the processor 25 determines the relationship between the silence duration of the third silence event and the first and second broadcast durations; if the silence duration of the third silence event is greater than the second broadcast duration, the processor 25 broadcasts the second voice broadcast information within the silence time corresponding to the third silence event; if the silence duration of the third silence event is greater than or equal to the first broadcast duration but less than the second broadcast duration, the processor 25 broadcasts the first voice broadcast information within the silence time corresponding to the third silence event.

[0165] In an optional embodiment, the processor 25 predicts the duration of silence for the first, second, or third silence event based on the user's speech prosody decay characteristics before the first, second, or third silence event, the semantic boundary characteristics of the current text information, and the user's historical silence behavior patterns.

[0166] In an optional embodiment, the complete intent understanding model includes a second semantic encoding layer, a complete intent recognition layer, a business routing layer, and a third response generation layer. The processor 25 uses the complete intent understanding model to perform complete intent recognition on all text information, and calls the target business system matching the complete intent through the business routing engine to perform corresponding business operations. Based on the execution result of the business operation, it outputs third response information to achieve a complete response at the business level. This includes: inputting all text information into the second semantic encoding layer, performing keyword extraction, sentence segmentation, paragraph division, and central semantic summarization to obtain multiple keywords, multiple sentences, at least one text paragraph, and central semantics; and vectorizing the multiple keywords, multiple sentences, at least one text paragraph, and central semantics to obtain multiple word vectors, multiple sentence vectors, at least one text segment vector, and central semantics. Semantic vectors: Multiple word vectors, multiple sentence vectors, at least one text segment vector, and a central semantic vector are input into the complete intent recognition layer. Intent recognition is performed in parallel across the keyword, sentence, text segment, and central semantic dimensions. The intent at the keyword dimension and the intent at the sentence dimension are fused in a first layer to obtain a first fused intent. The first fused intent is then fused in a second layer with the intent at the text segment dimension to obtain a second fused intent. The second fused intent is then fused in a third layer with the intent at the central semantic dimension to obtain a complete intent. The complete intent is input into the business routing layer, which matches the target business system based on the complete intent and calls the target business system through the business routing engine to execute the corresponding business operation, obtaining the execution result. The execution result is input into the third response generation layer, which generates third response information based on the execution result for the user's complete business needs.

[0167] In an optional embodiment, the processor 25 outputs third response information, including: calling an AI voice generation model to convert the third response information into third voice broadcast information, adding the third voice broadcast information to the voice output queue, and broadcasting the third voice broadcast information.

[0168] Furthermore, such as Figure 2 As shown, the electronic device also includes other components such as a communication component 26, a display 27, a power supply component 28, and an audio component 29. Figure 2 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 2 The components shown.

[0169] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be performed by an electronic device in the above method embodiments.

[0170] The above Figure 2The communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0171] The above Figure 2 The display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action, but also the duration and pressure associated with the touch or swipe operation.

[0172] The above Figure 2 The power supply component provides power to the various components of the device in which it resides. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which it resides.

[0173] The above Figure 2 The audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0174] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0175] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0176] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0177] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0178] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0179] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0180] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0181] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0182] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An information processing method for an intelligent telephone system, characterized in that, The system includes a streaming speech recognition engine and three parallel channels: a fast response channel based on an emotion perception model, a pre-communication channel based on a partial intent understanding model, and a business response channel based on a complete intent understanding model; the method includes: During a user's call with a smart phone system via a terminal, a streaming speech recognition engine converts the user's speech into text information in real time, and inputs the text information and its metadata into three channels in parallel; the metadata includes the cumulative number of characters in the text information and the description information of the silent event when a silent event occurs. When the cumulative number of characters reaches the first threshold, the fast response channel is triggered. The emotion perception model is used to analyze the user's emotional state based on the current text information, and the first response information is output in combination with the emotion response strategy library to achieve a fast response at the emotion level. When the cumulative number of characters reaches a second threshold greater than the first threshold, the pre-communication channel is triggered. The local intent understanding model is used to simultaneously perform local intent recognition and historical intent association based on the current text information and the dialogue history memory bank to obtain the local fusion intent. Based on the local fusion intent, the second response information is output to achieve semantic-level interactive feedback. When a silence event occurs with a duration exceeding the third threshold, the business response channel is triggered. The complete intent understanding model is used to perform complete intent recognition on all text information. The business routing engine then calls the target business system that matches the complete intent to perform the corresponding business operation. Based on the execution result of the business operation, the third response information is output, thus achieving a complete response at the business level.

2. The method according to claim 1, characterized in that, The first threshold and the second threshold are dynamically determined, and the method further includes: The user's real-time speaking speed is calculated based on the number of characters that increase in the text information within at least one unit of time. Predicting background noise levels in the call environment based on the spectral characteristics of user speech; The real-time speech rate and background noise level are input into the dynamic threshold prediction model to predict the character number threshold in real time, and the first threshold and the second threshold are output.

3. The method according to claim 2, characterized in that, The dynamic threshold prediction model includes a feature input layer, a feature fusion layer, a threshold prediction layer, and a threshold output layer. The real-time speech rate and background noise level are input into the dynamic threshold prediction model to perform real-time prediction of the character count threshold, and the model outputs a first threshold and a second threshold, including: The real-time speech rate and background noise level are input into the feature input layer, and the real-time speech rate and background noise level are standardized and feature extracted to obtain speech rate features and noise features. The speech rate feature and the noise feature are input into the feature fusion layer for joint modeling, and the fusion feature reflecting semantic density and recognition stability is extracted. The fusion features are input into the threshold prediction layer, and a character number threshold adjustment parameter is generated based on the relationship between the fusion features and the preset speech rate threshold and the preset noise threshold. The character count threshold adjustment parameter is input into the threshold output layer to adjust the preset baseline character count threshold and output the first threshold and the second threshold.

4. The method according to claim 1, characterized in that, The emotion perception model includes a feature encoding layer, an emotion feature extraction layer, an emotion classification layer, and a first response generation layer. It analyzes the user's emotional state based on current text information and outputs first response information in conjunction with an emotion response strategy library to achieve rapid emotional response, including: The current text information is input into the feature encoding layer for encoding processing at three levels: intonation, speech rate, and semantics, resulting in a word vector sequence including intonation vector, speech rate vector, and semantic vector. The word vector sequence is input into the emotion feature extraction layer. Based on the intonation vector, emotion intensity features reflecting the fluctuation of the user's tone are extracted; based on the speech rate vector, features reflecting the speed of the user's speech rhythm are extracted; and based on the semantic vector, semantic emotion features reflecting the emotional tendency of the text are extracted. The emotion features of the three levels are fused to obtain the emotion context feature vector. The emotional context feature vector is input into the emotional classification layer. The similarity between the emotional context feature vector and multiple emotional state category vectors learned during model training is calculated. A target emotional state category vector whose similarity to the emotional context feature vector meets the set conditions is selected. The emotional state expressed by the target emotional state category vector is taken as the user's emotional state. The emotional state is input into the first response generation layer. Combined with preset emotional response rules, the layer queries the emotional response strategy library for a response strategy that matches the emotional state. Based on the expression style and soothing goal defined by the response strategy, text is generated to obtain the first response information corresponding to the emotional state.

5. The method according to claim 1, characterized in that, The local intent understanding model includes a first semantic encoding layer, a local intent recognition layer, an intent fusion layer, and a second response generation layer. Based on current text information and a dialogue history memory, the local intent understanding model simultaneously performs local intent recognition and historical intent association to obtain a locally fused intent. Based on this locally fused intent, it outputs second response information to achieve semantic-level interactive feedback, including: The current text information is input into the first semantic encoding layer for local semantic encoding to obtain a local semantic feature vector. The local semantic feature vector is input into the local intent recognition layer, and the local semantic feature vector is decoded to obtain the local intent. The local intent is input into the intent fusion layer. Based on the local intent, historical intents related to the user's voice are extracted from the dialogue history memory. The local intent and the historical intent are fused using a cross-self-attention mechanism to obtain the local fused intent. The local fusion intent is input into the second response generation layer, and the local fusion intent is used to generate text to obtain the second response information.

6. The method according to any one of claims 1-5, characterized in that, The first response information output includes: The AI ​​voice generation model is invoked to convert the first response information into first voice broadcast information, the first voice broadcast information is added to the voice output queue, and the first broadcast duration required to broadcast the first voice broadcast information is predicted. If a first silence event occurs after the cumulative number of characters reaches the first threshold and before the second threshold, and the silence duration is greater than or equal to the first broadcast duration, then the first voice broadcast information is broadcast during the silence time corresponding to the first silence event. If no silence event occurs after the cumulative number of characters reaches the first threshold and before the second threshold is reached, or if no first silence event with a duration greater than or equal to the first broadcast duration occurs, the broadcast of the first voice broadcast information is delayed until another silence event with a duration greater than or equal to the first broadcast duration occurs before a silence event with a duration greater than the third threshold occurs.

7. The method according to claim 6, characterized in that, Output the second response information, including: The AI ​​voice generation model is invoked to convert the second response information into second voice broadcast information, the second voice broadcast information is added to the voice output queue, and the second broadcast duration required to broadcast the second voice broadcast information is predicted. If the voice output queue contains only the second voice broadcast information, then if a second silence event occurs before the silence duration exceeds the third threshold, and the silence duration is greater than or equal to the second broadcast duration, then the second voice broadcast information will be broadcast within the silence time corresponding to the second silence event.

8. The method according to claim 7, characterized in that, Also includes: If the voice output queue includes both the first voice broadcast information and the second voice broadcast information, then before the silence duration exceeds the third threshold, if a third silence event occurs with a silence duration greater than or equal to the sum of the first and second broadcast durations, then the first voice broadcast information and the second voice broadcast information will be broadcast sequentially within the silence time corresponding to the third silence event. If the silence duration of the third silence event is less than the sum of the first broadcast duration and the second broadcast duration, then the semantic coherence score between the first voice broadcast information and the second voice broadcast information is calculated. If the semantic coherence score is greater than or equal to a preset coherence threshold, the first voice broadcast information and the second voice broadcast information are semantically fused to obtain fused voice broadcast information with a broadcast duration less than the silence duration of the third silence event. Then, the fused voice broadcast information is broadcast during the silence time corresponding to the third silence event.

9. The method according to claim 8, characterized in that, Also includes: If the semantic coherence score is less than the preset coherence threshold, then the relationship between the silence duration of the third silent event and the first and second broadcast durations is determined. If the duration of the third silence event is greater than the duration of the second broadcast, then the second voice broadcast information is broadcast during the silence time corresponding to the third silence event. If the silence duration of the third silence event is greater than or equal to the first broadcast duration but less than the second broadcast duration, then the first voice broadcast information is broadcast during the silence time corresponding to the third silence event.

10. The method according to claim 9, characterized in that, Also includes: For the first, second, or third silence events, the silence duration of the first, second, or third silence events is predicted based on the user's speech prosody decay characteristics before the occurrence of the first, second, or third silence events, the semantic boundary characteristics of the current text information, and the user's historical silence behavior patterns.

11. The method according to any one of claims 1-10, characterized in that, The complete intent understanding model includes a second semantic encoding layer, a complete intent recognition layer, a business routing layer, and a third response generation layer. It utilizes the complete intent understanding model to perform complete intent recognition on all text information, and uses the business routing engine to call the target business system matching the complete intent to execute the corresponding business operation. Based on the execution result of this business operation, it outputs third response information, achieving a complete response at the business level, including: All text information is input into the second semantic encoding layer for keyword extraction, sentence segmentation, paragraph division, and central semantic summarization, resulting in multiple keywords, multiple sentences, at least one text paragraph, and central semantics. The multiple keywords, multiple sentences, at least one text paragraph, and central semantics are then vectorized to obtain multiple word vectors, multiple sentence vectors, at least one text paragraph vector, and central semantic vector. The multiple word vectors, multiple sentence vectors, at least one text segment vector, and the central semantic vector are input into the complete intent recognition layer, and intent recognition is performed in parallel on the keyword, sentence, text segment, and central semantic dimensions. The intent on the keyword dimension and the intent on the sentence dimension are fused in the first layer to obtain the first fused intent. The first fused intent is fused in the second layer with the intent on the text segment dimension to obtain the second fused intent. The second fused intent is fused in the third layer with the intent on the central semantic dimension to obtain the complete intent. The complete intent is input into the business routing layer, the target business system is matched according to the complete intent, and the corresponding business operation is executed by calling the target business system through the business routing engine to obtain the execution result. The execution result is input into the third response generation layer, and a third response information tailored to the user's complete business needs is generated based on the execution result.

12. The method according to claim 11, characterized in that, Output the third response information, including: The AI ​​voice generation model is invoked to convert the third response information into third voice broadcast information, which is then added to the voice output queue and broadcast.

13. An electronic device, characterized in that, include: A processor and a memory, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1-12.

14. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 1-12.

15. A computer program product, characterized in that, Includes a computer program / instruction that, when executed by a processor, causes the processor to perform the steps of the method according to any one of claims 1-12.

Images