Voice customer service processing method, system and device based on large model and storage medium

Abstract

The invention provides a voice customer service processing method, system and device based on a large model and a storage medium, and the method comprises the steps: receiving input data from a user, and carrying out the shunting processing of the input data based on the feature information related to the user; establishing a session corresponding to the user, and executing identity verification and voice activity detection on the voice input to obtain an effective voice segment; performing voice recognition on the effective voice segments to generate text information, and constructing input data for large model processing based on the text information and session context; the large model processes the input data to generate a response text for representing customer service response content, and determines a corresponding service processing action; generating voice output based on the response text, and executing a service processing action in the session process; and judging whether to end the session based on the session state, if a condition is met, ending the session, otherwise, continuously processing the user input. According to the invention, the voice customer service processing flow with high efficiency, low time delay and higher reliability is realized.

Description

Technical Field

[0001] This application relates to the fields of speech processing and artificial intelligence technology, and in particular to a speech customer service processing method, system, device and storage medium based on a large model. Background Technology

[0002] With the rapid development of artificial intelligence technology, voice interaction has been widely applied in the field of intelligent customer service. Traditional interactive voice response (IVR) systems typically rely on preset voice menus, using telephone keypad (DTMF) or simple voice commands to select and route user intents, which is insufficient to meet the complex and ever-changing business consultation needs. To improve service efficiency, some customer service systems have begun to introduce artificial intelligence technologies such as automatic speech recognition (ASR), text-to-speech (TTS), and large language models (LLM) to enhance the system's semantic understanding and natural language response capabilities.

[0003] However, in practical enterprise applications, existing intelligent customer service systems still struggle to meet the automation needs of high-frequency business scenarios. Many repetitive issues still rely on manual processing, leading to a continuous increase in service costs and pressure. Simultaneously, as business scale expands, user requests exhibit characteristics such as diversification, continuity, and real-time demands, revealing significant shortcomings in traditional technology systems regarding processing efficiency, semantic understanding, and business process flexibility. Existing publicly available literature, such as patent document (publication number: CN119474280A), proposes an intelligent customer service system based on a large-scale artificial intelligence model. This system employs a modular architecture to handle multimodal inputs and utilizes knowledge graphs and business rules to achieve semantic understanding and task processing. While this solution can improve the processing capability for complex inputs to some extent, it still suffers from the following shortcomings:

[0004] 1. The lack of an effective traffic distribution mechanism means that all requests are processed through a unified entry point, which can easily lead to excessive computational load and exacerbate semantic ambiguity, affecting overall processing efficiency and accuracy of understanding.

[0005] 2. It relies heavily on knowledge graphs and manual rule construction, resulting in high upfront costs and complex updates and maintenance, making it difficult to meet the agile deployment needs of high-frequency business scenarios.

[0006] 3. The processing link is long and the latency is high, making it difficult to meet the requirements of voice customer service for low latency and high real-time performance, thus affecting the smoothness of interaction.

[0007] In summary, existing voice customer service solutions still have shortcomings in terms of request routing, high-frequency business automation, system construction costs, and real-time voice interaction. Therefore, there is an urgent need to provide an intelligent customer service processing solution that can efficiently route user requests, reduce system construction and maintenance costs, and is suitable for high real-time voice interaction requirements. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this application provides a voice customer service processing method, system, device, and storage medium based on a large model, which at least solves the technical problems of insufficient intent recognition accuracy, business processing efficiency, and real-time response capability in existing voice customer service systems.

[0009] To achieve the above objectives and other advantages, some embodiments of this application provide the following aspects:

[0010] Firstly, some embodiments of this application provide a voice customer service processing method based on a large model, including:

[0011] Receive input data from the user and perform split processing on the input data based on feature information related to the user;

[0012] Establish a session corresponding to the user, and perform identity verification and voice activity detection on the voice input in the input data to obtain effective voice segments for semantic parsing;

[0013] The effective speech segments are used to generate text information through speech recognition, and input data for large model processing is constructed based on the text information and the session context.

[0014] The input data is processed based on the large model to generate response text that represents the content of customer service response, and to determine the business processing action corresponding to the response text.

[0015] The system generates voice output based on the response text and performs the business processing action during the session in which the voice output is generated.

[0016] The session state determines whether the session termination condition is met. If it is met, the session is terminated; otherwise, the session is maintained and user input data is continuously processed.

[0017] Secondly, some embodiments of this application also provide a voice customer service processing system based on a large model, including:

[0018] The softphone module is used to receive input data from the user and perform stream processing on the input data based on feature information related to the user;

[0019] The voice processing module is used to establish a conversation with the user and perform identity verification and voice activity detection on the voice input in the input data to obtain effective voice segments for semantic parsing.

[0020] The speech recognition module is used to generate text information from the effective speech segments through speech recognition;

[0021] The business scheduling module is used to construct input data for large model processing based on the text information and session context;

[0022] The large model module is used to process the input data based on the large model to generate response text that represents the content of customer service response, and to determine the business processing action corresponding to the response text.

[0023] The speech synthesis module is used to generate speech output based on the response text;

[0024] The service scheduling module is also used to execute the service processing action during the session in which the voice output is generated, and to receive the voice output and send it to the voice processing module so that the voice processing module can play the voice output to the user through the softphone module;

[0025] The voice processing module is also used to determine whether the session termination condition is met based on the session state. If the condition is met, the session is terminated; if not, the session is maintained and the user's input data is continuously processed.

[0026] Thirdly, some embodiments of this application also provide an electronic device, the electronic device comprising:

[0027] One or more processors; and a memory storing computer program instructions that, when executed, cause the processors to perform the large-model-based voice customer service processing method as described above.

[0028] Fourthly, some embodiments of this application also provide a computer-readable storage medium having a computer program and / or instructions stored thereon, which, when executed by a processor, implement the large-model-based voice customer service processing method as described above.

[0029] Compared with existing technologies, the solution provided in this application, through user feature-driven input diversion, a session-level dynamic voice management mechanism, and a structured input construction method for large models, enables the system to complete the differentiation of business paths and data constraints in the early stages of receiving user requests, thereby reducing invalid parsing and improving overall processing efficiency. Combined with voice activity detection and voiceprint verification, it achieves real-time extraction of valid voice segments and identity security control, ensuring the accuracy of speech recognition and semantic parsing in a dynamic interactive environment. By fusing text information with session context to form a unified structured input, it ensures that the understanding ability of the large model remains consistent with the business state, reducing the risk of misjudgment due to missing context. During model inference, response text and business processing actions are obtained synchronously, and then business execution is advanced in parallel during the voice output stage, allowing users to receive immediate business processing feedback while listening to system announcements, significantly reducing waiting time and enhancing the smoothness of interaction. Simultaneously, through continuous session state monitoring and control, it ensures the logical consistency of multi-turn interactions and the closed-loop nature of the service process. The aforementioned collaborative mechanism demonstrates significant technical advantages in reducing system construction costs, improving business access agility, reducing response latency, enhancing voice understanding capabilities, and improving the overall intelligence level of services. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other implementation methods can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is one of the flowcharts illustrating a voice customer service processing method based on a large model provided in this application embodiment;

[0032] Figure 2 This is the second flowchart of a voice customer service processing method based on a large model provided in the embodiments of this application;

[0033] Figure 3 This is a schematic diagram of the structure of a voice customer service processing system based on a large model, provided in an embodiment of this application;

[0034] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] The following terms are used in this document.

[0037] IVR (Interactive Voice Response) refers to an automatic voice interaction system based on preset voice flow and input parsing rules. It is used to collect and parse the user's key input, voice input, or routing parameters after the user connects to the communication link, and to perform preliminary traffic diversion and process guidance for the business scenario accordingly.

[0038] Automatic Speech Recognition (ASR) is a technology that uses acoustic models, language models, and feature extraction algorithms to automatically convert continuous speech signals input by a user into corresponding text sequences. Its basic process typically includes steps such as speech signal preprocessing, feature parameter extraction, acoustic modeling, and decoding search. By analyzing the time-domain, frequency-domain, and acoustic features of the speech, it achieves the recognition and transcription of speech content.

[0039] TTS (Text-to-Speech) refers to the technology that automatically converts input text content into playable speech signals using text analysis, speech synthesis models, and acoustic parameter generation algorithms. Its core process includes text normalization, prosody prediction, acoustic modeling, and vocoder generation. By learning a large number of correspondences between natural speech and text, it synthesizes speech output with naturalness and intelligibility in terms of timbre, speech rate, pauses, and stress.

[0040] LLM (Large Language Model) refers to a large-scale parameterized language model built using deep learning techniques. It typically employs neural network structures such as the Transformer and learns semantic relationships, contextual logic, and reasoning patterns through pre-training on massive amounts of text data. LLMs possess capabilities such as natural language understanding, generation, summarization, reasoning, and dialogue management, and can automatically generate task-related text output based on input content.

[0041] Some embodiments of this application relate to a voice customer service processing method based on a large model, see reference. Figure 1 As shown, the method may include the following steps:

[0042] Step S1: Receive input data from the user and perform split processing on the input data based on user-related feature information.

[0043] Input data can be the user's voice stream during a call, such as natural voice content used to inquire about order status, request after-sales processing, or provide complaint feedback; it can also include DTMF (Dual-Tone Multi-Frequency) signals entered by the user on the phone keypad, such as pressing "1" to select a service category or pressing "9" to select human assistance; furthermore, it can include call metadata carried at the time of call access, such as the user's mobile phone number, location information, call time, and whether it is a repeat call. Through any one or a combination of these types of input data, the user's current service request entry point can be obtained.

[0044] In a preferred embodiment, step S1 specifically includes:

[0045] Step S101: Obtain user-related feature information based on input data, and perform multi-source feature fusion processing on the feature information to extract underlying related features that can reflect user business needs.

[0046] After receiving user input data (such as voice input, keypad input, or call metadata), the system retrieves user-related feature information from multiple data sources. This feature information can be extracted from multiple data sources on the enterprise side, including: the user's historical business behavior records (such as recent returns, frequent complaints, and whether there are any incomplete work orders); the user's membership level attributes (such as ordinary users, VIP users, and enterprise customers); risk identification tags (such as black market risks, short-term high-frequency calls, and abnormal refund records); the user's preferred business characteristics (such as mainly handling logistics inquiries, recharge services, or accounting inquiries); and semantic features automatically extracted from historical dialogue data (such as the user's commonly used expressions and intent distribution characteristics).

[0047] The aforementioned heterogeneous features are transformed into structured data representations. Categorical features are embedded to form dense vector representations, numerical features are normalized for scale alignment, and textual features are transformed into semantic vectors using a lightweight semantic encoder, achieving operability of features from different sources within a unified feature space. After completing the feature structuring process, multi-source features are fused to extract underlying relational features of user business needs. These underlying relational features characterize the deep semantic relationship between input data and user profiles, reflecting the user's potential business needs and interaction intentions in the current business scenario. Fusion methods can include vector concatenation, feature weighted aggregation, attention-based relevance modeling, or multi-layer nonlinear mapping. For example, by constructing attention weights between features, the system can dynamically focus on the user's historical behavior or contextual state most relevant to the current business need based on the semantic intent of the input; or a lightweight deep network can be used to compress and reconstruct the concatenated feature vectors to extract user business preferences, request intent trends, and scenario relevance hidden behind multi-source data.

[0048] Step S102: Construct business routing labels to express user business needs based on underlying association features.

[0049] Intent relevance analysis is performed on the underlying associated features. By calculating the matching degree of associated features under different business categories, it is determined which business scenarios are highly relevant to the user's request. Relevance analysis can be implemented using statistical models, lightweight classification networks, or feature matching models based on attention mechanisms. For example, by inputting associated features into a multi-class business routing classifier, the probability distribution of each candidate business scenario is output, and the most likely business request of the current user is determined based on the candidate set with the highest probability or after rule filtering.

[0050] Building upon this foundation, user profile tags, risk levels, account levels, and feature tags from the front-end routing chain can be further combined to correct the initially identified business scenarios, ensuring that the business routing tags accurately reflect the user's current business processing priority. For example, if the classifier identifies the business scenario as "logistics consultation," but the user profile contains a "high-value member" tag, the business routing tag can be adjusted to "premium logistics consultation" according to a preset strategy; if a user is identified as a black market risk user, the business routing tag can be set to "risk interception process." This business routing tag can be in string, structured object, or vector encoding form, and can contain fields such as business type, user priority, risk category, and context state. The final generated business routing tag serves as a guiding identifier for subsequent processing chains and is passed to the next stage of the processing chain used to determine voice input.

[0051] Step S103: Adaptively determine the processing link for voice input based on the service routing label.

[0052] In a specific implementation, multiple selectable voice processing links are predefined, such as: standard consultation link, complaint review link, identity-sensitive business link, risk interception link, and member-exclusive link. Each link corresponds to different voice processing strategies, model call configurations, response strategies, and business scheduling methods. Upon receiving a business routing tag, the system will select a matching basic link from the predefined link set based on the business type field contained in the tag (such as "order consultation," "logistics tracking," "return application," "risk control verification," etc.).

[0053] The initially selected basic links can be further refined and the strategies adjusted based on the user level field, risk level field, and context status in the business routing label. For example, when the business routing label indicates that the user is a high-value member, a high-priority processing link can be enabled to improve the accuracy of speech recognition and the allocation of large model resources; if the business routing label includes a risk interception scenario, a speech processing link with enhanced verification and strict action restrictions will be enabled to reduce potential risky operations.

[0054] Through steps S101 to S103, the system can automatically identify user business needs based on multi-source features before the voice content enters the complex understanding and reasoning stage, and generate business routing labels that accurately indicate business categories and processing priorities. This preprocessing framework can structurally classify the business attributes, risk characteristics, and processing intentions of user input at the initial stage of data flow, thereby ensuring that subsequent business scheduling processes operate under clearly defined business scenario constraints. This mechanism not only avoids redundant computation and resource overhead caused by general processing links, but also improves the differentiated processing capabilities under different business scenarios, enabling it to execute voice customer service tasks with lower latency and higher reliability.

[0055] Step S2: Establish a session corresponding to the user, and perform identity verification and voice activity detection on the voice input in the input data to obtain valid voice segments for semantic parsing.

[0056] In a preferred embodiment, step S2 specifically includes:

[0057] Step S201: Create a session instance to identify the user interaction session, and maintain the session identifier, session state, and session context accessible across modules in the session instance.

[0058] Upon receiving input data from the user, a session instance is created to identify the current user interaction flow. This session instance can be constructed using structured data objects, such as key-value pairs that record the session identifier (SessionID), creation timestamp, and current processing status, thus uniquely representing the interaction session corresponding to the user's current call.

[0059] When a session instance is created, the session state is initialized. The initial state can include one of the predefined states such as "Session on hold," "Waiting for voice input," "Voice input in progress," or "System response in progress," and is stored through a state identifier field for subsequent session flow management. Additionally, a session context accessible across modules is maintained within the session instance. This session context can be defined using an extensible structured data format (such as a JSON object or key-value map) to record user identity information, historical dialogue content, current business processing stage, and business-level context parameters derived from business routing tags. To ensure that different processing modules (e.g., speech recognition module, large model module, business scheduling module) can access and update this context data, the session context is uniformly managed by the session instance and accessed through a shared data bus or inter-module interface.

[0060] Step S202: Perform speech activity detection on the speech input and separate active speech segments containing speech content.

[0061] After establishing a session instance, the speech stream is extracted from the input data, and Voice Activity Detection (VAD) is performed on it to identify and extract active speech segments containing valid language content. Specifically, the received speech signal is first segmented into frames, and a pre-defined voice activity detection model is invoked to analyze acoustic indicators such as energy distribution, spectral characteristics, and signal-to-noise ratio for each frame.

[0062] Based on the judgment results of the speech activity detection model, and combined with preset parameters such as the minimum speech energy threshold, the shortest speech duration, and the maximum silence duration threshold, speech frames are classified. When consecutive frames meet the speech activity conditions, they are marked as active speech segments; when insufficient energy, excessively short speech duration, or silence time exceeding the threshold is detected, the corresponding segments are identified as non-speech regions, including silent segments or background noise segments, and are filtered out.

[0063] Step S203: Extract acoustic feature vectors based on active speech segments, and compare the similarity between the acoustic feature vectors and the user's associated registered voiceprint feature vectors to complete user identity verification.

[0064] When a user's current business processing scenario involves property security or personal privacy operations, a voiceprint verification process is initiated to ensure that the voice interaction subject is consistent with the registered user's identity. From the active voice segments obtained in step S202, segments that meet the minimum duration and feature integrity requirements are selected as voiceprint comparison samples, and a pre-trained voiceprint feature extraction model is invoked to encode the voice sample to generate an acoustic feature vector representing the user's vocal characteristics.

[0065] Based on the business identifier (such as mobile phone number, user ID, or account information) contained in the input data, the registered voiceprint feature vector of the corresponding user is retrieved from the voiceprint database. To improve the robustness of the comparison, both the target feature vector and the registered feature vector are subjected to L2 normalization, and cosine similarity is used as the similarity evaluation method. Its calculation method can be expressed as:

[0066]

[0067] Here, "score" represents the similarity score. The speaker feature vector to be tested is extracted from active speech segments; This represents the registered voiceprint feature vector corresponding to the user identifier in the voiceprint database; "·" indicates the vector dot product operation.

[0068] Based on the business configuration, an adaptive threshold is used to determine the similarity score. When the score is greater than a preset threshold (e.g., 0.65), it is determined that the active voice segment and the registered voiceprint belong to the same speaker, and the voiceprint verification passes. If the score does not reach the threshold, it is considered an identity risk, and the user will be prohibited from directly entering business processes involving privacy or financial security. The user may be prompted for secondary verification or transferred to manual processing. Through the above voiceprint verification mechanism, this embodiment can achieve real-time, seamless user identity verification in the voice interaction link, ensuring the security of sensitive business operations.

[0069] Step S204: When the session is in the system response state and an active voice segment is detected, interrupt the current voice playback and switch the state of the session instance to the voice input state.

[0070] The session is currently in the "System Response" state, meaning that while the system is playing response content to the user, it continues to monitor the input audio stream. If a new active voice segment is detected during playback, it is determined that the user is making new voice input. An interrupt is immediately executed, stopping the currently playing voice output to prevent overlap between system output and user input, which could affect interactive understanding. Simultaneously, the session state field in the session instance is switched from "System Response" to "Voice Input," causing the overall processing flow to re-enter the state of receiving and parsing user voice input.

[0071] When a conversation is in a "conversation on hold" state and an active voice segment is detected, the conversation state will also be switched to "voice input" to ensure that any valid user input can be processed in a timely manner. Subsequently, the detected active voice segment is transmitted to the speech recognition module for speech-to-text processing to ensure that the user's latest voice content can enter the semantic parsing process without delay.

[0072] Through steps S201-S204, structured session instances are constructed and a session context accessible across modules is maintained, ensuring the continuity and consistency of multi-turn dialogues in semantic understanding and business processes. Accurate extraction of valid speech segments via voice activity detection enables the system to stably acquire high-quality voice input suitable for semantic parsing even in noisy environments. Combined with a voiceprint verification mechanism, reliable identity security is ensured for processing privacy-related or financial transactions, reducing the risk of identity theft. Furthermore, the current playback is interrupted and the system switches to voice input mode in real time when the user issues new voice input, allowing the user's intent to be promptly entered into the processing chain, improving the smoothness and responsiveness of the interaction. Therefore, this processing flow improves the granularity of session management, enhances the accuracy and security of voice input processing, and improves the real-time interactive experience between the system and the user.

[0073] Step S3: Generate text information from valid speech segments through speech recognition, and construct input data for large model processing based on the text information and conversation context.

[0074] In a preferred embodiment, step S3 specifically includes:

[0075] Step S301: Perform speech recognition on valid speech segments to generate text information to represent user input content;

[0076] Step S302: Determine the business data to be invoked based on the business routing label;

[0077] Step S303: Maintain the session context in a structured manner to represent user identity, historical interaction records, business status, session status and business parameters related to the current business scenario, and fuse text information, business data and session context to generate structured input data for large model processing.

[0078] The system uses a speech recognition model to decode valid speech segments, converting continuous audio signals into readable text information. This text information directly expresses the user's current intent and can include various types such as business inquiries, order queries, and operation instructions.

[0079] Business routing tags, as the initial screening result of multi-source fusion of user input features, user profile information, and historical interaction behavior, can indicate the user's current potential business processing path, such as "bill inquiry," "order processing," "account security," and "preferential benefits." After obtaining the business routing tags, a pre-defined business data routing mapping table is used to retrieve the mapping relationship between the tags and various business data sources, thereby determining the data type to be accessed and its corresponding data providing module, such as: user account data, order information, billing records, processing progress, risk control information, or business rule configuration. The system can automatically select the minimum necessary dataset based on the fine granularity of the tags to avoid performance overhead caused by loading irrelevant data.

[0080] In some implementations, when the business scenario indicated by the business routing label involves dynamic data, a real-time data query process can also be triggered to pull the latest business fields from external business systems, cache components, or databases, so as to ensure that the large model inference process can be understood and generated based on the latest and most accurate business status.

[0081] The session context is organized using structured data in JSON format, and its fields include at least: a session ID to uniquely identify the current dialogue session, user identification information to represent the user's identity, historical interaction records to record the semantic interaction content of previous rounds, business status to reflect the business process execution stage, session status to reflect changes in the session input and output stages, and business parameters related to the current business scenario (such as order number, account information, transaction amount, or user permission level).

[0082] In actual deployment, short-term session contexts are stored in high-speed memory to achieve millisecond-level access latency during multi-round voice interactions, thereby ensuring rapid retrieval of the context required for large model inference. Simultaneously, critical business states (such as operation status, verification results, or records of executed actions) and important semantic fragments related to the session are asynchronously written to a database for persistent storage in the background. This ensures that in the event of system failure, user interruption, or the need to resume the session, the complete business context can be reloaded, guaranteeing business continuity and security.

[0083] After obtaining the text information representing the user input and the business data determined by the business routing labels, the text information undergoes semantic preprocessing. Stop word filtering, time and numerical entity labeling are completed through a preset text analysis component, and key business semantic fragments are identified to form standardized text input suitable for subsequent semantic reasoning. Before writing, the retrieved business data undergoes field filtering and format standardization, retaining only valid fields related to the current intent recognition and business logic judgment.

[0084] The system reads the current session identifier, user identity information, historical interaction records, previous model output, current business status, and parsed temporary business parameters from the session context. For historical interaction records, it automatically extracts summaries of the most recent rounds of dialogue or compressed semantic representations based on the context window management strategy to reduce invalid information and control the input scale. After collecting and preprocessing text information, business data, and session context, the above information is fused according to a unified structured pattern to form a multi-source composite input object containing user input, conversation context, business status, and business parameters. The fused structured input data is submitted to the large model module as a complete semantic environment, enabling the large model to perform accurate intent reasoning, response generation, and business action selection based on the current business scenario, user status, and historical interaction content.

[0085] Through steps S301 to S303, business data associated with business routing tags is introduced immediately after speech recognition. This eliminates the need for additional data retrieval and state inference during large-scale model inference, thereby reducing ambiguous judgments and improving the accuracy of business interpretation. Simultaneously, by uniformly and structurally modeling user identity, historical interaction records, business status, and current business parameters, the system provides the large-scale model with semantically coherent, context-complete, and business process-aligned comprehensive input, significantly reducing the risk of misjudgment due to missing context or fragmented information.

[0086] Step S4: Process the input data based on the large model to generate response text that represents the content of customer service response, and determine the business processing action corresponding to the response text.

[0087] In a preferred embodiment, step S4 specifically includes:

[0088] Step S401: Use a large model to perform intent recognition on the structured input data to obtain intent labels that represent the current business scenario.

[0089] Structured input data is fed into a pre-defined large-scale model inference engine, where it is vectorized using the engine's internal semantic encoder. The large-scale model's intent recognition submodule combines the semantic features of the input text, historical interaction records from the session context, and business parameters related to the current business state to perform multi-dimensional feature fusion and deep semantic inference, thereby identifying the user's actual business needs in the current round of interaction.

[0090] The fused semantic vector is classified by an intent classification head set within the model. It selects the semantic category that best matches the current input from a predefined set of business intents (e.g., order inquiry, logistics tracking, refund application, identity verification, account change, etc.) and outputs this category as an intent label in the form of a standardized identifier. This intent label represents the user's business request category or operational intent identified by the large model in the current dialogue round and is a key semantic identifier driving business process decisions.

[0091] Step S402: Use the large model to perform business logic detection on the structured input data to determine whether the business processing actions corresponding to the current business scenario meet the preset security rules and business process rules.

[0092] The large model internally includes a security rules submodule and a business process rules submodule, used to identify potential risk factors and inconsistencies in business semantics, respectively. The security rules submodule analyzes user identity information, historical interaction records, and business parameters in the structured input data to determine if the current input exhibits abnormal behavior, such as account risk, frequent refunds, high-risk complaints, or suspected black market activities. If risk characteristics are detected, the current business action is marked as unexecutable. Simultaneously, the business process rules submodule performs process consistency checks on the input data based on the business process specifications corresponding to the intent tags, determining whether the user's current business request conforms to the business processing sequence, whether the required conditions are complete, and whether necessary identity or parameters are missing. Through this dual-dimensional logical detection, it can comprehensively determine whether the business processing action corresponding to the current business scenario meets the preset security rules and business process rules, and use the detection results as triggering conditions for subsequent response generation and business action determination.

[0093] Step S403: When the business logic detection result indicates that the business processing action is executable, the large model generates response text to represent the customer service response content.

[0094] In a preferred embodiment, step S403 specifically includes:

[0095] Step S4031: The large model retrieves business information fragments related to the structured input data from the preset domain knowledge base, and merges the business information fragments with the structured input data to generate retrieval enhancement data for enhancing response generation;

[0096] Step S4032: Perform semantic understanding and multi-turn dialogue reasoning based on the retrieved augmented data to generate response text.

[0097] When the business logic detection result in step S402 indicates that the corresponding business processing action is executable, the response generation process based on the large model is triggered. Based on the text information, business routing tags, intent features, and contextual business parameters contained in the structured input data, a preset semantic retrieval algorithm is used to filter business information fragments highly relevant to the current business scenario from the domain knowledge base. The domain knowledge base may include internal enterprise process specification documents, historical high-satisfaction customer service dialogue records, quality-checked operation guidelines, common problem knowledge sets, and external professional knowledge entries related to the target business. These business information fragments are then fused with the structured input data. Fusion methods may include appending search results to the large model input, weighted concatenation of search content, or embedding them into the model's contextual hint structure in the form of knowledge prompts. This yields enhanced search data to strengthen the subsequent response generation process, enabling the large model to reason and generate content based on more reliable factual evidence, thereby reducing the risk of model illusion and improving the accuracy of professional responses.

[0098] To enhance the professionalism and reliability of the response content, the data sources in the domain knowledge base include customer service dialogue records rated as "excellent" or "high satisfaction" by the internal quality inspection system. These records undergo automated tagging and structuring processing through the company's existing business process system, service scenario classification logic, and quality inspection evaluation system, creating a high-confidence business dataset without the need for additional manual annotation. Because this dataset originates from real-world business scenarios, its business expressions are accurate and its processes are highly consistent, providing robust business context constraints for large models. This effectively reduces data preparation costs and improves the credibility and stability of the model's responses.

[0099] The large model then leverages its multi-turn dialogue reasoning capabilities to perform correlation analysis between the current input and the conversation history, determining the specific context of the user's question, the stage of business processing, and possible subsequent operational paths. During the reasoning process, the model combines factual information from the retrieved augmented data to generate customer service response content that conforms to the business process, is semantically coherent, and is actionable. The final output is a response text representing the customer service response content, providing a basis for subsequent speech generation and business action execution.

[0100] Step S404: Select candidate service processing actions from the preset action mapping relationship based on intent tags, and match the candidate service processing actions with the service parameters in the session context to determine the service processing action corresponding to the response text.

[0101] The intent tag determines the set of actions corresponding to the intent, and this set of actions is used as candidate business processing actions. These candidate actions are further filtered and matched using business parameters from the session context. Business parameters include user identity information, historical interaction records, current business processing stage, and key business fields in the context (such as order number, product type, verification status, etc.). The feasibility, necessity, and execution order of candidate actions are determined based on the business parameters. For example, if the user has already completed identity verification, the verification action is skipped; if a valid order number exists in the session context, the order query can be executed directly; if the current business processing stage is the second step of the refund process, only actions matching that stage are allowed into the final action set.

[0102] Through steps S401 to S404, on the one hand, by performing security checks and process consistency judgments on the input data, risky operations and actions that do not conform to business rules are avoided from being triggered, thereby improving the security and reliability of the system in the execution phase. On the other hand, while generating the response text, the system matches candidate business processing actions with intent tags and session context, forming a consistent logical chain between response generation and action derivation, and achieving real-time synchronization between response content and business operations. Therefore, this embodiment can accurately identify user business intent based on structured input data and automatically determine the business scenario based on intent tags, enabling the system to maintain a continuous understanding of the business context during multi-turn dialogues.

[0103] Step S5: Generate voice output based on the response text, and perform business processing actions during the session in which the voice output is generated.

[0104] The response text is passed to a text-to-speech model, which then performs speech synthesis to generate natural speech output that matches the dialogue context. To reduce speech generation latency, the text-to-speech model can achieve millisecond-level startup time through warm start, caching commonly used phoneme sequences, and employing parallel decoding strategies, enabling the speech output to be returned to the user as quickly as possible.

[0105] Simultaneously, business processing actions corresponding to the response text are triggered concurrently during speech synthesis. For example, if the response text indicates "Checking your order status," the system will call the corresponding business interface to perform the order query operation while generating the speech output; or it may submit a refund request, change the delivery address, or trigger an SMS notification while generating an explanatory reply. To this end, the system relies on the business scheduling module to manage the business call chain, enabling speech output and business operations to be executed in parallel within the same session cycle. During the execution of business actions, the business status field in the session instance is continuously updated, allowing the large model to generate subsequent responses based on the latest business information in multi-turn interactions.

[0106] Furthermore, after generating the response text, based on the business processing action type output by the large model, the system invokes a pre-existing set of business capabilities to perform the corresponding business operations. This set of capabilities includes services such as order inquiry, logistics status inquiry, refund processing, account changes, and sending push messages, enabling interaction with the enterprise's backend system via APIs or message queues. By reusing this set of capabilities, there's no need to rebuild the logical flow for each new business scenario; instead, actions are dynamically mapped to existing capabilities, thus achieving rapid integration and unified management of business processes.

[0107] Step S6: Determine whether the session termination condition is met based on the session state. If it is met, terminate the session; otherwise, maintain the session and continue processing the user's input data.

[0108] After completing response generation and business processing actions, the session state determination phase begins to determine whether the current user interaction should end. First, the session state information maintained in the session instance is read, and combined with the intent tags identified by the large model in step S4 (such as "End Session," "Continue Query," "Enter Next Business Process," etc.), a determination is made as to whether the session meets the termination conditions. Termination conditions may include, but are not limited to: the user explicitly expressing the intention to end (such as "No more questions," "Goodbye"), the business process has been completed and there are no subsequent tasks, no new active voice segments are detected in the session within a preset time, or the user is identified as having hung up. When any termination condition is met, the session state is updated to "Ended," and resources associated with the session are released; if the termination condition is not met, the session remains in the "Connected" or "Voice Input" state, and continues to wait for new user input data before entering the next round of processing. This approach ensures the continuity of interaction while avoiding resource waste caused by unnecessary waiting.

[0109] In one embodiment, such as Figure 2 The voice customer service process shown may include the following steps:

[0110] Step A101: The user enters the system by dialing the preset service number. After receiving the incoming call request, the system plays a voice navigation menu to prompt the user to select a service.

[0111] Step A102: Receive the key selection made by the user based on the voice menu, and combine it with user profile information (such as membership level, historical business behavior, etc.) to perform routing processing on the current request in order to determine the corresponding business processing channel.

[0112] Step A103: Based on the routing results, play a guiding script that matches the current business scenario to the user; at the same time, load data related to the scenario in the background, including user business records, billing information, rights information and other contextual content, for use in subsequent interactions.

[0113] Step A104: Start receiving and recording the user's voice input, and use it as the input signal for the subsequent voice processing module.

[0114] Step A105: Create a session instance for the current user and maintain the session identifier, session state, and context data in the session instance; at the same time, use voiceprint comparison technology to verify the user's identity by verifying the user's voice to confirm the user's identity attributes.

[0115] Step A106: Call the automatic speech recognition engine to recognize and process the active speech segments input by the user, and convert the audio signal into corresponding text information.

[0116] Step A107: Summarize the text information, contextual data related to the business scenario, and information such as user profiles and business status recorded in the session instance, and construct them into structured input data for large model processing.

[0117] Step A108: The large model module processes the structured input data, including understanding user intent, determining business scenarios, checking risk conditions, and validating processes based on preset business rules to determine the scope of subsequent executable business operations.

[0118] Step A109: The large model module generates the corresponding response text based on the processing result of step A108, and determines the business processing actions to be performed, such as business queries, status changes, or other operation instructions related to the current scenario.

[0119] Step A110: Based on the response text generated in step A109, call the speech synthesis module to generate a speech output signal, and at the same time as generating the speech output, execute the business processing action determined in step A109.

[0120] Step A111: Play the voice output generated in step A110 to the user to complete the voice feedback for the current round.

[0121] Step A112: Determine whether the current service process has ended based on the session state in the session instance; if the end condition is not met, the process jumps to step A104 to continue receiving user input; if the end condition is met, the system terminates the session or performs subsequent operations such as hanging up or transferring to a human agent.

[0122] In summary, the large-model-based voice customer service processing method provided in this application enables the system to differentiate business paths and constrain data in the early stages of receiving user requests through user feature-driven input diversion, a conversation-level dynamic voice management mechanism, and a structured input construction method oriented towards the large model. This reduces invalid parsing and improves overall processing efficiency. Combined with voice activity detection and voiceprint verification, it achieves real-time extraction of valid voice segments and identity security control, ensuring the accuracy of speech recognition and semantic parsing in a dynamic interactive environment. By fusing text information with conversation context to form a unified structured input, it ensures that the large model's understanding ability remains consistent with the business state, reducing the risk of misjudgment due to missing context. During model inference, response text and business processing actions are obtained simultaneously, and then business execution is advanced in parallel during the voice output stage. This allows users to receive immediate business processing feedback while listening to system announcements, significantly reducing waiting time and enhancing the smoothness of interaction. Simultaneously, continuous conversation state monitoring and control ensure logical consistency and service process closure in multi-turn interactions. The aforementioned collaborative mechanism demonstrates significant technical advantages in reducing system construction costs, improving business access agility, reducing response latency, enhancing voice understanding capabilities, and improving the overall intelligence level of services.

[0123] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0124] Some embodiments of this application also relate to a voice customer service processing system based on a large model, see reference. Figure 3 As shown, the system includes:

[0125] The softphone module is used to receive input data from users and to perform segmentation processing on the input data based on user-related feature information;

[0126] The voice processing module is used to establish a conversation with the user and perform identity verification and voice activity detection on the voice input in the input data to obtain effective voice segments for semantic parsing.

[0127] The speech recognition module is used to generate text information from valid speech segments through speech recognition;

[0128] The business scheduling module is used to construct input data for large model processing based on text information and session context;

[0129] The large model module is used to process input data based on the large model to generate response text that represents the content of customer service responses and to determine the business processing actions corresponding to the response text.

[0130] The speech synthesis module is used to generate speech output based on the response text;

[0131] The service scheduling module is also used to perform service processing actions during the session that generates voice output, and to receive voice output and send it to the voice processing module so that the voice processing module can play the voice output to the user through the softphone module.

[0132] The voice processing module is also used to determine whether the session termination condition is met based on the session state. If the condition is met, the session is terminated; if not, the session is maintained and the user's input data is continuously processed.

[0133] Specifically, the softphone module receives input data from users, which may include voice streams, key input (DTMF signals), caller ID, user identifiers, and other information. The softphone module performs initial data routing based on user-related feature information (such as user profile, historical behavior, membership level, etc.) and can obtain user profiles by calling the service interface of the business scheduling module to execute corresponding business routing decisions, such as routing user requests to specified business processes or model processing nodes. Furthermore, the softphone module also handles session state management, processing call connection, hold, transfer, and hang-up status transitions, and is responsible for sending the final voice output stream to the user. The softphone module can integrate IVR menu guidance functionality to pre-route user requests before model processing.

[0134] The voice processing module is used for standardization, voiceprint verification, voice activity detection (VAD), and session management of voice input from the softphone module. This module extracts the voice stream from the input data and uses a VAD model to analyze acoustic indicators such as energy, spectral characteristics, and signal-to-noise ratio of the input audio to identify active voice segments containing valid language content, while filtering out inactive segments (such as silence and noise). For security scenarios involving accounts and funds, the voice processing module performs voiceprint comparison based on active voice segments, verifying user identity through 1:1 or 1:N voiceprint retrieval. When the system is in the voice playback phase and detects a new active voice segment, the voice processing module can perform interruption management, stopping the current voice playback and switching to voice input mode.

[0135] The speech recognition module receives valid speech segments from the speech processing module and converts them into text information. This module preprocesses the audio signal, extracts acoustic features, and decodes it using both acoustic and language models to generate a text sequence representing the user's semantic content. The generated text is then sent back to the speech processing module.

[0136] The business scheduling module is responsible for internal business data processing, session context management, user profile construction and updating, and business process scheduling. Specifically, after receiving text information, key input information, and basic user data from the voice processing module, the business scheduling module first aggregates user-related data sources, including the user's historical business records, membership level information, risk identification tags, behavioral preference characteristics, and past interaction logs. Based on preset feature extraction rules, it generates a user profile to characterize the user's traits. This user profile includes the user's business intent preferences, frequently handled business types, risk level, service priority, and personalized features related to the current scenario, and is dynamically updated with new input data in each round of interaction.

[0137] After constructing the user profile, the business scheduling module jointly processes text information, key input information, and the user profile, determining the business data resources required for the current business scenario based on the preceding business routing tags. Simultaneously, it maintains a structured session context, which records user identity, historical interaction records, business status, session status, business parameters, and user characteristic information obtained through the user profile construction. The business scheduling module provides unified data input to the large model module based on text information, business data, and session context, and is responsible for triggering the corresponding backend business processing actions after the large model module generates the response text.

[0138] During the execution of the business process, the business scheduling module is also used to update and manage the business process status, that is, to switch states for different stages of the business processing. This state switching can be dynamically completed based on user input, intent tags identified by the large model, and business logic detection results. This allows the business scheduling module to accurately determine the current stage of the user's business processing and load the corresponding business data, update the session context, and generate the input structure required for the next round of model inference, thereby ensuring the continuous advancement of multiple rounds of business processes during voice interaction.

[0139] Furthermore, the business scheduling module is also used to trigger backend business reuse capabilities, such as pushing messages, querying logistics, and executing order cancellations, and submits the response text to the speech synthesis module for speech generation.

[0140] The large model module, serving as the core of the system's intelligent decision-making and semantic reasoning, receives structured input data constructed by the business scheduling module and performs operations such as intent recognition, semantic understanding, knowledge retrieval, business logic detection, and multi-turn dialogue reasoning. In some embodiments, the large model module can retrieve knowledge fragments related to the current business from a preset domain knowledge base and fuse them with the structured input data to construct retrieval-enhanced data to improve the accuracy of content generation. Based on the input content and business rules, the large model module generates response text to represent customer service response content and determines the corresponding business processing actions.

[0141] In addition, the large model module is equipped with a model scheduling mechanism that can dynamically switch between general large models and lightweight domain models according to business scenario characteristics, interaction complexity and real-time requirements, thereby achieving a balance between service effect, response latency and system resource consumption.

[0142] The text-to-speech (TTS) module receives the response text output from the large model module or the business scheduling module and converts it into a speech signal. This module supports dynamic adjustment of parameters such as speech rate, timbre, and emotion to adapt to the broadcasting needs of different user groups or business scenarios. The speech stream generated by the TTS module is forwarded to the speech processing module via the business scheduling module and finally played to the user by the softphone module. To reduce generation latency, this module implements a warm-start and pre-run data loading mechanism, maintaining the text-to-speech process in millisecond-level response times. Simultaneously, it forms a parallel execution architecture with the business scheduling module; that is, after the business scheduling module receives the output from the large model module, it can simultaneously perform speech generation and business processing actions, without relying on sequential completion of business processes, thereby shortening the overall response path and avoiding long waiting times.

[0143] In one embodiment, the service scheduling module is further configured to perform service processing actions during the session in which voice output is generated, and to receive the voice stream output by the speech synthesis module and send it to the speech processing module to complete the audio playback process. The speech processing module is further configured to determine whether the current session state meets the termination condition, thereby deciding whether to terminate the session or continue processing subsequent user input.

[0144] The content of the above embodiments of the voice customer service processing method based on large models is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above embodiments of the voice customer service processing method based on large models, and the beneficial effects achieved are also the same as those achieved in the above embodiments of the voice customer service processing method based on large models. To reduce repetition, further details are omitted here.

[0145] Furthermore, some embodiments of this application also provide an electronic device. The electronic device can be various forms of digital computer, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device can also be various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0146] The electronic device includes: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform a large-model-based voice customer service processing method as provided in any one or more of the above embodiments. Figure 4 An exemplary structural diagram of the electronic device is disclosed. The electronic device includes one or more processors 1101, a memory 1102, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0147] The electronic device may further include an input device 1103 and an output device 1104. The processor 1101, memory 1102, input device 1103, and output device 1104 may be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0148] Input device 1103 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1104 may include a display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, a liquid crystal display, a light-emitting diode display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0149] To provide interaction with the user, the electronic device can be a computer. The computer has: a display device (e.g., a cathode ray tube or LCD monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback); and input from the user can be received in any form (e.g., voice input or tactile input).

[0150] In this embodiment, a computer-readable medium stores a computer program / instruction, which, when executed by a processor, implements a large-model-based voice customer service processing method provided in any one or more of the above embodiments. The computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into that device. The computer-readable medium carries one or more computer-readable instructions.

[0151] The memory 1102 can serve as a non-transitory computer-readable storage medium, used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 1102, thereby implementing the program instructions / modules corresponding to the methods provided in any one or more of the embodiments described above in this application.

[0152] The memory 1102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 1102 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1102 may optionally include memory remotely located relative to the processor 1101, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0153] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0154] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technologies, read-only optical discs, digital versatile optical discs or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0155] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including local area networks (LANs) or wide area networks (WANs), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0156] In the above embodiments, all or part of the implementation can be achieved through software, hardware, firmware, or any combination thereof. For example, it can be implemented using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the above steps or functions. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. In addition, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0157] The computer program product provided in this application includes one or more computer programs / instructions. When executed by a processor, these computer programs / instructions generate, in whole or in part, the processes or functions described in this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0158] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0159] The scope of this application is defined by the appended claims rather than the foregoing description, and is therefore intended to encompass all variations falling within the meaning and scope of equivalents of the claims. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device in software or hardware. Terms such as "first," "second," etc., are used only for distinguishing descriptions and do not indicate any particular order, nor should they be construed as indicating or implying relative importance.

[0160] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily made by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. A voice customer service processing method based on a large model, characterized in that, include: Receive input data from the user and perform split processing on the input data based on feature information related to the user; Establish a session corresponding to the user, and perform identity verification and voice activity detection on the voice input in the input data to obtain effective voice segments for semantic parsing; The effective speech segments are used to generate text information through speech recognition, and input data for large model processing is constructed based on the text information and the session context. The input data is processed based on the large model to generate response text that represents the content of customer service response, and to determine the business processing action corresponding to the response text. The system generates voice output based on the response text and performs the business processing action during the session in which the voice output is generated. The session state determines whether the session termination condition is met. If it is met, the session is terminated; otherwise, the session is maintained and user input data is continuously processed.

2. The voice customer service processing method based on a large model according to claim 1, characterized in that, The step of receiving input data from a user and performing triage processing on the input data based on feature information related to the user includes: Based on the input data, feature information related to the user is obtained, and multi-source feature fusion processing is performed on the feature information to extract underlying related features that can reflect the user's business needs. Based on the underlying association features, a business routing label is constructed to express the user's business requirements; The processing link for the voice input is adaptively determined based on the service routing label.

3. The voice customer service processing method based on a large model according to claim 1, characterized in that, The steps of establishing a session corresponding to the user and performing identity verification and voice activity detection on the voice input in the input data to obtain valid voice segments for semantic parsing include: Create a session instance to identify the user interaction session, and maintain the session identifier, session state, and session context accessible across modules in the session instance; Perform voice activity detection on the voice input to separate active voice segments containing voice content; Based on the active speech segments, acoustic feature vectors are extracted, and the acoustic feature vectors are compared with the similarity of the user's associated registered voiceprint feature vectors to complete the user identity verification. When a session is in a system response state and an active voice segment is detected, the current voice playback is interrupted, and the state of the session instance is switched to voice input state.

4. The voice customer service processing method based on a large model according to claim 2, characterized in that, The step of generating text information from the effective speech segments through speech recognition, and constructing input data for large model processing based on the text information and the session context, includes: The valid speech segments are subjected to speech recognition to generate text information representing the user's input. The required business data is determined based on the business routing label; The session context is maintained in a structured manner to represent user identity, historical interaction records, business status, session status and business parameters related to the current business scenario. The text information, the business data and the session context are then fused to generate structured input data for large model processing.

5. The voice customer service processing method based on a large model according to claim 4, characterized in that, The steps of processing the input data based on the large model to generate response text representing customer service response content and determining the business processing action corresponding to the response text include: The large model is used to perform intent recognition on the structured input data to obtain intent labels that characterize the current business scenario; The large model is used to perform business logic detection on the structured input data to determine whether the business processing actions corresponding to the current business scenario meet the preset security rules and business process rules. When the business logic detection result indicates that the business processing action is executable, the large model generates response text to characterize the customer service response content; Based on the intent tag, candidate service processing actions are selected from the preset action mapping relationship, and the candidate service processing actions are matched with the service parameters in the session context to determine the service processing action corresponding to the response text.

6. The voice customer service processing method based on a large model according to claim 5, characterized in that, The steps of generating response text to represent customer service response content using the large model include: The large model retrieves business information fragments related to the structured input data from a preset domain knowledge base, and merges the business information fragments with the structured input data to generate retrieval enhancement data for enhancing response generation; Based on the retrieved enhanced data, semantic understanding and multi-turn dialogue reasoning are performed to generate response text.

7. The voice customer service processing method based on a large model according to claim 5, characterized in that, Also includes: The current business scenario is encoded based on the business routing label and the intent label to generate a business scenario identifier; The session context is embedded based on the business scenario identifier, and the session state is controlled to transition between different business processing stages.

8. A voice customer service processing system based on a large model, characterized in that, include: The softphone module is used to receive input data from the user and perform stream processing on the input data based on feature information related to the user; The voice processing module is used to establish a conversation corresponding to the user and perform identity verification and voice activity detection on the voice input in the input data to obtain effective voice segments for semantic parsing. The speech recognition module is used to generate text information from the effective speech segments through speech recognition; The business scheduling module is used to construct input data for large model processing based on the text information and session context; The large model module is used to process the input data based on the large model to generate response text that represents the content of customer service response, and to determine the business processing action corresponding to the response text. The speech synthesis module is used to generate speech output based on the response text; The service scheduling module is also used to execute the service processing action during the session in which the voice output is generated, and to receive the voice output and send it to the voice processing module so that the voice processing module can play the voice output to the user through the softphone module; The voice processing module is also used to determine whether the session termination condition is met based on the session state. If the condition is met, the session is terminated; if not, the session is maintained and the user's input data is continuously processed.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; and a memory storing computer program instructions, which, when executed, cause the processors to perform the large-model-based voice customer service processing method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program and / or instructions stored thereon, characterized in that, When the computer program and / or instructions are executed by the processor, they implement the large-model-based voice customer service processing method as described in any one of claims 1-7.

Images