Intelligent customer service voice interaction method and system based on large model cascade architecture

Abstract

The present application relates to the technical field of voice interaction, in particular to a kind of intelligent customer service voice interaction method and system based on large model cascade architecture.It includes receiving user voice input and converting into target problem text, synchronous output token recognition confidence sequence;Through lightweight field adaptation model, structured semantic analysis is executed, and structured semantic representation including intent, business object, constraint condition, interaction history context and the independent confidence of each semantic element is output, and decision chain is recorded;Conduct confidence-driven dynamic routing decision, and according to fine-grained confidence, request is distributed to symbolic rule processing path, clarification process or large model inference path;When entering clarification process, low-confidence constraint condition is attributed to analysis, reason category is determined and differentiated clarification strategy is selected, and user answer is fused and then routed again;After generating response text, it is output by voice synthesis.

Description

Technical Field

[0001] This invention relates to the field of voice interaction technology, and more specifically, to an intelligent customer service voice interaction method and system based on a large model cascade architecture. Background Technology

[0002] Traditional intelligent customer service voice interaction systems generally adopt a cascaded architecture of VAD-ASR-NLP-TTS, where each module is trained independently and executed serially, using text as a relay baton to transmit information. This architecture suffers from drawbacks such as high latency, poor user experience due to interruptions, loss of paralinguistic information, service fragmentation, and low recognition rate in noisy environments. Existing technologies attempt to improve performance by modeling each module into larger models, but routing decisions are mostly based on global confidence, making it difficult to distinguish scenarios where the intent is clear but local constraints are ambiguous; clarification strategies are usually selected only based on confidence intervals, without attribution analysis of the causes of low confidence; and there is a lack of feedback fusion and rerouting loop after clarification, resulting in limited service accuracy and user experience.

[0003] However, existing technologies have the following drawbacks:

[0004] Existing solutions only output overall confidence scores and cannot identify local uncertainties in specific semantic elements (such as constraints). When the intent is clear but local constraints are ambiguous, the system can only either scale up to a large model or fail outright, increasing both the cost of invocation and the user experience.

[0005] Existing clarification mechanisms are mostly triggered based on confidence level ranges, without distinguishing the specific reasons for low confidence (speech recognition errors, semantic ambiguity, incomplete expression, or unclear intent). The clarification questions are not targeted enough, resulting in more interaction rounds and decreased user patience.

[0006] Existing solutions fail to integrate user responses to clarification with the original semantic representation before re-performing routing decisions, thus failing to progressively eliminate uncertainty. Furthermore, the lack of structured contextual communication between lightweight models, symbolic engines, and large models makes it difficult to balance interpretability, accuracy, and complex reasoning capabilities.

[0007] Therefore, an intelligent customer service voice interaction method and system based on a large model cascade architecture is designed. Summary of the Invention

[0008] The purpose of this invention is to provide an intelligent customer service voice interaction method and system based on a large model cascade architecture, so as to solve the problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention aims to provide an intelligent customer service voice interaction method based on a large model cascade architecture, comprising: step S1, receiving user voice input, converting the voice input into target question text through automatic speech recognition, and synchronously outputting the recognition confidence sequence of each token in the target question text.

[0010] The automatic speech recognition described in step S1 adopts a streaming output method, and the incremental inference of the lightweight domain adaptation model in step S2 is started when a partial transcription result is generated.

[0011] Step S2: Input the target question text into the lightweight domain adaptation model, perform structured semantic parsing, and output a structured semantic representation. The structured semantic representation includes the following semantic elements: intent classification, business object, constraints, interaction history context, and a confidence evaluation value calculated independently for each of the above semantic elements. At the same time, the decision chain in the parsing process is recorded.

[0012] The record structure of the decision chain in step S2 is as follows: each decision chain consists of several decision steps, and each decision step records the decision content of the step, the input features or intermediate results that trigger the decision, the natural language reason for the decision, and the confidence level of the step.

[0013] The decision chain is persistently stored in the database. When the output of the lightweight domain adaptation model needs further reasoning, the large language model can query the decision chain to locate the failed step and perform targeted reasoning.

[0014] The decision chain is also used for offline analysis to statistically analyze common failure modes of lightweight adaptation models, providing data support for model fine-tuning.

[0015] The lightweight domain adaptation model satisfies the following characteristics:

[0016] The total number of parameters ranges from 100M to 500M, and can be deployed on edge servers or local computing environments.

[0017] The training data comes from 50,000 to 100,000 labeled dialogues in a specific customer service domain, including intent tags and slot tags.

[0018] The model output includes intent classification results, constraint extraction results, and confidence scores calculated independently for each decision element. These confidence scores are obtained through softmax output scores, prediction entropy values, or calibrated confidence scores.

[0019] The model also generates a decision chain record, which contains the input features, intermediate results, output decisions and the rules or patterns triggered by each step in the parsing process.

[0020] Step S3, Confidence-driven dynamic routing decision: Compare the independent confidence assessment value of each semantic element with multiple preset confidence thresholds, and determine the processing path corresponding to the current level of understanding based on the comparison results.

[0021] Step S4: Execute the corresponding processing path based on the decision result of step S3, and generate a response text.

[0022] When the confidence level of the intent is higher than the first threshold and the confidence levels of all key constraints are higher than the second threshold, the symbol rule processing path is entered.

[0023] When the confidence level of the intent is higher than the first threshold, but the confidence level of at least one key constraint is between the clarification threshold and the second threshold, the clarification process is initiated.

[0024] When the confidence level of intent is lower than the first threshold, or when the confidence level of key constraints is lower than the clarification threshold, the system enters the large model inference path.

[0025] Step S5: Based on the independent confidence of each semantic element obtained in step S2, prioritize and adaptively organize the response content, prioritize the presentation of content related to high confidence constraints, and adjust the level of detail of the response according to the complexity of the intent.

[0026] Step S6: The response text is converted into a speech signal and output through the speech synthesis module; wherein, the speech synthesis module starts speech synthesis and plays the response text as soon as it receives the first semantically complete segment of the response text, rather than waiting for the complete response text to be generated.

[0027] Step S6 is followed by a cascading consistency check step:

[0028] The factual assertions in the response text are compared one by one with the search results generated by the search enhancement module.

[0029] If there are factual assertions that are not supported by search results, the verification is deemed to fail.

[0030] When the verification fails, a degradation strategy is executed, which includes: returning the processing result of the lightweight domain adaptation model or symbol engine, generating clarification scripts, or transferring to human customer service.

[0031] Step S7: Collect explicit and implicit user feedback on this interaction, analyze the failure cases of the lightweight domain adaptation model in step S2, dynamically optimize the confidence threshold accordingly, and expand the intent definition and symbol rule library.

[0032] The dynamic optimization of the confidence threshold described in step S7 specifically includes:

[0033] Statistics on the clarification accuracy and symbol engine failure rate for each constraint type.

[0034] The values ​​of the first threshold, second threshold, and clarification threshold are automatically adjusted based on statistical results to achieve a dynamic balance between accuracy and user experience.

[0035] When a user has already made multiple clarifications, the threshold for subsequent clarifications is increased, and requests are prioritized for processing in larger models to avoid excessive clarification rounds that could negatively impact the user experience.

[0036] As a further improvement to this technical solution, in step S2, the confidence evaluation value of each semantic element is calculated independently, specifically including: for intent classification, the confidence value of the intent is calculated by the softmax output of the intent classification head and the highest score is taken as the confidence value of the intent.

[0037] For business objects and constraints, sequence labeling is used to calculate the confidence level of the corresponding element by averaging the labeled probability sequence. If the same constraint is extracted by multiple independent recognizers, the weighted average of the scores of each recognizer is taken.

[0038] For the token recognition confidence sequence transmitted by speech recognition, when a certain semantic element spans multiple low-confidence tokens, the confidence of that semantic element decreases accordingly.

[0039] Simultaneously, the overall reliability score of the lightweight domain adaptation model is calculated, and the overall reliability score is obtained by aggregating the average, minimum or weighted comprehensive values ​​of multidimensional confidence.

[0040] As a further improvement to this technical solution, in the structured semantic representation, key constraints are pre-configured according to the business type and risk level to which the constraints belong; when the absence or error of a certain constraint will cause the business operation to fail or generate business risks, the constraint is marked as a key constraint.

[0041] As a further improvement to this technical solution, the symbol rule processing path is specifically as follows: based on the intent and constraints extracted in step S2, query the predefined intent-business system mapping table to obtain the corresponding database query template.

[0042] Perform normalization and data validation on constraints; execute database queries or call business system interfaces to obtain query results.

[0043] If the query is successful and the result is not empty, proceed to step S5; if the query fails or the result is empty, transfer the task to the large model for processing, and pass the reason for the query failure as context.

[0044] The normalization of constraints described in the symbol rule processing path includes:

[0045] Standardize the time constraints of natural language and convert fuzzy time expressions into specific date ranges.

[0046] Value mapping is performed on enumeration class constraints to map user verbal expressions to business system code.

[0047] Perform a range check on the numerical constraints.

[0048] Standardize the matching of location and institution constraints, and handle synonyms.

[0049] All anomalies or unnormalizable values ​​found during the normalization process are marked as low confidence, triggering a clarification process or transitioning to the large model inference path.

[0050] The integration method between the symbol decision rule engine and the backend business system in the symbol rule processing path is as follows:

[0051] Define standardized interface specifications, and each business system implements functional contracts including query interfaces, data verification interfaces, and intent template query interfaces.

[0052] Perform permission verification before executing the query, and de-identify or encrypt sensitive fields.

[0053] Cache frequently used query results for a short period to reduce the pressure on the backend system.

[0054] When a business system is temporarily unavailable, explain the fault to the user and suggest retrying later, or transfer the request to a large language model to generate temporary suggestions based on historical or general knowledge.

[0055] As a further improvement to this technical solution, the clarification process specifically includes: identifying key constraints with insufficient confidence but not completely failed; generating targeted clarification questions based on the specific type of the constraint, the user's interaction history, and the current business context, wherein the clarification questions include a natural language expression of the constraint and / or a list of candidate options; converting the clarification questions into speech output; receiving the user's answer to the clarification questions, converting the answer into structured information, and fusing it with the original structured semantic representation obtained in step S2 to update the confidence of the affected constraints, wherein the fused confidence is the larger value between the original confidence and the confidence of the clarification answer; and returning to step S3 to re-execute the routing decision after the fusion is completed.

[0056] As a further improvement to this technical solution, the following sub-steps are performed when entering the clarification process:

[0057] Attribution Analysis: Attribution analysis is performed on key constraints with confidence levels below the second threshold to determine the category of reasons for the low confidence level of the key constraint. The category of reasons includes at least: speech recognition error, semantic ambiguity, incomplete expression, and unclear intent. The input to the attribution analysis includes: the confidence value of the constraint, the token recognition confidence sequence corresponding to the constraint, the parsing steps of the constraint in the decision chain, and the previous state of the constraint in the dialogue history.

[0058] The attribution analysis is implemented using a lightweight attribution classifier, which employs a multilayer perceptron structure or a rule-based decision tree structure. Its input features include: the confidence value of the constraint, the minimum / mean / variance of the token recognition confidence value corresponding to the constraint, the number of semantic parsing rules that hit the constraint in the decision chain, and the number of times the constraint was clarified in the dialogue history. The output of the attribution classifier is the cause category label.

[0059] The attribution classifier employs a three-layer multilayer perceptron, with an input layer dimension of 8 (corresponding to 8 input features), hidden layers of 32 and 16 dimensions, and an output layer dimension of 4 (corresponding to 4 cause categories). The activation function is ReLU, and the output layer uses softmax. Training data consists of 5000 manually labeled low-confidence samples from historical customer service conversations. In the rule engine, when multiple rules are applied simultaneously, the priority order is: speech recognition error > semantic ambiguity > incomplete expression > unclear intent.

[0060] Differentiated clarification strategy selection: Select the corresponding clarification strategy based on the cause category output by the attribution analysis: When the cause category is "speech recognition error", generate a clarification question requesting the user to repeat the statement.

[0061] When the cause category is "semantic ambiguity", a clarifying question is generated that provides candidate options.

[0062] When the reason category is "incomplete expression", generate a clarification question to guide the user to supplement the information.

[0063] When the cause category is "ambiguous intent", no clarification question is generated, and the process directly proceeds to the large model inference path.

[0064] The differentiated clarification strategy mentioned in the step-by-step differentiated clarification strategy selection further includes:

[0065] For the "semantic ambiguity" cause category, the generated clarification question contains at least two candidate options, which are determined based on multiple candidate values ​​of the constraint in the structured semantic representation and their confidence ranking.

[0066] For the "incomplete expression" reason category, the generated clarification question includes guiding prompts, which are determined based on the mandatory fields of the constraint in the business template and the information obtained from the dialogue history.

[0067] For the "speech recognition error" cause category, the generated clarification question only requires the user to repeat the speech segment corresponding to the constraint, rather than requiring the user to restate the entire question.

[0068] Clarification Feedback Fusion: The generated clarification question is converted into voice output; the user's answer to the clarification question is received, the answer is converted into structured information and fused with the original structured semantic representation obtained in step S2, the confidence of the affected constraints is updated, and the confidence of the fused confidence is the larger value between the original confidence and the confidence of the clarification answer; after the fusion is completed, the process returns to step S3 to re-execute the routing decision.

[0069] The step-by-step clarification feedback fusion, which involves fusing user responses with the original structured semantic representation, further includes:

[0070] Convert users' answers to clarification questions into structured information.

[0071] Locate the position of the constraint corresponding to the answer in the original structured semantic representation.

[0072] Update the value and confidence of the constraint with the merged confidence.

[0073] Determine whether there are other key constraints with confidence levels below the second threshold in the fused structured semantic representation. If they exist, return to attribution analysis and continue performing attribution analysis and differential clarification. If they do not exist, re-execute the routing decision.

[0074] As a further improvement to this technical solution, the attribution analysis is based on rules, specifically including: when there are multiple consecutive tokens with confidence scores lower than the token threshold in the token recognition confidence sequence corresponding to the constraint condition, the cause category is determined to be "speech recognition error".

[0075] When the parsing step of the constraint in the decision chain hits multiple candidate semantic patterns and the confidence difference is less than the difference threshold, the cause category is determined to be "semantic ambiguity".

[0076] When the constraint is missing or only partially filled in the structured semantic representation, and the user has not provided complete information in the dialogue history, the reason category is "incomplete expression".

[0077] When the confidence level of the intent is lower than the intent ambiguity threshold, and the low confidence level of the constraint is associated with intent uncertainty, the cause category is determined to be "intent ambiguity".

[0078] As a further improvement to this technical solution, the large model inference path specifically includes: constructing an input context package for the large language model, which includes: the original target question text, the complete structured semantic representation of step S2, the independent confidence decomposition of each semantic element, the decision chain record, and the symbol engine query results and failure reasons; calling the large language model to perform inference and generate the response text.

[0079] The input context package of the large language model in the large model inference path is a structured data package containing the following fields: the original user input text, the identifier of each semantic element in the structured semantic representation and its confidence decomposition value, each step record of the decision chain, the query attempt status and failure reasons of the symbol engine, and the attribution analysis results of the step attribution analysis (if they exist).

[0080] The large language model performs inference based on the structured context package, rather than directly processing the raw text, so that its output focuses on understanding complex needs, explaining the reasons for failure, handling follow-up questions, and generating personalized suggestions.

[0081] The large language model inference path described in the large model inference path further includes a dynamic model activation step:

[0082] During inference, when the correlation between the current inference state and the intermediate semantic representation of a candidate model not included in the inference path exceeds a preset backtracking threshold, the current inference path is paused.

[0083] The candidate model is activated and added to the inference path by using the currently generated context information as a condition.

[0084] The remaining inference steps are completed by the updated inference path in a parallel or alternating manner.

[0085] As a further improvement to this technical solution, a multi-round dialogue context maintenance step is also included: after each round of interaction is completed, the intent, business object and key constraints of this round are recorded in the interaction history context.

[0086] In subsequent rounds, the lightweight adaptation model automatically reuses the slot values ​​extracted from previous rounds, unless the user explicitly requests new values.

[0087] Calculate the continuity score between the current round intent and the previous round intent. When the score is higher than a set threshold, inherit the previous confidence and query results. When the score is lower than a set threshold, clear the relevant inherited slot values.

[0088] Detect logical contradictions in multi-turn dialogues; if any are found, trigger a clarification process or mark the scenario as a complex one requiring the intervention of a large language model.

[0089] The continuity score is calculated using any one or a combination of the following methods:

[0090] A direct match of the intent identifier returns 1.0 if they are the same, and 0.0 if they are completely different.

[0091] The similarity between different intents is calculated using a predefined intent relationship graph.

[0092] The intent is vectorized using a semantic similarity model, and cosine similarity is calculated.

[0093] A differentiated strategy is implemented based on the continuity score: when the score is higher than the first continuity threshold, the previous slot value is automatically inherited; when the score is between the first and second continuity thresholds, the inherited value is marked as low trust and the user is prompted for confirmation; when the score is lower than the second continuity threshold, it is determined to be an intent conversion and the relevant slot value is cleared.

[0094] In a second aspect, the present invention provides an intelligent customer service voice interaction system based on a large model cascade architecture, comprising: a voice recognition module, used to receive user voice input, convert the voice input into target question text through automatic voice recognition, and synchronously output the recognition confidence sequence of each token in the target question text.

[0095] The structured semantic parsing module is used to input the target question text into the lightweight domain adaptation model, perform structured semantic parsing, and output a structured semantic representation. The structured semantic representation includes the following semantic elements: intent classification, business object, constraints, interaction history context, and a confidence evaluation value calculated independently for each of the above semantic elements, while recording the decision chain in the parsing process.

[0096] The dynamic routing decision module compares the independent confidence assessment value of each semantic element with multiple preset confidence thresholds, and determines the processing path corresponding to the current level of understanding based on the comparison results.

[0097] The processing path execution module is used to execute the corresponding processing path according to the decision result of the dynamic routing decision module and generate a response text.

[0098] The response organization module is used to prioritize and adaptively organize the response content based on the independent confidence of each semantic element obtained by the structured semantic parsing module, prioritizing the presentation of content related to high confidence constraints, and adjusting the level of detail of the response according to the complexity of the intent.

[0099] The speech synthesis module is used to convert response text into speech signals and output them.

[0100] The learning optimization module is used to collect explicit and implicit user feedback on this interaction, analyze the failure cases of the lightweight domain adaptation model, dynamically optimize the confidence threshold accordingly, and expand the intent definition and symbol rule library.

[0101] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0102] This invention independently calculates the confidence score for each semantic element, such as intent, business object, and constraints, to accurately pinpoint the source of uncertainty. Based on fine-grained confidence scores, it performs dynamic routing through three paths: symbol rules, clarification, and large model, significantly reducing the cost of calling the large model while maintaining accuracy.

[0103] This invention uses attribution analysis to determine the specific reasons for low confidence (speech recognition error, semantic ambiguity, incomplete expression or unclear intent), and generates targeted clarification questions such as requesting repetition, providing options, and guiding supplementation, thereby reducing invalid interaction rounds and improving user experience.

[0104] In this invention, user responses to clarification are integrated into the original semantic representation, and the confidence level is updated. Routing decisions are then re-executed, gradually eliminating local uncertainties. A lightweight model outputs a structured semantic representation, a symbolic engine executes deterministic business logic, and a large model handles complex reasoning. The three work in a layered and collaborative manner, achieving interpretability, traceability, and efficiency. Attached Figure Description

[0105] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0106] Figure 1 This is a schematic diagram of the implementation steps of the method of the present invention.

[0107] Figure 2 This is a schematic diagram of the system structure connection of the present invention. Detailed Implementation

[0108] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0109] Example: Please refer to Figure 1As shown, an intelligent customer service voice interaction method based on a large model cascade architecture is provided, including: step S1, receiving user voice input, converting the voice input into target question text through automatic speech recognition, and synchronously outputting the recognition confidence sequence of each token in the target question text.

[0110] The automatic speech recognition described in step S1 adopts a streaming output method, and the incremental inference of the lightweight domain adaptation model in step S2 is started when a partial transcription result is generated.

[0111] Step S2: Input the target question text into the lightweight domain adaptation model, perform structured semantic parsing, and output a structured semantic representation. The structured semantic representation includes the following semantic elements: intent classification, business object, constraints, interaction history context, and a confidence evaluation value calculated independently for each of the above semantic elements. At the same time, the decision chain in the parsing process is recorded.

[0112] In one specific embodiment, each semantic element in step S2 independently calculates a confidence evaluation value, specifically including: for intent classification, the confidence value is calculated by using the softmax output of the intent classification head, and the highest score is taken as the confidence value of the intent.

[0113] For business objects and constraints, sequence labeling is used to calculate the confidence level of the corresponding element by averaging the labeled probability sequence. If the same constraint is extracted by multiple independent recognizers, the weighted average of the scores of each recognizer is taken.

[0114] For the token recognition confidence sequence transmitted by speech recognition, when a certain semantic element spans multiple low-confidence tokens, the confidence of that semantic element decreases accordingly.

[0115] Simultaneously, the overall reliability score of the lightweight domain adaptation model is calculated, and the overall reliability score is obtained by aggregating the average, minimum or weighted comprehensive values ​​of multidimensional confidence.

[0116] In one specific embodiment, in the structured semantic representation, key constraints are pre-configured according to the business type and risk level to which the constraints belong; when the absence or error of a certain constraint will cause the business operation to fail or generate business risks, the constraint is marked as a key constraint.

[0117] The pre-configuration of key constraints is achieved through a predefined "business type - constraint - risk level" mapping table: For each customer service business type (such as insurance claims, financial transfers, telecommunications package applications, etc.), business experts or system administrators pre-list all constraints involved in the business and mark each constraint with a risk level (such as high, medium, low). When the risk level of a constraint reaches the preset high-risk threshold, or when the business rules determine that its absence or error will directly lead to the inability to execute the business operation, financial loss, compliance violations, or security incidents, the system marks the constraint as a key constraint. This configuration can be pre-set based on regulatory requirements, historical failure case statistics, and business expert experience, and supports dynamic updates during system operation based on the launch of new businesses or adjustments to risk strategies.

[0118] Specific example: Taking the "car insurance claims" business type as an example, the pre-configured constraints and risk levels are as follows: Policy number (high risk, missing it makes it impossible to locate the policy, marked as a key constraint), accident time (high risk, missing it makes it impossible to determine whether it is within the insurance period, marked as a key constraint), accident location (high risk, missing it affects damage assessment and rescue dispatch, marked as a key constraint), accident type (medium risk, missing it can be supplemented later, marked as a key constraint), vehicle information (medium risk, missing it can be transferred to manual verification, not marked as a key constraint), driver information (medium risk, missing it can be supplemented later, not marked as a key constraint), vehicle color (low risk, does not affect the claims process, not marked as a key constraint). When a user voice inputs "I want to report a case, I was rear-ended on the highway yesterday afternoon," the structured semantic parsing module extracts the intent as "car insurance claims," ​​where the confidence level for the accident time "yesterday afternoon" is 0.78, the confidence level for the accident location "highway" is 0.62, and the confidence level for the accident type "rear-end collision" is 0.91. Since the accident time and location are pre-configured as key constraints, and the confidence level of the accident location (0.62) is lower than the second threshold but higher than the clarification threshold, the dynamic routing decision module determines to enter the clarification process. The attribution analysis module further determines that the reason for the low confidence level of the accident location is "semantic ambiguity" ("highway" may refer to an unclear specific highway name). The differentiated clarification strategy selection module generates clarification questions that provide candidate options (such as "Was the rear-end collision on the G4 Beijing-Hong Kong-Macau Expressway or the G45 Daqing-Guangzhou Expressway?"), thereby accurately eliminating the uncertainty of the key constraints, rather than requiring the user to re-describe the entire accident.

[0119] The record structure of the decision chain in step S2 is as follows: each decision chain consists of several decision steps, and each decision step records the decision content of the step, the input features or intermediate results that trigger the decision, the natural language reason for the decision, and the confidence level of the step.

[0120] The decision chain is persistently stored in the database. When the output of the lightweight domain adaptation model needs further reasoning, the large language model can query the decision chain to locate the failed step and perform targeted reasoning.

[0121] The decision chain is also used for offline analysis to statistically analyze common failure modes of lightweight adaptation models, providing data support for model fine-tuning.

[0122] The lightweight domain adaptation model satisfies the following characteristics:

[0123] The total number of parameters ranges from 100M to 500M, and can be deployed on edge servers or local computing environments.

[0124] The training data comes from 50,000 to 100,000 labeled dialogues in a specific customer service domain, including intent tags and slot tags.

[0125] The model output includes intent classification results, constraint extraction results, and confidence scores calculated independently for each decision element. These confidence scores are obtained through softmax output scores, prediction entropy values, or calibrated confidence scores.

[0126] The model also generates a decision chain record, which contains the input features, intermediate results, output decisions and the rules or patterns triggered by each step in the parsing process.

[0127] The lightweight domain-adapted model uses a BERT-based architecture as its foundation, comprising a 12-layer Transformer encoder with 12 attention heads per layer, a hidden layer dimension of 768, and a total of approximately 110M parameters. At the top of the model are an intent classification head and a sequence labeling head: the intent classification head uses a fully connected layer with softmax, and the sequence labeling head uses a BiLSTM-CRF structure. The training data consists of 80,000 labeled dialogues from the insurance claims domain, using the Adam optimizer with a learning rate of 2e-5, a batch size of 32, and training for 10 epochs. Confidence scores are obtained through softmax output and calibrated using temperature scaling. The model is compressed from a larger teacher model through knowledge distillation.

[0128] Step S3, Confidence-driven dynamic routing decision: Compare the independent confidence assessment value of each semantic element with multiple preset confidence thresholds, and determine the processing path corresponding to the current level of understanding based on the comparison results.

[0129] Step S4: Execute the corresponding processing path based on the decision result of step S3, and generate a response text.

[0130] When the confidence level of the intent is higher than the first threshold and the confidence levels of all key constraints are higher than the second threshold, the symbol rule processing path is entered.

[0131] In one specific embodiment, the symbol rule processing path is as follows: based on the intent and constraints extracted in step S2, query the predefined intent-business system mapping table to obtain the corresponding database query template.

[0132] Perform normalization and data validation on constraints; execute database queries or call business system interfaces to obtain query results.

[0133] If the query is successful and the result is not empty, proceed to step S5; if the query fails or the result is empty, transfer the task to the large model for processing, and pass the reason for the query failure as context.

[0134] The normalization of constraints described in the symbol rule processing path includes:

[0135] Standardize the time constraints of natural language and convert fuzzy time expressions into specific date ranges.

[0136] Value mapping is performed on enumeration class constraints to map user verbal expressions to business system code.

[0137] Perform a range check on the numerical constraints.

[0138] Standardize the matching of location and institution constraints, and handle synonyms.

[0139] All anomalies or unnormalizable values ​​found during the normalization process are marked as low confidence, triggering a clarification process or transitioning to the large model inference path.

[0140] The integration method between the symbol decision rule engine and the backend business system in the symbol rule processing path is as follows:

[0141] Define standardized interface specifications, and each business system implements functional contracts including query interfaces, data verification interfaces, and intent template query interfaces.

[0142] Perform permission verification before executing the query, and de-identify or encrypt sensitive fields.

[0143] Cache frequently used query results for a short period to reduce the pressure on the backend system.

[0144] When a business system is temporarily unavailable, explain the fault to the user and suggest retrying later, or transfer the request to a large language model to generate temporary suggestions based on historical or general knowledge.

[0145] When the confidence level of the intent is higher than the first threshold, but the confidence level of at least one key constraint is between the clarification threshold and the second threshold, the clarification process is initiated.

[0146] In one specific embodiment, the clarification process includes: identifying key constraints with insufficient confidence but not completely failed; generating targeted clarification questions based on the specific type of the constraint, the user's interaction history, and the current business context, wherein the clarification questions include a natural language description of the constraint and / or a list of candidate options; converting the clarification questions into speech output; receiving the user's answer to the clarification questions, converting the answer into structured information, and fusing it with the original structured semantic representation obtained in step S2 to update the confidence of the affected constraints, wherein the fused confidence is the larger value between the original confidence and the confidence of the clarification answer; and returning to step S3 to re-execute the routing decision after the fusion is completed.

[0147] In one specific embodiment, when entering the clarification process, the following sub-steps are performed: Attribution analysis: Attribution analysis is performed on key constraints with confidence levels below a second threshold to determine the category of reasons for the low confidence level of the key constraint. The category of reasons includes at least: speech recognition error, semantic ambiguity, incomplete expression, and unclear intent. The inputs to the attribution analysis include: the confidence value of the constraint, the token recognition confidence sequence corresponding to the constraint, the parsing step record of the constraint in the decision chain, and the previous state of the constraint in the dialogue history.

[0148] The attribution analysis is implemented using a lightweight attribution classifier, which employs a multilayer perceptron structure or a rule-based decision tree structure. Its input features include: the confidence value of the constraint, the minimum / mean / variance of the token recognition confidence value corresponding to the constraint, the number of semantic parsing rules that hit the constraint in the decision chain, and the number of times the constraint was clarified in the dialogue history. The output of the attribution classifier is the cause category label.

[0149] Differentiated clarification strategy selection: Select the corresponding clarification strategy based on the cause category output by the attribution analysis: When the cause category is "speech recognition error", generate a clarification question requesting the user to repeat the statement.

[0150] When the cause category is "semantic ambiguity", a clarifying question is generated that provides candidate options.

[0151] When the reason category is "incomplete expression", generate a clarification question to guide the user to supplement the information.

[0152] When the cause category is "ambiguous intent", no clarification question is generated, and the process directly proceeds to the large model inference path.

[0153] The differentiated clarification strategy mentioned in the step-by-step differentiated clarification strategy selection further includes:

[0154] For the "semantic ambiguity" cause category, the generated clarification question contains at least two candidate options, which are determined based on multiple candidate values ​​of the constraint in the structured semantic representation and their confidence ranking.

[0155] For the "incomplete expression" reason category, the generated clarification question includes guiding prompts, which are determined based on the mandatory fields of the constraint in the business template and the information obtained from the dialogue history.

[0156] For the "speech recognition error" cause category, the generated clarification question only requires the user to repeat the speech segment corresponding to the constraint, rather than requiring the user to restate the entire question.

[0157] Clarification Feedback Fusion: The generated clarification question is converted into voice output; the user's answer to the clarification question is received, the answer is converted into structured information and fused with the original structured semantic representation obtained in step S2, the confidence of the affected constraints is updated, and the confidence of the fused confidence is the larger value between the original confidence and the confidence of the clarification answer; after the fusion is completed, the process returns to step S3 to re-execute the routing decision.

[0158] The step-by-step clarification feedback fusion, which involves fusing user responses with the original structured semantic representation, further includes:

[0159] Convert users' answers to clarification questions into structured information.

[0160] Locate the position of the constraint corresponding to the answer in the original structured semantic representation.

[0161] Update the value and confidence of the constraint with the merged confidence.

[0162] Determine whether there are other key constraints with confidence levels below the second threshold in the fused structured semantic representation. If they exist, return to attribution analysis and continue performing attribution analysis and differential clarification. If they do not exist, re-execute the routing decision.

[0163] In one specific embodiment, the attribution analysis is based on rules, and the specific rules include: when there are multiple consecutive tokens with confidence scores lower than the token threshold in the token recognition confidence sequence corresponding to the constraint condition, the cause category is determined to be "speech recognition error".

[0164] When the parsing step of the constraint in the decision chain hits multiple candidate semantic patterns and the confidence difference is less than the difference threshold, the cause category is determined to be "semantic ambiguity".

[0165] When the constraint is missing or only partially filled in the structured semantic representation, and the user has not provided complete information in the dialogue history, the reason category is "incomplete expression".

[0166] When the confidence level of the intent is lower than the intent ambiguity threshold, and the low confidence level of the constraint is associated with intent uncertainty, the cause category is determined to be "intent ambiguity".

[0167] When the confidence level of intent is lower than the first threshold, or the confidence level of key constraints is lower than the clarification threshold, the system enters the large model inference path.

[0168] In one specific embodiment, the large model inference path specifically includes: constructing an input context package for a large language model, the input context package including: the original target question text, the complete structured semantic representation of step S2, the independent confidence decomposition of each semantic element, the decision chain record, and the symbol engine query results and failure reasons; calling the large language model to perform inference and generate response text.

[0169] The input context package of the large language model in the large model inference path is a structured data package containing the following fields: the original user input text, the identifier of each semantic element in the structured semantic representation and its confidence decomposition value, each step record of the decision chain, the query attempt status and failure reasons of the symbol engine, and the attribution analysis results of the step attribution analysis (if they exist).

[0170] The large language model performs inference based on the structured context package, rather than directly processing the raw text, so that its output focuses on understanding complex needs, explaining the reasons for failure, handling follow-up questions, and generating personalized suggestions.

[0171] The large language model inference path described in the large model inference path further includes a dynamic model activation step:

[0172] During inference, when the correlation between the current inference state and the intermediate semantic representation of a candidate model not included in the inference path exceeds a preset backtracking threshold, the current inference path is paused.

[0173] The candidate model is activated and added to the inference path by using the currently generated context information as a condition.

[0174] The remaining inference steps are completed by the updated inference path in a parallel or alternating manner.

[0175] The relevance is calculated using cosine similarity: the current inference state is represented as a vector h_current (based on the average pooling of the last hidden state of the current model), and the semantic representation of the intermediate layer of the candidate model is represented as a vector h_candidate (based on the average pooling of the 6th hidden state of the candidate model). The cosine similarity is then calculated as the relevance. The backtracking trigger threshold is set to 0.75. The candidate models are multiple pre-registered domain expert models, including an insurance claims expert model, a financial business expert model, and a telecommunications package expert model. When backtracking is triggered, the current model's inference is paused, and the generated context information is used as a condition input to the candidate model. The candidate model continues to generate from the current inference state, and the final result is a weighted fusion of the outputs of multiple models.

[0176] The large language model uses the open-source model Qwen2-7B-Instruct with 7 bytes of parameters. The input context package is in JSON format and contains the following fields: user_input (raw user input), semantic_representation (structured semantic representation), confidence_scores (confidence scores of each semantic element), decision_chain (decision chain record), symbolic_result (symbolic engine query result), and failure_reason (failure reason). The prompt word template is: "You are a customer service assistant. Please understand the user's needs and generate a response based on the following structured context information. Context information: {context_json}. Please note: For constraints with a confidence score below 0.7, if you cannot determine the answer, please generate a clarifying question instead of a direct answer." The model is deployed using the vLLM inference framework and generates response text in a streaming output manner.

[0177] Step S5: Based on the independent confidence of each semantic element obtained in step S2, prioritize and adaptively organize the response content, prioritize the presentation of content related to high confidence constraints, and adjust the level of detail of the response according to the complexity of the intent.

[0178] In step S5, the level of detail in the response is adjusted according to the complexity of the intent. This is achieved in the following ways: The system pre-configures a complexity level for each intent type, or calculates the intent complexity in real time based on the number of constraints involved, the number of business operation steps, and whether multiple business systems need to be called. When the intent complexity is simple, the response text adopts a concise and direct single sentence or short sentence format, containing only the core result. When the intent complexity is medium, the response text adds key step descriptions or effective conditions on the basis of the core result. When the intent complexity is complex, the response text further elaborates on the steps and adds precautions, required materials, or subsequent operation guidelines to meet the user's understanding needs of complex business processes.

[0179] Specific examples: Taking a telecom customer service scenario as an example—a user checking their "phone bill balance" is a simple intent, and the system response is "Your current phone bill balance is 58 yuan."; a user requesting a "package change" is a medium intent, and the system response is "Your package change has been processed. The new package will take effect on the 1st of next month, with a monthly fee of 58 yuan, including 20GB of data and 200 minutes of calls."; a user reporting a "broadband fault" is a complex intent, and the system response is "Your broadband fault report has been registered. Please cooperate by completing the following steps: First, check if the optical modem's power indicator light is on normally; second, confirm that the fiber optic interface is firmly plugged in; third, if you still cannot access the internet, please keep the optical modem powered on. A repair technician will contact you within 2 hours. Please note that you should not disassemble the equipment yourself during the repair process."

[0180] Step S6: The response text is converted into a speech signal and output through the speech synthesis module; wherein, the speech synthesis module starts speech synthesis and plays the response text as soon as it receives the first semantically complete segment of the response text, rather than waiting for the complete response text to be generated.

[0181] Step S6 is followed by a cascading consistency check step:

[0182] The factual assertions in the response text are compared one by one with the search results generated by the search enhancement module.

[0183] If there are factual assertions that are not supported by search results, the verification is deemed to fail.

[0184] When the verification fails, a degradation strategy is executed, which includes: returning the processing result of the lightweight domain adaptation model or symbol engine, generating clarification scripts, or transferring to human customer service.

[0185] Step S7: Collect explicit and implicit user feedback on this interaction, analyze the failure cases of the lightweight domain adaptation model in step S2, dynamically optimize the confidence threshold accordingly, and expand the intent definition and symbol rule library.

[0186] The dynamic optimization of the confidence threshold described in step S7 specifically includes:

[0187] Statistics on the clarification accuracy and symbol engine failure rate for each constraint type.

[0188] The values ​​of the first threshold, second threshold, and clarification threshold are automatically adjusted based on statistical results to achieve a dynamic balance between accuracy and user experience.

[0189] When a user has already made multiple clarifications, the threshold for subsequent clarifications is increased, and requests are prioritized for processing in larger models to avoid excessive clarification rounds that could negatively impact the user experience.

[0190] In one specific embodiment, a multi-round dialogue context maintenance step is also included: after each round of interaction is completed, the intent, business object and key constraints of this round are recorded in the interaction history context.

[0191] In subsequent rounds, the lightweight adaptation model automatically reuses the slot values ​​extracted from previous rounds, unless the user explicitly requests new values.

[0192] Calculate the continuity score between the current round intent and the previous round intent. When the score is higher than a set threshold, inherit the previous confidence and query results. When the score is lower than a set threshold, clear the relevant inherited slot values.

[0193] Detect logical contradictions in multi-turn dialogues; if any are found, trigger a clarification process or mark the scenario as a complex one requiring the intervention of a large language model.

[0194] The continuity score is calculated using any one or a combination of the following methods:

[0195] A direct match of the intent identifier returns 1.0 if they are the same, and 0.0 if they are completely different.

[0196] The similarity between different intents is calculated using a predefined intent relationship graph.

[0197] The intent is vectorized using a semantic similarity model, and cosine similarity is calculated.

[0198] A differentiated strategy is implemented based on the continuity score: when the score is higher than the first continuity threshold, the previous slot value is automatically inherited; when the score is between the first and second continuity thresholds, the inherited value is marked as low trust and the user is prompted for confirmation; when the score is lower than the second continuity threshold, it is determined to be an intent conversion and the relevant slot value is cleared.

[0199] See Figure 2 As shown, an intelligent customer service voice interaction system based on a large model cascade architecture is provided, including: a voice recognition module, used to receive user voice input, convert the voice input into target question text through automatic voice recognition, and synchronously output the recognition confidence sequence of each token in the target question text.

[0200] The automatic speech recognition module employs a Conformer-based streaming ASR model, comprising 12 Conformer layers, each containing a multi-head self-attention module and a convolutional module, with a hidden layer dimension of 512. The model is jointly trained using CTC-Attention, and streaming recognition is implemented in a chunk-based manner with a chunk size of 640ms and a look-ahead window of 320ms. Token recognition confidence is obtained through the posterior probability of CTC, taking the maximum softmax probability at each time step as the recognition confidence for that token. The model is adapted to training data at an 8kHz sampling rate for telephone channels and undergoes data augmentation in a noisy call center environment.

[0201] The structured semantic parsing module is used to input the target question text into the lightweight domain adaptation model, perform structured semantic parsing, and output a structured semantic representation. The structured semantic representation includes the following semantic elements: intent classification, business object, constraints, interaction history context, and a confidence evaluation value calculated independently for each of the above semantic elements, while recording the decision chain in the parsing process.

[0202] The dynamic routing decision module compares the independent confidence assessment value of each semantic element with multiple preset confidence thresholds, and determines the processing path corresponding to the current level of understanding based on the comparison results.

[0203] The processing path execution module is used to execute the corresponding processing path according to the decision result of the dynamic routing decision module and generate a response text.

[0204] The response organization module is used to prioritize and adaptively organize the response content based on the independent confidence of each semantic element obtained by the structured semantic parsing module, prioritizing the presentation of content related to high confidence constraints, and adjusting the level of detail of the response according to the complexity of the intent.

[0205] The speech synthesis module is used to convert response text into speech signals and output them.

[0206] The speech synthesis module uses FastSpeech2 as the acoustic model and HiFi-GAN as the vocoder. Streaming synthesis is achieved by segmenting the response text into semantically complete fragments. Each fragment is generated and immediately fed into the acoustic model and vocoder. The synthesized audio is then pushed to the user terminal through a streaming output interface. The determination of semantically complete fragments is based on punctuation and the semantic integrity model. When a period, question mark, exclamation mark, or semantically complete clause is detected, it is determined to be a semantically complete fragment.

[0207] The learning optimization module is used to collect explicit and implicit user feedback on this interaction, analyze the failure cases of the lightweight domain adaptation model, dynamically optimize the confidence threshold accordingly, and expand the intent definition and symbol rule library.

[0208] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A method for intelligent customer service voice interaction based on a large model cascade architecture, characterized in that, include: Step S1: Receive user voice input, convert the voice input into target question text through automatic speech recognition, and simultaneously output the recognition confidence sequence of each token in the target question text; Step S2: Input the target question text into the lightweight domain adaptation model, perform structured semantic parsing, and output a structured semantic representation. The structured semantic representation includes the following semantic elements: intent classification, business object, constraints, interaction history context, and a confidence evaluation value calculated independently for each of the above semantic elements. At the same time, the decision chain in the parsing process is recorded. Step S3, Confidence-driven dynamic routing decision: Compare the independent confidence assessment value of each semantic element with multiple preset confidence thresholds, and determine the processing path corresponding to the current level of understanding based on the comparison results; Step S4: Execute the corresponding processing path based on the decision result of step S3, and generate a response text. When the confidence level of the intent is higher than the first threshold and the confidence levels of all key constraints are higher than the second threshold, the symbol rule processing path is entered. When the confidence level of the intent is higher than the first threshold, but the confidence level of at least one key constraint is between the clarification threshold and the second threshold, the clarification process is initiated. When the confidence level of intent is lower than the first threshold, or the confidence level of key constraints is lower than the clarification threshold, the large model inference path is entered. Step S5: Based on the independent confidence of each semantic element obtained in step S2, prioritize and adaptively organize the response content, prioritize the presentation of content related to high confidence constraints, and adjust the level of detail of the response according to the complexity of the intent. Step S6: Convert the response text into a speech signal through the speech synthesis module and output it; Step S7: Collect explicit and implicit user feedback on this interaction, analyze the failure cases of the lightweight domain adaptation model in step S2, dynamically optimize the confidence threshold accordingly, and expand the intent definition and symbol rule library.

2. The intelligent customer service voice interaction method based on a large model cascade architecture according to claim 1, characterized in that: In step S2, each semantic element independently calculates its confidence assessment value, specifically including: For intent classification, the confidence score is calculated using the softmax output of the intent classification header, and the highest score is taken as the confidence score of the intent. For business objects and constraints, a sequence labeling method is used to calculate the confidence level of the corresponding element by averaging the labeled probability sequence; if the same constraint is extracted by multiple independent recognizers, the weighted average of the scores of each recognizer is taken. For the confidence sequence of tokens transmitted by speech recognition, when a certain semantic element spans multiple low-confidence tokens, the confidence of that semantic element decreases accordingly. Simultaneously, the overall reliability score of the lightweight domain adaptation model is calculated, and the overall reliability score is obtained by aggregating the average, minimum or weighted comprehensive values ​​of multidimensional confidence.

3. The intelligent customer service voice interaction method based on a large model cascade architecture according to claim 1, characterized in that: In the structured semantic representation, key constraints are pre-configured according to the business type and risk level to which the constraints belong; When the absence or error of a constraint will cause business operations to fail or create business risks, the constraint is marked as a critical constraint.

4. The intelligent customer service voice interaction method based on a large model cascade architecture according to claim 1, characterized in that: The symbol rule processing path is as follows: Based on the intent and constraints extracted in step S2, query the predefined intent-business system mapping table to obtain the corresponding database query template; Perform normalization and data validation on constraints; execute database queries or call business system interfaces to obtain query results; If the query is successful and the result is not empty, proceed to step S5; if the query fails or the result is empty, transfer the task to the large model for processing, and pass the reason for the query failure as context.

5. The intelligent customer service voice interaction method based on a large model cascade architecture according to claim 1, characterized in that: The clarification process is as follows: Identify key constraints that have insufficient confidence but have not completely failed; based on the specific type of the constraint, the user's interaction history, and the current business context, generate targeted clarification questions, which include a natural language description of the constraint and / or a list of candidate options; The clarification question is converted into speech output; the user's answer to the clarification question is received, the answer is converted into structured information and then fused with the original structured semantic representation obtained in step S2, the confidence of the affected constraints is updated, and the fused confidence is the larger value between the original confidence and the confidence of the clarification answer; after the fusion is completed, the process returns to step S3 to re-execute the routing decision.

6. The intelligent customer service voice interaction method based on a large model cascade architecture according to claim 1, characterized in that: When entering the clarification process, the following sub-steps are executed: Attribution Analysis: Attribution analysis is performed on key constraints with confidence levels below the second threshold to determine the category of reasons for the low confidence level of the key constraint. The category of reasons includes at least: speech recognition error, semantic ambiguity, incomplete expression, and unclear intent. The input to the attribution analysis includes: the confidence value of the constraint, the token recognition confidence sequence corresponding to the constraint, the parsing steps of the constraint in the decision chain, and the previous state of the constraint in the dialogue history. Differentiated clarification strategy selection: Select the corresponding clarification strategy based on the cause category output by the attribution analysis. When the cause category is "speech recognition error", generate a clarification question requesting the user to repeat their statement; When the cause category is "semantic ambiguity", generate a clarifying question that provides candidate options; When the reason category is "incomplete expression", generate a clarifying question to guide the user to supplement the information; When the cause category is "ambiguous intent", no clarification question is generated, and the process directly proceeds to the large model inference path; Clarification Feedback Fusion: The generated clarification question is converted into voice output; the user's answer to the clarification question is received, the answer is converted into structured information and fused with the original structured semantic representation obtained in step S2, the confidence of the affected constraints is updated, and the confidence of the fused confidence is the larger value between the original confidence and the confidence of the clarification answer; after the fusion is completed, the process returns to step S3 to re-execute the routing decision.

7. The intelligent customer service voice interaction method based on a large model cascade architecture according to claim 6, characterized in that: The attribution analysis is rule-based, and the specific rules include: When there are multiple consecutive tokens with confidence scores below the token threshold in the token recognition confidence sequence corresponding to this constraint, the cause category is determined to be "speech recognition error"; When the parsing step of the constraint in the decision chain hits multiple candidate semantic patterns and the confidence difference is less than the difference threshold, the cause category is determined to be "semantic ambiguity". When the constraint is missing or only partially filled in the structured semantic representation, and the user has not provided complete information in the dialogue history, the reason category is "incomplete expression"; When the confidence level of the intent is lower than the intent ambiguity threshold, and the low confidence level of the constraint is associated with intent uncertainty, the cause category is determined to be "intent ambiguity".

8. The intelligent customer service voice interaction method based on a large model cascade architecture according to claim 1, characterized in that: The inference path of the large model is as follows: Construct the input context package for the large language model, which includes: the original target question text, the complete structured semantic representation of step S2, the independent confidence decomposition of each semantic element, the decision chain record, and the query results and failure reasons of the symbol engine; call the large language model to perform reasoning and generate the response text.

9. The intelligent customer service voice interaction method based on a large model cascade architecture according to claim 1, characterized in that: It also includes multi-turn dialogue context maintenance steps: After each round of interaction is completed, the intent, business object, and key constraints of this round are recorded in the interaction history context. In subsequent rounds, the lightweight adaptation model automatically reuses the slot values ​​extracted in previous rounds, unless the user explicitly requests new values; Calculate the continuity score between the current round intent and the previous round intent. When the score is higher than a set threshold, inherit the previous confidence and query results. When the score is lower than a set threshold, clear the relevant inherited slot values. Detect logical contradictions in multi-turn dialogues; if any are found, trigger a clarification process or mark the scenario as a complex one requiring the intervention of a large language model.

10. An intelligent customer service voice interaction system based on a large model cascade architecture, used to execute the intelligent customer service voice interaction method based on a large model cascade architecture as described in any one of claims 1-9, characterized in that: include: The speech recognition module is used to receive user voice input, convert the voice input into target question text through automatic speech recognition, and simultaneously output the recognition confidence sequence of each token in the target question text; The structured semantic parsing module is used to input the target question text into the lightweight domain adaptation model, perform structured semantic parsing, and output a structured semantic representation. The structured semantic representation includes the following semantic elements: intent classification, business object, constraints, interaction history context, and a confidence evaluation value calculated independently for each of the above semantic elements, while recording the decision chain in the parsing process. The dynamic routing decision module is used to compare the independent confidence evaluation value of each semantic element with multiple preset confidence thresholds, and determine the processing path corresponding to the current level of understanding based on the comparison results. The processing path execution module is used to execute the corresponding processing path according to the decision result of the dynamic routing decision module and generate a response text; The response organization module is used to prioritize and adaptively organize the response content based on the independent confidence of each semantic element obtained by the structured semantic parsing module, prioritizing the presentation of content related to high confidence constraints, and adjusting the level of detail of the response according to the complexity of the intent. The speech synthesis module is used to convert the response text into a speech signal and output it. The learning optimization module is used to collect explicit and implicit user feedback on this interaction, analyze the failure cases of the lightweight domain adaptation model, dynamically optimize the confidence threshold accordingly, and expand the intent definition and symbol rule library.

Images