A dialogue method and device based on multi-agent driving

Abstract

The application discloses a dialogue method and device based on multi-agent driving, which is applied to a customer service dialogue system and includes the following steps: obtaining user features corresponding to current user input in a sample conversation, gold medal customer service input and a standard dialogue strategy; pre-training an expert model to predict a combined dialogue strategy corresponding to the user features and a decision sensitivity factor; taking the combined dialogue strategy as a constraint, pre-training a customer service model to generate a quasi-customer service reply corresponding to the user features; based on the gold medal customer service input, the quasi-customer service reply, the decision sensitivity factor, the standard dialogue strategy and the combined dialogue strategy, respectively optimizing the pre-trained customer service model and the pre-trained expert model to generate a quasi-customer service model and a quasi-expert model. Thus, the embodiment solves the problems of model decision and execution coupling and imprecise combined dialogue strategy execution in the prior art based on a double-model hierarchical collaborative architecture and a hard constraint transmission mechanism of the combined dialogue strategy.

Description

Technical Field

[0001] This application belongs to the field of intelligent dialogue system technology, and more specifically, relates to a dialogue method and apparatus based on multi-agent driving. Background Technology

[0002] Currently, enterprises are facing an "efficiency trap" in intelligent customer service / shopping guide systems in core business scenarios such as e-commerce shopping guides and after-sales service. For example, although advanced technologies such as large language models have been introduced, and the system has improved in natural language understanding and response fluency, its contribution to core business KPIs is not satisfactory, and it has failed to truly achieve the leap from "being eloquent" to "being effective and convertible".

[0003] Existing intelligent dialogue systems have core business pain points, mainly manifested in the following aspects:

[0004] 1. The fundamental misalignment between optimization goals and the ultimate state of customer service operations. Existing intelligent systems prioritize technical metrics (such as recommendation relevance and semantic understanding accuracy) over hard metrics for the end-stage of customer service operations (such as order conversion rate, positive user sentiment conversion rate, and one-time problem resolution rate). For example, CN202310082420.8 uses a named entity model to identify product entities and concatenate expanded words for recommendations. Its optimization goal is "relevance of recommended products," neglecting end-stage metrics like "positive user sentiment conversion rate" and "order completion rate," leading to a one-sided bias in preference alignment. CN202510608863.5 uses a large language model to generate training samples to balance weaker context categories, but its sample quality assessment relies solely on process metrics such as "questioning order" and "emotional expression content," lacking a result-oriented reward design based on whether business goals are achieved, resulting in a singular reward signal. Consequently, the system may successfully recommend "relevant" products or provide "accurate" information, but due to a lack of dynamic sales strategies and compliant communication, it ultimately fails to facilitate transactions or resolve problems, resulting in significant traffic waste and customer churn. Therefore, the disconnect between optimization goals and customer service business KPIs, as well as the inherent limitations of reinforcement learning in multi-turn dialogue scenarios, make it impossible for the model to accurately attribute "which round of strategy selection" or "which reply" to the final business failure, resulting in a misalignment between the model iteration direction and business objectives.

[0005] 2. Strategy rigidity and user experience disruption in complex, multi-turn services In complex scenarios involving multiple rounds of interaction, emotional fluctuations, or information collection (such as users refusing to provide order numbers or continuously expressing dissatisfaction with recommended products), existing systems lack deep modeling of user intent evolution and emotional changes, making it impossible to dynamically and coherently adjust service strategies. For example, CN202311175405.4 processes shopping guide tasks through a "intent recognition → product recall → dialogue generation" process, but its SOP nodes (such as "recommending products" and "answering knowledge-based questions") are preset fixed branches, unable to dynamically adjust the process chain based on user status (such as worsening emotions or intent shifts). For instance, it cannot automatically insert a node to guide users to find their order numbers when they refuse to provide them, thus lacking the ability to dynamically assemble SOPs. Existing patents generally employ a single agent to handle the entire process, failing to separate the roles of strategy and execution. For example, CN201911387758.4 processes user requests through a linear process of "intent recognition → dialogue management → language generation." A single module must simultaneously complete the dual tasks of "determining the next SOP direction" and "generating a compliant response," leading to coupling of strategy logic and execution constraints. That is, when a user refuses to provide information, the system must simultaneously determine whether to "appease emotions" or "continue guiding," and immediately generate a response text that conforms to the strategy. This conflation of the two can easily lead to a disconnect between strategy selection and actual response. Although CN202410191767.0 enhances personalization capabilities through "user profile analysis + historical dialogue summary," it still relies on a single processing layer for strategy formulation and response arrangement. This cannot guarantee the continuity of the strategy chain of "appease emotions in the first round → guide to find the order number in the second round," easily resulting in "breaks in logic between rounds," leading to a lack of continuity across multiple rounds. Consequently, conversations can easily fall into a vicious cycle of "mechanical repetition and logical breaks" (such as mechanically requesting information when the user's emotions are already bad), severely damaging the user experience, reducing customer satisfaction, and prolonging service time. Therefore, a single agent architecture cannot effectively separate the responsibilities of "policy" and "response execution," and lacks a forward-looking, dynamic SOP assembly mechanism.

[0006] 3. Uncontrolled compliance risks and unstable service quality Because the responsibilities of strategy logic and response generation are intertwined, and there is a lack of structured hard constraints on the content generated by the large model, existing systems struggle to ensure that customer service responses strictly adhere to the company's pre-set SOPs and compliance requirements (such as prohibitions on promises and the requirement to include disclaimers). For example, while the routing agent in CN202510889849.7 is responsible for intent analysis and target agent allocation, it does not impose strong constraints on customer service responses at the execution layer (e.g., the target agent). This makes it impossible to guarantee that generated responses strictly follow the keyword requirements of the selected SOPs (e.g., "soothing emotions" must include "understanding" and "apology") and prohibits certain topics, resulting in a loss of compliance supervision at the execution layer. Consequently, customer service responses may deviate from the established service process and even contain prohibited phrases, leading not only to inconsistent service quality but also potential compliance and legal risks. Therefore, the lack of a constraint transmission mechanism from strategy to response execution prevents the strategy layer's intent from being translated into hard constraints at the execution layer. Summary of the Invention

[0007] The main purpose of this application is to provide a dialogue method and device based on multi-agent driving, which can achieve seamless standard operating procedures, strict compliance and no deviation in customer service responses, and precise alignment of business goals in multi-round interactions.

[0008] To achieve the above objectives, according to the first aspect of this application, a multi-agent-driven dialogue method is proposed for application in a customer service dialogue system, comprising: acquiring user features corresponding to the current user input in a sample session, a top-tier customer service input, and a standard dialogue strategy; wherein the user features include at least user intent and emotional state; a pre-trained expert model predicts a combined dialogue strategy and a decision sensitivity factor corresponding to the user features; using the combined dialogue strategy as a constraint, a pre-trained customer service model generates a quasi-customer service response corresponding to the user features; based on the top-tier customer service input, the quasi-customer service response, the decision sensitivity factor, the standard dialogue strategy, and the combined dialogue strategy, the pre-trained customer service model and the pre-trained expert model are optimized respectively; and the model optimization process is iterated continuously to generate a quasi-customer service model and a quasi-expert model.

[0009] Optionally, the pre-trained expert model predicts the combined dialogue strategy and decision sensitivity factor corresponding to the user features; including: the pre-trained expert model matching the combined dialogue strategy corresponding to the user features from the strategy library; based on the user features and the combined dialogue strategy, the pre-trained expert model performing multi-path strategy simulation to obtain the expected reward corresponding to the combined dialogue strategy and the expected loss corresponding to at least one counterfactual dialogue strategy; and generating the decision sensitivity factor for the current dialogue round based on the expected reward and the expected loss.

[0010] Optionally, the step of generating quasi-customer service responses corresponding to the user characteristics using the combined dialogue strategy as a constraint includes: using the combined dialogue strategy as a constraint, the pre-trained customer service model predicts responses and generates several candidate customer service responses; and feeds them back to the pre-trained expert model; based on rules of strategy compliance and semantic consistency, the pre-trained expert model scores and ranks the several candidate customer service responses, generates a ranking result, and feeds it back to the pre-trained customer service model; based on the ranking result, the pre-trained customer service model selects quasi-customer service responses from the several candidate customer service responses and generates an endogenous contrast reward.

[0011] Optionally, the method further includes: detecting the user's response status to the prospective customer service reply; if the response status indicates that the user has executed all business objectives indicated by the prospective customer service reply, then the sample session is terminated; if the response status indicates that at least one business objective among all business objectives has not been executed, then the response status and the prospective customer service reply are used together as the next round of user input for the sample session, and the model optimization process continues.

[0012] Optionally, the method further includes: obtaining the current user input in the sample session; parsing and processing the current user input using a dialogue analysis module to output the user intent and emotional state; and determining the user intent and emotional state as user features.

[0013] Optionally, the method further includes: determining the corresponding gold-medal customer service input based on the current user input; and matching the corresponding standard dialogue strategy from the strategy library based on the current user input and the gold-medal customer service input.

[0014] Optionally, the step of optimizing the pre-trained customer service model and the pre-trained expert model based on the gold-medal customer service input, the quasi-customer service response, the decision sensitivity factor, the standard dialogue strategy, and the combined dialogue strategy, and iterating the model optimization process to generate a quasi-customer service model and a quasi-expert model, includes: constructing an expert reward function based on the standard dialogue strategy, the decision sensitivity factor, and the combined dialogue strategy; generating a customer service reward function based on the gold-medal customer service input and the quasi-customer service response; weighting the expert reward function and the customer service reward function to construct a target reward function; and optimizing the pre-trained customer service model and the pre-trained expert model, respectively. The process involves constructing a corresponding loss function for the customer service model; generating a customer service loss function and an expert loss function; wherein the customer service loss function includes at least: policy gradient loss, policy compliance loss, contrast loss, and diversity loss; the expert loss function includes at least: policy loss and entropy term loss; training the pre-trained customer service model based on the policy gradient method; when the corresponding customer service loss function tends to be minimized, the long-term cumulative reward of the target reward function is maximized, generating a quasi-customer service model; training the pre-trained expert model based on the policy gradient method; when the corresponding expert loss function tends to be minimized, the long-term cumulative reward of the target reward function is maximized, generating a quasi-expert model.

[0015] Optionally, the step of optimizing the pre-trained customer service model and the pre-trained expert model based on the gold-medal customer service input, the quasi-customer service response, the decision sensitivity factor, the standard dialogue strategy, and the combined dialogue strategy, and iterating the model optimization process to generate a quasi-customer service model and a quasi-expert model, includes: constructing an expert reward function based on the standard dialogue strategy, the decision sensitivity factor, and the combined dialogue strategy; generating a customer service reward function based on the gold-medal customer service input and the quasi-customer service response; and constructing corresponding loss functions for the pre-trained customer service model and the pre-trained expert model, respectively, to generate a customer service reward function. The system employs a customer service loss function and an expert loss function. The customer service loss function includes at least: policy gradient loss, policy compliance loss, contrast loss, and diversity loss. The expert loss function includes at least: policy loss and entropy loss. The pre-trained customer service model is trained using the policy gradient method. When the customer service loss function approaches its minimum, the long-term cumulative reward of the customer service reward function is maximized, generating a quasi-customer service model. The pre-trained expert model is then trained using the policy gradient method. When the expert loss function approaches its minimum, the long-term cumulative reward of the expert reward function is maximized, generating a quasi-expert model.

[0016] Optionally, constructing an expert reward function based on the standard dialogue strategy, the decision sensitivity factor, and the combined dialogue strategy includes: performing a matching evaluation based on the standard dialogue strategy and the combined dialogue strategy to generate a strategy matching reward; evaluating the combined dialogue strategy to generate a task completion reward and a dialogue efficiency penalty; generating a correction reward based on the decision sensitivity factor and the combined dialogue strategy; and weighting the strategy matching reward, the task completion reward, the correction reward, and the dialogue efficiency penalty to generate the expert reward function.

[0017] Optionally, generating a customer service reward function based on the input of the top-performing customer service representative and the response of the prospective customer service representative includes: performing a matching evaluation based on the input of the top-performing customer service representative and the response of the prospective customer service representative to generate a response matching reward; performing a quality evaluation on the response of the prospective customer service representative to generate a response quality reward; using the combined dialogue strategy as a constraint, pre-training a customer service model to generate an endogenous contrast reward; and performing weighted processing on the response matching reward, the response quality reward, and the endogenous contrast reward to generate a customer service reward function. Alternatively, based on the input of the top-performing customer service representative and the response of the prospective customer service representative, a matching evaluation is performed to generate a response matching reward; using the combined dialogue strategy as a constraint, a pre-trained customer service model generates an endogenous contrast reward; the response of the prospective customer service representative is evaluated for quality to generate a response quality reward; based on the response matching reward, the response quality reward, and the endogenous contrast reward, a weighted average is performed to generate a customer service reward function; Optionally, the method further includes: obtaining the current user input in the test session and the user features corresponding to the current user input; performing matching processing on the user features using a quasi-expert model to output a combined dialogue strategy; and using the combined dialogue strategy as a constraint, generating a quasi-customer service response corresponding to the user features using a quasi-customer service model based on the user features.

[0018] Optionally, the method further includes: obtaining the dialogue turn when each business objective in the test session is completed; wherein the test session includes at least one business objective; for any one of the at least one business objective: determining the completion indicator of the business objective based on the dialogue turn, the decay coefficient, and the evaluation value corresponding to the completion of the business objective; and determining the probability that the test session is successfully recommended based on the completion indicator of each of the at least one business objective.

[0019] To achieve the above objectives, according to a second aspect of this application, a multi-agent driven dialogue device is also provided, which is applied to a customer service dialogue system; comprising: an acquisition module, used to acquire user features, top-level customer service input, and standard dialogue strategy corresponding to the current user input in a sample session; wherein the user features include at least user intent and emotional state; a prediction module, used to pre-train an expert model to predict a combined dialogue strategy and a decision sensitivity factor corresponding to the user features; a generation module, used to generate a quasi-customer service response corresponding to the user features using the combined dialogue strategy as a constraint; and a model training module, used to optimize the pre-trained customer service model and the pre-trained expert model based on the top-level customer service input, the quasi-customer service response, the decision sensitivity factor, the standard dialogue strategy, and the combined dialogue strategy; and to continuously iterate the model optimization process to generate a quasi-customer service model and a quasi-expert model.

[0020] To achieve the above objectives, according to a third aspect of this application, a computer-readable medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the dialogue method as described in the first aspect.

[0021] This embodiment provides a multi-agent driven dialogue method applied to a customer service dialogue system. The method includes: first, acquiring user features, top-performing customer service input, and standard dialogue strategies corresponding to the current user input in a sample session; wherein the user features include at least user intent and emotional state; second, a pre-trained expert model predicts a combined dialogue strategy and decision sensitivity factor corresponding to the user features; then, using the combined dialogue strategy as a constraint, a pre-trained customer service model generates a quasi-customer service response corresponding to the user features; finally, based on the top-performing customer service input, quasi-customer service response, decision sensitivity factor, standard dialogue strategy, and the combined dialogue strategy, the pre-trained customer service model and the pre-trained expert model are optimized respectively; and the model optimization process is iterated continuously to generate a quasi-customer service model and a quasi-expert model. Therefore, this embodiment, based on a dual-model hierarchical collaborative architecture and a hard constraint transmission mechanism for combined dialogue strategies, solves the problems of model decision-making and execution coupling and lax SOP execution in existing technologies. Through a hierarchical reward mechanism and a loss function design aligned with business objectives, it solves the problem of misalignment between existing model optimization objectives and business objectives. Attached Figure Description

[0022] The accompanying drawings, which form part of this application, are used to provide a further understanding of the application and to make other features, objects, and advantages of the application more apparent. The illustrative embodiments and descriptions of this application are used to explain the application and do not constitute an undue limitation of the application. In the drawings: Figure 1 A flowchart illustrating a multi-agent-driven dialogue method provided in an embodiment of this application; Figure 2 A flowchart illustrating the generation of a quasi-customer service model and a quasi-expert model according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a dialogue device based on multi-agent driving, provided as an embodiment of this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] like Figure 1 The diagram shown is a flowchart of a dialogue method based on multi-agent driving provided in an embodiment of this application.

[0025] A dialogue method based on multi-agent driving includes at least the following steps: S101, Obtain the user characteristics, top customer service input, and standard dialogue strategy corresponding to the current user input in the sample session; wherein, the user characteristics include at least the user intent and emotional state; S102, The pre-trained expert model predicts the combined dialogue strategy and decision sensitivity factor corresponding to the user characteristics; S103, using combined dialogue strategies as constraints, pre-trained customer service models generate quasi-customer service responses corresponding to user characteristics; S104 optimizes the pre-trained customer service model and the pre-trained expert model based on gold-medal customer service input, near-customer service response, decision sensitivity factor, standard dialogue strategy, and combined dialogue strategy; and iterates the model optimization process to generate near-customer service model and near-expert model.

[0026] In S101, there are no restrictions on how user characteristics, top customer service input, and standard dialogue strategies are obtained. For example, user characteristics, top customer service input, and standard dialogue strategies corresponding to the current user input can be determined through preset rules or models.

[0027] "Top-tier customer service input" refers to existing, high-quality, real-world customer service dialogue records in the training data. These dialogues come from top-performing customer service personnel in actual business scenarios, representing ideal response methods and service quality standards. During training, these real, high-quality dialogues are used as reference standards to guide the model in learning what constitutes a good response. In other words, the dialogues of real, good customer service representatives are used as correct answers for the model to imitate and learn from.

[0028] For example, the current user input in a sample session is obtained; the current user input is parsed and processed using a dialogue analysis module to output the user intent and emotional state; the user intent and emotional state are determined as user features. Based on the current user input, the corresponding "top customer service" input is determined; based on the current user input and the "top customer service" input, a strong model is used to match the corresponding standard dialogue strategy from the strategy library.

[0029] For example, the current user input is sent to the dialogue analysis module, which extracts the core information and sentiment of the user's needs, and outputs the user's intent and emotional state. Thus, by performing preliminary analysis of the current user input through the dialogue analysis module, key features can be provided for subsequent strategy generation.

[0030] The standard dialogue strategy output by the strong model is mainly used for subsequent evaluation of the improvement effect of the pre-trained expert model.

[0031] In S102 and S103, both the pre-trained expert model and the pre-trained customer service model are open-source dialogue models such as GPT, Qwen, and DeepSeek. The policy library includes several dialogue policies, which indicate the standard operating procedure (SOP) for dialogue. Each SOP includes at least one node. User features are input into the pre-trained expert model, which matches a combined dialogue policy corresponding to the user features from the policy library. The policy library indicates a set of dialogue policies extracted from historical logs; the dialogue policies indicate the SOP for dialogue. Based on the user features and the combined dialogue policies, the pre-trained expert model performs multi-path policy simulation to obtain the expected reward corresponding to the combined dialogue policy and the expected loss corresponding to at least one counterfactual dialogue policy. Based on the expected reward and the expected loss, a decision sensitivity factor for the current dialogue round is generated. Counterfactual is typically used to describe a hypothetical situation, that is, analyzing the possible consequences of the system if a key decision or action had not occurred, but instead an alternative choice was made, while keeping other conditions unchanged. In research fields such as causal inference and reinforcement learning, counterfactual reasoning is often used to evaluate the actual contribution of a decision. It measures the value of the current decision by comparing the difference between the "actual outcome" and the "possible outcome if other actions were taken." Here, a counterfactual dialogue strategy indicates the consequences of executing other strategy nodes instead of the selected combined dialogue strategy nodes. For example, when a user inquires about returning or exchanging a product, the combined dialogue strategy might require customer service to first execute the standard operating procedure (SOP) of "verifying order information → confirming return / exchange conditions → guiding the user to submit a return / exchange request." The counterfactual dialogue strategy corresponds to other dialogue nodes that might be taken if this procedure is not followed, such as directly providing a vague reply or directly recommending other products. To construct the counterfactual dialogue strategy, a model with fixed parameters is introduced into the pre-trained expert model as a regular strategy customer service role. Samples of historical customer service replies that are artificially labeled as ordinary replies (not high-quality replies) are used as minority sample examples and input as prompt words into the regular strategy customer service model. This model simulates selecting and outputting a standard SOP path from a strategy library. By comparing the strategy SOP output by the pre-trained expert model t with the strategy SOP output by the ordinary strategy customer service model, the degree of advantage of the expert strategy over the ordinary strategy can be characterized, thus providing a reference for subsequent strategy decision-making.

[0032] The combined dialogue strategy and user characteristics are input into the pre-trained customer service model. The pre-trained customer service model uses the combined dialogue strategy as a constraint and outputs a quasi-customer service response corresponding to the user characteristics.

[0033] Specifically, the pre-trained expert model, acting as the strategy maker, primarily executes the selection and combination of strategies to guide the dialogue to completion efficiently and smoothly. The pre-trained customer service model, acting as the executor, mainly generates the most natural and effective customer service responses under the constraints of the combined dialogue strategies selected by the pre-trained expert model. Here, customer service responses can be in text or voice format.

[0034] The pre-trained expert model first analyzes user characteristics in real time (e.g., when a user refuses to provide information, it identifies whether the user's emotions have worsened or their intentions have shifted). Then, it selects dialogue decisions from the policy library that correspond to the user's characteristics (e.g., selecting "soothe emotions" when the user is agitated, and selecting "recommend products" after the user has clarified their needs). Based on these dialogue decisions, it dynamically adjusts the original dialogue strategy, generating combined dialogue strategies (e.g., a coherent transition from "collecting order number → user refusal → guiding the search for order number") and a decision sensitivity factor for the current dialogue round. Thus, the pre-trained expert model in this embodiment dynamically assembles multiple SOPs into a coherent strategy chain and supports dynamic adjustment and insertion of remedial SOP strategy nodes based on real-time user feedback (e.g., continuous refusal, escalation of emotions). This solves the problems of existing SOP technologies. This addresses the issues of process breaks and dialogue loops caused by fixed nodes and single-step jumps; it enables seamless and uninterrupted standard operating procedures and precise alignment of business objectives across multiple rounds of interaction. The pre-trained customer service model generates quasi-customer service responses based on the combined dialogue strategies selected by the pre-trained expert model; it also controls the style (e.g., conversational, polite) and length (e.g., conciseness suited to the customer service scenario) of the output responses. For example, if the pre-trained expert model selects "guide to find order number," the pre-trained customer service model will not generate a quasi-customer service response of "direct refund." This prevents customer service responses from deviating from the combined dialogue strategy, ensuring the pre-trained customer service model strictly adheres to the constraints of the combined dialogue strategy, thereby generating quasi-customer service responses that conform to scenario norms and flow naturally, thus improving the user experience.

[0035] This embodiment uses a pre-trained expert model to handle the judgment and decision-making of multi-turn dialogue strategies, while a pre-trained customer service model focuses on generating responses that conform to the constraints of the dialogue strategies. Therefore, this embodiment, through parameter space decoupling and structured hard constraint propagation, ensures that both the generation of combined dialogue strategies and the execution of responses are unidirectional control flows. This solves the technical problems in existing single-model architectures caused by the coupling between dialogue strategies and the generation of potential customer service responses, as well as the disconnect between strategy selection and potential customer service responses, thus improving the accuracy of combined dialogue strategy execution and the compliance of potential customer service responses.

[0036] In S104, each sample session includes at least one round of user input; each round of user input has a corresponding output, which includes at least: a top-tier customer service representative input, a quasi-customer service representative response, a standard dialogue strategy, and the combined dialogue strategy. This embodiment optimizes the pre-trained expert model and the pre-trained customer service model based on the outputs corresponding to each round of user input in several sample sessions, until the models reach their optimal state, generating a quasi-customer service model and a quasi-expert model.

[0037] For example, the step of optimizing the pre-trained customer service model and the pre-trained expert model based on the gold-medal customer service input, the quasi-customer service response, the decision sensitivity factor, the standard dialogue strategy, and the combined dialogue strategy; and iterating the model optimization process to generate a quasi-customer service model and a quasi-expert model; includes: constructing an expert reward function based on the standard dialogue strategy, the decision sensitivity factor, and the combined dialogue strategy; generating a customer service reward function based on the gold-medal customer service input and the quasi-customer service response; and constructing corresponding loss functions for the pre-trained customer service model and the pre-trained expert model to generate a customer service reward function. The system employs a customer service loss function and an expert loss function. The customer service loss function includes at least: policy gradient loss, policy compliance loss, contrast loss, and diversity loss. The expert loss function includes at least: policy loss and entropy loss. The pre-trained customer service model is trained using the policy gradient method. When the customer service loss function approaches its minimum, the long-term cumulative reward of the customer service reward function is maximized, generating a quasi-customer service model. The pre-trained expert model is then trained using the policy gradient method. When the expert loss function approaches its minimum, the long-term cumulative reward of the expert reward function is maximized, generating a quasi-expert model.

[0038] As further exemplified, the step of constructing an expert reward function based on the standard dialogue strategy, the decision sensitivity factor, and the combined dialogue strategy includes: performing a matching evaluation based on the standard dialogue strategy and the combined dialogue strategy to generate a strategy matching reward; evaluating the combined dialogue strategy to generate a task completion reward and a dialogue efficiency penalty; generating a correction reward based on the decision sensitivity factor and the combined dialogue strategy; and weighting the strategy matching reward, the task completion reward, the correction reward, and the dialogue efficiency penalty to generate the expert reward function.

[0039] The step of generating a customer service reward function based on the input of the top-performing customer service representative and the response of the prospective customer service representative includes: performing a matching evaluation based on the input of the top-performing customer service representative and the response of the prospective customer service representative to generate a response matching reward; performing a quality evaluation on the response of the prospective customer service representative to generate a response quality reward; and performing a weighted average of the response matching reward and the response quality reward to generate a customer service reward function. This embodiment employs a hierarchical reinforcement learning method. A pre-trained expert model learns a macro-level strategy (i.e., generates a combined dialogue strategy) to maximize the expert reward function; a pre-trained customer service model learns micro-level execution (i.e., generates customer service responses) to maximize the customer service reward function, and the generation of micro-level execution is constrained by the macro-level strategy.

[0040] Furthermore, in this embodiment, the pre-trained expert model transforms the combined dialogue strategy into an executable hard constraint, directly embedding it into the generation process of the pre-trained customer service model. Through counterfactual reasoning, the pre-trained expert model possesses predictive corrective capabilities, accurately identifying key decision points that influence the final state of the business, significantly improving the system's decision robustness in complex scenarios. This ensures that the customer service responses generated by the pre-trained customer service model strictly adhere to the combined dialogue strategy selected by the pre-trained expert model, thereby reducing the risk of compliance failure to zero.

[0041] like Figure 2 The diagram shown is a flowchart illustrating the process of generating a quasi-customer service model and a quasi-expert model according to an embodiment of this application.

[0042] Generating near-customer service models and near-expert models includes at least the following steps: S201 constructs an expert reward function based on standard dialogue strategies, decision sensitivity factors, and combined dialogue strategies; and generates a customer service reward function based on the input of top-performing customer service representatives and the responses of quasi-customer service representatives. S202, We weight the expert reward function and the customer service reward function to construct the target reward function; S203, construct corresponding loss functions for the pre-trained customer service model and the pre-trained expert model respectively; generate customer service loss function and expert loss function; wherein, the customer service loss function includes at least: policy gradient loss, policy compliance loss, contrast loss, and diversity loss; the expert loss function includes at least: policy loss and entropy term loss; S204. The pre-trained customer service model is trained based on the policy gradient method. When the corresponding customer service loss function tends to be minimized, the long-term cumulative reward of the objective reward function is maximized, thus generating a quasi-customer service model. S205. The pre-trained expert model is trained based on the policy gradient method. When the corresponding expert loss function tends to be minimized, the long-term cumulative reward of the objective reward function is maximized, thus generating a quasi-expert model.

[0043] For example, the goal of a pre-trained expert model is to select the correct combination of indicators, specifically as follows: To ensure that the indicator accurately reflects the key tasks in real business scenarios, the inventors conducted systematic mining and analysis of large-scale e-commerce customer service dialogue data. First, 10 potential key indicators for sales guides were extracted from business practices. Then, statistical analysis was used to verify their effectiveness in actual dialogues, thereby driving the completion of dialogue tasks and managing user emotions. The reward of the pre-trained expert model consists of four parts: Strategy Matching Rewards The pre-trained expert model selects combinatorial dialogue strategies and compares them with standard strategies, scoring them using Mean Average Precision (MAP). MAP is a classic publicly available metric in information retrieval and ranking evaluation, primarily used to measure the consistency of the ranking between the evaluated sequence and the standard sequence. It assigns higher natural weights to matching nodes at the beginning of the sequence, aligning with the business logic in multi-turn dialogues where "the first strategy node executed has a greater impact on the dialogue's direction." The score range is... A higher score indicates a higher degree of matching between the strategy sequence to be evaluated and the standard strategy sequence. For single-round strategy sequence matching evaluation scenarios, MAP is first calculated through... Calculate the matching accuracy of the first k positions in the policy sequence to be evaluated, where k is the execution position of the policy node in the sequence (k=1,2,3). ), and then through Calculate the final score, where P represents the set of positions of all nodes in the strategy sequence to be evaluated that match the standard strategy, and M represents the total number of nodes in the standard strategy sequence.

[0044] For example, the standard strategy sequence is [recommend product, introduce selling points, facilitate order], with a total of 3 nodes M. The expert combination strategy sequence to be evaluated is [recommend product, offer discounts, introduce selling points]. The matching nodes are recommended product (k=1) and introduced selling points (k=3), with a position set P={1,3}. First, calculate the accuracy of each matching position. Then calculate the final MAP score. This score indicates a moderate match between the two rankings. If they are completely identical, MAP=1; if no nodes match, MAP=0. When the core node's recommended product matches at positions k=1 and k=3, the corresponding values ​​are [recommended product, no action, no action] and [no action, no action, recommended product], respectively. The calculated MAP scores are as follows: and It is evident that the impact of matching nodes in preceding positions on the overall score is far greater than that of subsequent nodes, fully aligning with the business logic characteristics of multi-turn dialogues. Besides MAP, this method can also employ other publicly available and universally accepted ranking metrics such as Discounted Cumulative Gain (DCG), Mean Reciprocal Rank (MRR), and Kendall's Rank Correlation Coefficient to evaluate the consistency of strategy sequence ranking, all achieving equivalent evaluation results.

[0045] Task completion reward Sparse positive rewards are given when a dialogue task (such as information gathering or closing a deal) is completed; that is, a fixed positive value (such as +1) is given only in the round in which the dialogue task is completed, and 0 in the remaining rounds.

[0046] Dialogue efficiency penalty Each round of dialogue is rewarded with a small negative value (e.g., -0.01) to encourage the pre-trained expert model to complete the task as quickly as possible. That is, a small negative value (e.g., -0.01) is deducted for each round of dialogue, and the more rounds there are, the more the value is deducted, forcing the model to complete the task as quickly as possible.

[0047] Correction Rewards The corrective reward is used to characterize the difference in strategy selection quality between the pre-trained expert model and the ordinary customer service strategy model. Specifically, let... This represents the sequence of strategy operation (SOP) selected by the expert model. This represents the sequence of Standard Operating Procedures (SOPs) selected by the standard customer service strategy model. The difference between the strategy SOPs selected by the pre-trained expert model and the standard customer service strategy is measured by calculating the Hamming distance. A larger difference indicates that the expert strategy deviates further from the typical decision-making path of a standard customer service representative, thus warranting a higher reward; a smaller difference indicates that the expert strategy is closer to standard customer service behavior, corresponding to a lower reward. This is also combined with a decision sensitivity factor. The reward is weighted to enhance the strategy correction capability at key decision-making junctures. For example, it can improve the effectiveness of the reward at critical junctures such as user demand identification, product matching, and recommendation strategy selection. The value of is chosen to enhance the incentive for the degree of policy deviation. The formula is as follows: Equation (1); in, Let Hamming distance function be used. and These represent the lengths of the corresponding policy sequences. This represents the larger of the two policy sequence lengths, used for normalizing the Hamming distance.

[0048] For example, a typical customer service strategy model: =[Inquire about budget range, recommend product models, and send product links], Expert Strategy SOP: =[Inquire about budget range, recommend product models, introduce key selling points, and guide the customer service strategy sequence to place an order]. The length of the standard customer service strategy sequence is [length missing]. =3, the expert model policy sequence length is =4, therefore the normalized length is: =4. A positional comparison of the differences between the two policy nodes yields: 1st: Inquire about budget range / Inquire about budget range (same); Second place: Recommended product model / Recommended product model (same); 3rd position: Send product link / Introduce core selling points (different); 4th position: Empty / Guide to place an order (different); Therefore, the Hamming distance is: =2; if the current decision sensitivity factor is =0.8, then the correction reward is: =0.8 × 2 / 4 = 0.4. This result indicates that the output strategy of the pre-trained expert model differs from the ordinary customer service strategy at some key nodes, thus obtaining corresponding correction rewards. Expert reward function The expression for is shown in equation (2): Equation (2); Here, λ1 and λ2 are the weight hyperparameters for strategy matching reward and task completion reward, respectively, used to balance the pre-trained expert model's focus on process compliance and task completion. The specific values ​​of λ1 and λ2 are determined through experimental tuning, and there are no normalization constraints between them; their relative magnitudes reflect the priority preference for strategy compliance and task progress.

[0049] The goal of a pre-trained customer service model is to generate high-quality, compliant customer service responses under the constraints of combined dialogue strategies. The reward for a pre-trained customer service model consists of three parts: Reply quality reward Send the customer service responses to a large model (such as GPT) and let it score them based on dimensions such as politeness, relevance, strategy compliance, and sales conversion effect, and finally summarize them into a comprehensive score; Reply Match Rewards This measures the text similarity between customer service replies and the input of top-performing customer service representatives (e.g., BERTScore or cosine similarity); the greater the similarity, the higher the score. Endogenous contrast reward. After obtaining the combined dialogue strategy, the pre-trained customer service model generates k candidate customer service responses under the constraints of this strategy. The system then prioritizes and ranks the candidate responses. A pre-trained expert model predicts the benefits of each response and generates a preference ranking. An endogenous contrastive reward is constructed by comparing the consistency of these rankings; the formula is as follows: Equation (3); The corresponding reward is .in and The preference rankings were generated by the customer service model and the expert model, respectively.

[0050] In practical applications, since the customer service model only outputs the highest-priority response, this response can be compared with the optimal response evaluated by the expert model to calculate the reward. The formula is as follows: , Equation (4) For example, in a pre-sales scenario where a user inquires whether a product is worth buying, the customer service model generates three candidate responses under the constraint of a combined dialogue strategy: "This product is currently quite popular; you might consider buying it." "This product is characterized by its long battery life and high cost-performance ratio. If you are concerned about battery life, you should definitely consider it." "The product link has been sent to you; you can view the details yourself." The customer service model prioritizes the following: The preference ranking obtained by the pre-trained expert model based on strategy compliance and sales conversion effectiveness is as follows: It can be seen that the expert model believes... This aligns better with the strategic intent because it includes product selling points, thus having a higher ranking priority. Since the customer service model ultimately outputs its highest-priority response, when calculating rewards, we can simply compare whether the Top-1 response selected by the customer service model matches the optimal response ranked by the expert model, thereby calculating the corresponding intrinsic contrastive reward. 1. Then at this time =-1. This indicates that the response selected by the pre-trained customer service model is inconsistent with the preferences of the pre-trained expert model, thus receiving a negative reward. This guides the pre-trained customer service model to gradually adjust its response ranking during training, making it more consistent with the strategy judgments of the pre-trained expert model. This embodiment utilizes a "pre-trained expert model" with strategy judgment capabilities as a "referee." The pre-trained customer service model generates multiple candidate responses for the same strategy, and the pre-trained expert model ranks these candidate responses based on its preferences for "strategy compliance" and "semantic consistency." This ranking signal is used as an intrinsic reward to drive the customer service model's preference alignment. This ensures a high degree of alignment between the generated responses and the strategic intent at a deep semantic level, improving the accuracy of the responses.

[0051] The expression for the customer service reward function is shown in equation (5) below: Equation (5); Among them, μ1, μ2, and μ3 are hyperparameters that balance the relative weights of response quality, response matching, and endogenous contrastive reward.

[0052] The expression for the target reward function is shown in equation (6) below: Equation (6); in, and It is a weight used to balance the importance of strategy selection and customer service response generation.

[0053] The expression for the expert loss function is shown in equation (7) below: Equation (7); The core of training a pre-trained expert model is maximizing the long-term cumulative reward, corresponding to the ascent direction of the policy gradient; however, the loss function... The addition of a negative sign means that the optimizer actually performs gradient descent to minimize this loss, which is equivalent to performing gradient ascent on the original policy objective. (Entropy term loss) Then the strategy loss Both are minimized to encourage policy exploration. L_ Policy Policy gradient loss. This is calculated by multiplying the advantage (current reward minus the baseline estimate) by the log probability of the action, then taking a negative sign. The advantage is calculated using a moving average of the baseline, i.e., the difference between the current reward and the historical average reward. Taking a negative sign transforms the maximization objective into a minimization objective, making the optimizer's gradient descent equivalent to gradient ascent with respect to the policy objective.

[0054] Entropy-regularized loss. This calculates the information entropy of the probability distribution of the expert policy output and multiplies it by a negative weighting coefficient. Higher entropy indicates a more diversified and exploratory policy. Adding this term minimizes the total loss while simultaneously encouraging the policy to maintain a degree of exploratory activity, preventing premature convergence to a single action.

[0055] The customer service loss function consists of three parts: Policy gradient loss This approach uses a policy gradient loss based on the customer service reward function to optimize response generation. Specifically, it multiplies the customer service reward by the log probability of generating a response, takes the negative sign, and then calculates the mean. The principle is the same as above, transforming reward maximization into loss minimization.

[0056] Strategy compliance losses : To measure the degree to which customer service responses adhere to a combined dialogue strategy, use (Bertscore) is used as a penalty factor; that is, the Bertscore F1 value between the customer service-generated response and the SOP description selected by the expert is calculated, and then 1 - F1 is used as the distance. The more the response conforms to the SOP description, the higher the F1 and the smaller the penalty.

[0057] Diversity loss This approach uses a negative Bertscore to penalize the similarity of consecutive responses. The higher the similarity between two adjacent responses, the larger the Bertscore value, and the stronger the penalty in the loss term. This, in turn, pushes the model to break free from the inertia of repetitive responses and generate differentiated, targeted replies, balancing compliance with natural fluency. In other words, it calculates the Bertscore similarity between two adjacent customer service responses, with higher similarity resulting in a greater penalty. This forces the model to generate differentiated responses in consecutive rounds, avoiding repetitive language.

[0058] To assess the comparative loss, a pre-trained expert model is used to rank the revenue prediction results of the multiple candidate customer service responses, and the comparative loss is calculated based on the ranking results.

[0059] Thus, this embodiment achieves low-cost self-evolution of the system. Through the endogenous closed loop of "expert-guided execution," the problem of expensive and inconsistent reward signal acquisition in reinforcement learning is solved, ensuring a high degree of alignment between response generation and policy intent in deep semantics.

[0060] The customer service loss function expression is shown in equation (8) below: Equation (8); It should be noted that Bertscore is a text similarity metric based on a pre-trained language model. It calculates the bidirectional optimal cosine similarity between the candidate and reference text token embeddings, and then combines this with precision and recall to obtain an F1 score, thus measuring the deep semantic matching degree between texts. Where λ... RL , λ SOP , λ Div Three weighted hyperparameters control the relative weights of the three losses: policy gradient, SOP compliance, and diversity. These are determined through experimental tuning.

[0061] This embodiment establishes a two-dimensional reward mechanism and incorporates business objectives (e.g., positive user sentiment conversion rate, order conversion rate) into a weighted target reward function. This not only solves the problem that existing models' optimization objectives are limited to technical metrics and misaligned with business goals, but also addresses the long-term credit allocation dilemma in reinforcement learning through reward separation; thus significantly improving the order conversion rate.

[0062] In a preferred embodiment of this example, the method further includes: acquiring the current user input in the test session and the user features corresponding to the current user input; performing matching processing on the user features using a quasi-expert model to output a combined dialogue strategy; and using the combined dialogue strategy as a constraint, generating a customer service response corresponding to the user features using a quasi-customer service model based on the user features.

[0063] Specifically, in the prediction phase, user features from the user input in the test session are fed into a quasi-expert model, which outputs a combined dialogue strategy. The dialogue strategy and user features are then fed into a quasi-customer service model, which outputs a customer service response. In this embodiment, the pre-trained expert model or quasi-expert model deeply analyzes the evolution trajectory of user intent and the curve of emotion changes through time-series dynamic feature modeling. Based on the intent transition probability matrix, it proactively predicts subsequent user behavior and prepares backup SOPs in advance. This solves the problems of shallow context modeling and policy adjustment lagging behind changes in user state in existing technologies, ensuring that the policy is switched in advance before the user's state deteriorates, thus improving the one-time problem resolution rate.

[0064] In a preferred embodiment of this example, the method further includes: detecting the user's response status to the customer service reply; if the response status indicates that the user has executed all business objectives indicated by the customer service reply, then the sample session is terminated; if the response status indicates that at least one business objective among all business objectives indicated by the customer service reply has not been executed, then the response status and the customer service reply are used together as the next round of user input for the sample session, and the model optimization process continues.

[0065] In a preferred embodiment of this example, the method further includes: obtaining the dialogue round when each business objective in the test session is completed; wherein the test session includes at least one business objective; for any one of the at least one business objective: determining the completion index of the business objective based on the dialogue round, the decay coefficient, and the evaluation value corresponding to the completion of the business objective; Based on the completion metrics of each of the at least one business objective, the probability of the test session being successfully recommended is determined.

[0066] For example, when applying the method of this embodiment to a shopping guide scenario, the probability that the test session is successfully recommended is the shopping guide success rate. The formula for "shopping guide success rate" is derived from the frequency and combination patterns mined from real data, thus providing data support for selecting the three core tasks of sequential conditional shopping guide success rate. Based on the business logic of e-commerce shopping guides, the following 10 candidate indicators are defined for this purpose: Metric 1 (Accuracy of Demand Clarification): The degree to which potential user needs (budget, preferences, usage scenarios) are accurately captured; Metric 2 (Pain Point Solution Delivery Rate): The number of times the corresponding product selling points are effectively delivered to address user concerns (such as "fear of durability"); Metric 3 (Scarcity Information Reach Rate): The number of times information such as limited stock and limited-time offers is successfully delivered to users to encourage them to make a purchase; Indicator 4 (Personalized Recommendation Fit): The accuracy of recommending and matching products based on user needs; Indicator 5 (Completion rate of actions to lower decision-making threshold): Completion rate of actions to reduce concerns about placing an order, such as providing return and exchange guarantees and trial policies; Indicator 6 (Perceived Value Delivery): The degree to which users perceive the core value of a product, such as its cost-effectiveness and unique advantages; Metric 7 (Instant Response Rate to User Inquiries): The rate at which user-submitted product-related inquiries are answered without delay; Metric 8 (Adaptation to Scenario Relevance): The degree of accuracy in linking products with users' actual usage scenarios (such as "for commuting" or "for gifting"). Indicator 9 (Repurchase Intention Stimulation): The degree to which users' willingness to repurchase is stimulated by conveying information about after-sales service, membership benefits, etc. Indicator 10 (Positive Improvement in User Emotions): The degree to which users' emotions are transformed into pleasant satisfaction through polite responses, patient explanations, and empathetic communication.

[0067]

[0068]

[0069]

[0070]

[0071] While Indicator 7 ranks first with a single-indicator achievement rate of 62.52%, it only reflects basic service capabilities. Indicator 1, on the other hand, has the highest participation rate in high-value combinations; combinations containing Indicator 1 account for over 60% of the total pre-sales scenarios, and all three top-conversion combinations driving orders include Indicator 1. Indicators 2 and 3 are equally crucial: the inclusion of Indicator 2 significantly improves the combination conversion rate, and the combination of Indicator 3 and Indicator 7 ranks third in the complete pre-sales scenario with 4.92%. In contrast, other indicators, such as Indicators 9 and 10, are either limited to niche scenarios or exist as auxiliary items like Indicators 4 and 8, lacking universal value. Based on the above analysis, the three core tasks are defined as follows: Task 1 (demand matching) corresponds to Indicator 1, Task 2 (value delivery) corresponds to Indicator 2, and Task 3 (proactive guidance) corresponds to Indicator 3. This design aligns with business logic and has been validated by real data.

[0072] The sales conversion success rate measures the ability of a model (e.g., a near-expert model or a near-customer service model) to complete key sales conversion tasks promptly and effectively during a conversation. This metric doesn't simply count whether "the task was completed," but rather incorporates the concept of "time decay." Because in a sales conversion scenario, a user's patience decreases with each round of conversation, completing key tasks earlier has a higher value for the final conversion.

[0073]

[0074] Equation (9); in, It represents the success rate of sales referrals; a core indicator used to evaluate the business value of vertical market scenarios. This indicates the importance weight of business objectives, reflecting their priority. This represents a function indicating whether a business objective has been achieved, used to make a binary judgment on whether the business objective has been achieved. This indicates the number of dialogue rounds required to achieve a business objective, and is used to calculate time costs. This represents the round decay coefficient, reflecting the business logic that "the later the round, the lower the value."

[0075] Application Example: Suppose a sales consultant's test session has 3 business objectives, and the weights of these business objectives are set as follows: attenuation coefficient If business objective 1 is achieved in the second round, then... The attenuation term is The evaluation value corresponding to the completion of business objective 1 Contribution value If business objective 2 is achieved in round 5, then The attenuation term is The evaluation value corresponding to the completion of business objective 2. Contribution value If business objective 3 is achieved in round 8, then The attenuation term is The evaluation value corresponding to the achievement of business objective 3. Contribution value .final Therefore, the probability of the tested session being successfully recommended is determined to be 0.855. If business objective 2 is delayed until the 10th round, the decay term will become... The corresponding contribution value will decrease significantly, and the probability of the test session being successfully recommended will also decrease.

[0076] For example: The current user's input is: Please recommend a face cream for dry skin.

[0077] The three business objectives of this session are: recommending a suitable face cream, providing information on the selling points of the recommended face cream, and expediting the order.

[0078] Customer service reply: Hello! We recommend this repairing face cream. It moisturizes and locks in moisture for dry skin in one step, and it's hydrating without being sticky. It's also suitable for sensitive skin. Order now and you'll receive an exclusive discount! Based on user responses, we can determine the dialogue rounds corresponding to the completion of three different business objectives.

[0079] The following section provides a detailed description of a dialogue method based on multi-agent driving, provided in this embodiment, using specific application scenarios as examples.

[0080] S1. Obtain the current user input in the sample session; parse and process the current user input using the dialogue analysis module to output the user intent and emotional state; determine the user intent and emotional state as user features. Based on the current user input, determine the corresponding top-performing customer service input; based on the current user input and the top-performing customer service input, the strong model matches the corresponding standard dialogue strategy from the strategy library.

[0081] S2, a pre-trained expert model predicts a combined dialogue strategy and a decision sensitivity factor corresponding to the user characteristics; wherein, the strategy library is used to indicate the set of dialogue strategies extracted from historical logs; using the combined dialogue strategy as a constraint, a pre-trained customer service model generates a quasi-customer service response corresponding to the user characteristics.

[0082] S3, detect the user's response status to the customer service reply; if the response status indicates that the user has executed all the business objectives indicated by the prospective customer service reply, then end the sample session; if the response status indicates that at least one of the business objectives indicated by the prospective customer service reply has not been executed, then use the response status and the prospective customer service reply together as the next round of user input for the sample session, and continue the model optimization process.

[0083] S4, for any round of current user input in the sample session: obtain the gold-level customer service input, the near-customer service response, the decision sensitivity factor, the standard dialogue strategy, and the combined dialogue strategy corresponding to the current user input; perform matching evaluation based on the standard dialogue strategy and the combined dialogue strategy to generate a strategy matching reward; evaluate the combined dialogue strategy to generate a task completion reward and a dialogue efficiency penalty; generate a correction reward based on the decision sensitivity factor and the combined dialogue strategy. The strategy matching reward, the task completion reward, the correction reward, and the dialogue efficiency penalty are weighted and processed to generate an expert reward function.

[0084] Based on the input from the top-performing customer service representative and the response from the prospective customer service representative, a matching evaluation is performed to generate a response matching reward; an endogenous contrast reward corresponding to the response from the prospective customer service representative is obtained; the quality of the response from the prospective customer service representative is evaluated to generate a response quality reward; based on the response matching reward, the response quality reward, and the endogenous contrast reward, a weighted average is performed to generate a customer service reward function. The expert reward function and the customer service reward function are weighted to construct a target reward function. Corresponding loss functions are constructed for the pre-trained customer service model and the pre-trained expert model respectively; a customer service loss function and an expert loss function are generated; wherein, the customer service loss function includes at least: policy gradient loss, policy compliance loss, contrast loss, and diversity loss; the expert loss function includes at least: policy loss and entropy term loss.

[0085] S5. The pre-trained customer service model is trained based on the policy gradient method. When the corresponding customer service loss function tends to be minimized, the long-term cumulative reward of the target reward function is maximized, generating a quasi-customer service model. The pre-trained expert model is trained based on the policy gradient method. When the corresponding expert loss function tends to be minimized, the long-term cumulative reward of the target reward function is maximized, generating a quasi-expert model.

[0086] S6, obtain the current user input in the test session and the user features corresponding to the current user input; use a quasi-expert model to perform matching processing on the user features and output a combined dialogue strategy; use the combined dialogue strategy as a constraint and use a quasi-customer service model to generate a quasi-customer service response corresponding to the user features based on the user features.

[0087] S7, Obtain the dialogue round when each business objective in the test session is completed; wherein, the test session includes at least one business objective; for any one of the at least one business objective: based on the dialogue round, the decay coefficient, and the evaluation value corresponding to the completion of the business objective, determine the completion index of the business objective; based on the completion index of each of the at least one business objective, determine the probability that the test session is successfully recommended.

[0088] Specifically, this embodiment employs a hierarchical reinforcement learning architecture, where two models work collaboratively to complete the dialogue task. The pre-trained expert model acts as the policy maker, generating discrete macro-level policies; the pre-trained customer service model acts as the executor, generating continuous micro-level actions (e.g., customer service responses).

[0089] Phase 1: Dialogue Input and Feature Extraction This stage is the starting point of the process, responsible for receiving user input and performing preliminary analysis, providing key features for subsequent strategy assembly.

[0090] Table 4 Dialogue Input and Feature Extraction

[0091] Phase Two: Generation and Comparison of Combined Dialogue Strategies (Dual-track parallel approach) This stage generates two dialogue strategies simultaneously for subsequent comparison and evaluation.

[0092] Table 5 Generation and Comparison of Combined Dialogue Strategies

[0093] Phase Three: Customer Service Model Response and Evaluation This phase uses a "combined dialogue strategy" to generate actual responses and conducts a dual evaluation of the combined dialogue strategy and the quality of the responses.

[0094] Table 6. Responses and Evaluations of the Pre-trained Customer Service Model

[0095] Phase Four: Reward Calculation and Dialogue Control This stage calculates the final reward and determines the next step in the dialogue.

[0096] Table 7 Reward Calculation and Dialogue Control

[0097] First, upon a user's call, the dialogue analysis module is activated. The current user input is used as input, and the module parses and outputs the corresponding user intent and emotional state. Next, the module selects a combined dialogue strategy: on one hand, a pre-trained expert model receives the user intent and emotional state and outputs a combined dialogue strategy; on the other hand, a strong model (control group) generates a standard dialogue strategy based on the current user input and the input from a top-performing customer service representative. Then, the pre-trained customer service model receives the combined dialogue strategy, user intent, and emotional state provided by the pre-trained expert model and generates a customer service response that conforms to the combined dialogue strategy. Simultaneously, the evaluation process for the combined dialogue strategy is initiated, using the standard dialogue strategy and the combined dialogue strategy output by the pre-trained expert model as input, calculating and outputting the expert reward function for the strategy dimension. Then, through a response evaluation mechanism, combining the large model's score for the customer service response and its matching degree with the top-performing customer service representative's reference response, the customer service reward function is obtained. Finally, a reward weighting operation is performed, weighting and integrating the expert reward function and the customer service reward function into a target reward function, which is then fed back to both the pre-trained expert model and the pre-trained customer service model. The next dialogue loop then begins, continuing until the entire dialogue process is complete.

[0098] To address the shortcomings of traditional single-model optimization methods in current customer service scenarios, such as "conflicted decision-making and execution responsibilities," leading to issues like "broken logic in multi-round processes, uncontrolled SOP compliance, and weak alignment of business objectives (e.g., positive emotion conversion, order conversion)," this invention proposes a dual-model collaborative architecture that separates "strategy decision-making" and "response execution." A specially designed expert model is employed to handle "multi-round SOP process judgment and strategy direction decision-making," while the customer service model focuses on "response generation that conforms to SOP constraints." Simultaneously, a hierarchical state interaction mechanism, a joint reward calculation model, and a customized loss function adapted to both models are developed. Ultimately, this achieves the customer service AI optimization goal of "unbroken SOP processes in multi-round interactions, strict compliance and no deviation in customer service responses, and precise alignment of business final states (e.g., user emotions, order results)." Furthermore, the generalization of this dual-model architecture across different customer service scenarios (e.g., e-commerce refunds, hotel bookings) and its deployment feasibility on low-resource devices (e.g., client-side and edge-side) are verified, providing a technical paradigm for multi-model collaborative optimization in complex service scenarios.

[0099] like Figure 3 The diagram shown is a structural schematic of a dialogue device based on multi-agent driving provided in this application.

[0100] A multi-agent driven dialogue device is applied to a customer service dialogue system. The device 300 includes: a first acquisition module 301, used to acquire user features, a top-tier customer service input, and a standard dialogue strategy corresponding to the current user input in a sample session; wherein the user features include at least user intent and emotional state; a prediction module 302, used to pre-train an expert model to predict a combined dialogue strategy and a decision sensitivity factor corresponding to the user features; a generation module 303, used to generate a quasi-customer service response corresponding to the user features using the combined dialogue strategy as a constraint; and a model training module 304, used to optimize the pre-trained customer service model and the pre-trained expert model based on the top-tier customer service input, the quasi-customer service response, the decision sensitivity factor, the standard dialogue strategy, and the combined dialogue strategy; and to iterate the model optimization process to generate a quasi-customer service model and a quasi-expert model.

[0101] In a preferred embodiment of this example, the prediction module includes: a matching unit, configured to pre-train an expert model to match a combined dialogue strategy corresponding to the user features from a policy library; wherein the policy library indicates a set of dialogue strategies extracted from historical logs; a simulation unit, configured to perform multi-path strategy simulation based on the user features and the combined dialogue strategy, using the pre-trained expert model to obtain the expected reward corresponding to the combined dialogue strategy and the expected loss corresponding to at least one counterfactual dialogue strategy; and a generation unit, configured to generate a decision sensitivity factor for the current dialogue round based on the expected reward and the expected loss.

[0102] In a preferred embodiment of this example, the generation module includes: a first generation unit, used to predict responses using a pre-trained customer service model with the combined dialogue strategy as a constraint, generate several candidate customer service responses, and feed them back to a pre-trained expert model; a second generation unit, used to score and rank the several candidate customer service responses based on rules of strategy compliance and semantic consistency, generate a ranking result, and feed it back to the pre-trained customer service model; and a third generation unit, used to select quasi-customer service responses from the several candidate customer service responses based on the ranking result.

[0103] In a preferred embodiment of this example, the device further includes: a detection module for detecting the user's response status to the quasi-customer service reply; an end module for ending the sample session if the response status indicates that the user has executed all business objectives indicated by the quasi-customer service reply; and an execution module for using the response status and the quasi-customer service reply together as the next round of user input for the sample session and continuing the model optimization process if the response status indicates that at least one business objective indicated by the customer service reply has not been executed.

[0104] In a preferred embodiment of this invention, the device further includes: a second acquisition module, used to acquire current user input in a sample session; a parsing and processing module, used to parse and process the current user input using a dialogue analysis module, and output user intent and emotional state; and a first determination module, used to determine the user intent and the emotional state as user features.

[0105] In a preferred embodiment of this invention, the device further includes: a second determining module, configured to determine the corresponding gold-medal customer service input based on the current user input; and a matching module, configured to match the corresponding standard dialogue strategy from the strategy library using a strong model based on the current user input and the gold-medal customer service input.

[0106] In a preferred embodiment of this example, the model training module includes: a first construction unit, used to construct an expert reward function based on the standard dialogue strategy, the decision sensitivity factor, and the combined dialogue strategy; and to generate a customer service reward function based on the gold-medal customer service input and the quasi-customer service response; a second construction unit, used to weight the expert reward function and the customer service reward function to construct a target reward function; a generation unit, used to construct corresponding loss functions for the pre-trained customer service model and the pre-trained expert model respectively; and to generate a customer service loss function and an expert loss function; wherein the customer service loss function includes at least: policy gradient loss, contrast loss, policy compliance loss, and diversity loss; and the expert loss function includes at least: policy loss and entropy term loss; a customer service model optimization unit, used to train the pre-trained customer service model based on the policy gradient method, and when the corresponding customer service loss function tends to be minimized, the long-term cumulative reward of the target reward function is maximized, generating a quasi-customer service model; and an expert model optimization unit, used to train the pre-trained expert model based on the policy gradient method, and when the corresponding expert loss function tends to be minimized, the long-term cumulative reward of the target reward function is maximized, generating a quasi-expert model.

[0107] In a preferred embodiment of this example, the model training module includes: a first construction unit, used to construct an expert reward function based on the standard dialogue strategy, the decision sensitivity factor, and the combined dialogue strategy; and to generate a customer service reward function based on the gold-medal customer service input and the quasi-customer service response; a generation unit, used to construct corresponding loss functions for the pre-trained customer service model and the pre-trained expert model respectively, generating a customer service loss function and an expert loss function; wherein the customer service loss function includes at least: policy gradient loss, policy compliance loss, contrast loss, and diversity loss; the expert loss function includes at least: policy loss and entropy term loss; a customer service model optimization unit, used to train the pre-trained customer service model based on the policy gradient method, and when the customer service loss function tends to be minimized, the long-term cumulative reward of the customer service reward function is maximized, generating a quasi-customer service model; and an expert model optimization unit, used to train the pre-trained expert model based on the policy gradient method, and when the expert loss function tends to be minimized, the long-term cumulative reward of the expert reward function is maximized, generating a quasi-expert model.

[0108] In a preferred embodiment of this example, the first construction unit includes: a first matching subunit, used to perform matching evaluation based on the standard dialogue strategy and the combined dialogue strategy, and generate a strategy matching reward; a first evaluation subunit, used to evaluate the combined dialogue strategy, and generate a task completion reward and a dialogue efficiency penalty; a first generation subunit, used to generate a correction reward based on the decision sensitivity factor and the combined dialogue strategy; and a second generation subunit, used to perform weighted processing on the strategy matching reward, the task completion reward, the correction reward, and the dialogue efficiency penalty to generate an expert reward function.

[0109] In a preferred embodiment of this example, the first construction unit includes: a second matching subunit, used to perform matching evaluation based on the gold-medal customer service input and the prospective customer service response, and generate a response matching reward; a second evaluation subunit, used to perform quality evaluation on the prospective customer service response, and generate a response quality reward; and a second generation subunit, used to perform weighted processing based on the response matching reward and the response quality reward, and generate a customer service reward function. Alternatively, the first construction unit includes: a second matching subunit, used to perform matching evaluation based on the gold-medal customer service input and the prospective customer service response, and generate a response matching reward; a second evaluation subunit, used to perform quality evaluation on the prospective customer service response, and generate a response quality reward; an acquisition subunit, used to acquire the endogenous comparison reward corresponding to the prospective customer service response; and a second generation subunit, used to perform weighted processing based on the response matching reward, the response quality reward, and the endogenous comparison reward, and generate a customer service reward function.

[0110] In a preferred embodiment of this example, the device further includes: a third acquisition module, further configured to acquire the current user input in the session to be tested, and the user features corresponding to the current user input; a third matching module, configured to perform matching processing on the user features using a quasi-expert model, and output a combined dialogue strategy; and a second generation module, configured to use the combined dialogue strategy as a constraint condition, and use a quasi-customer service model to generate a quasi-customer service response corresponding to the user features based on the user features.

[0111] In a preferred embodiment of this example, the device further includes: a fourth acquisition module, further configured to acquire the dialogue turn when each business objective in the test session is completed; wherein the test session includes at least one business objective; a third determination module, configured to, for any one of the at least one business objective: a fourth determination module, configured to, based on the dialogue turn, the decay coefficient, and the evaluation value corresponding to the completion of the business objective, determine the completion index of the business objective; and a fifth determination module, configured to, based on the completion index of each of the at least one business objective, determine the probability that the test session is successfully recommended.

[0112] The aforementioned multi-agent-driven dialogue device can execute a multi-agent-driven dialogue method provided in an embodiment of the present invention, and possesses the corresponding functional modules and beneficial effects for executing a multi-agent-driven dialogue method. Technical details not described in detail in this embodiment can be found in the multi-agent-driven dialogue method provided in an embodiment of the present invention.

[0113] The present invention also provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement a multi-agent driven dialogue method as described in the present invention.

[0114] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0115] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0116] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to the following embodiments of this application described in the "Exemplary Methods" section above.

[0117] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0118] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0119] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0120] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0121] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0122] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

[0123] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0124] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0125] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A dialogue method based on multi-agent driving, characterized in that, Applications in customer service dialogue systems include: Obtain the user characteristics, top customer service input, and standard dialogue strategy corresponding to the current user input in the sample session; wherein, the user characteristics include at least the user intent and emotional state; The pre-trained expert model predicts the combined dialogue strategy and decision sensitivity factor corresponding to the user characteristics; Using the combined dialogue strategy as a constraint, the pre-trained customer service model generates quasi-customer service responses corresponding to the user characteristics; Based on the gold-medal customer service input, the near-customer service response, the decision sensitivity factor, the standard dialogue strategy, and the combined dialogue strategy, the pre-trained customer service model and the pre-trained expert model are optimized respectively; and the model optimization process is continuously iterated to generate near-customer service model and near-expert model.

2. The method according to claim 1, characterized in that, The pre-trained expert model predicts combined dialogue strategies and decision sensitivity factors corresponding to the user characteristics; including: The pre-trained expert model matches a combined dialogue policy corresponding to the user features from a policy library; wherein the policy library is used to indicate a set of dialogue policies extracted from historical logs; Based on the user characteristics and the combined dialogue strategy, a pre-trained expert model is used to simulate multi-path strategies to obtain the expected revenue corresponding to the combined dialogue strategy and the expected loss corresponding to at least one counterfactual dialogue strategy. Based on the expected gains and expected losses, a decision sensitivity factor for the current dialogue round is generated.

3. The method according to claim 1, characterized in that, The step of generating a quasi-customer service response corresponding to the user characteristics using the combined dialogue strategy as a constraint includes: Using the combined dialogue strategy as a constraint, the pre-trained customer service model predicts responses and generates several candidate customer service responses; these responses are then fed back to the pre-trained expert model. Based on the rules of policy compliance and semantic consistency, the pre-trained expert model scores and ranks the candidate customer service responses, generates a ranking result, and feeds it back to the pre-trained customer service model. Based on the ranking results, the pre-trained customer service model selects a potential customer service response from several candidate customer service responses.

4. The method according to claim 1, characterized in that, Also includes: Detect the user's response status to the quasi-customer service reply; If the response status indicates that the user has executed all the business objectives indicated by the quasi-customer service response, then the sample session ends; If the response status indicates that at least one business objective has not been executed, then the response status and the prospective customer service reply are used together as the next round of user input for the sample session, and the model optimization process continues.

5. The method according to claim 1, characterized in that, Also includes: Get the current user input in the sample session; The dialogue analysis module is used to parse and process the current user input, and output the user's intent and emotional state. The user's intent and emotional state are identified as user characteristics.

6. The method according to claim 1, characterized in that, Also includes: Based on the current user input, determine the corresponding gold-medal customer service input; Based on the current user input and the gold-medal customer service input, the strong model matches the corresponding standard dialogue strategy from the strategy library.

7. The method according to claim 1, characterized in that, The model optimization is performed on the pre-trained customer service model and the pre-trained expert model based on the gold-medal customer service input, the near-customer service response, the decision sensitivity factor, the standard dialogue strategy, and the combined dialogue strategy. The model optimization process is continuously iterated to generate near-customer service models and near-expert models; including: Based on the standard dialogue strategy, the decision sensitivity factor, and the combined dialogue strategy, an expert reward function is constructed; and based on the input of the top customer service representative and the response of the quasi-customer service representative, a customer service reward function is generated. The expert reward function and the customer service reward function are weighted and processed to construct the target reward function; Construct corresponding loss functions for the pre-trained customer service model and the pre-trained expert model respectively; generate customer service loss function and expert loss function; wherein, the customer service loss function includes at least: policy gradient loss, policy compliance loss, contrast loss, and diversity loss; the expert loss function includes at least: policy loss and entropy term loss; The pre-trained customer service model is trained based on the policy gradient method; when the corresponding customer service loss function tends to be minimized, the long-term cumulative reward of the target reward function is maximized, thus generating a quasi-customer service model. The pre-trained expert model is trained using the policy gradient method; when the corresponding expert loss function approaches its minimum, the long-term cumulative reward of the objective reward function is maximized, thus generating a quasi-expert model.

8. The method according to claim 1, characterized in that, The model optimization is performed on the pre-trained customer service model and the pre-trained expert model based on the gold-medal customer service input, the near-customer service response, the decision sensitivity factor, the standard dialogue strategy, and the combined dialogue strategy. The model optimization process is continuously iterated to generate near-customer service models and near-expert models; including: Based on the standard dialogue strategy, the decision sensitivity factor, and the combined dialogue strategy, an expert reward function is constructed; based on the input of the top customer service representative and the response of the quasi-customer service representative, a customer service reward function is generated. Construct corresponding loss functions for the pre-trained customer service model and the pre-trained expert model respectively, generating a customer service loss function and an expert loss function; wherein, the customer service loss function includes at least: policy gradient loss, policy compliance loss, contrast loss, and diversity loss; the expert loss function includes at least: policy loss and entropy term loss; The pre-trained customer service model is trained based on the policy gradient method; when the customer service loss function tends to be minimized, the long-term cumulative reward of the customer service reward function is maximized, thus generating a quasi-customer service model. The pre-trained expert model is trained using the policy gradient method; when the expert loss function approaches its minimum, the long-term cumulative reward of the expert reward function is maximized, thus generating a quasi-expert model.

9. The method according to claim 7 or 8, characterized in that, The construction of an expert reward function based on the standard dialogue strategy, the decision sensitivity factor, and the combined dialogue strategy includes: Based on the standard dialogue strategy and the combined dialogue strategy, a matching evaluation is performed to generate a strategy matching reward. The combined dialogue strategy is evaluated to generate task completion rewards and dialogue efficiency penalties; Based on the decision sensitivity factor and the combined dialogue strategy, a corrective reward is generated. The strategy matching reward, the task completion reward, the correction reward, and the dialogue efficiency penalty are weighted and processed to generate an expert reward function.

10. The method according to claim 7 or 8, characterized in that, The function for generating customer service rewards based on the input of the top-performing customer service representative and the response of the junior customer service representative includes: Based on the input of the top customer service representative and the response of the prospective customer service representative, a matching evaluation is performed to generate a response matching reward; The quality of the responses from the prospective customer service representatives is evaluated, and a response quality reward is generated. Using the combined dialogue strategy as a constraint, a pre-trained customer service model generates an endogenous contrastive reward; based on the response matching reward, the response quality reward, and the endogenous contrastive reward, a weighted processing is performed to generate a customer service reward function; Alternatively, based on the input of the top-performing customer service representative and the response of the prospective customer service representative, a matching evaluation is performed to generate a response matching reward; an endogenous contrast reward corresponding to the response of the prospective customer service representative is obtained; the quality of the response of the prospective customer service representative is evaluated to generate a response quality reward; the combined dialogue strategy is used as a constraint to pre-train a customer service model to generate an endogenous contrast reward; and a weighted average is performed on the response matching reward, the response quality reward, and the endogenous contrast reward to generate a customer service reward function.

11. The method according to claim 1, characterized in that, Also includes: Obtain the current user input in the session to be tested, and the user features corresponding to the current user input; The user features are matched using a quasi-expert model to output a combined dialogue strategy; Using the combined dialogue strategy as a constraint, a quasi-customer service model is used to generate quasi-customer service responses corresponding to the user characteristics.

12. The method according to claim 1, characterized in that, Also includes: Obtain the dialogue turn when each business objective is completed in the test session; wherein, the test session includes at least one business objective; For any one of the at least one business objectives: determine the completion indicator of the business objective based on the dialogue rounds, the decay coefficient, and the evaluation value corresponding to the completion of the business objective; Based on the completion metrics of each of the at least one business objective, the probability that the session to be tested will be successfully recommended is determined.

13. A dialogue device based on multi-agent driving, characterized in that, Applications in customer service dialogue systems; including: The first acquisition module is used to acquire user features, top customer service input, and standard dialogue strategy corresponding to the current user input in the sample session; wherein, the user features include at least user intent and emotional state; The prediction module is used to pre-train an expert model to predict the combined dialogue strategy and decision sensitivity factor corresponding to the user characteristics; the generation module is used to pre-train a customer service model to generate a quasi-customer service response corresponding to the user characteristics, using the combined dialogue strategy as a constraint. The model training module is used to optimize the pre-trained customer service model and the pre-trained expert model based on the gold-medal customer service input, the quasi-customer service response, the decision sensitivity factor, the standard dialogue strategy, and the combined dialogue strategy; and to continuously iterate the model optimization process to generate quasi-customer service models and quasi-expert models.

14. A computer-readable medium, characterized in that, It stores a computer program that, when executed by a processor, implements the dialogue method as described in any one of claims 1 to 12.

Images