Virtual user simulation method and system for dialogue test

Abstract

The invention provides a virtual user simulation method and system for dialogue testing, and the method comprises the steps: obtaining a plurality of pieces of effective multi-round dialogue data from a to-be-tested system, and carrying out the multi-label labeling, so as to construct a basic sample data set containing user portrait labels, the key sample data set further comprises user emotion tags; constructing a group optimization strategy model, performing cold start training based on the basic sample data set, and performing intensive training by using a multi-dimensional composite reward function based on the key sample data set to obtain a dialogue generation model with an emotion cognition-verbal skill expression decoupling architecture; and taking a historical round of dialogue of the to-be-tested system aiming at the current interaction as input of a dialogue generation model, and generating a next round of user reply consistent with the user emotion and the user portrait. According to the invention, deep, efficient and automatic adversarial testing is carried out on the task-oriented dialogue system, so that system defects which are difficult to find by a traditional testing method are effectively exposed.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and specifically to a virtual user simulation method and system for dialogue testing. Background Technology

[0002] With the rapid development of Large Language Model (LLM) technology, task-oriented dialogue systems (Task- Oriented Dialogue Systems (TOD) are increasingly used in customer service, intelligent assistants, and vertical industry consulting. Traditional methods for evaluating and optimizing the performance of these systems rely on large-scale manual testing or static rule-based automated testing. However, manual testing is costly, inefficient, and struggles to cover long-tail scenarios; rule-based testing lacks flexibility and fails to simulate the language variations and logical transitions of real users in complex interactions. In recent years, data-driven user simulators have become the mainstream approach to address these issues. Early user simulators were mostly based on sequence-to-sequence (Seq2Seq) models or reinforcement learning (RL) frameworks, generating user responses by mimicking historical dialogue data. While these methods have achieved some degree of automated interaction, they still have significant limitations in practical applications.

[0003] First, existing simulators generally lack consistency between cognition and expression. Traditional end-to-end models directly map dialogue history to responses, ignoring changes in the user's internal psychological state (such as satisfaction and emotional fluctuations). This often leads to logical disconnects in simulators; for example, when the system fails to resolve a problem, the simulator outputs a thank-you message, failing to accurately reflect the user's negative emotions or persistent behavior when encountering service disruptions. Second, long-term intent management capabilities are insufficient. Real users often hold complex intent stacks in dialogues and dynamically adjust priorities based on system feedback. Existing models often forget the initial goal after multiple rounds of interaction or prematurely jump to subsequent topics before completing prerequisite tasks, causing the dialogue flow to deviate from the real scenario. Furthermore, personalized and diverse simulations are relatively rigid. Existing methods often struggle to finely control the differentiated expressions of different personalities (such as "impatient" and "mild") within the same model. Meanwhile, high-quality labeled dialogue data (especially data containing user emotional states and intent jump logic) is extremely scarce. Traditional supervised fine-tuning (SFT) or reinforcement learning methods that rely on reward models (such as GRPO) often require massive amounts of data or high training costs, making it difficult to achieve efficient alignment with sparse, high-value samples.

[0004] Therefore, how to construct a user simulator that has explicit cognitive reasoning capabilities, can maintain a long-term intent stack, and can achieve precise control of personality and emotion through efficient reinforcement learning strategies with a small number of high-value samples has become a technical challenge that urgently needs to be solved in the field of dialogue system evaluation. Summary of the Invention

[0005] Therefore, this invention provides a virtual user simulation method and system for dialogue testing, aiming to solve the technical problems of existing virtual user simulators lacking realism, logical consistency, and dynamic emotion simulation capabilities.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: According to a first aspect of the present invention, the present invention provides a virtual user simulation method for dialogue testing, the method comprising: Acquire multiple valid multi-turn dialogue data from the system under test; Multiple labels are applied to each of the effective multi-turn dialogue data to construct a basic sample dataset containing user profile labels, and a key sample dataset containing the user profile labels and user emotion labels; An initial model based on a group optimization strategy is constructed, and the initial model is cold-started and trained on the basic sample dataset to obtain an intermediate model. Based on the key sample dataset, the intermediate model is reinforced and trained using a multi-dimensional composite reward function to obtain a dialogue generation model with a decoupled architecture of emotion cognition and verbal expression. The system under test takes historical dialogue data from the current interaction as input, uses the dialogue generation model to determine the user's emotion, and generates the next round of user response that is consistent with the user's emotion and user profile.

[0007] Furthermore, the step of using the dialogue generation model to determine the user's emotion and generating a next-round user response consistent with the user's emotion and user profile includes: Based on the historical dialogue data, the dialogue generation model is used to analyze the emotional cognitive state that represents the user's emotions. When the historical responses of the system under test continue to cause users to have a negative emotional state, the system generates the next round of user questions that are consistent with the user profile and have increasing emotional intensity, in order to simulate the evolution of user emotions.

[0008] Furthermore, the acquisition of multiple valid multi-turn dialogue data from the system under test includes: Multiple question-and-answer dialogue data from the system under test are collected as raw dialogue data. The original dialogue data is cleaned, filtered, and quality-screened using a large language model, and single-turn original dialogue data is removed to obtain multiple valid multi-turn dialogue data.

[0009] Furthermore, the step of performing multi-label annotation on each of the effective multi-turn dialogue data to construct a basic sample dataset containing user profile labels, and a key sample dataset containing the user profile labels and user emotion labels, includes: The effective multi-turn dialogue data was labeled with user profile tags containing personality type and intent information using a large language model; The effective multi-turn dialogue data is associated with the corresponding user profile tags to obtain the basic sample dataset; Filter a predetermined proportion of basic sample data from the basic sample dataset; determine the user sentiment labels of the basic sample data through manual annotation; the user sentiment includes positive, neutral, and negative. The basic sample data is associated with the corresponding user emotion tags to obtain the key sample dataset.

[0010] Further, the step of performing cold start training on the initial model based on the basic sample dataset to obtain an intermediate model includes: The initial model is trained in a supervised manner using the basic sample dataset to obtain the intermediate model with structured dialogue output capability.

[0011] Furthermore, the composite reward function includes multiple rule-based rewards, specifically including: A format reward characterizing whether the output of the intermediate model conforms to a preset structural format; and / or, A reward for accuracy in determining whether the intermediate model accurately identifies user emotions; and / or, The response emotion cognition-verbal expression consistency reward characterizes whether the emotional polarity of the next round of user responses output by the intermediate model is consistent with the user's emotion; and / or, A personality fit reward characterizing whether the next round of user responses output by the intermediate model matches the personality type in the user profile; and / or, The intermediate model output represents whether the user's emotion is positive or neutral, and whether it detects the intent advancement reward for the next intent information in the intent stack.

[0012] Furthermore, based on the key sample dataset, the intermediate model is reinforced and trained using a multi-dimensional composite reward function to obtain a dialogue generation model with a decoupled architecture of emotion cognition and verbal expression. The reward value of each rule reward is determined using a reward assignment rule that matches each of the rule rewards. The composite reward function is calculated by weighted summation of the reward values ​​of each rule reward. The intra-group dominance function is calculated using the composite reward function, and its mathematical expression is as follows:

[0013] Among them, A i The within-group advantage function represents the importance of the i-th model's output; r i This represents the composite reward function corresponding to the output of the i-th model; This represents the mean of the composite reward function output by all models within the group; This represents the standard deviation of the composite reward function output by all models within the group; The intra-group advantage function is applied to the loss function of the intermediate model, and the intermediate model that minimizes the loss function is taken as the trained dialogue generation model.

[0014] Furthermore, the reward assignment rules include: A positive formatting reward is given when the output of the intermediate model contains standard thinking tags and response tags; a negative formatting reward is given when the output of the intermediate model does not contain standard thinking tags or response tags; and / or, When the user emotion determined by the intermediate model matches the manually labeled user emotion tag, a positive reward is given for accuracy; when the user emotion determined by the intermediate model does not match the manually labeled user emotion tag, a negative reward is given for accuracy; and / or, When the emotional polarity of the next round of user responses output by the intermediate model matches the user's emotion, a positive reward for consistency between emotional cognition and verbal expression is given; when the emotional polarity of the next round of user responses output by the intermediate model does not match the user's emotion, a negative reward for consistency between emotional cognition and verbal expression is given; and / or, Determine whether the next round of user responses output by the intermediate model matches the personality type in the user profile. If they match, a positive personality fit reward is given; if they do not match, a lower or negative personality fit reward is given. And / or, When the user's emotion is positive or neutral, if the output of the intermediate model contains the next intent information of the intent stack or there is no next intent, a positive intent advancement reward is given; if the output of the intermediate model does not detect the next intent information of the intent stack, a negative intent advancement reward is given.

[0015] Furthermore, determining whether the next round of user responses output by the intermediate model matches the personality type in the user profile includes: When the personality type is the first preset personality type, determine whether the next round of user responses containing pause words and / or hesitant tone features match the personality type in the user profile; and / or; When the personality type is the first preset personality type, determine whether the next round of user responses that meet the preset conditions of average sentence length and / or pause word frequency match the personality type in the user profile; and / or; When the personality type is the second preset personality type, it is determined that the next round of user responses containing the same core demand repeatedly reiterated in the historical dialogue data are consistent with the personality type in the user profile.

[0016] According to a second aspect of the present invention, the present invention provides a virtual user simulation system for dialogue testing, the system comprising: The data acquisition module is used to acquire multiple valid multi-turn dialogue data from the system under test; The labeling module is used to perform multi-labeling on each of the effective multi-turn dialogue data to construct a basic sample dataset containing user profile labels, and a key sample dataset containing the user profile labels and user emotion labels; The model building module is used to build an initial model based on the group optimization strategy, and to perform cold start training on the initial model based on the basic sample dataset to obtain an intermediate model; The reinforcement training module is used to reinforce the intermediate model based on the key sample dataset and using a multi-dimensional composite reward function to obtain a dialogue generation model with a decoupled architecture of emotion cognition and verbal expression. The simulation generation module is used to take the historical rounds of dialogue data of the system under test for the current interaction as input, use the dialogue generation model to determine the user's emotion, and generate the next round of user response that is consistent with the user's emotion and user profile.

[0017] The present invention, by adopting the above technical solution, has at least the following beneficial effects: This invention proposes a virtual user simulation method for dialogue testing. It involves acquiring multiple valid multi-turn dialogue data from the system under test; labeling each valid multi-turn dialogue data with multiple tags to construct a basic sample dataset containing user profile tags, and a key sample dataset containing both user profile tags and user emotion tags; constructing an initial model based on a group optimization strategy; performing cold-start training on the initial model based on the basic sample dataset to obtain an intermediate model; and using a multi-dimensional composite reward function to reinforce the intermediate model based on the key sample dataset to obtain a dialogue generation model with a decoupled architecture of emotion cognition and verbal expression. Using historical dialogue data from the system under test for the current interaction as input, the dialogue generation model is used to determine the user's emotion and generate the next round of user response consistent with the user's emotion and user profile. This constructs a user simulator with explicit cognitive reasoning capabilities, the ability to maintain a long-range intent stack, and the ability to achieve precise control of personality and emotion through efficient reinforcement learning strategies with a small number of high-value samples. This simulator enables in-depth, efficient, and automated adversarial testing of task-oriented dialogue systems, effectively exposing system defects that are difficult to detect using traditional testing methods.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a virtual user simulation method for dialogue testing according to an embodiment of the present invention is shown. Figure 2 The diagram shows a signaling timing diagram of the interaction between a dialogue generation model provided in an embodiment of the present invention and the system under test. Figure 3 A schematic diagram of the structure of a virtual user simulation system for dialogue testing provided in an embodiment of the present invention is shown. Detailed Implementation

[0021] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0022] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0023] This invention provides a virtual user simulation method for dialogue testing, such as... Figure 1 As shown, it may include at least the following steps S101~S105: Step S101: Obtain multiple valid multi-turn dialogue data from the system under test.

[0024] The system to be tested in this embodiment of the invention can be an online TOD system, namely a task-oriented dialogue system. Specifically, multiple question-and-answer dialogue data from the system to be tested can be collected as raw dialogue data; the raw dialogue data can be cleaned, filtered, and quality-screened using a large language model, and single-turn raw dialogue data can be removed to obtain multiple valid multi-turn dialogue data.

[0025] In practical applications, we can first extract 100,000 real, multi-turn question-and-answer dialogues between users (passengers) and the online TOD system. Then, a large language model is used to perform initial quality screening of the dialogues, removing a large amount of noise, dialogues lacking substantial interaction, or those with confused logic. Subsequently, to focus on meaningful multi-turn interactions, we further eliminate independent question-and-answer dialogues in a single turn, retaining only multi-turn dialogues containing two or more rounds of interaction. After the above screening process, we can finally construct a dataset containing 50,000 high-quality multi-turn dialogues, which will serve as the basis for subsequent model training.

[0026] Step S102: Perform multi-label annotation on each valid multi-turn dialogue data to construct a basic sample dataset containing user profile labels, and a key sample dataset containing user profile labels and user emotion labels.

[0027] Understandably, this annotation step aims to provide a foundation for subsequent supervised learning and reinforcement training. In an optional embodiment, this step may specifically include: using a large language model to annotate effective multi-turn dialogue data with user profile labels containing personality type and intent information; associating the effective multi-turn dialogue data with the corresponding user profile labels to obtain a basic sample dataset; selecting a predetermined proportion of basic sample data from the basic sample dataset; determining user emotion labels for the basic sample data through manual annotation; and associating the basic sample data with the corresponding user emotion labels to obtain a key sample dataset.

[0028] In this embodiment of the invention, the user profile may include personality type and intent information. Personality type refers to classifying users into various preset personality profiles based on characteristics such as tone, word choice, and speech rate in the dialogue, such as "reasonable," "quick-witted," "hesitant," or "stubborn." Intent information refers to the extracted preset intent stack, which involves analyzing the entire dialogue flow to identify a series of pending intents that the user wishes to complete and organizing them into an ordered list. For example, a user's intent stack might be "check order status - request order cancellation - apply for a refund - complain about driver service attitude," thus obtaining a preset intent stack containing pending intents.

[0029] In practical applications, user profiling is performed on all valid multi-turn dialogue data to obtain a basic sample dataset containing user profile labels. To provide high-quality reward signals for the reinforcement learning phase, 4,000 high-value dialogues are carefully selected from the aforementioned 50,000 valid multi-turn dialogues. These high-value dialogues are typically representative of the scenario; for example, 2,000 dialogues where the user's final request is perfectly resolved (high satisfaction), and 2,000 dialogues where the user's problem remains unresolved or is repeatedly and ineffectively addressed (extreme dissatisfaction). For each response in these 4,000 dialogues, a human annotator evaluates it, providing a three-way judgment of satisfaction with the previous system response, corresponding to positive, neutral, and negative user emotions. Specifically, the three-factor judgment levels include: "Perfect," indicating that the response fully satisfies the user's current intention, allowing the current inquiry to end and prompting follow-up questions; "Neutral," indicating that the response advances the conversation to some extent but may not completely resolve the issue, allowing for normal and polite follow-up questions; and "Stalling," indicating that the response is ineffective, repetitive, or off-topic, causing the conversation to stall, allowing for feedback with a urging or impatient tone. This yields a key sample dataset containing user profile labels and user emotion labels.

[0030] After completing the above data preparation and labeling, we proceed to the model training stage shown in steps S103 to S104.

[0031] Step S103: Construct an initial model based on the group optimization strategy, and perform cold start training on the initial model based on the basic sample dataset to obtain an intermediate model.

[0032] Step S104: Based on the key sample dataset, the intermediate model is reinforced and trained using a multi-dimensional composite reward function to obtain a dialogue generation model with a decoupled architecture of emotion cognition and verbal expression.

[0033] The core of steps S103 and S104 lies in constructing a dialogue generation model with a decoupled architecture of "emotion cognition - verbal expression". The dialogue generation model is configured to generate an intermediate representation of its internal cognitive state before generating the final natural language dialogue response. Specifically, the model's output is designed as a specific type of structured text containing a... <thought>Part and one <response>Part of it. Among them, <thought>This part is an intermediate representation, the core of which is the aforementioned three-dimensional judgment of response satisfaction; while <response>Part of it is based on <thought>The dialogue response generated by the cognitive judgments in the part is highly consistent with the emotion and content.

[0034] The following is an example of model input: [System Prompt]: You are a person, and you are also a ride-hailing passenger. [Persona]: Quick-witted and outspoken (Impatient) [Intent Stack]: [Driver no-show (incomplete), cancellation fee issue (incomplete)] [Dialogue History]:Q1A1Q2A2Q3A3 [System Last Reply]: "A: Please wait, I am querying..." (Assuming this is the 3rd repetition) The following is an example of the mode output: <thought> Status: Stalling Reason: The system is repeating the same problem without solving it. Strategy: to urge, to express dissatisfaction Intent stack check: < / thought> <response> Have you finished your investigation? Don't waste my time! For model training, this embodiment of the invention first uses a basic sample dataset to perform supervised fine-tuning training (SFT) on the initial model to obtain an intermediate model with structured dialogue output capabilities. It should be noted that during model training, judgments and generation must be based on complete dialogue history data, not just the last turn. The goal of this stage is to enable the model to initially learn to generate dialogues containing... <thought>and <response>Understand the basic format of tags and the fundamental relationships between conversation history, user personality, and intent stack.

[0035] Furthermore, following supervised fine-tuning, the intermediate model is further trained using 4000 rounds of finely labeled high-value dialogue data (i.e., the key sample dataset). This step aims to enable the model to learn more complex anthropomorphic interaction logic, particularly accurate cognitive judgment and emotional expression capabilities. Specifically, the GRPO model employs a multi-dimensional composite reward function calculated in real-time during training.

[0036] Specifically, the composite reward function can include multiple rule-based rewards: for example, a format reward R that characterizes whether the output of the intermediate model conforms to a preset structural format. format The accuracy of the intermediate model's judgment of user sentiment is represented by the reward R. judgment The emotional polarity of the next round of user responses, representing the output of the intermediate model, is consistent with the user's emotions. This is indicated by the consistency between response emotion cognition and verbal expression, and the reward R. consistency The reward R represents the consistency between the next round of user responses from the intermediate model output and the personality type in the user profile. persona The output of the intermediate model represents whether the user's emotion is positive or neutral, indicating whether the model has detected the next intent information in the intent stack. The intent advancement reward R represents this. intent .

[0037] Furthermore, the reward value of each rule reward can be determined by using the reward assignment rules that match the rewards of each rule.

[0038] Specifically, this can be achieved when the output of the intermediate model contains standard thought labels. <thought>and reply tags <response>When the output of the intermediate model does not contain standard thinking or response tags, a positive (e.g., +1) format reward is given; when the output of the intermediate model does not contain standard thinking or response tags, a negative (e.g., -1) format reward is given.

[0039] To ensure the simulator "recognizes the situation," user sentiment (extracted) can be used as an intermediate model for assessment. <thought>When the user sentiment (Status) in the intermediate model matches the manually labeled user sentiment (Ground Truth: Perfect / Neutral / Stalling), a positive (e.g., +2) reward is given for accuracy; when the user sentiment judged by the intermediate model does not match the manually labeled user sentiment, a negative (e.g., -1) reward is given for accuracy.

[0040] As the core reward, when the emotional polarity of the next round of user responses output by the intermediate model matches the user's current status, a positive (e.g., +1) reward for consistency between emotional cognition and verbal expression is given; when the emotional polarity of the next round of user responses output by the intermediate model does not match the user's current status, a negative (e.g., -1) reward for consistency between emotional cognition and verbal expression is given. For example, if <thought>The Status in the detection is Stalling. <response>The emotional polarity is assessed to determine if it is negative / aggressive, for example, whether the content contains words expressing negative emotions such as questioning or urging; if the Status is Perfect, then the detection... <response>Does it contain positive words such as confirmation or gratitude? This setting is a mandatory requirement in model design; the generated response must strictly follow the emotion-oriented ternary judgment. For example, "Stalling" must be followed by an impolite follow-up question to ensure the authenticity of the emotional evolution.

[0041] To ensure that the user's response matches the user's persona in historical conversations, the system determines whether the next round of user responses output by the intermediate model matches the personality type in the user profile. If they match, a positive (e.g., +0.5) persona consistency reward is given; otherwise, a lower or negative (e.g., 0 or -0.5) persona consistency reward is given. Embodiments of this invention can be based on different personality types categorized for users, such as "reasonable," "quick-witted," "hesitant," or "stubborn." In practical applications, when the personality type is "hesitant and indecisive," the next round of user responses containing pause words (e.g., "um...") are judged to match the personality type in the user profile. When the personality type is "quick-witted and outspoken," the next round of user responses that meet the preset conditions (preferring shorter, more direct, and concise sentences) are judged to match the personality type in the user profile. When the personality type is "stubborn and inflexible," the next round of user responses that contain the same core demand repeatedly reiterated in historical dialogue data are judged to match the personality type in the user profile. When the personality type is "reasonable and understanding," the next round of user responses will prefer polite and objective expressions.

[0042] To ensure that the generated dialogue response helps the user complete their task, a positive intent advancement reward is given when the user's emotion is positive or neutral, provided the intermediate model's output contains information about the next intent in the intent stack or if there is no next intent. Conversely, a negative intent advancement reward is given if the intermediate model's output does not detect information about the next intent in the intent stack. In other words, when... <thought>When the Status is Perfect or Neutral, check <response>If the next intent in the intent stack is detected, the reward value is increased by 1; if no next intent is detected, the reward value is decreased by 1.

[0043] During model training, for each input, different outputs from the same group (G=8 or 6) can be sampled. For each output, the reward value for each rule is determined using the above reward assignment rules, and then a weighted sum is performed. =w1×R format +w2×R judgment +w3×R consistency +w4×R persona +w5×R intent The composite reward function for each model output can be calculated mathematically as follows:

[0044] Among them, A i The within-group advantage function represents the importance of the i-th model's output; r i This represents the composite reward function corresponding to the output of the i-th model; This represents the mean of the composite reward function output by all models within the group; This represents the standard deviation of the composite reward function output by all models within the group.

[0045] Furthermore, the within-group advantage function can be applied to the loss function of the intermediate model, and the intermediate model that minimizes the loss function is taken as the trained dialogue generation model. The model loss function can be defined as:

[0046] Among them, L GRPO The loss function is represented by G; the number of generated answers is represented by π. θ (o i |q) represents the current policy distribution, indicating the model output o generated by the current model under condition q. i The probability of π old (o i |q) represents the old policy distribution, indicating that the model output was generated under condition q in the previous iteration. i The probability of A; i This indicates the measurement of the model output o. i The importance of the within-group advantage function; clip(·) represents the clipping function; D represents the constraint parameter; KL KL divergence is represented by β; weight parameter π is represented by β. ref Indicates a reference strategy.

[0047] It is understandable that the intra-group advantage function A i As scalar weights, a value greater than 0 indicates that the model's output is better than the group average, and the loss function encourages the model to increase the probability of generating that output. A value less than 0 indicates that the model's output is lower than the group average, and the loss function inhibits the model from increasing the probability of generating that output. The loss function updates the model parameters by maximizing an objective function that includes a pruning function, a policy probability ratio, and an advantage function. A KL divergence penalty term prevents new policies from deviating too far from the supervised fine-tuning initial policy, thus ensuring training stability. In this way, the model is guided to generate more outputs that yield high combinatorial rewards, thereby quickly aligning complex anthropomorphic interaction logic on a small number of high-value samples.

[0048] In this embodiment of the invention, the training of the dialogue generation model is determined by minimizing the loss function, and the model enters the interaction simulation stage with the system under test, i.e., step S105.

[0049] Step S105: Take the historical dialogue data of the system under test for the current interaction as input, use the dialogue generation model to judge the user's emotion, and generate the next round of user response that is consistent with the user's emotion and user profile.

[0050] like Figure 2 The diagram shows the signaling sequence diagram for the interaction between the dialogue generation model and the system under test. In the initial dialogue phase, a preset intent stack (e.g., "The driver did not arrive on time, request cancellation and refund") and a user personality (e.g., "Quick-witted") can be set for the virtual user. Based on the first pending intent in the preset intent stack, the dialogue generation model uses a heuristic method to generate the first sentence from the seed database and sends it to the system under test.

[0051] In each subsequent round of interaction, after receiving a response from the system under test, the dialogue generation model analyzes the emotional cognitive state (ternary judgment) representing the user's emotions based on all historical dialogue data, that is, it judges the virtual user's satisfaction with the latest response from the system under test. Based on the ternary judgment result, it generates the next round of user questions that are completely consistent with the user's emotions and user profile. In addition to considering the ternary judgment state, the dialogue generation model also considers the completion progress of the initial intent stack when generating the next round of user questions. That is, in a satisfied or neutral state, it can actively advance unfinished intents.

[0052] Understandably, the dialogue generation model trained in the above way has biological stress response. That is, when the answer of the system under test is obviously inconsistent with the passenger's wishes and continues to cause the user to have a negative emotional cognitive state, it generates the next round of user questions that are consistent with the user profile and have increasing emotional intensity, so as to simulate the evolution of the user's emotions. For example, it gives a clear overreaction (giving a set, gradually increasing anger value), which is more like a real person, ensuring the robustness and authenticity of the simulated behavior.

[0053] For example, regarding the intent to "request a refund," the system under test provides irrelevant or invalid responses in two consecutive rounds. In the first round, the system responds with "It's my pleasure to help you. How can I help you?", which the dialogue generation model might interpret as... <thought> Status: neutral< / thought> Because the conversation didn't progress but also didn't completely stray from the topic, it generated... <response> Hello, as I just mentioned, I want a refund. How do I do that?< / response> In the second round, the system under test again replied, "We provide a variety of services, including…", which was a completely invalid response. At this point, the system under test judged it as… <thought> Status: stalling< / thought> Because this is continuous negative feedback, the user's emotional state (e.g., quantifiable as a cumulative negative emotion value) increases accordingly. Consequently, the dialogue generation model generates a response with progressively increasing emotional intensity, consistent with a "quick-witted" personality, such as... <response> Are you even listening? I just want a refund! Tell me exactly what to do, don't give me any other information!< / response> If invalid responses are still received, the response may be escalated to achieve automated stress testing and adversarial testing of the system under test, effectively exposing its shortcomings in handling users' negative emotions and complex issues.

[0054] As an optional embodiment of the present invention, in order to improve testing efficiency, a dialogue round target can be set for the dialogue generation model, encouraging the completion of core intent testing within a set range such as 6 rounds or 12 rounds.

[0055] This invention provides a virtual user simulation method for dialogue testing, which, compared with the prior art, offers at least the following advantages: 1) Improved the realism and logic of the simulation: By using a "cognition-expression" decoupling architecture, the model is forced to make internal state judgments before expressing them, which solves the problem of inconsistency in logic of traditional end-to-end models. This makes the behavior of virtual users more consistent with real human logic, and the explicit cognitive state makes the motivation of the simulator's behavior more interpretable. By forcing the model to output "three-element judgment (satisfaction state)" before outputting "response", the "explicitization of implicit state" is realized. 2) Reduced training costs and technical barriers: The rule-based reward reinforcement learning strategy proposed in this invention can efficiently utilize a small amount of high-value labeled data to complete model alignment, avoiding the need to train expensive and unstable independent reward models, and significantly reducing the technical implementation cost of building high-quality virtual users; 3) Achieved deep automated adversarial testing: Through dynamic intent stack management and emotional evolution loop, the simulator has "anger value accumulation" and personality specificity, and can automatically conduct stress tests and adversarial tests, thereby more realistically and comprehensively exposing the boundary defects and robustness issues of the system under test when dealing with complex scenarios and users with negative emotions.

[0056] Furthermore, as Figure 1 In specific implementation, embodiments of the present invention provide a virtual user simulation system for dialogue testing, such as... Figure 3 As shown, the system may include: a data acquisition module 310, a labeling module 320, a model building module 330, a reinforcement training module 340, and a simulation generation module 350.

[0057] The data acquisition module 310 can be used to acquire multiple valid multi-turn dialogue data from the system under test. The labeling module 320 can be used to perform multi-label annotation on each valid multi-turn dialogue data to build a basic sample dataset containing user profile labels, as well as a key sample dataset containing user profile labels and user emotion labels. The model building module 330 can be used to build an initial model based on a group optimization strategy, perform cold start training on the initial model based on a basic sample dataset, and obtain an intermediate model. The reinforcement training module 340 can be used to reinforce the intermediate model based on key sample datasets and utilize a multi-dimensional composite reward function to obtain a dialogue generation model with a decoupled architecture of emotion cognition and verbal expression. The simulation generation module 350 can be used to take the historical rounds of dialogue data of the system under test for the current interaction as input, use the dialogue generation model to judge the user's emotions, and generate the next round of user response that is consistent with the user's emotions and user profile.

[0058] It should be noted that other corresponding descriptions of the functional modules involved in the virtual user simulation system for dialogue testing provided in this embodiment of the invention can be found in [reference needed]. Figure 1 The corresponding description of the method shown will not be repeated here.

[0059] Those skilled in the art will clearly understand that the specific working process of the systems, devices, modules and units described above can be referred to the corresponding process in the foregoing method embodiments. For the sake of brevity, it will not be repeated here.

[0060] Furthermore, the functional units in the various embodiments of the present invention can be physically independent of each other, or two or more functional units can be integrated together, or all functional units can be integrated into one processing unit. The integrated functional units described above can be implemented in hardware, or in software or firmware.

[0061] Those skilled in the art will understand that if the integrated functional unit is implemented in software and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or all or part of it, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computing device (e.g., a personal computer, server, or network device) to execute all or part of the steps of the methods described in the embodiments of the present invention when running the instructions. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] Alternatively, all or part of the steps of the foregoing method embodiments can be implemented by hardware (such as a computing device, personal computer, server, or network device) related to program instructions. The program instructions can be stored in a computer-readable storage medium. When the program instructions are executed by the processor of the computing device, the computing device executes all or part of the steps of the methods described in the various embodiments of the present invention.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that within the spirit and principles of the present invention, modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the corresponding technical solutions to depart from the protection scope of the present invention.< / response> < / thought> < / response> < / response> < / thought> < / thought> < / response> < / thought> < / response> < / thought> < / response> < / thought> < / response> < / thought> < / response> < / thought>

Claims

1. A method for virtual user simulation for conversational testing, characterized in that, The method comprises: obtaining a plurality of effective multi-turn conversation data from a system to be tested; multi-label labeling each of the effective multi-turn conversation data to construct a basic sample data set containing user portrait labels and a key sample data set containing the user portrait labels and user emotion labels; constructing an initial model based on a group optimization strategy, cold-start training the initial model based on the basic sample data set to obtain an intermediate model; based on the key sample data set, using a multi-dimensional composite reward function to reinforce training of the intermediate model to obtain a conversation generation model with emotion cognition-speech expression decoupling architecture; using the conversation generation model to judge the user emotion and generate the next round of user replies consistent with the user emotion and user portrait based on the historical round conversation data of the current interaction of the system to be tested.

2. The method of claim 1, wherein, The use of the conversation generation model to judge the user emotion and generate the next round of user replies consistent with the user emotion and user portrait comprises: based on the historical conversation data, using the conversation generation model to analyze the emotion cognition state representing the user emotion; when the historical replies of the system to be tested continuously lead to the user producing a negative emotion cognition state, generating the next round of user questions consistent with the user portrait and increasing in emotion intensity to simulate user emotion evolution.

3. The method of claim 1, wherein, The obtaining of the plurality of effective multi-turn conversation data from the system to be tested comprises: collecting a plurality of one-question-one-answer conversation data from the system to be tested as original conversation data; using a large language model to clean, filter and quality screen the original conversation data and eliminate single-turn original conversation data to obtain a plurality of effective multi-turn conversation data.

4. The method of claim 1, wherein, The multi-label labeling of each of the effective multi-turn conversation data to construct a basic sample data set containing user portrait labels and a key sample data set containing the user portrait labels and user emotion labels comprises: using a large language model to label the effective multi-turn conversation data with user portrait labels containing personality type and intent information; associating the effective multi-turn conversation data with the corresponding user portrait labels to obtain the basic sample data set; screening a preset proportion of the basic sample data in the basic sample data set; determining the user emotion labels of the basic sample data through artificial labeling; the user emotion includes positive, neutral and negative; associating the basic sample data with the corresponding user emotion labels to obtain the key sample data set.

5. The method of claim 1, wherein, The cold-start training of the initial model based on the basic sample data set to obtain an intermediate model comprises: using the basic sample data set to perform supervised fine-tuning training on the initial model to obtain the intermediate model with structured conversation output capability.

6. The method of claim 1, wherein, The composite reward function comprises a plurality of rule rewards, specifically comprising: a format reward representing whether the output of the intermediate model conforms to a preset structure format; and / or, a judgment accuracy reward representing whether the user emotion judged by the intermediate model is accurate; and / or, a reply emotion cognition-utterance expression consistency reward representing whether the sentiment polarity of the next round of user reply output by the intermediate model is consistent with the user emotion or not; and / or, a character type coincidence reward representing whether the next round of user reply output by the intermediate model is consistent with the character type in the user portrait or not; and / or, an intention promotion reward representing whether the output of the intermediate model detects the next intention information in the intention stack when the user emotion is positive or neutral.

7. The method of claim 6, wherein, The intermediate model is trained by using a multi-dimensional composite reward function based on the key sample dataset, so as to obtain a dialogue generation model with emotion cognition-utterance expression decoupling architecture. The reward values of the rule rewards are determined by using reward assignment rules matched with the rule rewards, and the composite reward function is calculated by weighted summation of the reward values of the rule rewards. The in-group advantage function is calculated by using the composite reward function, and the mathematical expression is as follows: wherein A i represents an intra-group advantage function measuring the importance of the i-th model output; r i represents a composite reward function corresponding to the i-th model output; represents the mean of the composite reward functions of all model outputs within the group; represents the standard deviation of the composite reward functions of all model outputs within the group; The in-group advantage function is applied to the loss function of the intermediate model, and the intermediate model with minimized loss function is taken as the trained dialogue generation model.

8. The method of claim 6, wherein, The reward assignment rules include: a positive format reward is given when the output of the intermediate model contains standard thinking labels and reply labels, and a negative format reward is given when the output of the intermediate model does not contain standard thinking labels or reply labels; and / or, a positive judgment accuracy reward is given when the user emotion judged by the intermediate model is consistent with the user emotion label artificially annotated, and a negative judgment accuracy reward is given when the user emotion judged by the intermediate model is not consistent with the user emotion label artificially annotated; and / or, a positive emotion cognition-utterance expression consistency reward is given when the sentiment polarity of the next round of user reply output by the intermediate model is consistent with the user emotion, and a negative emotion cognition-utterance expression consistency reward is given when the sentiment polarity of the next round of user reply output by the intermediate model is not consistent with the user emotion; and / or, a positive character type coincidence reward is given when the next round of user reply output by the intermediate model is consistent with the character type in the user portrait, and a lower or negative character type coincidence reward is given when the next round of user reply output by the intermediate model is not consistent with the character type in the user portrait; and / or, a positive intention promotion reward is given when the next intention information of the intention stack is included in the output of the intermediate model or there is no next intention when the user emotion is positive or neutral, and a negative intention promotion reward is given when the next intention information of the intention stack is not detected in the output of the intermediate model.

9. The method of claim 8, wherein, The judgment of whether the next round of user reply output by the intermediate model is consistent with the character type in the user portrait includes: when the character type is a first preset character type, it is judged that the next round of user reply containing pause words and / or hesitation tone features is consistent with the character type in the user portrait; and / or; when the character type is a first preset character type, it is judged that the next round of user reply with average sentence length and / or pause word frequency satisfying a preset condition is consistent with the character type in the user portrait; and / or; When the personality type is a second preset personality type, it is determined whether a next round of user reply containing a same core appeal repeatedly reiterated in the historical dialogue data conforms to the personality type in the user portrait.

10. A virtual user simulation system for conversational testing, characterized by The system comprises: a data acquisition module configured to acquire a plurality of pieces of valid multi-round dialogue data from a system to be tested; a label annotation module configured to perform multi-label annotation on each of the valid multi-round dialogue data to construct a basic sample data set containing a user portrait label and a key sample data set containing the user portrait label and a user emotion label; a model construction module configured to construct an initial model based on a group optimization strategy, perform cold start training on the initial model based on the basic sample data set, and obtain an intermediate model; a reinforcement training module configured to perform reinforcement training on the intermediate model based on the key sample data set by using a multi-dimensional composite reward function to obtain a dialogue generation model with an emotion cognition-speech expression decoupling architecture; a simulation generation module configured to use the dialogue generation model to judge user emotion and generate a next round of user reply consistent with the user emotion and user portrait by taking historical round dialogue data of the system to be tested for current interaction as input.

Images