Scripted dialogue flow control method
By controlling the large language model to generate a set of dialogue frameworks and combining them with user responses to calculate similarity, the problem of omissions and repetitions in the large language model's autonomous random selection of dialogues for examination is solved, thus achieving the completeness and controllability of dialogue examination.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN CITY ZHITONG INFORMATION TECHNOLOGY CO LTD
- Filing Date
- 2026-01-19
- Publication Date
- 2026-05-26
AI Technical Summary
Existing questioning designs that rely on large language models to randomly select questions are prone to issues such as missing or repeated questions.
By configuring dialogue scenario information and dialogue scripts, a large language model is called to generate a dialogue framework for scenario roles and associate it with the dialogue scripts. The dialogue framework dialogue script set is managed according to the correspondence between the scenario role dialogue framework and the dialogue scripts. Similarity calculation is performed based on the user's current round of replies to determine the next dialogue script to be examined. The large language model is then called to generate dialogue text output to control the logic of the dialogue script examination process.
This ensures the completeness and orderliness of the verbal communication test, avoids missing or duplicate questions, improves the controllability of the test, simplifies the task of the large language model, and reduces the reliance on complex prompts.
Smart Images
Figure CN121542396B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and language processing technology, and in particular to a method for controlling the dialogue flow of speech assessment. Background Technology
[0002] The rapid development of artificial intelligence technology, especially the breakthrough in Large Language Models (LLMs), has made highly human-like role-playing dialogues possible. In the field of professional skills training, using LLMs to simulate conversations with specific roles (such as customers) and users (such as salespeople) has become an efficient method for skills training and assessment. In this application scenario, LLMs simulate real customer conversations and pose standard script questions to users, thereby dynamically assessing their mastery of professional scripts, their on-the-spot adaptability, and their communication skills. However, traditional human-computer dialogue mainly relies on LLMs to autonomously and randomly determine when and which script questions to ask. This design of script assessment, where the LLM autonomously and randomly selects questions, is prone to problems such as missed questions and repeated questions. Summary of the Invention
[0003] The technical problem to be solved by the present invention is to provide a method for controlling the dialogue process of dialogue assessment, so as to solve the problems of question omission and repeated questioning in the existing dialogue assessment design that uses a large language model to autonomously and randomly select questions.
[0004] To solve the above-mentioned technical problems, the purpose of this invention is achieved through the following technical solution: A method for controlling the dialogue flow of script assessment is provided, comprising the following steps: configuring dialogue scenario information and scripts, wherein the scripts include questions and answers; calling a large language model to generate a scenario role dialogue framework based on the dialogue scenario information and scripts, and associating it with the scripts; managing the scripts according to the order of the scenario role dialogue frameworks based on the correspondence between the scenario role dialogue frameworks and the scripts, thereby obtaining a dialogue framework script set; calculating the similarity based on the dialogue framework script set output by the large language model and the received user's current round reply, determining the next script to be assessed; and generating and outputting the dialogue text based on the next script to be assessed using the large language model.
[0005] The beneficial technical effects of this invention are as follows: The dialogue flow control method of this invention, by calling a large language model to generate a scene role dialogue framework based on the configured dialogue scene information and dialogue, and associating it with the configured dialogue, manages the dialogue according to the order of the scene role dialogue framework based on the correspondence between the scene role dialogue framework and the dialogue, obtaining a dialogue framework dialogue set. Similarity calculation is performed based on the received user's current round reply to determine the next dialogue to be examined. Based on the next dialogue to be examined, the large language model is called to generate dialogue text and output it. This decouples the dialogue examination flow control logic from the large language model, allowing precise control by the control program. This ensures that all preset dialogues are examined completely, orderly, and without repetition, resulting in stronger controllability. It effectively avoids and reduces missed and repeated questions, and can intelligently filter covered questions, making the dialogue logic more reasonable. It also simplifies the task of the large language model and reduces reliance on complex prompts. Attached Figure Description
[0006] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0007] Figure 1 This is a flowchart illustrating the dialogue process control method for assessing verbal communication provided in an embodiment of the present invention. Detailed Implementation
[0008] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0009] Please see Figure 1 As shown, Figure 1 This is a flowchart illustrating the dialogue process control method for script assessment provided in an embodiment of the present invention. The dialogue process control method for script assessment includes the following steps:
[0010] Step S10: Configure dialogue scenario information and dialogue scripts, wherein the dialogue scripts include questions and answers;
[0011] Step S20: The large language model generates scene role dialogue frameworks based on the dialogue scenario information and scripts, and then associates them with the scripts. Based on the correspondence between the scene role dialogue frameworks and scripts, the scripts are managed according to the order of the scene role dialogue frameworks, resulting in a dialogue framework script set. The dialogue scenario information includes the dialogue background and dialogue roles. A scene role dialogue framework refers to a dialogue framework based on the dialogue roles within the dialogue background. For example, if the dialogue background is about selling goods, the dialogue roles include salesperson and customer, then the scene role dialogue framework is a dialogue framework for salesperson and customer. In this case, the large language model acts as the customer, and the user acts as the salesperson. The dialogue role assumed by the large language model based on the dialogue background is the customer role, specified by prompts. The large language model can also act as the questioner in the scripts to examine the user's responses. The dialogue framework script set is an ordered script set, where each element is the collection of scripts corresponding to each sequence number of the scene role dialogue framework. The dialogue framework dialogue set is obtained through a large language model. It is a structured dialogue set that is readable and usable by the engineering code logic (program). The order of the dialogue framework dialogue set corresponds to the order of the scene role dialogue frameworks in the scene role dialogue framework list.
[0012] Step S30: Based on the dialogue framework set output by the large language model, calculate the similarity with the current round of user responses to determine the next dialogue to be examined. Then, use the large language model to generate dialogue text and output it based on the next dialogue to be examined.
[0013] Among them, Large Language Model (LLM) refers to a deep learning model trained on a large amount of text data, which can generate natural language text or understand the meaning of language text. Dialogue scene information and dialogue configuration can be configured by writing prompt words for the LLM. The prompt words can specify the role of the LLM based on the dialogue scene, so as to call the LLM to generate a scene role dialogue framework. The dialogue flow control method for script assessment decouples the script assessment process from the large language model. It generates scene role dialogue frameworks based on configured dialogue scenario information and scripts, and associates these frameworks with the configured scripts. Then, based on the correspondence between the scene role dialogue frameworks and the scripts, the scripts are managed in the order of the scene role dialogue frameworks, resulting in a dialogue framework script set. Similarity calculations are performed using the received user responses in the current round to determine the next script to be assessed. Based on the next script to be assessed, the large language model is invoked to generate and output the dialogue text. This process, involving a control program invoking the large language model to generate the dialogue framework script set and performing similarity calculations using the user responses in the current round to determine the next script to be assessed, decouples the script assessment flow control logic from the large language model. The control program precisely controls this process, ensuring that all preset scripts are assessed completely, orderly, and without repetition. This enhances controllability, effectively reduces missed or duplicate questions, and intelligently filters covered questions, making the dialogue logic more reasonable. It also simplifies the task of the large language model and reduces reliance on complex prompts.
[0014] Specifically, step S10, namely configuring the dialogue scenario information and dialogue script, includes:
[0015] Determine the dialogue background information; specifically, by determining the dialogue background information according to the requirements of the dialogue assessment task, the specific business scenario in which the simulated dialogue takes place is established, thus setting the context boundaries for the dialogue. Determining the dialogue background information includes: identifying the product or service, and setting a basic scenario; the basic scenario can be a customer's initial inquiry, a customer follow-up, or a customer complaint, etc.
[0016] Setting dialogue roles; specifically, setting dialogue roles according to the requirements of the dialogue assessment task to clarify the positions, goals, and behavioral tendencies of both parties in the dialogue, ensuring the authenticity of the dialogue simulation. Setting dialogue roles includes: defining a customer role and defining a user role; defining the customer role to describe the customer's needs and personality, and defining the user role to determine the user's position and dialogue assessment objectives; configuring dialogue scenario information by determining dialogue background information and setting dialogue roles.
[0017] Compile a script list. This involves creating a script list based on the requirements of the script assessment task and the defined dialogue roles. This breaks down the assessment requirements into specific and measurable question-and-answer pairs, serving as a direct basis for subsequent control logic and questioning. The script list compilation includes: listing questions and writing standard answers for each question; the listed questions are those that customers might ask in the corresponding dialogue scenarios, extracted from business experience; the standard answers include reference answers or key points for response.
[0018] Specifically, step S20, which involves calling the large language model to generate a scene role dialogue framework based on the dialogue scene information and the dialogue script, and then associating it with the dialogue script, includes:
[0019] Structured prompts are constructed based on dialogue scenario information and questions in the dialogue script. The dialogue scenario information includes the dialogue background and dialogue roles. Structured prompts are constructed by integrating the dialogue background and dialogue roles and using all questions in the dialogue script as key nodes.
[0020] Based on structured prompts, a list of scene role dialogue frames is generated by calling the large language model through the Application Programming Interface (API). The list of scene role dialogue frames is an ordered list, and the order of the scene role dialogue frames in the list is the development order of the scene.
[0021] Associate the scene character dialogue frames in the scene character dialogue frame list with the dialogue scripts to obtain the correspondence between the scene character dialogue frames and the dialogue scripts.
[0022] Specifically, in step S20, the dialogue scripts associated with the scene role dialogue framework are managed according to the order of the scene role dialogue framework to obtain a dialogue framework script set, including:
[0023] The dialogue scripts in the dialogue script list are managed according to the order of the scene role dialogue frames in the scene role dialogue frame list, based on the correspondence between the obtained scene role dialogue frames and the dialogue scripts, to obtain a dialogue frame dialogue script set.
[0024] The generated scene role dialogue framework list contains four scene role dialogue frameworks, denoted as a1 to a4, and the script list contains nine scripts, denoted as q1 to q9. The correspondence between scene role dialogue frameworks and scripts is as follows: a1 is associated with q1 and q2, a2 with q3, q4, and q5, a3 with q6 and q7, and a4 with q8 and q9. The resulting ordered dialogue framework script set can be denoted as {[q1,q2],[q3,q4,q5],[q6,q7],[q8,q9]}. Since the scene role dialogue frameworks and scripts use a non-sequential mapping relationship, the scripts in the set corresponding to a single scene role dialogue framework are dynamically ordered and not bound by a predefined order.
[0025] Specifically, step S30, which involves calculating the similarity of the dialogue framework script set output by the large language model with the received user's current round response to determine the next script to be examined, includes:
[0026] Obtain the set of dialogue framework utterances output by the large language model;
[0027] Receive the user's reply in the current round;
[0028] The similarity calculation is performed between the received user's current round response and the answers to the dialogue responses in the unexamined dialogue set within the dialogue framework's dialogue script set. This determines whether the user's current round response answered the question requiring a response in the current round and whether it prematurely answered a question that hasn't yet been asked. Answered dialogue responses are filtered out, resulting in an updated unexamined dialogue script set. This ensures that answered dialogue responses are not re-examined in subsequent rounds, avoiding duplicate questioning. The dialogue framework's dialogue script set is divided based on whether a dialogue response has been answered, into an answered dialogue script set and an unexamined dialogue script set. The unexamined dialogue script set includes the dialogue responses requiring a response in the current round and all dialogue responses that haven't yet been asked. Therefore, the answers to the dialogue responses in the unexamined dialogue script set include the answers to the dialogue responses requiring a response in the current round and the answers to all dialogue responses that haven't yet been asked.
[0029] Based on the current round's scenario-based dialogue framework and a preset selection strategy, the next dialogue to be examined is determined from the updated set of dialogues to be examined.
[0030] Preferably, the step of calculating the similarity between the received user's current round response and the answers to the dialogue responses in the dialogue framework's set of responses to be examined includes:
[0031] The received user's current round reply is split into text to obtain reply clauses; the text splitting can be based on punctuation marks in the text.
[0032] The answers to the dialogues in the set of dialogues to be examined are split into text to obtain the dialogue answer clauses;
[0033] Calculate the basic similarity between the response clause and the prepared answer clause;
[0034] The phrase with the highest basic similarity value to the reply phrase is selected as the similar phrase to that reply phrase, and the corresponding phrase and basic similarity value are recorded. The recorded phrase is the phrase corresponding to the similar phrase to the reply phrase, and the recorded basic similarity value is the basic similarity between the reply phrase and its similar phrase. This basic similarity value can serve as the similarity between the user's current round of replies and the corresponding phrase, providing a basis for determining the next phrase to be examined.
[0035] Preferably, a bag-of-words model can be used to calculate the basic similarity between the response clause and the word answer clause. The basic similarity includes character overlap similarity, word overlap similarity, pinyin similarity, edit distance similarity, and / or n-gram similarity.
[0036] Preferably, determining whether the user's current response answers the questions required for the current round and whether it answers questions that have not yet been asked includes:
[0037] Compare the similarity score with a preset current answer similarity threshold to determine whether the user's current response answers the question that needs to be answered in the current round.
[0038] By comparing the similarity score with a preset threshold for similarity between answers to unassigned questions, it can be determined whether the user's current response has preemptively answered a question that has not yet been asked.
[0039] Specifically, when the similarity is not lower than the preset current answer similarity threshold, it is determined that the user's current round reply has answered the question that needs to be answered in the current round; when the similarity is not lower than the preset unanswered question similarity threshold, it is determined that the user's current round reply has answered the question that has not yet been asked.
[0040] Preferably, the preset current answer similarity threshold is less than the preset non-question answer similarity threshold. Specifically, the preset current answer similarity threshold can be 0.6, the preset non-question answer similarity threshold can be 0.8, and the preset non-question answer similarity threshold is greater than the preset current answer similarity threshold. This ensures that the assessment of non-question responses is more rigorous than that of responses with questions, thereby improving the accuracy of the response assessment.
[0041] Specifically, after obtaining the updated set of dialogues to be examined, the dialogue examination process control method further includes:
[0042] If a user's response in the current round does not answer the question that requires a response in the current round, then the next question to be examined is determined to be the question that requires a response in the current round.
[0043] Preferably, the filtering of already answered statements to obtain an updated set of statements to be examined includes:
[0044] Remove the already answered statements from the statement set to be examined, resulting in an updated statement set. The already answered statements include statements that need to be answered in the current round and statements for which answers have already been given.
[0045] Specifically, the step of calculating the similarity between the received user's current round response and the answers to the dialogue responses in the dialogue framework's set of responses to be examined further includes:
[0046] The Aho-Corasick algorithm is used to identify fragments of the user's current round response that correspond to the responses in the dialogue framework's utterance set. The similarity between each fragment and the responses in the corresponding utterance set is calculated based on the order of the utterances in the dialogue framework. The Aho-Corasick algorithm is a highly efficient multi-pattern string matching algorithm.
[0047] Specifically, the process of determining the next dialogue item to be examined from the updated set of dialogue items, based on the current round's scenario-role dialogue framework and a preset selection strategy, includes:
[0048] Based on the current round's scenario-role dialogue framework, obtain the unexamined dialogues associated with the corresponding scenario-role dialogue framework from the updated set of dialogues to be examined. Perform semantic matching based on the received user's current round response, and select the dialogue with the highest matching relevance from the obtained unexamined dialogues associated with the corresponding scenario-role dialogue framework as the next dialogue to be examined.
[0049] Preferably, before performing semantic matching based on the received user's current round of responses, the dialogue flow control method for assessing the script further includes:
[0050] If the updated set of dialogue options does not contain any unexamined dialogue options related to the current round's scenario / role dialogue framework, then the next unexamined dialogue option related to the scenario / role dialogue framework is retrieved from the updated set of dialogue options. Specifically, if the updated set of dialogue options does not contain any unexamined dialogue options related to the current round's scenario / role dialogue framework (meaning all dialogue options related to the current scenario / role dialogue framework have been examined), then the next unexamined dialogue option related to the scenario / role dialogue framework is selected sequentially.
[0051] Specifically, before performing semantic matching based on the received user's current round of responses, the dialogue flow control method for evaluating the script further includes:
[0052] When the updated set of dialogues to be examined is empty, the next set of dialogues to be examined is set to empty. An empty updated set of dialogues to be examined means that all dialogues have been examined.
[0053] Specifically, after selecting the dialogue with the highest matching relevance from the unexamined dialogues in the obtained corresponding scene role dialogue framework as the next dialogue to be examined, the dialogue examination process control method further includes:
[0054] If semantic matching does not yield a valid result, a random selection algorithm is used to obtain the next dialogue to be examined from the unexamined dialogues associated with the corresponding scene role dialogue framework.
[0055] Preferably, the step of generating and outputting the dialogue text based on the next utterance to be examined using a large language model specifically involves:
[0056] Based on the next utterance to be examined, a large language model invocation instruction is generated, the large language model is invoked to execute the corresponding invocation instruction, the dialogue text is generated, and the dialogue text is output for the next round of dialogue interaction.
[0057] Specifically, when the next dialogue to be examined is the dialogue that needs to be answered in the current round, a large language model calling instruction is generated based on the preset dialogue follow-up requirements and the next dialogue to be examined. In this case, the large language model calling instruction is a follow-up question calling instruction. When the next dialogue to be examined is a new dialogue, a large language model calling instruction is generated based on the preset dialogue questioning requirements and the next dialogue to be examined. In this case, the large language model calling instruction is a question calling instruction. When the next dialogue to be examined is empty, a large language model calling instruction is generated based on the preset dialogue summary requirements. In this case, the large language model calling instruction is a dialogue summary calling instruction.
[0058] Among them, new dialogue refers to dialogue that is different from the dialogue to be answered in the current round and has not been examined. The preset dialogue follow-up requirements include follow-up prompts, the preset dialogue question requirements include question prompts, and the preset dialogue summary requirements include summary prompts.
[0059] In summary, the dialogue flow control method for skills assessment of this invention generates a scene role dialogue framework based on the configured dialogue scenario information and dialogue scripts by calling a large language model. This framework is then associated with the configured dialogue scripts. Based on the correspondence between the scene role dialogue framework and the associated dialogue scripts, the dialogue scripts are managed in the order of the scene role dialogue frameworks to obtain a dialogue framework dialogue script set. Similarity calculation is performed based on the received user's current round response to determine the next dialogue script to be assessed. The large language model is then called to generate and output the dialogue text based on the next dialogue script to be assessed. This process decouples the dialogue assessment flow control logic from the large language model, allowing precise control by the control program. This ensures that all preset dialogue scripts are assessed completely, orderly, and without repetition, resulting in stronger controllability. It effectively avoids and reduces missed or repeated questions, intelligently filters covered questions, makes the dialogue logic more reasonable, simplifies the task of the large language model, and reduces reliance on complex prompts.
[0060] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for controlling the dialogue flow in a script assessment, characterized in that, Includes the following steps: Configure dialogue scenario information and scripts, including questions and answers; After generating a scene role dialogue framework based on the dialogue scenario information and the dialogue script by calling the large language model, the dialogue script is associated with the dialogue script. According to the correspondence between the scene role dialogue framework and the dialogue script, the dialogue script is managed in the order of the scene role dialogue framework to obtain a dialogue framework dialogue script set. The dialogue scenario information includes the dialogue background and the dialogue role. The scene role dialogue framework refers to the dialogue framework based on the dialogue role in the dialogue background. Based on the dialogue framework set output by the large language model, similarity is calculated by combining the received user's current round response to determine the next dialogue to be examined. Based on the next dialogue to be examined, the large language model is called to generate dialogue text and output it. The process of calculating similarity based on the dialogue framework set output by the large language model and the received user's current round response to determine the next dialogue to be examined includes: Obtain the set of dialogue framework utterances output by the large language model; Receive the user's reply in the current round; The similarity calculation is performed between the received user's current round reply and the answers of the words in the set of words to be examined in the dialogue framework. It is then determined whether the user's current round reply answers the question of the word to be answered in the current round and whether it answers the question of the word to be examined before it is asked. Answered words are filtered out to obtain an updated set of words to be examined. Based on the current round's scenario-role dialogue framework and combined with the preset selection strategy, the next dialogue to be examined is determined from the updated set of dialogues to be examined. The process of calling the large language model to generate a scene role dialogue framework based on dialogue scene information and utterances, and then associating it with the utterances, includes: Structured prompts are constructed based on dialogue scenario information and questions in the dialogue script; the dialogue scenario information includes the dialogue background and the dialogue characters; Based on structured prompts, a list of scene character dialogue frames is generated by calling a large language model through an application programming interface; the list of scene character dialogue frames is an ordered list of dialogue frames, and the order of the scene character dialogue frames in the list is the development order of the scene. Associate the scene character dialogue frames in the scene character dialogue frame list with the dialogue scripts to obtain the correspondence between the scene character dialogue frames and the dialogue scripts.
2. The dialogue flow control method for script assessment according to claim 1, characterized in that, The configuration of dialogue scenario information and dialogue scripts includes: Determine the background information of the conversation; Set the dialogue characters; Compile a list of conversation scripts.
3. The dialogue flow control method for script assessment according to claim 1, characterized in that, The step of calculating the similarity between the received user's current round response and the answers to the responses in the dialogue framework's set of responses to be examined includes: The received user's current round of replies is split into text to obtain reply clauses; The answers to the dialogues in the set of dialogues to be examined are split into text to obtain the dialogue answer clauses; Calculate the basic similarity between the response clause and the prepared answer clause; Select the phrase with the highest basic similarity value to the reply phrase as the similar phrase to the reply phrase, and record the corresponding phrase and basic similarity value.
4. The dialogue process control method for script assessment according to claim 3, characterized in that, The basic similarity includes character overlap similarity, word overlap similarity, pinyin similarity, and / or edit distance similarity.
5. The dialogue flow control method for script assessment according to claim 1, characterized in that, The determination of whether the user's current response answers the questions required for the current round and whether the user has answered questions that have not yet been asked includes: Compare the similarity score with a preset current answer similarity threshold to determine whether the user's current response answers the question that needs to be answered in the current round. By comparing the similarity score with a preset threshold for similarity between answers to unassigned questions, it can be determined whether the user's current response has preemptively answered a question that has not yet been asked.
6. The dialogue flow control method for script assessment according to claim 5, characterized in that, The preset current answer similarity threshold is less than the preset non-question answer similarity threshold.
7. The dialogue process control method for script assessment according to claim 5, characterized in that, After obtaining the updated set of dialogues to be examined, the dialogue examination process control method further includes: If a user's response in the current round does not answer the question that requires a response in the current round, then the next question to be examined is determined to be the question that requires a response in the current round.
8. The dialogue process control method for script assessment according to claim 1, characterized in that, The scenario-based dialogue framework for the current round, combined with a preset selection strategy, determines the next dialogue to be examined from the updated set of dialogues to be examined, including: Based on the current round's scenario-role dialogue framework, obtain the unexamined dialogues associated with the corresponding scenario-role dialogue framework from the updated set of dialogues to be examined. Perform semantic matching based on the received user's current round response, and select the dialogue with the highest matching relevance from the obtained unexamined dialogues associated with the corresponding scenario-role dialogue framework as the next dialogue to be examined.