Interaction model test method, customer service robot test method and electronic equipment

Abstract

The invention discloses an interaction model testing method, a customer service robot testing method and electronic equipment, and relates to the field of large model technologies and model testing. The method comprises the steps that historical interaction data of an interaction model are acquired, and the historical interaction data are used for representing data generated by multiple rounds of interaction between the interaction model and a user in historical interaction events; the historical interaction data are input into the large script generation model, an interaction scene script is generated through the large script generation model, and the interaction scene script guides the interaction model to conduct interaction testing by describing the demand information of the user and the interaction scene; based on the interaction scene script, guiding the questions to generate a large model, and performing multi-round interaction with the interaction model to obtain test interaction data; and testing the interaction capability of the interaction model based on the test interaction data to obtain a test result. According to the method and the device, the technical problem of relatively poor test effect on the model in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of large model technology and model testing, and specifically, to a testing method for an interactive model, a testing method for a customer service robot, and electronic equipment. Background Art

[0002] At present, with the introduction of large models, the service capabilities of customer service robots have been significantly enhanced, but they are also facing testing challenges. Traditional testing methods have limitations in dealing with the changing dialogue results dynamically generated by customer service robots, maintaining dialogue coherence, and accurately simulating user questioning habits. It is difficult to fully test the performance of dialogue models such as customer service robots in complex dialogue scenarios.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a testing method for an interactive model, a testing method for a customer service robot, and an electronic device, so as to at least solve the technical problem of poor testing effect on the model in the related art.

[0005] According to one aspect of an embodiment of the present application, a testing method for an interaction model is provided, including: obtaining historical interaction data of the interaction model, wherein the historical interaction data is used to represent data generated by multiple rounds of interactions between the interaction model and users in historical interaction events; inputting the historical interaction data into a script generation big model, and using the script generation big model to generate an interaction scenario script, wherein the interaction scenario script guides the interaction model to perform interaction testing by describing the user's demand information and the interaction scenario; guiding the question generation big model to perform multiple rounds of interactions with the interaction model based on the interaction scenario script to obtain test interaction data; and testing the interaction ability of the interaction model based on the test interaction data to obtain test results.

[0006] According to another aspect of the embodiment of the present application, a testing method for a customer service robot is also provided, including: obtaining historical conversation data of the customer service robot, wherein the historical conversation data is used to represent data generated by multiple rounds of conversations between the customer service robot and a user in a historical conversation event; inputting the historical conversation data into a script generation big model, and using the script generation big model to generate a conversation scenario script, wherein the conversation scenario script guides the customer service robot to conduct a conversation test by describing the user's demand information and the conversation scenario; based on the conversation scenario script, guiding the question generation big model to conduct multiple rounds of conversations with the customer service robot to obtain test conversation data; and testing the conversation ability of the customer service robot based on the test conversation data to obtain a test result.

[0007] According to another aspect of an embodiment of the present application, a testing method for an interaction model is also provided, including: responding to an input instruction acting on an operation interface, and displaying historical interaction data of the interaction model on the operation interface, wherein the historical interaction data is used to represent data generated by multiple rounds of interactions between the interaction model and the user in historical interaction events; responding to a processing instruction acting on the operation interface, and displaying a test result of the interaction model on the operation interface, wherein the test result is obtained by testing the interaction ability of the interaction model based on the test interaction data, and the test interaction data is generated based on an interaction scenario script to guide a question generation model to perform multiple rounds of interactions with the interaction model, and the interaction scenario script is generated by inputting the historical interaction data into the script generation model using the script generation model, and the interaction scenario script guides the interaction model to perform interaction testing by describing the user's demand information and the interaction scenario.

[0008] According to another aspect of an embodiment of the present application, a testing method for an interaction model is also provided, including: obtaining historical interaction data of the interaction model by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes historical interaction data, and the historical interaction data is used to represent data generated by multiple rounds of interactions between the interaction model and the user in historical interaction events; inputting the historical interaction data into a script generation model, and using the script generation model to generate an interaction scenario script, wherein the interaction scenario script guides the interaction model to perform interaction testing by describing the user's demand information and the interaction scenario; based on the interaction scenario script, guiding the question generation model to perform multiple rounds of interaction with the interaction model to obtain test interaction data; testing the interaction ability of the interaction model based on the test interaction data to obtain test results; and outputting the test results by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the test results.

[0009] According to another aspect of the embodiments of the present application, a computing device is further provided, including: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present application is executed when the program is running.

[0010] According to another aspect of the embodiments of the present application, an electronic device is also provided, including: a memory storing an executable program; a processor connected to the memory via a bus and used to run the program, wherein the method of each embodiment of the present application is executed when the program is running.

[0011] According to another aspect of an embodiment of the present application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in each embodiment of the present application.

[0012] According to another aspect of the embodiments of the present application, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present application is implemented.

[0013] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present application is implemented.

[0014] According to another aspect of the embodiments of the present application, a computer program is also provided, and when the computer program is executed by a processor, the methods in the various embodiments of the present application are implemented.

[0015] In the embodiment of the present application, the historical interaction data of the interaction model is obtained, wherein the historical interaction data is used to represent the data generated by the interaction model and the user in multiple rounds of interaction in the historical interaction event; the historical interaction data is input into the script generation model, and the script generation model is used to generate the interaction scenario script, wherein the interaction scenario script guides the interaction model to perform interaction testing by describing the user's demand information and the interaction scenario; based on the interaction scenario script, the big model is guided by the question generation model to perform multiple rounds of interaction with the interaction model to obtain the test interaction data; based on the test interaction data, the interaction ability of the interaction model is tested to obtain the test results, thereby improving the test effect of the interaction model; it is easy to notice that the previous multi-round dialogue information can be obtained to ensure that the test is based on actual user interaction. The script generation model is used to analyze the historical data and generate the interaction scenario script to guide the test. Based on the script guidance, a simulated dialogue is conducted to test the response of the interaction model and collect the test interaction data. Through the test interaction data, the performance of the interaction model is comprehensively evaluated and the test results are output; this process makes full use of the richness of historical data and the guidance of the script generation, and effectively tests the continuous dialogue ability and problem-solving ability of the interaction model by simulating real multi-round dialogues, solves the problem of poor effect of previous testing methods, and improves the test effect. This solves the technical problem of poor testing effect on the model in related technologies.

[0016] It is easy to notice that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1is a schematic diagram of a test application scenario of an interaction model according to an embodiment of the present application;

[0019] Figure 2 is a flow chart of a method for testing an interaction model according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a conversation scoring of a customer service robot according to an embodiment of the present application;

[0021] Figure 4 is a flow chart of a method for testing a customer service robot according to an embodiment of the present application;

[0022] Figure 5 is a flow chart of a testing method of an interaction model according to an embodiment of the present application;

[0023] Figure 6 is a flow chart of a testing method of an interaction model according to an embodiment of the present application;

[0024] Figure 7 is a schematic diagram of a testing device for an interaction model according to an embodiment of the present application;

[0025] Figure 8 is a schematic diagram of a test device for a customer service robot according to an embodiment of the present application;

[0026] Fig. 9 is a schematic diagram of a testing device for an interaction model according to an embodiment of the present application;

[0027] Fig.10 is a schematic diagram of a testing device for an interaction model according to an embodiment of the present application;

[0028] Fig.11 is a structural block diagram of a computing device according to an embodiment of the present application;

[0029] Fig.12 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] The technical solution provided in this application is mainly implemented by large-scale model technology. The large model here refers to a deep learning model with large-scale model parameters, which can usually contain hundreds of millions, tens of billions, hundreds of billions, trillions or even more than 10 trillion model parameters. The large model can also be called a cornerstone model / foundation model (Foundation Model). The large model is pre-trained through large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks, and the model has good generalization ability, such as large-scale language model (Large Language Model, LLM), multi-modal pre-training model (multi-modal pre-training model), etc.

[0033] It should be noted that when the big model is actually applied, the pre-trained model can be fine-tuned through a small number of samples, so that the big model can be applied to different tasks. For example, the big model can be widely used in natural language processing (NLP), computer vision, speech processing and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image caption (IC), image generation, etc. It can also be widely used in natural language processing tasks such as text-based sentiment classification, text summary generation, machine translation, etc. Therefore, the main application scenarios of the big model include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. In the embodiment of the present application, the test of the interactive model by generating a big model through a script and generating a big model through questions in a test scenario is explained as an example,

[0034] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0035] Customer service robots refer to artificial intelligence robots used in customer service systems, which can automatically identify user questions and provide corresponding answers and suggestions.

[0036] A big model refers to a large artificial intelligence model that can be used to handle complex natural language processing tasks such as question answering and dialogue generation.

[0037] A script refers to the script used to guide conversations between customer service robots and users, including conversation scenarios, conversation content, conversation processes, etc.

[0038] User issues refer to problems encountered by users when using the platform, including product quality, delivery time, refunds, etc.

[0039] Emotional changes refer to the changes in users’ emotions when talking to customer service robots, such as anger, disappointment, satisfaction, etc.

[0040] Multi-round dialogue refers to the conversation between the customer service robot and the user, including the raising of questions, answers, and provision of suggestions.

[0041] Conversation scoring refers to scoring the conversation between the customer service robot and the user to evaluate the quality and effectiveness of the conversation.

[0042] At present, with the support of big models, customer service robots are more human and smarter than before. This progress has brought considerable challenges to the testing of dialogues, which are mainly reflected in the following aspects: the same user question may get different answers after being generated by the big model, and the expected value is no longer enumerable, requiring semantic judgment; the big model will guide the description of the question and ask questions in the process of answer generation, and the content of the user's question should change with the guidance of the model. This means that when you want to restore the online dialogue, using the same user question will make the content of the entire dialogue lose coherence; the customer service robot dialogue is multi-round. When you want to test the customer intelligent robot's problem understanding and problem-solving capabilities in a certain scenario, you need to design some user questions by yourself. Due to the limitations of testing thinking, the design of the dialogue content may not conform to the real questioning habits of consumers.

[0043] The current solution calls the backend interface for online conversation requests, provides the system with the same input parameters, and verifies the return effect under the same input parameters. The disadvantage of this solution is that the conversation is in a single-round mode and lacks conversation context information. It can only verify the performance of the customer service robot in one conversation, but the conversation between the user and the customer service robot is usually multi-round, and it may take multiple conversations to clarify a problem. The single-round mode makes it difficult to verify the overall effect of the conversation.

[0044] The target solution can also directly use the questions asked by online users to restore the conversation during the multi-round conversation of the customer service robot. The disadvantage of this solution is that although it is a multi-round mode, during the conversation, due to the changes in the answers returned by the dialogue system (the randomness of the internal model responses and the changes brought about by the model iteration, etc.), the conversation process is incoherent and lacks reference value.

[0045] According to an embodiment of the present application, a testing method for an interactive model is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0046] Considering the huge number of model parameters of the large model and the limited computing resources of the mobile terminal, the above method provided in the embodiment of the present application can be applied to Figure 1 The application scenarios shown are not limited to these. Figure 1 is a schematic diagram of a test application scenario of an interaction model according to an embodiment of the present application. Figure 1 In the application scenario shown, the large model is deployed in the server 10, and the server 10 can be connected to one or more client devices 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client device 20 here may include but is not limited to: a smart phone, a tablet computer, a laptop computer, a PDA, a personal computer, a smart home device, a vehicle-mounted device, etc. The client device 20 can communicate with the user through a graphical user interface to implement the call of the large model, thereby implementing the method provided in the embodiment of the present application.

[0047] In the embodiment of the present application, the system composed of the client device and the server can perform the following steps: the client device performs the step of obtaining the historical interaction data of the interaction model. The server performs the step of inputting the historical interaction data into the script generation model, and using the script generation model to generate the interaction scenario script; based on the interaction scenario script, the question generation model is guided to perform multiple rounds of interaction with the interaction model to obtain test interaction data; based on the test interaction data, the interaction ability of the interaction model is tested to obtain the test result.

[0048] It should be noted that, with the rapid development of high-performance computing units, in other application scenarios, the above method provided in the embodiment of the present application can also be applied to the model all-in-one machine. In an optional embodiment, a plurality of models are built into the model all-in-one machine, and the user can choose to adjust a model as needed to obtain the user's own model, so that the high-performance computing unit built into the model all-in-one machine can directly call the adjusted model to execute the above method provided in the embodiment of the present application. In another optional embodiment, a trained model is built into the large model all-in-one machine, so that the high-performance computing unit built into the model all-in-one machine can directly call the model to execute the above method provided in the embodiment of the present application.

[0049] Furthermore, when users need to train their own models, they can also upload their own data sets through the client, which are sent by the client to the server, so that the server can adjust the pre-trained model with the data set to obtain the user's own model and then deploy it to the production environment. In order to facilitate users' needs for model adjustment, the server can provide complete adjustment tools, development frameworks and processes, and can support multiple adjustment strategies, so that the adjusted model can better adapt to applications in different fields and achieve a high degree of customization.

[0050] Under the above operating environment, this application provides Figure 2 Test methods for the interaction model shown. Figure 2 FIG. 1 is a flow chart of a method for testing an interactive model according to an embodiment of the present application. Figure 2 As shown, the method may include the following steps:

[0051] Step S202, obtaining historical interaction data of the interaction model;

[0052] The historical interaction data is used to represent the data generated by multiple rounds of interactions between the interaction model and the user in historical interaction events.

[0053] The above interaction model is a software system or algorithm designed to understand and respond to user input (such as text, voice, etc.) and to conduct multiple rounds of dialogue. It covers everything from simple question-answering models to complex dialogue generation and management models, and can respond dynamically and coherently based on user questions, instructions or situations, aiming to simulate the natural dialogue process between people and provide a personalized and intelligent interactive experience.

[0054] The interaction model can parse the user's input and understand the intention, emotion and information needs, which is the first step to achieve effective dialogue. This requires the model to have a certain natural language processing capability and be able to recognize and process complex language structures and meanings. In multiple rounds of dialogue, the interaction model needs to keep a memory of the dialogue history, understand the context of the input, and ensure the coherence of the response and the adaptability of the situation. This is an important mark that distinguishes simple question-and-answer systems from complex dialogue systems. Based on the results of understanding, the interaction model can generate contextual and meaningful responses, including but not limited to answering questions, providing suggestions, and emotional comfort, to meet user needs.

[0055] High-quality interaction models have the ability to learn and adapt, and can continuously optimize their understanding and generation strategies based on dialogue feedback and user behavior, providing a more accurate interaction experience that is closer to user needs. In complex dialogue scenarios, the interaction model needs to be able to manage the dialogue process, including guiding the dialogue, switching topics, resolving conflicts, etc., to ensure the smooth progress of the dialogue.

[0056] The above-mentioned interaction models include but are not limited to intelligent customer service, virtual assistants, smart home control, online education platforms, and medical consultation platforms, and this application does not make specific limitations.

[0057] The above historical interaction data refers to a record that covers all the details of previous conversations with the interaction model, including but not limited to user questions, model responses, conversation context information, user emotion changes, etc. These data reflect the actual communication between the interaction model and the user in multiple rounds of conversations, and provide empirical materials for testing.

[0058] The above stages are the preparation for the test process, which involves collecting all records of the interaction model's communication with the user in historical dialogue events. Historical interaction data may include but is not limited to user questions, the interaction model's responses, contextual information during the dialogue, changes in user emotions, etc. These data provide key reference information for script generation and subsequent interaction capability testing, ensuring that the construction of test scenarios is based on actual user behavior and model performance.

[0059] You can collect conversation records of the interactive model in actual operation, including user questions, model responses, contextual information during the conversation, and user emotion changes. By analyzing this data, you can find the advantages and disadvantages of the interactive model in multi-round conversations, providing basic support for subsequent testing.

[0060] For example, a user may first ask about the details of a product and then ask about the return policy. This consistent inquiry pattern and the change in user emotions (from curiosity to eagerness) will be recorded and used for script generation.

[0061] Step S204, inputting the historical interaction data into the script generation model, and using the script generation model to generate an interactive scenario script;

[0062] Among them, the interaction scenario script guides the interaction model to conduct interaction testing by describing the user's demand information and interaction scenarios.

[0063] The above-mentioned script generation model is a type of artificial intelligence model, which is responsible for generating interactive scenario scripts that can guide testing based on historical interaction data. The script contains a detailed description and simulation of user needs and dialogue processes, and is a framework for testing the performance of the interactive model.

[0064] The historical interaction data can be input into the script generation model, and the generated script contains the user demand information and the description of the dialogue process. The role of the script is to provide guidance for subsequent multi-round interaction simulations to ensure that the simulated dialogue scenarios meet the actual needs and communication habits of users. For example, the script will simulate the entire dialogue process from asking for basic product information, expressing dissatisfaction, to the final problem resolution.

[0065] In an optional embodiment, the collected historical interaction data is input into a large script generation model, which analyzes the user needs and dialogue processes in the data and generates one or more sets of interaction scenario scripts. The generation of the script is intended to simulate the user's demand information and interaction process in a specific scenario, provide guidance for the testing of the interaction model, ensure that the test scenario is highly consistent with the actual user needs, and thus more accurately evaluate the performance of the interaction model in real dialogues.

[0066] Step S206, based on the interactive scenario script, the large model is asked questions to perform multiple rounds of interactions with the interactive model to obtain test interaction data;

[0067] The above-mentioned question generation model is also an artificial intelligence model. Its function is to generate simulated user questions in real time based on the script and the context information of the previous round of dialogue for multi-round dialogue testing.

[0068] Based on the interactive scenario script, the question generation model can simulate a conversation with the interactive model. Each round of questions and feedback is based on the content of the previous round of conversation, ensuring the continuity of the conversation and the authenticity of the scene. The test interaction data generated by this process is the key to the subsequent evaluation of the performance of the interactive model. For example, the script sets that the user has questions about the product description. The question generation model simulates the user's questions, the interactive model tries to explain, and the question generation model generates new questions (such as details about quality) based on the explanation until the problem is solved.

[0069] In an optional embodiment, based on the guidance of the script, the question generation model and the interaction model conduct multiple rounds of simulated dialogues, and each round of questions and the response of the interaction model are closely related to ensure the coherence of the dialogue and the authenticity of the situation. The test interaction data generated in this process provides a direct basis for subsequent performance evaluation.

[0070] Step S208: testing the interaction capability of the interaction model based on the test interaction data to obtain a test result.

[0071] The above-mentioned interaction ability test refers to the process of quantitatively evaluating the interaction model's ability to understand user needs, provide reasonable feedback, solve user problems and maintain conversation consistency by simulating the conversation process.

[0072] Based on the test interaction data, the overall performance of the interaction model in multiple rounds of dialogue is evaluated, including problem understanding ability, communication skills, ability to solve user problems, and dialogue coherence and rationality. This evaluation provides direct feedback for optimizing the interaction model. For example, the evaluation is whether the interaction model can accurately identify the user's conversion from product information to return policy, whether the solution it provides is reasonable and meets user needs, and whether the dialogue is natural and coherent.

[0073] In an optional embodiment, the collected test interaction data is used to comprehensively evaluate the interaction model's interaction capabilities, including but not limited to problem understanding, situation adaptation, solution provision, and dialogue coherence. The output of this testing process, i.e., the test results, provides direct feedback information for the optimization and iteration of the interaction model.

[0074] It should be noted that the test interaction data can be evaluated by a pre-set evaluation method to achieve the purpose of testing the interaction ability of the interaction model. Optionally, the test interaction data can be evaluated by a pre-set evaluation model to obtain a test result, or the test interaction data can be output and evaluated by a user to obtain a test result.

[0075] Through the above steps, we can collect historical interaction data, use the script to generate a large model to summarize the characteristics of the conversation, then generate a large model by asking questions to simulate multiple rounds of conversation, and finally use the test interaction data to evaluate the performance of the interaction model. The whole process is closely linked, aiming to fully test the ability of the interaction model by simulating the complexity of actual multiple rounds of conversations to ensure its application effect in real scenarios.

[0076] For example, in a model test scenario where the interaction model is a customer service robot, historical interaction data shows that users often ask for detailed information about the return and exchange policy after purchasing goods. Based on these data, the script generation model generates a script that includes user inquiries from product inquiries to return and exchange process inquiries. The question generation model conducts a simulated dialogue with the customer service robot, starting with questions about product details and gradually transitioning to detailed inquiries about return and exchange policies. Each question and the customer service robot's response are based on the content of the previous round of dialogue, ensuring the consistency and authenticity of the dialogue. After completing multiple rounds of dialogue, the customer service robot is evaluated through an interactive ability test to evaluate the accuracy of understanding, fluency of communication, and effectiveness of problem solving when dealing with user return and exchange policy issues. This testing process not only tests the customer service robot's ability to face specific scenarios, but also ensures the consistency and rationality of the dialogue in complex situations. The final output test results provide key data support for the improvement and improvement of the customer service robot.

[0077] In this way, the application solves the problem of incoherent and unreasonable conversations that may occur during the testing process of the customer service robot, verifies the effectiveness and rationality of the customer service robot in multiple rounds of conversations, and significantly improves the test results.

[0078] Through the above-mentioned testing method, this application effectively solves the problem of incoherent and unreasonable dialogue caused by the diversity of dialogue results during the test process of the dialogue system. The use of historical interaction data and large model technology not only ensures the authenticity and diversity of the test scenarios, but also improves the efficiency and effectiveness of the test, so that the dialogue system can better adapt to the needs of real users and complex and changing dialogue situations. This method has significant benefits for the performance testing and optimization of interactive models such as customer service dialogue robots, intelligent assistants, and virtual customer service, ensuring the stability of the system and the improvement of user experience.

[0079] Through the above steps, the historical interaction data of the interaction model is obtained, wherein the historical interaction data is used to represent the data generated by the interaction model and the user in multiple rounds of interaction in the historical interaction event; the historical interaction data is input into the script generation model, and the script generation model is used to generate the interaction scenario script, wherein the interaction scenario script guides the interaction model to perform interaction testing by describing the user's demand information and the interaction scenario; based on the interaction scenario script, the big model is guided by the question generation model to perform multiple rounds of interaction with the interaction model to obtain the test interaction data; based on the test interaction data, the interaction ability of the interaction model is tested to obtain the test results, thereby improving the test effect of the interaction model; it is easy to notice that the information of the previous multiple rounds of dialogue can be obtained to ensure that the test is based on the actual user interaction. The script generation model is used to analyze the historical data and generate the interaction scenario script to guide the test. Based on the script guidance, a simulated dialogue is conducted to test the response of the interaction model and collect the test interaction data. Through the test interaction data, the performance of the interaction model is comprehensively evaluated and the test results are output; this process makes full use of the richness of historical data and the guidance of the script generation, and effectively tests the continuous dialogue ability and problem-solving ability of the interaction model by simulating real multiple rounds of dialogue, solves the problem of poor effect of previous testing methods, and improves the test effect. This solves the technical problem of poor testing effect on the model in related technologies.

[0080] In the above embodiments of the present application, the question generation big model is guided by the interactive scenario script to perform multiple rounds of interactions with the interactive model to obtain test interaction data, including: in the first round of interaction between the question generation big model and the interactive model, initial question data is generated based on the interactive scenario script, and initial feedback data is generated based on the initial question data using the interactive model; in multiple rounds of interactions after the first round of interaction between the question generation big model and the interactive model, multiple rounds of interactions are guided by the initial feedback data and the interactive scenario script to perform multiple rounds of interactions with the interactive model to obtain multiple rounds of interaction data; and test interaction data is constructed based on the initial question data, the initial feedback data and multiple rounds of interaction data.

[0081] The above initial question data can be the first round of user question information extracted from the interactive scenario script, which is used as the starting point for initiating multiple rounds of dialogue testing. It can be based on the description of user needs and dialogue situations in the script, and is the basis for subsequent multiple rounds of interaction.

[0082] The initial feedback data mentioned above can be the first response generated by the interaction model according to its internal logic and algorithm after receiving the initial question data. This step tests the interaction model's ability to understand and respond to the user's initial question.

[0083] The above multi-round interaction data is the data generated by the continuous dialogue between the question generation model and the interaction model after the first round of interaction. The generation of multi-round interaction data depends on the feedback data of the previous round and the guidance of the script, ensuring the coherence of the dialogue and the authenticity of the situation.

[0084] At the beginning of multiple rounds of interactive testing, the script generation model is used to summarize the user's questioning patterns in specific scenarios and generate initial question data that conforms to the scenario. This process ensures that the test reflects real user behavior. For example, in the scenario where the user inquires about the product return policy, the script generation model will generate questions that the user may ask about return conditions, processes, etc. based on historical data analysis.

[0085] Input the initial question data into the interaction model and observe the feedback data it generates. This step tests the interaction model's initial understanding of the user's question without contextual information. In the past, the user asked about the return policy, and the interaction model needs to be able to correctly understand this requirement and provide relevant information.

[0086] After the first round of interaction, based on the initial feedback data and the guidance of the script, the question generation model continues to generate follow-up questions that match the dialogue context, conducts multiple rounds of interactions with the interaction model, and generates multiple rounds of interaction data. This process simulates the real dialogue process and verifies the performance of the interaction model in the dialogue context. For example, after the user obtains preliminary return policy information, he or she may ask questions about the refund period, whether he or she needs to bear the shipping costs, etc. The question generation model will simulate the user's possible questions based on the preset user behavior in the script and the feedback from the previous round.

[0087] The initial question data, initial feedback data and multiple rounds of interaction data can be integrated to form a complete test interaction data set. This data set is the key material for evaluating the performance of the interaction model and is used for subsequent interaction capability testing.

[0088] In an optional embodiment, initial question data that matches user needs and dialogue context is first generated based on the interactive scenario script, and then the first round of feedback is performed using the interactive model, followed by multiple rounds of interactive cycles, with each round of question generation based on the feedback data and script guidance of the previous round until the simulated dialogue ends. The construction of test interaction data ensures the comprehensiveness and authenticity of the test cases, and provides a detailed data foundation for subsequent interactive capability evaluation.

[0089] For example, in the test of the customer service robot, it is assumed that the script generation model summarizes the common needs and scenarios of users asking about the return policy after purchasing goods. First, the script generation model generates the first round of question data about the return conditions based on the scenario. After receiving this question, the customer service robot generates initial feedback data and explains the basic return policy. Subsequently, based on this feedback and the subsequent guidance in the script, the question generation model generates follow-up data about the refund cycle and return process, and interacts with the customer service robot for multiple rounds. The data of each interaction is collected and eventually constitutes a complete set of test interaction data, which is used to evaluate the customer service robot's interactive ability in handling return policy inquiries.

[0090] This testing method significantly improves the coherence and situational adaptability of the test by simulating the user's real conversation process, avoiding incoherent and unreasonable conversation problems, and ensuring that the test results can truly reflect the interactive capabilities of the customer service robot, thereby providing strong data support for optimizing and upgrading the algorithm model of the customer service robot. Through multiple rounds of interactive testing, the conversation quality of the customer service robot has been improved, better meeting user needs and enhancing user experience.

[0091] Through the above methods, this application not only improves the efficiency and effectiveness of dialogue system testing, but also provides a more scientific and systematic method for the continuous optimization and iteration of customer service robots, especially when dealing with complex and changing user needs and situations. It can more accurately evaluate and improve the robot's interaction capabilities.

[0092] In the above-mentioned embodiment of the present application, the question generation big model is guided to perform multiple rounds of interaction with the interaction model based on the initial feedback data and the interaction scene script to obtain multiple rounds of interaction data, including: inputting the initial feedback data as the preset feedback data into the question generation big model, and using the question generation big model to generate intermediate question data based on the preset feedback data and the interaction scene script; inputting the intermediate question data into the interaction model, using the interaction model to generate intermediate feedback data, and inputting the intermediate feedback data as the preset feedback data into the question generation big model; looping and executing the steps of using the question generation big model to generate intermediate question data based on the preset feedback data and the interaction scene script, inputting the intermediate question data into the interaction model, using the interaction model to generate intermediate feedback data, and inputting the intermediate feedback data as the preset feedback data into the question generation big model, until the end condition of the interaction scene script is reached; and constructing multiple rounds of interaction data based on the intermediate question data and the intermediate feedback data.

[0093] The preset feedback data mentioned above may refer to the feedback information generated by the interaction model in a round of dialogue during the multi-round dialogue simulation process and used as input for the next round of question generation. The preset feedback data is crucial for building a coherent multi-round dialogue, ensuring that the large question generation model can generate reasonable subsequent questions based on the model's previous response.

[0094] The above-mentioned intermediate question data can be the question information generated by the question generation model based on the feedback data of the current round and the guidance of the interactive scenario script during the iteration process of multiple rounds of dialogue simulation. The intermediate question data reflects the user's possible follow-up questions or situation changes in the dialogue process, and is used to promote the progress of the dialogue.

[0095] The above-mentioned intermediate feedback data can be feedback information generated by the interaction model after receiving the intermediate question data. This data not only reflects the interaction model's understanding and response to the intermediate question data, but is also used as input for the next round of question generation to ensure the continuity of the conversation.

[0096] The above-mentioned multi-round interaction data can be composed of intermediate question data and intermediate feedback data, which is a data set reflecting the multi-round dialogue simulation process. This data set is the key material for testing the performance of the interaction model and is used for subsequent interaction capability evaluation.

[0097] Based on the first round of feedback (initial feedback data) of the interaction model and the preset interaction scenario script, the question generation model starts the iterative dialogue simulation. In each round, the question generation model will take the feedback data (initial feedback data or intermediate feedback data) of the previous round as input, and combine it with the interaction scenario and user demand information set in the script to generate the next round of questions (intermediate question data). After that, the intermediate question data is input into the interaction model, and the interaction model generates corresponding feedback (intermediate feedback data). This feedback is recorded and used as the basis for the next round of question generation by the question generation model until the end condition of the interaction scenario script is met. The construction of multiple rounds of interaction data ensures that the test cases cover the complete dialogue process and verifies the performance of the interaction model in multiple rounds of dialogue.

[0098] In the case where the interaction model is a dialogue system, this application solves the common problems of incoherent and unreasonable dialogues in dialogue system testing, and significantly improves the accuracy and efficiency of the test. Taking the test of the customer service robot as an example, suppose the script sets a scenario where the user starts with product consultation and gradually transitions to understanding the return and exchange policy until the conversation ends with satisfaction. In the first round of interaction, the customer service robot gave feedback based on questions about product information. This feedback was used as input for the question generation model to generate specific questions about the return and exchange process. In subsequent rounds of dialogue, each feedback from the customer service robot will prompt the question generation model to generate questions that are closer to the real situation, until the user's problem is solved or the conversation reaches the end condition set by the script.

[0099] This testing process not only verifies the customer service robot's ability to continuously understand and respond to user needs, but also ensures the coherence and logic of the entire conversation process. Through the simulation of multiple rounds of conversations, the effectiveness of the customer service robot in complex conversation scenarios can be more comprehensively evaluated, providing strong data support for improving its conversation quality and user experience. In addition, this method can also proactively discover and fix possible problems in the model, such as misunderstandings of specific problems or irrational solutions, thereby promoting the continuous optimization of the interaction model.

[0100] The test method of this application effectively solves the key problems in the dialogue system test through iterative multi-round dialogue simulation, ensures the authenticity and comprehensiveness of the test cases, and provides a scientific and efficient method for the performance evaluation and optimization of the interaction model. This method is not only suitable for the testing of customer service robots, but can also be widely used in the performance evaluation of various interaction models, contributing to improving the user experience of artificial intelligence dialogue systems.

[0101] In the above embodiment of the present application, the method also includes: establishing a communication connection between a first device where the interaction model is located and a second device where the question generation model is located based on a two-way communication mechanism; and executing, based on the communication connection, a step of guiding the question generation model to conduct multiple rounds of interaction with the interaction model based on an interactive scenario script to obtain test interaction data.

[0102] The above-mentioned two-way communication mechanism is a communication method that allows data to flow freely in two or more directions. The two-way communication mechanism can ensure that the large model for question generation and the interactive model can exchange data in real time and effectively, and support the continuity and fluency of multiple rounds of dialogue. Among them, the two-way communication mechanism can be implemented through a communication protocol (websocke), which is a protocol for full-duplex communication on a single transmission control protocol connection, which can establish a long connection between the client and the server and support real-time data exchange.

[0103] In an optional embodiment, a two-way communication connection can be established between the second device where the question generation model is located and the first device where the interaction model is located through the websocket communication protocol. This connection is the basis for realizing multi-round interaction simulation, ensuring that questions and feedback can be transmitted in real time, simulating the real dialogue process. Next, based on the established communication connection, the question generation model and the interaction model can conduct multiple rounds of dialogue according to the guidance of the interaction scene script. In each round of dialogue, the question generation model generates new questions based on the script information and the previous round of interaction feedback, and sends them to the interaction model through websocket; after receiving the question, the interaction model generates feedback and sends it back to the question generation model through websocket, and this process is repeated until the dialogue ends or the end condition set by the script is reached.

[0104] For example, in the test scenario of the customer service robot, the test engineer first uses websocket to establish a real-time communication connection with the server where the customer service robot is located. Then, according to the pre-designed interactive scenario script about the return and exchange policy, the question generation model begins to generate the first round of questions of the simulated user, such as "Does this product support returns?" and sends it to the customer service robot through websocket. After receiving the question, the customer service robot generates feedback, such as "Yes, as long as the product is unused and the packaging is intact, you can apply for a return within 15 days after receiving the product." This feedback is sent back to the question generation model through websocket, and the model generates the next round of questions based on the feedback and the guidance of the script, such as "What documents are required for returns?" The whole process is repeated until the simulated dialogue ends, and all interactive data is collected for subsequent test analysis. The above values ​​are for example only and are not specifically limited.

[0105] Through the two-way communication mechanism established by websocket, the test method of this application can ensure real-time data exchange between the question generation model and the interaction model, effectively supporting the continuity and fluency of multiple rounds of dialogue. This mechanism overcomes the problems of incoherent dialogue and low test efficiency caused by data transmission delays or unstable communication in traditional testing methods.

[0106] The two-way communication mechanism in this application provides an efficient and stable data exchange method for testing multi-round dialogues, ensuring the continuity of the dialogue and the adaptability of the situation, thereby more accurately evaluating the performance of the interaction model and improving the user experience of the dialogue system. In scenarios such as intelligent customer service, virtual assistants, and online education that require multi-round dialogue interactions, the application of this mechanism is particularly critical. It can significantly improve the accuracy and efficiency of the test and accelerate the process of product iteration and optimization.

[0107] In the above embodiment of the present application, historical interaction data is input into a script generation model, and an interactive scenario script is generated using the script generation model, including: inputting historical interaction data into the script generation model, and using the script generation model to extract a user portrait of the user, wherein the user portrait is used to represent the user's questioning pattern and language style; and using the script generation model to generate an interactive scenario script based on the user portrait and historical interaction data.

[0108] In the field of data analysis and artificial intelligence, user portraits are comprehensive models that represent user characteristics by collecting and analyzing data such as user behavior, preferences, and interaction history. They can capture personalized information such as user question patterns and language styles, helping the system better understand and predict user needs.

[0109] First, the historical interaction dataset is used as input to the script generation model. This historical data may contain multiple rounds of conversations between users in different situations, including user questions, responses from the interaction model, user feedback, and other information. The script generation model analyzes this historical data, extracts the user's questioning patterns and language style features, and constructs a virtual portrait of the user.

[0110] Next, the script generation model uses the constructed user portraits and historical interaction data to start generating interactive scenario scripts. The script generation not only refers to the specific historical dialogue content, but also integrates the common patterns of user questions and specific language styles to ensure that the generated script can accurately simulate the user's actual conversation habits, thereby improving the authenticity and effectiveness of subsequent dialogue tests.

[0111] For example, in the test scenario of the customer service robot, the script generation model analyzed the user's historical conversations in scenarios such as returns and exchanges, product consultations, and logistics tracking, and found that users tend to use a polite tone and provide more details when describing problems. Based on these characteristics, when generating scripts, the script generation model not only simulates specific user questions (such as "The size of the clothes I bought is not suitable, how do I apply for a return?"), but also incorporates the user's language style (such as the polite language used and the way of describing the problem in detail), ensuring the naturalness of the script and the authenticity of the situation.

[0112] By incorporating the concept of user portraits into the generation of interactive scenario scripts, the testing method of this application can significantly improve the authenticity and effectiveness of dialogue testing. The construction of user portraits enables the script generation model to not only reproduce specific historical interaction cases, but also capture and simulate the user's questioning patterns and language style, ensuring that the test scenario is highly consistent with real user needs, thereby more accurately evaluating the interaction model's ability to understand and respond in complex dialogue situations.

[0113] The script generation method of this application constructs user portraits and accurately simulates the user's questioning mode and language style, providing more realistic and diverse scenarios for the testing of interactive models, improving the efficiency and effectiveness of the test, and having significant benefits for improving the user experience and performance of the dialogue system. This method is particularly suitable for testing interactive models such as customer service dialogue robots and intelligent assistants that require high personalization and situational adaptability, ensuring that the model can provide a smooth, natural, and effective dialogue experience when facing real users.

[0114] In the above embodiments of the present application, the interaction ability of the interaction model is tested based on the test interaction data to obtain the test results, including: inputting the test interaction data into the interaction scoring big model, using the interaction scoring big model to score the interaction ability of the interaction model to obtain the scoring results; generating the test results based on the test interaction data and the scoring results.

[0115] The above-mentioned interactive scoring model is an artificial intelligence model specially designed to evaluate the interactive ability of the dialogue system. It is trained based on a large amount of dialogue data and can understand and score key elements in the dialogue, such as question understanding accuracy, response rationality, dialogue coherence, user experience, etc. In this application, the interactive scoring model is used to receive test interaction data, evaluate the performance of the interactive model, and give specific scoring results to provide a basis for subsequent optimization and improvement.

[0116] The above-mentioned test interaction data is simulated conversation data generated during the interaction model testing process, which includes user questions, responses of the interaction model, context information, etc., and is used to evaluate the performance of the interaction model in a specific scenario.

[0117] The above test results are a comprehensive evaluation report generated after the interaction model test is completed, based on the test interaction data and the scoring results of the interaction scoring model. It provides specific indicators of the interaction model performance, such as question understanding score, response quality score, user experience score, etc., to help test engineers quickly locate problems and optimization directions.

[0118] In an optional embodiment, the test interaction data generated in the multi-round dialogue test is input into the interaction scoring model. This data contains the content of multiple rounds of dialogue simulated according to the script and user portrait, including user questions, responses of the interaction model, context information, etc. Then, the interaction scoring model evaluates the interaction ability of the interaction model based on the input test interaction data, including but not limited to the accuracy of question understanding, rationality of response, dialogue coherence, user experience and other indicators, and gives a specific scoring result. Finally, the test system generates a detailed test result report based on the test interaction data and scoring results. This report lists in detail the performance of the interaction model in the multi-round dialogue test, including specific scoring items, existing problems, potential optimization points, etc., providing clear guidance for subsequent model iteration and user experience improvement.

[0119] By introducing the interactive scoring model, the test method of this application can quantitatively evaluate the performance of the interactive model, overcome the limitations of subjective evaluation in traditional test methods, and provide an objective and comprehensive test and scoring system. This not only helps to quickly locate the defects and deficiencies of the model, but also can quantify the performance improvement of different iterative versions, accelerating the optimization of the model and the product iteration process.

[0120] The test method of this application uses a large interactive scoring model to objectively and comprehensively evaluate the test interaction data, generate detailed test results, and provide strong data support and improvement strategies for the performance optimization of the interactive model and the improvement of user experience. This method is particularly suitable for intelligent dialogue systems that require continuous iteration and optimization, such as customer service robots and virtual assistants, ensuring that the model can provide high-quality dialogue services when facing real user needs.

[0121] In the above embodiment of the present application, the interaction ability of the interaction model is scored using the interaction scoring big model to obtain a scoring result, including: using the interaction scoring big model to score the interaction ability of the interaction scoring based on the scoring dimensions to obtain the scoring values ​​corresponding to the scoring dimensions, wherein the scoring dimensions include at least one of the following: risk scoring dimension, accuracy scoring dimension, coherence scoring dimension, integrity scoring dimension, response speed scoring dimension, and scenario adaptability scoring dimension; and determining the scoring result based on the scoring values ​​corresponding to the scoring dimensions.

[0122] In the testing and evaluation of dialogue systems, the above-mentioned scoring dimensions refer to specific standards or indicators used to measure the quality of dialogue or the performance of interaction models. In this application, scoring dimensions such as risk, accuracy, coherence, completeness, response speed, and scenario adaptability are mentioned, which correspond to different aspects of the dialogue. For example, the risk scoring dimension focuses on the negative consequences or potential risks that may be brought about in the dialogue; the accuracy scoring dimension focuses on the accuracy of the interaction model's understanding and response to user questions; the coherence scoring dimension evaluates the logic and fluency of the dialogue content; the completeness scoring dimension ensures that the dialogue covers all necessary information points; the response speed scoring dimension examines the processing speed of the interaction model; and the scenario adaptability scoring dimension evaluates the adaptability and flexibility of the interaction model in different scenarios.

[0123] Among them, the risk scoring dimension is used to evaluate whether the response of the interaction model has potential emotional risks, such as whether the response contains sensitive information, whether it may cause user dissatisfaction, etc. The accuracy scoring dimension is used to measure the accuracy of the interaction model's understanding and response to user questions. The coherence scoring dimension is used to evaluate the contextual relationship and logical coherence in multiple rounds of dialogue. The completeness scoring dimension is used to ensure that the interaction model covers all necessary information points or completes all necessary tasks in the dialogue. The response speed scoring dimension is used to examine the response speed of the interaction model in processing user questions to ensure that the system can provide responses within the time expected by the user. The scenario adaptability scoring dimension is used to evaluate the adaptability and flexibility of the interaction model in different dialogue scenarios, including whether it can correctly identify the scenario and whether it can provide effective responses related to the scenario. Each scoring dimension will receive a specific score value, which comprehensively reflects the comprehensive performance and user experience of the interaction model.

[0124] By scoring the interactive model's interactive capabilities under multiple scoring dimensions, the test method of this application can provide a comprehensive and detailed evaluation perspective, ensuring that the dialogue system can not only respond accurately and coherently when facing users, but also handle dialogues quickly, safely, and adapt to the situation, thereby providing a high-quality user experience. This method overcomes the limitations of single-dimensional evaluation in traditional testing, can promptly discover and solve problems that may exist in many aspects of the interaction model, and accelerates the optimization and iteration of dialogue system performance.

[0125] The multi-dimensional scoring method of this application not only improves the test quality and efficiency of the dialogue system by comprehensively evaluating the performance of the interaction model, but also promotes the continuous optimization of the user experience, which is of great significance for promoting the development and application of intelligent dialogue systems. Especially in areas such as customer service robots and intelligent assistants that require high personalization and situational adaptability, this method can more accurately identify and solve complex problems that the model may encounter in practical applications, ensuring the stability of the system and user satisfaction.

[0126] Figure 3 is a schematic diagram of a conversation scoring of a customer service robot according to an embodiment of the present application, such as Figure 3 As shown, it includes a script generation module, a dialogue simulation module, and a dialogue scoring module.

[0127] In the script generation module, the online conversation is input into the script generation model, which summarizes the online conversation and obtains the script. Optionally, the system collects multiple rounds of conversation data between online users and customer service robots, analyzes and summarizes these conversations using the big model, and generates a dialogue script for testing. The script covers the scene, content, process, etc. of the conversation, as well as the emotional changes and questioning methods of users at different stages, providing guidance for subsequent conversation simulations.

[0128] The dialogue simulation module includes users, question generation models and customer service robots. In the first round of questions, the user asks questions without going through the question generation model. The customer service robot directly analyzes the user's first round of questions and returns the results to the question generation model. The question generation model can generate the user's question content based on the returned results and feedback it to the user. In multiple rounds of dialogue, the question generation model is used to generate the user's question content based on the user's reply content and send it to the customer service robot. The customer service robot analyzes the question content and feeds the returned results back to the question generation model. The question generation model can generate the user's question content based on the returned results. The multiple rounds of dialogue can be executed in a loop. After the user's question content is generated, the conversation is not over. In the last round of dialogue, the question generation model is used to generate the user's question content based on the user's reply content and send it to the customer service robot. The customer service robot analyzes the question content and feeds the returned results back to the question generation model. The question generation model can generate the user's question content based on the returned results. The conversation ends after the last round of dialogue. After the user's question content is generated, the conversation ends. The above-mentioned dialogue simulation module simulates multiple rounds of dialogue based on the generated script. The simulation process is implemented through communication protocol (WebSocket) to ensure real-time and efficient message transmission. During the simulation, according to the script settings and the responses of the customer service robot in the previous round, the question generation model is used to construct the user's question message in real time to form a coherent and reasonable dialogue process.

[0129] In the dialogue scoring module, the entire dialogue content in the dialogue simulation module can be submitted to the dialogue scoring model to conduct a comprehensive evaluation of the customer service robot's problem understanding, communication strategy, solution effectiveness, and rationality of the dialogue, and provide a scoring result for analyzing and improving the dialogue quality of the customer service robot.

[0130] The above-mentioned multi-round dialogue drive can communicate based on a preset communication protocol and rewrite the upstream content of the communication protocol. Here, the generation results of the big model are mainly used to construct the user's question message, and the downstream message is provided to the big model for generating the next round of user questions, that is, based on the content of the script and the return results of the previous round of customer service robot.

[0131] The above-mentioned script generation model is based on online user conversations and summarizes the content of each round of user conversations, including questions, emotions, etc.

[0132] The above-mentioned question generation model is based on the script design, and the customer service robot generates the user's question content in real time based on the reply content of the previous round.

[0133] The above-mentioned dialogue scoring model can score based on the generated dialogue content after the script is played, and is used to evaluate the effectiveness and rationality of the dialogue. For example, whether the entire dialogue is in line with the script, whether the customer service robot has correctly understood the user's problem, whether the solution provided by the customer service robot solves the user's problem, and whether the customer service robot's words are reasonable.

[0134] This application uses a large model to help solve the problems of incoherent and unreasonable dialogues when restoring online dialogues due to changes in dialogue results during the dialogue system testing process. Specifically, this application mainly includes three parts: first, summarizing the online dialogue content and generating a dialogue script; second, generating user question content that meets the current scenario settings based on the script, and driving the execution of the dialogue through a communication protocol (web socket) channel; third, after multiple rounds of dialogue, evaluating the effectiveness and rationality of the entire dialogue.

[0135] This application can be applied to the daily test scenarios of the customer service robot dialogue system to verify the customer service robot's problem understanding and problem solving capabilities, evaluate the rationality of the dialogue, and proactively discover risk issues that do not meet service specifications and may cause public opinion issues. This application can be applied to the evaluation of the customer service robot dialogue algorithm model. The large model in the customer service robot dialogue system is in the process of continuous iteration on a daily basis. This solution can also be used for the evaluation of the large model in the dialogue system to evaluate the effect of the new model.

[0136] This application uses a multi-round model to solve the problem that the overall dialogue effect cannot be verified in a single-round mode. In order to solve the problem of incoherent dialogue and lack of reference value caused by directly restoring online dialogues, this application introduces a large model to generate user questions and evaluate dialogues, achieving the following effects: summarizing online dialogues through script generation large models to ensure that the effect of the entire dialogue is in line with the real demands and questioning habits of consumers; during the dialogue process, through question generation large models, based on the context of the dialogue and the design of the script, the user's question content is generated in real time, making the dialogue process smoother and more reasonable; after the dialogue, the dialogue scoring large model is used to judge the customer service robot's problem understanding ability, communication and problem-solving ability in the dialogue, whether there are high-risk replies, etc., and make a scoring evaluation of the entire dialogue process.

[0137] This application uses a large model to summarize the script and generate real-time user questions to improve the coherence and effectiveness of the conversation; add specific processing in the message sending and receiving stage to achieve real-time generation of user question content; use the conversation scoring large model to score based on the generated conversation content to evaluate the effectiveness and rationality of this conversation. Through the script method, the problem that multiple rounds of online conversations cannot be restored during the test process is solved; based on the script and the above content in the multiple rounds of conversations, the large model is used to generate user questions that fit the conversation scene and content; the conversation scoring large model is used to evaluate the effectiveness and rationality of the entire conversation process, thereby better improving the user experience.

[0138] According to an embodiment of the present application, a testing method for a customer service robot is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here. Figure 4 is a flow chart of a method for testing a customer service robot according to an embodiment of the present application, such as Figure 4 As shown, the method includes:

[0139] Step S402, obtaining historical conversation data of the customer service robot;

[0140] Among them, historical conversation data is used to represent the data generated by multiple rounds of conversations between the customer service robot and the user in historical conversation events.

[0141] First, the data generated by the customer service robot and the user in historical conversation events are collected. This data contains multiple rounds of conversation records between the customer service robot and the user, including both the user's original inquiries and the customer service robot's responses and feedback.

[0142] Historical conversation data is the basis for script generation and conversation ability testing in subsequent steps. It provides samples of real conversations with customer service robots, covers various user needs and conversation scenarios, and is important material for learning and evaluating customer service robots.

[0143] Step S404, inputting the historical dialogue data into the script generation model, and using the script generation model to generate a dialogue scene script;

[0144] Among them, the dialogue scenario script guides the customer service robot to conduct dialogue testing by describing the user's demand information and dialogue scenarios.

[0145] After collecting historical conversation data, the system inputs this data into a trained script generation model. The model can analyze historical conversation data and extract the characteristics of user needs, including question patterns, language style, emotional changes, etc.

[0146] The script generation model uses these features and historical data to generate a series of dialogue scenario scripts, each of which describes in detail the user demand information and dialogue process in a specific situation. These scripts are used to guide subsequent dialogue tests to ensure that the test scenarios are highly matched with real user needs.

[0147] Step S406, based on the dialogue scenario script, the large model is asked questions to conduct multiple rounds of dialogue with the customer service robot to obtain test dialogue data;

[0148] The system generates a large model by asking questions and conducts multiple rounds of interactions with the customer service robot, simulating user questions and the customer service robot's responses to generate test conversation data.

[0149] The question generation model generates questions that meet user habits and needs based on the settings in the script, and sends them to the customer service robot through the real-time communication mechanism. The customer service robot responds based on these questions. This process can be executed repeatedly until all dialogue scenarios in the script are covered.

[0150] These multiple rounds of conversations not only tested the customer service robot’s ability to answer specific questions, but also evaluated its response, coherence, and situational adaptability under different emotions.

[0151] Step S408: testing the conversation capability of the customer service robot based on the test conversation data to obtain a test result.

[0152] Based on the test conversation data, the system can conduct a comprehensive test and evaluation of the customer service robot's conversational capabilities, including but not limited to accuracy of question understanding, rationality of responses, conversation coherence, emotion recognition capabilities, scenario adaptability, etc. The test results are usually quantified based on multiple scoring dimensions to form a comprehensive report that points out the strengths and weaknesses of the customer service robot in the test and provides suggestions for improvement.

[0153] For example, test results may indicate that the customer service robot has a low conversation coherence score when handling a complex refund process, or has defects in emotion recognition and fails to correctly respond to user emotional changes.

[0154] This application provides a systematic and automated conversational ability testing method that can comprehensively evaluate the performance of customer service robots in various complex scenarios, including problem-solving ability, situational adaptability, and user experience. By using script generation and question generation models, the test process can simulate a variety of real user behaviors and needs to ensure the accuracy and depth of conversation testing. The final test results provide specific goals and directions for the optimization and iteration of customer service robots, which helps to improve their service quality and efficiency in practical applications.

[0155] Through the above steps, the historical dialogue data of the customer service robot is obtained, wherein the historical dialogue data is used to represent the data generated by the customer service robot and the user in multiple rounds of dialogue in the historical dialogue event; the historical dialogue data is input into the script generation model, and the script generation model is used to generate a dialogue scenario script, wherein the dialogue scenario script guides the customer service robot to conduct dialogue testing by describing the user's demand information and dialogue scenarios; based on the dialogue scenario script, the question generation model is guided to conduct multiple rounds of dialogue with the customer service robot to obtain test dialogue data; based on the test dialogue data, the dialogue ability of the customer service robot is tested to obtain test results, thereby improving the test effect of the customer service robot; it is easy to notice that the information of previous multiple rounds of dialogue can be obtained to ensure that the test is based on actual user interaction. The script generation model is used to analyze historical data and generate interactive scenario scripts to guide the test. Based on the script guidance, simulated dialogues are conducted to test the response of the customer service robot and collect test interaction data. Through the test interaction data, the performance of the customer service robot is comprehensively evaluated and the test results are output; this process makes full use of the richness of historical data and the guidance of script generation, and effectively tests the continuous dialogue ability and problem-solving ability of the customer service robot by simulating real multiple rounds of dialogue, solves the problem of poor effect of previous testing methods, and improves the test effect. This solves the technical problem of poor testing effect on the model in related technologies.

[0156] According to an embodiment of the present application, a testing method for an interactive model is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here. Figure 5 is a flow chart of a testing method for an interaction model according to an embodiment of the present application, such as Figure 5 As shown, the method includes:

[0157] Step S502, in response to an input instruction acting on the operation interface, displaying historical interaction data of the interaction model on the operation interface;

[0158] The historical interaction data is used to represent the data generated by multiple rounds of interactions between the interaction model and the user in historical interaction events.

[0159] When the tester or relevant engineer triggers a specific input command on the operation interface, the system responds to this operation and presents the data generated by the interaction model and the user in the historical interaction events on the interface. These historical interaction data cover the real multi-round dialogue records in the past, including the user's questions, the response of the interaction model, and the emotional changes and scenario information during the dialogue. The purpose of displaying this data is to allow testers to understand the performance of the interaction model in actual applications, identify common user needs and dialogue patterns, and better design test cases and scenarios.

[0160] Step S504, in response to the processing instruction acting on the operation interface, the test result of the interactive model is displayed on the operation interface.

[0161] Among them, the test results are obtained by testing the interactive ability of the interactive model based on the test interaction data. The test interaction data is generated based on the interactive scenario script to guide the question generation big model and the interactive model to conduct multiple rounds of interaction. The interactive scenario script is generated by inputting historical interaction data into the script generation big model and using the script generation big model. The interactive scenario script describes the user's demand information and the interactive scenario to guide the interactive model to conduct interactive testing.

[0162] After the tester executes the test on the interaction model, the system displays the test results on the interface through the processing instructions on the operation interface. The test results include the interaction scenario script generated by the historical interaction data, and the subsequent multiple rounds of interaction data between the large model and the interaction model generated by asking questions under the guidance of the script. By analyzing these test data, the system can evaluate the performance of the interaction model in specific scenarios, including question comprehension ability, answer accuracy, dialogue coherence and scenario adaptability. The test results are usually presented in the form of charts, lists or text reports, so that testers can see the score and performance of the interaction model at a glance and quickly locate the problem.

[0163] Through the above steps, in response to the input instruction on the operation interface, the historical interaction data of the interaction model is displayed on the operation interface, wherein the historical interaction data is used to represent the data generated by the interaction model and the user in multiple rounds of interaction in the historical interaction event; in response to the processing instruction on the operation interface, the test result of the interaction model is displayed on the operation interface, wherein the test result is obtained by testing the interaction ability of the interaction model based on the test interaction data, and the test interaction data is generated based on the interaction scenario script to guide the question generation model and the interaction model to perform multiple rounds of interaction, and the interaction scenario script is generated by inputting the historical interaction data into the script generation model and using the script generation model to generate the interaction script, and the interaction scenario script describes the user's demand information and the interaction scenario to guide the interaction model to perform interaction testing, thereby improving the test effect of the interaction model; it is easy to notice that the information of multiple rounds of dialogue in the past can be obtained to ensure that the test is based on actual user interaction. The script generation model is used to analyze historical data and generate an interaction scenario script to guide the test. Based on the script guidance, a simulated dialogue is conducted to test the response of the interaction model and collect test interaction data. By testing the interactive data, the performance of the interactive model is comprehensively evaluated and the test results are output; this process fully utilizes the richness of historical data and the guidance of script generation, and effectively tests the continuous dialogue ability and problem-solving ability of the interactive model by simulating real multi-round dialogues, solving the problem of poor results of previous testing methods and improving the test results. This solves the technical problem of poor test results for models in related technologies.

[0164] According to an embodiment of the present application, a testing method for an interactive model is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here. Figure 6 is a flow chart of a testing method for an interaction model according to an embodiment of the present application, such as Figure 6 As shown, the method includes:

[0165] Step S602, acquiring historical interaction data of the interaction model by calling the first interface;

[0166] The first interface includes a first parameter, and the parameter value of the first parameter includes historical interaction data, and the historical interaction data is used to represent data generated by multiple rounds of interactions between the interaction model and the user in historical interaction events.

[0167] The above-mentioned first interface can be an interface for data interaction between the cloud server and the client, and the historical interaction data can be passed into the interface function as the first parameter of the interface function to achieve the purpose of uploading the historical interaction data to the cloud server.

[0168] Step S604, inputting the historical interaction data into the script generation model, and using the script generation model to generate an interactive scenario script;

[0169] Among them, the interaction scenario script guides the interaction model to conduct interaction testing by describing the user's demand information and interaction scenarios.

[0170] Step S606, based on the interactive scenario script, the large model is asked questions to perform multiple rounds of interactions with the interactive model to obtain test interaction data;

[0171] Step S608, testing the interaction capability of the interaction model based on the test interaction data to obtain a test result;

[0172] Step S610: output the test result by calling the second interface.

[0173] The second interface includes a second parameter, and a parameter value of the second parameter includes a test result.

[0174] The above-mentioned second interface can be an interface for data interaction between the cloud server and the client. The cloud server can pass the test result to the interface function as the second parameter of the interface function to achieve the purpose of sending the test result to the client.

[0175] Through the above steps, the historical interaction data of the interaction model is obtained by calling the first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes historical interaction data, and the historical interaction data is used to represent the data generated by the interaction model and the user in multiple rounds of interaction in the historical interaction event; the historical interaction data is input into the script generation model, and the script generation model is used to generate an interaction scenario script, wherein the interaction scenario script guides the interaction model to perform interaction testing by describing the user's demand information and the interaction scenario; based on the interaction scenario script, the question generation model is guided to perform multiple rounds of interaction with the interaction model to obtain test interaction data; based on the test interaction data, the interaction ability of the interaction model is tested to obtain the test result; the test result is output by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the test result, so as to improve the test effect of the interaction model; it is easy to notice that the information of multiple rounds of dialogue in the past can be obtained to ensure that the test is based on actual user interaction. The script generation model is used to analyze historical data and generate an interaction scenario script to guide the test. Based on the script guidance, a simulated dialogue is conducted to test the response of the interaction model and collect test interaction data. By testing the interactive data, the performance of the interactive model is comprehensively evaluated and the test results are output; this process fully utilizes the richness of historical data and the guidance of script generation, and effectively tests the continuous dialogue ability and problem-solving ability of the interactive model by simulating real multi-round dialogues, solving the problem of poor results of previous testing methods and improving the test results. This solves the technical problem of poor test results for models in related technologies.

[0176] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0177] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0178] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0179] According to an embodiment of the present application, a testing device for an interaction model for implementing the above-mentioned testing method for an interaction model is also provided. Figure 7 is a schematic diagram of a test device for an interactive model according to an embodiment of the present application, such as Figure 7 As shown, the device 700 includes: an acquisition module 702 , a generation module 704 , an interaction module 706 , and a testing module 708 .

[0180] Among them, the acquisition module is used to obtain the historical interaction data of the interaction model, wherein the historical interaction data is used to represent the data generated by multiple rounds of interaction between the interaction model and the user in the historical interaction events; the generation module is used to input the historical interaction data into the script generation model, and use the script generation model to generate an interaction scenario script, wherein the interaction scenario script guides the interaction model to perform interaction testing by describing the user's demand information and the interaction scenario; the interaction module is used to guide the question generation model to perform multiple rounds of interaction with the interaction model based on the interaction scenario script to obtain test interaction data; the test module is used to test the interaction ability of the interaction model based on the test interaction data to obtain test results.

[0181] It should be noted that the acquisition module 702, the generation module 704, the interaction module 706, and the test module 708 correspond to steps S202 to S204 in the above embodiment, and the four modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors, and the above modules can also be run as part of the device in the server 10 provided in the above embodiment.

[0182] In the above embodiments of the present application, the interaction module is also used to generate initial question data based on the interaction scenario script during the first round of interaction between the question generation model and the interaction model, and to generate initial feedback data based on the initial question data using the interaction model; in multiple rounds of interaction after the first round of interaction between the question generation model and the interaction model, guide the question generation model to perform multiple rounds of interaction with the interaction model based on the initial feedback data and the interaction scenario script to obtain multiple rounds of interaction data; and construct test interaction data based on the initial question data, initial feedback data and multiple rounds of interaction data.

[0183] In the above embodiments of the present application, the interaction module is also used to input the initial feedback data as preset feedback data into the question generation model, and use the question generation model to generate intermediate question data based on the preset feedback data and the interaction scenario script; input the intermediate question data into the interaction model, use the interaction model to generate intermediate feedback data, and input the intermediate feedback data as the preset feedback data into the question generation model; loop through the steps of using the question generation model to generate intermediate question data based on the preset feedback data and the interaction scenario script, input the intermediate question data into the interaction model, use the interaction model to generate intermediate feedback data, and input the intermediate feedback data as the preset feedback data into the question generation model, until the end condition of the interaction scenario script is reached; and construct multiple rounds of interaction data based on the intermediate question data and the intermediate feedback data.

[0184] In the above embodiments of the present application, the device further includes: a communication module.

[0185] Among them, the communication module is used to establish a communication connection between the first device where the interaction model is located and the second device where the question generation model is located based on a two-way communication mechanism; the interaction module is also used to execute, based on the communication connection, a step of guiding the question generation model to conduct multiple rounds of interaction with the interaction model based on the interaction scenario script to obtain test interaction data.

[0186] In the above embodiments of the present application, the interaction module is also used to input historical interaction data into the script generation model, and use the script generation model to extract the user's user portrait, wherein the user portrait is used to represent the user's questioning pattern and language style; and use the script generation model to generate an interaction scenario script based on the user portrait and historical interaction data.

[0187] In the above embodiments of the present application, the test module is also used to input the test interaction data into the interaction scoring model, use the interaction scoring model to score the interaction ability of the interaction model, and obtain a scoring result; and generate a test result based on the test interaction data and the scoring result.

[0188] In the above embodiments of the present application, the scoring module is also used to score the interactive ability of the interactive scoring based on the scoring dimensions using the interactive scoring big model to obtain the scoring values ​​corresponding to the scoring dimensions, wherein the scoring dimensions include at least one of the following: risk scoring dimension, accuracy scoring dimension, coherence scoring dimension, integrity scoring dimension, response speed scoring dimension, and scenario adaptability scoring dimension; the scoring result is determined based on the scoring values ​​corresponding to the scoring dimensions.

[0189] According to an embodiment of the present application, a testing device for a customer service robot for implementing the above-mentioned testing method for a customer service robot is also provided. Figure 8 is a schematic diagram of a test device for a customer service robot according to an embodiment of the present application, such as Figure 8 As shown, the device 800 includes: an acquisition module 802 , a generation module 804 , a dialogue module 806 , and a testing module 808 .

[0190] Among them, the acquisition module is used to obtain the historical dialogue data of the customer service robot, wherein the historical dialogue data is used to represent the data generated by multiple rounds of dialogue between the customer service robot and the user in the historical dialogue events; the generation module is used to input the historical dialogue data into the script generation model, and use the script generation model to generate a dialogue scene script, wherein the dialogue scene script guides the customer service robot to conduct dialogue testing by describing the user's demand information and dialogue scenes; the dialogue module is used to guide the question generation model to conduct multiple rounds of dialogue with the customer service robot based on the dialogue scene script to obtain test dialogue data; the test module is used to test the dialogue ability of the customer service robot based on the test dialogue data to obtain test results.

[0191] It should be noted that the acquisition module 802, the generation module 804, the dialogue module 806, and the test module 808 correspond to steps S402 to S408 in the above embodiment, and the four modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors, and the above modules can also be run in the server 10 provided in the above embodiment as part of the device.

[0192] According to an embodiment of the present application, a testing device for an interaction model for implementing the above-mentioned testing method for an interaction model is also provided. Fig. 9 is a schematic diagram of a test device for an interactive model according to an embodiment of the present application, such as Fig. 9 As shown, the device 900 includes: a first display module 902 and a second display module 904 .

[0193] Among them, the first display module is used to respond to input instructions acting on the operation interface, and display historical interaction data of the interaction model on the operation interface, wherein the historical interaction data is used to represent data generated by multiple rounds of interactions between the interaction model and the user in historical interaction events; the second display module is used to respond to processing instructions acting on the operation interface, and display the test results of the interaction model on the operation interface, wherein the test results are obtained by testing the interaction ability of the interaction model based on the test interaction data, and the test interaction data is generated based on the interaction scenario script to guide the question generation model and the interaction model to perform multiple rounds of interactions. The interaction scenario script is generated by inputting historical interaction data into the script generation model using the script generation model, and the interaction scenario script guides the interaction model to perform interaction testing by describing the user's demand information and the interaction scenario.

[0194] It should be noted that the first display module 902 and the second display module 904 correspond to steps S502 to S504 in the above embodiment, and the two modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors, and the above modules can also be run as part of the device in the server 10 provided in the above embodiment.

[0195] According to an embodiment of the present application, a testing device for an interaction model for implementing the above-mentioned testing method for an interaction model is also provided. Fig.10 is a schematic diagram of a test device for an interactive model according to an embodiment of the present application, such as Fig.10 As shown, the device 1000 includes: an acquisition module 1002 , a generation module 1004 , an interaction module 1006 , a test module 1008 , and an output module 1010 .

[0196] Among them, the acquisition module is used to obtain the historical interaction data of the interaction model by calling the first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter includes the historical interaction data, and the historical interaction data is used to represent the data generated by multiple rounds of interactions between the interaction model and the user in the historical interaction events; the generation module is used to input the historical interaction data into the script generation model, and use the script generation model to generate an interaction scenario script, wherein the interaction scenario script guides the interaction model to perform interaction testing by describing the user's demand information and the interaction scenario; the interaction module is used to guide the question generation model to perform multiple rounds of interaction with the interaction model based on the interaction scenario script to obtain test interaction data; the test module is used to test the interaction ability of the interaction model based on the test interaction data to obtain test results; the output module is used to output the test results by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the test results.

[0197] It should be noted that the acquisition module 1002, the generation module 1004, the interaction module 1006, the test module 1008, and the output module 1010 correspond to steps S602 to S610 in the above embodiment, and the five modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above embodiment. It should be noted that the above modules or units can be hardware components or software components stored in a memory and processed by one or more processors, and the above modules can also be run in the server 10 provided in the above embodiment as part of the device.

[0198] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in the above embodiments, as well as the application scenario and implementation process, but is not limited to the scheme provided in the above embodiments.

[0199] An embodiment of the present application may provide a computing device. Fig.11 is a structural block diagram of a computing device according to an embodiment of the present application. Fig.11 As shown, the computing device 100 may include: one or more ( Fig.11 (only one is shown) processor 102, memory 104, storage controller, and peripheral interfaces.

[0200] The above-mentioned computing device can be understood as an integrated intelligent terminal, including but not limited to a server, a desktop computer, a PC (Personal Computer), a model all-in-one machine, etc., and the computing device can be pre-installed with the model in the above-mentioned embodiment of the present application.

[0201] Specifically, the computing device can pre-set multiple types of models, including but not limited to models in the fields of natural language processing, visual processing, speech processing, code processing, multimodal task processing, etc., so as to provide a variety of model options. In different product forms, the computing device can support one or more model usage methods, including but not limited to model training, model calling, model fine-tuning, model deployment, model reasoning and application, etc. In some product forms, the computing device also supports model management, including but not limited to multi-type model management (supporting the management of multiple types of models such as discriminants and generative models), model version control (supporting the control of different model versions), model evaluation (based on model evaluation tools, evaluating the performance and effect of the model), etc. In other product forms, the computing device can also generate applications based on the model, provide API calling capabilities, and can call the model to the generated application through the API interface, while providing application management tools to achieve management and monitoring of the application.

[0202] Furthermore, the computing device can also include data management (supporting the generation and management of model tuning data sets), a training center (providing rich training resources to help users learn and master AI technology), and basic management and control capabilities (providing enterprise-level basic management and control capabilities to ensure the security and efficient operation of the system). Through the above functions, a comprehensive, integrated AI development, training, deployment and application device is provided.

[0203] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the methods in the above embodiments. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0204] The processor may call the executable program stored in the memory through the transmission device to execute any method in the above embodiments.

[0205] An embodiment of the present application may provide an electronic device. Fig.12 is a structural block diagram of an electronic device according to an embodiment of the present application. Fig.12 As shown, the electronic device may include: an input / output device 112 ; a memory 114 and a processor 116 , wherein the processor 116 is connected to the input / output device 112 and the memory 114 via a bus 118 .

[0206] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the methods in the above embodiments. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0207] The processor may call the executable program stored in the memory through the transmission device to execute any method in the above embodiments.

[0208] Those skilled in the art will understand that Fig.11 The structure shown is for illustration only, and the computing device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, and other terminal devices. The figure does not limit the structure of the above computing device. For example, the computing device 100 may also include more or fewer components (such as a network interface, a display device, etc.) than those shown in the figure, or have a configuration different from that shown in the figure.

[0209] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0210] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the method provided in the above embodiment.

[0211] Optionally, in this embodiment, the above storage medium may be located in a computing device.

[0212] Optionally, in this embodiment, the computer-readable storage medium is configured to store an executable program, and when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the methods in the above embodiments.

[0213] The embodiment of the present application further provides a computer program product. Optionally, in this embodiment, the computer program product may include a computer program, and the computer program implements the method provided in the embodiment when executed by a processor.

[0214] The embodiments of the present application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which may be used to store a computer program, and when the computer program is executed by a processor, the method provided in the embodiments is implemented.

[0215] The embodiment of the present application further provides a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, the method provided in the above embodiment is implemented.

[0216] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0217] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0218] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0219] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0220] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0221] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A testing method for an interaction model, characterized in that: include: Acquire historical interaction data of the interaction model, wherein the historical interaction data is used to represent data generated by multiple rounds of interaction between the interaction model and the user in historical interaction events; Inputting the historical interaction data into a script generation model, and using the script generation model to generate an interaction scenario script, wherein the interaction scenario script guides the interaction model to perform interaction testing by describing the user's demand information and interaction scenarios; Based on the interactive scenario script, the large model is asked questions to conduct multiple rounds of interactions with the interactive model to obtain test interaction data; The interaction capability of the interaction model is tested based on the test interaction data to obtain a test result.

2. The test method of the interaction model according to claim 1, characterized in that: Based on the interactive scenario script, the question generation model is guided to perform multiple rounds of interaction with the interactive model to obtain test interaction data, including: In the first round of interaction between the question generation model and the interaction model, initial question data is generated based on the interaction scenario script, and initial feedback data is generated based on the initial question data using the interaction model; In a process of multiple rounds of interactions after the first round of interaction between the question generation big model and the interaction model, guiding the question generation big model to perform multiple rounds of interactions with the interaction model based on the initial feedback data and the interaction scenario script to obtain multiple rounds of interaction data; The test interaction data is constructed based on the initial question data, the initial feedback data and the multiple rounds of interaction data.

3. The test method of the interaction model according to claim 2, characterized in that: Based on the initial feedback data and the interactive scenario script, the question generation model is guided to perform multiple rounds of interaction with the interactive model to obtain multiple rounds of interaction data, including: Inputting the initial feedback data as preset feedback data into the question generation model, and using the question generation model to generate intermediate question data based on the preset feedback data and the interactive scenario script; Inputting the intermediate question data into the interaction model, generating intermediate feedback data using the interaction model, and inputting the intermediate feedback data into the question generation model as the preset feedback data; The steps of using the question generation macromodel to generate intermediate question data based on the preset feedback data and the interactive scenario script, inputting the intermediate question data into the interactive model, using the interactive model to generate intermediate feedback data, and inputting the intermediate feedback data into the question generation macromodel as the preset feedback data are executed in a loop until the end condition of the interactive scenario script is reached; The multiple rounds of interaction data are constructed based on the intermediate question data and the intermediate feedback data.

4. The test method of the interaction model according to claim 1, characterized in that: The method further comprises: Establishing a communication connection between the first device where the interactive model is located and the second device where the large question generation model is located based on a two-way communication mechanism; Based on the communication connection, the step of guiding the question generation large model based on the interaction scenario script to perform multiple rounds of interaction with the interaction model to obtain test interaction data is executed.

5. The method for testing an interactive model according to any one of claims 1 to 4, characterized in that: Inputting the historical interaction data into a script generation model, and using the script generation model to generate an interactive scenario script, including: Inputting the historical interaction data into the script generation model, and using the script generation model to extract a user portrait of the user, wherein the user portrait is used to represent the user's questioning pattern and language style; The script generation model is used to generate the interaction scenario script based on the user portrait and the historical interaction data.

6. The method for testing an interactive model according to any one of claims 1 to 4, characterized in that: The interactive capability of the interactive model is tested based on the test interaction data to obtain a test result, including: Inputting the test interaction data into the interaction scoring model, and using the interaction scoring model to score the interaction ability of the interaction model to obtain a scoring result; The test result is generated based on the test interaction data and the scoring result.

7. The method for testing an interactive model according to claim 6, characterized in that: The interaction ability of the interaction model is scored using the interaction scoring model to obtain a scoring result, including: Using the interaction scoring big model to score the interaction ability of the interaction scoring based on the scoring dimension, and obtaining the scoring value corresponding to the scoring dimension, wherein the scoring dimension includes at least one of the following: risk scoring dimension, accuracy scoring dimension, coherence scoring dimension, integrity scoring dimension, response speed scoring dimension, and scenario adaptability scoring dimension; The scoring result is determined based on the scoring value corresponding to the scoring dimension.

8. A method for testing a customer service robot, characterized in that: include: Acquire historical conversation data of the customer service robot, wherein the historical conversation data is used to represent data generated by multiple rounds of conversations between the customer service robot and the user in historical conversation events; Inputting the historical dialogue data into a script generation model, and using the script generation model to generate a dialogue scenario script, wherein the dialogue scenario script guides the customer service robot to perform a dialogue test by describing the user's demand information and dialogue scenarios; Based on the dialogue scenario script, a large model is generated to conduct multiple rounds of dialogue with the customer service robot to obtain test dialogue data; The conversation capability of the customer service robot is tested based on the test conversation data to obtain a test result.

9. A method for testing an interactive model, characterized in that: include: In response to an input instruction acting on the operation interface, displaying historical interaction data of the interaction model on the operation interface, wherein the historical interaction data is used to represent data generated by multiple rounds of interaction between the interaction model and the user in historical interaction events; In response to a processing instruction acting on the operation interface, a test result of the interaction model is displayed on the operation interface, wherein the test result is obtained by testing the interaction ability of the interaction model based on test interaction data, and the test interaction data is generated based on an interaction scenario script to guide a question generation model to perform multiple rounds of interactions with the interaction model, and the interaction scenario script is generated by inputting the historical interaction data into the script generation model and using the script generation model, and the interaction scenario script guides the interaction model to perform interaction testing by describing the user's demand information and interaction scenarios.

10. A method for testing an interactive model, characterized in that: include: Acquire historical interaction data of the interaction model by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter includes the historical interaction data, and the historical interaction data is used to represent data generated by multiple rounds of interaction between the interaction model and the user in historical interaction events; Inputting the historical interaction data into a script generation model, and using the script generation model to generate an interaction scenario script, wherein the interaction scenario script guides the interaction model to perform interaction testing by describing the user's demand information and interaction scenarios; Based on the interactive scenario script, the large model is asked questions to conduct multiple rounds of interactions with the interactive model to obtain test interaction data; Testing the interaction capability of the interaction model based on the test interaction data to obtain a test result; The test result is output by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter includes the test result.

11. A computing device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 10 when running.

12. An electronic device, characterized in that: include: A memory storing an executable program; A processor, connected to the memory via a bus, and configured to run the program, wherein the program executes the method described in any one of claims 1 to 10 when running.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 10.

14. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images