Test method, test device and equipment of dialogue robot, storage medium and program product

Abstract

The invention provides a dialogue robot testing method and device, equipment, a storage medium and a program product, and can be applied to the field of robots and artificial intelligence, the method comprises the steps that a dialogue test data uploading page is displayed, and the dialogue test data uploading page comprises a plurality of uploading components; each uploading component is configured to upload a dialogue process file in a data format corresponding to the uploading component; in response to the test instruction, performing identification fusion processing on the at least one dialogue process file uploaded through the uploading component by using the target multi-modal model to obtain complete dialogue process test data; performing data enhancement processing on the dialogue process test data by utilizing a first language model to obtain a plurality of dialogue test cases; and testing the dialogue robot based on the plurality of dialogue test cases and the expected classification result corresponding to each dialogue test case to obtain a test report.

Description

Technical Field

[0001] The present disclosure relates to the field of robots and artificial intelligence technology, and more specifically, to a testing method, a testing device, an electronic device, a computer-readable storage medium, and a computer program product for a conversational robot. Background Art

[0002] With the development of artificial intelligence, especially conversational robots, they have been applied to various fields to replace customer service for user guidance. A conversational robot is a program used to simulate human dialogue or chat. The task-based robot guides users to complete the input of information step by step by talking to them.

[0003] In the process of implementing the concept of the present disclosure, it was found that in the testing process of the conversation robot in the related technology, the testing accuracy of the conversation robot was poor due to the small number of test cases. Summary of the invention

[0004] In view of this, the present disclosure provides a testing method, a testing device, an electronic device, a computer-readable storage medium, and a computer program product for a conversational robot.

[0005] One aspect of the present disclosure provides a method for testing a conversational robot, comprising:

[0006] Displaying a dialog test data upload page, wherein the dialog test data upload page includes a plurality of upload components, each of which is configured to upload a dialog flow file in a data format corresponding to the upload component;

[0007] In response to the test instruction, at least one dialog flow file uploaded by the uploading component is identified and integrated by using the target multimodal model to obtain complete dialog flow test data;

[0008] Using the first language model to perform data enhancement processing on the above-mentioned dialogue process test data, to obtain multiple dialogue test cases;

[0009] The conversation robot is tested based on the multiple conversation test cases and the expected classification results corresponding to each of the conversation test cases to obtain a test report.

[0010] According to an embodiment of the present disclosure, before performing data enhancement processing, the method further includes:

[0011] Calling a format conversion tool to convert the above-mentioned dialogue flow test data to obtain a dialogue flow display diagram, so as to display the above-mentioned dialogue flow display diagram on a target display page;

[0012] In response to the modification operation on the target display page, the dialog flow display diagram and the dialog flow test data are modified in association with each other to obtain new dialog flow test data.

[0013] According to an embodiment of the present disclosure, the above-mentioned data format includes at least one of the following: an image format, a log format, a code format, and a text format.

[0014] According to an embodiment of the present disclosure, at least one dialog flow file uploaded by the above-mentioned upload component is identified and fused using a target multimodal model to obtain a complete dialog flow test data, including:

[0015] The target multimodal model is used to perform recognition and fusion processing on at least one of the above-mentioned dialogue flow files corresponding to at least one of the above-mentioned image format, the above-mentioned log format, the above-mentioned code format and the above-mentioned text format, so as to obtain the above-mentioned dialogue flow test data.

[0016] According to an embodiment of the present disclosure, the above-mentioned dialogue process test data includes multiple process paths and process nodes on each of the above-mentioned process paths, wherein the above-mentioned process nodes include conditional judgment nodes and multiple conditional classification nodes corresponding to the above-mentioned conditional judgment nodes.

[0017] According to an embodiment of the present disclosure, the first language model is used to perform data enhancement processing on the above-mentioned dialogue flow test data to obtain multiple dialogue test cases, including:

[0018] For each of the conditional classification nodes, data enhancement processing is performed on the conditional classification node using the first language model to obtain multiple conditional example information;

[0019] For each of the above conditional example information, add the above conditional example information to the above conditional classification node to obtain a complete node;

[0020] According to the above complete node, the conditional judgment node corresponding to the above complete node, and the process path associated with the above conditional judgment node, a dialogue test case is generated.

[0021] According to an embodiment of the present disclosure, the above-mentioned expected classification result is generated in the following manner:

[0022] For each of the above dialogue test cases, the above dialogue test case is processed using the second language model to obtain an initial classification result corresponding to the above dialogue test case;

[0023] Analyze the above initial classification results based on the expected classification results to obtain analysis results;

[0024] When the analysis result indicates that the initial classification result is different from the expected classification result, vectorization is performed on the dialogue test case and the plurality of preset classification results to obtain a dialogue test vector and a plurality of preset classification vectors;

[0025] Calculating the vector similarity between the dialogue test vector and each of the preset classification vectors;

[0026] The preset classification result corresponding to the maximum value among the plurality of the vector similarities is determined as the expected classification result of the dialogue test case.

[0027] According to an embodiment of the present disclosure, the target multimodal model is trained in the following manner:

[0028] Acquire a training sample set, wherein the training sample set includes a plurality of training samples and dialog flow identification data corresponding to the training samples, and the training samples include at least two of dialog flow training images, dialog flow training logs, flow code training data, and flow text training information;

[0029] For each of the above training samples, the initial multimodal model is used to process the above training samples to obtain predicted dialogue flow data;

[0030] Generate a target loss value according to the predicted dialogue process data and the dialogue process identification data corresponding to the predicted dialogue process data;

[0031] The model parameters of the initial multimodal model are iteratively adjusted based on the target loss value to obtain the target multimodal model.

[0032] According to an embodiment of the present disclosure, the conversational robot is tested based on the plurality of the conversational test cases and the expected classification results corresponding to each of the conversational test cases to obtain a test report, including:

[0033] For each of the above dialogue test cases, the above dialogue robot is tested using the above dialogue test case to obtain a prediction classification result;

[0034] The test report is generated based on the predicted classification results and the expected classification results of the multiple dialogue test cases.

[0035] According to an embodiment of the present disclosure, the above-mentioned predicted classification result includes the predicted result of each execution node, the above-mentioned expected classification result includes the expected result of each execution node, and the test process in the above-mentioned dialogue test case includes multiple of the above-mentioned execution nodes.

[0036] According to an embodiment of the present disclosure, the test report is generated according to the predicted classification results and the expected classification results of the plurality of the dialog test cases, including:

[0037] For each of the execution nodes, a test result is generated according to the prediction result of the execution node and the expected result;

[0038] The test report is generated according to the test results of the plurality of execution nodes of each of the dialogue test cases.

[0039] Another aspect of the present disclosure provides a testing device for a conversational robot, comprising:

[0040] A display module, used for displaying a dialog test data upload page, wherein the dialog test data upload page includes a plurality of upload components, each of which is configured to upload a dialog flow file in a data format corresponding to the upload component;

[0041] An identification and fusion module, which is used to respond to a test instruction and use a target multimodal model to perform identification and fusion processing on at least one dialog flow file uploaded by the uploading component to obtain complete dialog flow test data;

[0042] A data enhancement module, used to perform data enhancement processing on the above-mentioned dialogue process test data using the first language model to obtain multiple dialogue test cases;

[0043] The testing module is used to test the above-mentioned dialogue robot based on the multiple above-mentioned dialogue test cases and the expected classification results corresponding to each of the above-mentioned dialogue test cases to obtain a test report.

[0044] Another aspect of the present disclosure provides an electronic device, comprising:

[0045] one or more processors;

[0046] a memory for storing one or more programs,

[0047] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described above.

[0048] Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0049] Another aspect of the present disclosure provides a computer program product, which includes computer executable instructions, and when the instructions are executed, are used to implement the method as described above.

[0050] According to an embodiment of the present disclosure, by using different upload components to upload dialogue flow files in different data formats in a dialogue test data upload page, using a target multimodal model to perform recognition and fusion processing on at least one dialogue flow file uploaded by the upload component, complete dialogue flow test data is obtained, and then the dialogue flow test data is enhanced using a first language model to obtain multiple dialogue test cases, thereby testing a dialogue robot based on multiple dialogue test cases and the expected classification results corresponding to each dialogue test case to obtain a test report. Since the use of the target multimodal model to perform fusion recognition on dialogue flow files in different data formats can improve the integrity of the dialogue flow test data, and the use of the first language model for data enhancement effectively improves the coverage of dialogue test cases, thereby improving the test accuracy of the dialogue robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0052] Figure 1 An exemplary system architecture to which a testing method for a conversational robot can be applied according to an embodiment of the present disclosure is schematically shown;

[0053] Figure 2 A flowchart schematically shows a method for testing a conversational robot according to an embodiment of the present disclosure;

[0054] Figure 3 A flowchart schematically shows a method for testing a conversational robot according to another embodiment of the present disclosure;

[0055] Figure 4 Schematically shows a schematic diagram of uploading a dialog flow file in different data formats according to an embodiment of the present disclosure;

[0056] Figure 5 A schematic diagram schematically shows the generated dialog flow test data according to an embodiment of the present disclosure;

[0057] Figure 6 A schematic diagram schematically shows data enhancement according to an embodiment of the present disclosure;

[0058] Figure 7 A block diagram schematically shows a testing device for a conversational robot according to an embodiment of the present disclosure;

[0059] Figure 8 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0060] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0061] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0062] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0063] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0064] When generating test cases for a conversational robot, the business logic is obtained by identifying the flowchart. However, this method is more dependent on the business flowchart. If there is no flowchart, or the product flowchart is relatively simple, it is difficult to test the conversational robot. Moreover, since the entire business logic understanding is based on node type integration, strict requirements are required on the node type, which places strict requirements on the drawing of the flowchart. In addition, the test cases rely on some examples given in the flowchart, which leads to the problem of low test case coverage.

[0065] In view of this, the embodiments of the present disclosure provide a testing method, testing device, equipment, storage medium and program product for a conversational robot, which can be applied to the fields of robots and artificial intelligence. The method includes displaying a conversation test data upload page, wherein the conversation test data upload page includes multiple upload components, and each upload component is configured to upload a conversation flow file in a data format corresponding to the upload component; in response to a test instruction, using a target multimodal model to identify and fuse at least one conversation flow file uploaded by the upload component to obtain complete conversation flow test data; using a first language model to perform data enhancement processing on the conversation flow test data to obtain multiple conversation test cases; testing the conversational robot based on the multiple conversation test cases and the expected classification results corresponding to each conversation test case to obtain a test report.

[0066] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security, and national security.

[0067] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0068] Figure 1 The exemplary system architecture 100 to which the testing method of the conversational robot can be applied according to the embodiment of the present disclosure is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0069] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0070] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0071] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0072] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0073] It should be noted that the test method of the dialogue robot provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the test device of the dialogue robot provided in the embodiment of the present disclosure can generally be set in the server 105. The test method of the dialogue robot provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Correspondingly, the test device of the dialogue robot provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Alternatively, the test method of the dialogue robot provided in the embodiment of the present disclosure can also be executed by the first terminal device 101, the second terminal device 102 or the third terminal device 103, or can also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103. Correspondingly, the testing device of the conversational robot provided in the embodiment of the present disclosure may also be arranged in the first terminal device 101, the second terminal device 102 or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102 or the third terminal device 103.

[0074] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0075] Figure 2 The flowchart of the testing method of the conversational robot according to the embodiment of the present disclosure is schematically shown.

[0076] like Figure 2 As shown, the testing method of the conversational robot includes operations S201 to S204.

[0077] In operation S201, a dialog test data upload page is displayed, wherein the dialog test data upload page includes a plurality of upload components, each upload component being configured to upload a dialog flow file in a data format corresponding to the upload component;

[0078] In operation S202, in response to the test instruction, at least one dialog flow file uploaded by the upload component is identified and fused using the target multimodal model to obtain complete dialog flow test data;

[0079] In operation S203, the dialogue flow test data is enhanced using the first language model to obtain a plurality of dialogue test cases;

[0080] In operation S204, the dialogue robot is tested based on the multiple dialogue test cases and the expected classification results corresponding to each dialogue test case to obtain a test report.

[0081] According to the embodiments of the present disclosure, the conversational robot may be a robot in any industry and any field, such as a customer service robot in the financial services industry, a robot used to communicate with customers in the logistics industry, or a business robot in the production and manufacturing process in the physical manufacturing industry, a robot for software program verification, etc., wherein the robot may be a virtual robot AI program.

[0082] According to an embodiment of the present disclosure, the target multimodal model may be a deep learning model or a generalized linear model, such as a multimodal large model with relatively good effects such as gpt-4o and glm-4v. First language model. The first language model may be a GPT model, a BERT model, or a LLMs large language model, etc.

[0083] According to an embodiment of the present disclosure, the conversation test data upload page is a page in the conversation robot testing software, which includes multiple upload sub-pages. Each upload sub-page can contain an upload component, through which a conversation process file corresponding to its data format can be uploaded.

[0084] According to an embodiment of the present disclosure, after the dialogue flow files in different data formats are uploaded through the upload component, in response to the test instruction, the target multimodal model is used to identify and fuse at least one dialogue flow file uploaded through the upload component, and a complete dialogue flow test data can be obtained. The format of the dialogue flow test data can be JSON format.

[0085] It should be noted that you can upload only one format of dialogue flow file, and you can also use the target multimodal model to identify the dialogue flow test data.

[0086] According to an embodiment of the present disclosure, the first language model is combined with sophisticated context management and prompt engineering to integrate multiple enhancement strategies to perform data enhancement processing on the dialogue process test data, and multiple dialogue test cases that meet the scenario can be obtained. Thereafter, the dialogue test interface can be called to test the dialogue robot using the generated multiple dialogue test cases, thereby obtaining a test report for this test.

[0087] According to an embodiment of the present disclosure, by using different upload components to upload dialogue flow files in different data formats in a dialogue test data upload page, using a target multimodal model to perform recognition and fusion processing on at least one dialogue flow file uploaded by the upload component, complete dialogue flow test data is obtained, and then the dialogue flow test data is enhanced using a first language model to obtain multiple dialogue test cases, thereby testing a dialogue robot based on multiple dialogue test cases and the expected classification results corresponding to each dialogue test case to obtain a test report. Since the use of the target multimodal model to perform fusion recognition on dialogue flow files in different data formats can improve the integrity of the dialogue flow test data, and the use of the first language model for data enhancement effectively improves the coverage of dialogue test cases, thereby improving the test accuracy of the dialogue robot.

[0088] Figure 3 A flowchart of a method for testing a conversational robot according to another embodiment of the present disclosure is schematically shown.

[0089] According to the embodiments of the present disclosure, Figure 3 As shown, before data enhancement processing, it also includes:

[0090] Calling a format conversion tool to convert the dialog flow test data to obtain a dialog flow display diagram, so as to display the dialog flow display diagram on a target display page;

[0091] In response to the modification operation on the target display page, the dialog flow display diagram and the dialog flow test data are modified in association to obtain new dialog flow test data.

[0092] According to an embodiment of the present disclosure, the format conversion tool may be any tool that converts the data format of the dialog flow test data into an image, such as a react flow tool.

[0093] According to an embodiment of the present disclosure, after the target multimodal model outputs complete dialogue flow test data, a format conversion tool is called to convert the dialogue flow test data into a dialogue flow display diagram, and the dialogue flow display diagram is displayed on the target display page to facilitate staff to manually confirm the dialogue flow display diagram.

[0094] According to an embodiment of the present disclosure, if a staff member finds that there is an error in the dialogue flow display diagram, he or she can modify it in the target display page, so that the dialogue flow display diagram and the dialogue flow test data are modified synchronously, and then the new dialogue flow test data is input into the first language model for data enhancement after manual confirmation.

[0095] According to an embodiment of the present disclosure, the dialogue process test data is converted into a dialogue process display diagram through a format conversion tool, so that manual confirmation and modification can be performed during the display process, thereby ensuring the integrity and accuracy of the dialogue process, thereby further improving the testing accuracy of the dialogue robot.

[0096] Figure 4 The diagram schematically shows a dialog flow file upload diagram in different data formats according to an embodiment of the present disclosure. Figure 5 The diagram schematically shows the generated dialog flow test data according to an embodiment of the present disclosure.

[0097] According to the embodiments of the present disclosure, Figure 4 As shown, the data format includes at least one of the following: an image format, a log format, a code format, and a text format.

[0098] According to an embodiment of the present disclosure, at least one dialog flow file uploaded by the upload component is identified and fused using a target multimodal model to obtain a complete dialog flow test data, including:

[0099] The target multimodal model is used to perform recognition and fusion processing on at least one dialog flow file corresponding to at least one of an image format, a log format, a code format, and a text format to obtain dialog flow test data.

[0100] According to the embodiments of the present disclosure, the image format can be bmp, jpg, png, tif, gif and other formats. The dialogue flow file in log format refers to a record file that details a dialogue process and experience. The dialogue flow file in code format is a code file written based on the dialogue flow. The dialogue flow file in text format refers to a dialogue flow arranged in text, such as an Autoboft file.

[0101] According to the embodiment of the present disclosure, after the user uploads the dialogue flow files in different data formats, the dialogue flow files in multiple data formats uploaded by the target multimodal model are used for fusion recognition to obtain complete dialogue flow test data, see Figure 5 The dialogue flow display diagram shown uses complete dialogue flow test data for data enhancement to improve the test accuracy of the dialogue robot.

[0102] It should be noted that Figure 5 The dialogue flow test data is a dialogue flow display diagram obtained by converting the dialogue flow test data through a format conversion tool.

[0103] Figure 6 A schematic diagram of data enhancement according to an embodiment of the present disclosure is schematically shown.

[0104] According to an embodiment of the present disclosure, the dialog flow test data includes multiple flow paths and flow nodes on each flow path, wherein the flow nodes include conditional judgment nodes and multiple conditional classification nodes corresponding to the conditional judgment nodes.

[0105] According to an embodiment of the present disclosure, the first language model is used to perform data enhancement processing on the dialogue flow test data to obtain multiple dialogue test cases, including:

[0106] For each conditional classification node, the first language model is used to perform data enhancement processing on the conditional classification node to obtain multiple conditional example information;

[0107] For each conditional example information, add the conditional example information to the conditional classification node to obtain a complete node;

[0108] A dialog test case is generated according to the complete node, the conditional judgment node corresponding to the complete node, and the process path associated with the conditional judgment node.

[0109] According to the embodiments of the present disclosure, the dialogue process test data includes multiple process paths, and the process paths contain various process nodes, such as conditional judgment nodes, and conditional classification nodes corresponding to different judgment situations. For example, when a logistics customer service robot asks a customer "Is the order quantity greater than 50?", there may be multiple conditional classification nodes such as "greater than 50", "less than or equal to 50" and "unanswered", and there may be corresponding process paths for the above three conditional classification nodes.

[0110] According to an embodiment of the present disclosure, on each process path, the first language model traversally performs data enhancement on each conditional classification node. For example, for "greater than 50 orders", multiple conditional example information such as "51 orders", "60 orders", and "100 orders" can be generated. This data enhancement belongs to breadth enhancement, wherein the number of conditional example information can be used to set relevant parameters of the first language model in advance.

[0111] According to the embodiment of the present disclosure, "100 orders" are added to the conditional classification node to obtain a complete node, and data enhancement is performed on each conditional judgment node in turn on the process path where the complete node is located, that is, deep enhancement, thereby obtaining a complete dialogue test case, see Figure 6 .

[0112] According to an embodiment of the present disclosure, the process nodes on the process path are traversed in sequence through the first language model, and data enhancement is performed on the conditional judgment nodes, thereby providing a variety of dialogue test cases. The dialogue robot is tested using the multiple dialogue test cases obtained by data enhancement, which can ensure a large use case coverage rate during the testing process of the dialogue robot, thereby improving the testing accuracy of the dialogue robot.

[0113] According to an embodiment of the present disclosure, the expected classification result is generated in the following manner:

[0114] For each dialogue test case, the second language model is used to process the dialogue test case to obtain an initial classification result corresponding to the dialogue test case;

[0115] Analyze the initial classification results based on the expected classification results to obtain analysis results;

[0116] When the analysis result indicates that the initial classification result is different from the expected classification result, vectorization is performed on the dialogue test case and the plurality of preset classification results to obtain a dialogue test vector and a plurality of preset classification vectors;

[0117] Calculate the vector similarity between the dialogue test vector and each preset classification vector;

[0118] A preset classification result corresponding to a maximum value among the multiple vector similarities is determined as an expected classification result of the dialogue test case.

[0119] According to an embodiment of the present disclosure, the second language model may be a GPT model, a BERT model, or an LLMs large language model, etc. The expected classification result may be manually identified by a staff member, or may be predicted using a large language model, which indicates the final classification process path corresponding to the dialogue test case.

[0120] According to an embodiment of the present disclosure, before conducting a conversational robot test, it is necessary to determine the expected classification result of each conversational test case, thereby judging the test effect of the conversational robot according to the test classification result obtained from the test and the expected classification result during the test.

[0121] According to an embodiment of the present disclosure, for each dialogue test case, the dialogue test case is processed using a second language model to obtain an initial classification result of the dialogue test case, wherein multiple second language models can be used to process the dialogue test case simultaneously to obtain multiple classification results, and the classification results are classified according to the classification type, thereby determining the classification type with the largest number of classification results as the initial classification result.

[0122] According to an embodiment of the present disclosure, the initial classification result is compared and analyzed with the expected classification result. If the two are different, the dialogue test case and multiple preset classification results are first vectorized to obtain a dialogue test vector and multiple preset classification vectors. Thereafter, the vector similarity between the dialogue test vector and each preset classification vector is calculated, thereby determining the preset classification result corresponding to the maximum value in the vector similarity as the expected classification result of the dialogue test case.

[0123] According to an embodiment of the present disclosure, the expected classification results of dialogue test cases are determined by vector similarity, thereby avoiding the problem of inaccurate classification caused by manual classification, improving the accuracy of the classification results of dialogue test cases, and further improving the testing accuracy of the dialogue robot.

[0124] According to an embodiment of the present disclosure, the target multimodal model is trained in the following manner:

[0125] Acquire a training sample set, wherein the training sample set includes a plurality of training samples and dialog flow identification data corresponding to the training samples, and the training samples include at least two of dialog flow training images, dialog flow training logs, flow code training data, and flow text training information;

[0126] For each training sample, the initial multimodal model is used to process the training sample to obtain the predicted dialogue flow data;

[0127] generating a target loss value according to the predicted dialogue flow data and the dialogue flow identification data corresponding to the predicted dialogue flow data;

[0128] The model parameters of the initial multimodal model are iteratively adjusted based on the target loss value to obtain the target multimodal model.

[0129] According to an embodiment of the present disclosure, the target multimodal model may be an initial multimodal model. In order to further improve the fusion recognition accuracy of the target multimodal model, the initial multimodal model may be trained using a training sample set, thereby using the trained target multimodal model for fusion recognition.

[0130] According to an embodiment of the present disclosure, during the training process of the target multimodal model, the initial multimodal model is used to fuse and identify training samples in various data formats, thereby obtaining the predicted dialogue flow data of the training sample, and the predicted dialogue flow data and the dialogue flow identification data are input into the loss function to obtain the target loss value, and the model parameters of the initial multimodal model are iteratively adjusted based on the target loss value, thereby obtaining the target multimodal model. Among them, the loss function can be Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), etc.

[0131] According to an embodiment of the present disclosure, by using training samples in different data formats to train the initial multimodal model to obtain a target multimodal model, the fusion recognition accuracy of the target multimodal model can be improved, and using the trained target multimodal model to perform recognition and fusion on dialogue process files can provide dialogue process test data with higher accuracy, thereby improving the test accuracy of the dialogue robot.

[0132] According to an embodiment of the present disclosure, a dialog robot is tested based on a plurality of dialog test cases and an expected classification result corresponding to each dialog test case to obtain a test report, including:

[0133] For each dialogue test case, the dialogue robot is tested using the dialogue test case to obtain the prediction classification result;

[0134] Generate a test report based on the predicted classification results and expected classification results of multiple dialogue test cases.

[0135] According to an embodiment of the present disclosure, during the process of testing a conversational robot, a conversational test case is input into the conversational robot, and a predicted classification result is output. By comparing and analyzing the predicted classification result with the expected classification result, a test report for the conversational test case can be obtained.

[0136] According to an embodiment of the present disclosure, the test report can be displayed using a report display page, so that the staff can understand the security, relevance and rationality of the conversational robot in each link through the test report, and find out the problems existing in the test through in-depth testing of the conversational robot, so that the staff can adjust the test of the conversational robot in time, thereby ensuring that the conversational robot can run as expected after application or deployment, and avoiding potential risks to the greatest extent possible.

[0137] According to an embodiment of the present disclosure, the predicted classification result includes the predicted result of each execution node, the expected classification result includes the expected result of each execution node, and the test flow in the dialogue test case includes multiple execution nodes.

[0138] According to an embodiment of the present disclosure, a test report is generated according to the predicted classification results and expected classification results of multiple dialogue test cases, including:

[0139] For each execution node, generate test results based on the predicted results and expected results of the execution node;

[0140] Generate a test report based on the test results of multiple execution nodes of each dialogue test case.

[0141] According to the embodiments of the present disclosure, in order to further clarify the problems existing in the dialogue robot during the testing process, it is necessary to analyze and judge each execution node in the flow path of the dialogue test case during the testing process.

[0142] According to an embodiment of the present disclosure, for each execution node on the process path, the prediction result obtained during the dialogue robot test is combined with the expected result of the execution node in the expected classification result to analyze the test result of the execution node to obtain the test result of the execution node. The test results of multiple execution nodes of multiple dialogue test cases are used to generate a test report.

[0143] According to an embodiment of the present disclosure, the execution status of each dialogue test case at different execution nodes can be seen from the test report, thereby improving the dialogue robot according to whether the execution at each execution node is correct.

[0144] Figure 7 A block diagram of a testing device for a conversational robot according to an embodiment of the present disclosure is schematically shown.

[0145] like Figure 7 As shown, the testing device 700 of the dialogue robot includes a display module 710, a recognition and fusion module 720, a data enhancement module 730, and a testing module 740.

[0146] A display module 710 is used to display a dialog test data upload page, wherein the dialog test data upload page includes multiple upload components, each upload component is configured to upload a dialog flow file in a data format corresponding to the upload component;

[0147] The recognition and fusion module 720 is used to respond to the test instruction and use the target multimodal model to perform recognition and fusion processing on at least one dialogue flow file uploaded by the upload component to obtain complete dialogue flow test data;

[0148] A data enhancement module 730 is used to perform data enhancement processing on the dialogue flow test data using the first language model to obtain multiple dialogue test cases;

[0149] The testing module 740 is used to test the dialogue robot based on multiple dialogue test cases and the expected classification results corresponding to each dialogue test case to obtain a test report.

[0150] According to an embodiment of the present disclosure, by using different upload components to upload dialogue flow files in different data formats in a dialogue test data upload page, using a target multimodal model to perform recognition and fusion processing on at least one dialogue flow file uploaded by the upload component, complete dialogue flow test data is obtained, and then the dialogue flow test data is enhanced using a first language model to obtain multiple dialogue test cases, thereby testing a dialogue robot based on multiple dialogue test cases and the expected classification results corresponding to each dialogue test case to obtain a test report. Since the use of the target multimodal model to perform fusion recognition on dialogue flow files in different data formats can improve the integrity of the dialogue flow test data, and the use of the first language model for data enhancement effectively improves the coverage of dialogue test cases, thereby improving the test accuracy of the dialogue robot.

[0151] According to an embodiment of the present disclosure, the testing device 700 of the dialogue robot further includes:

[0152] A conversion module, used to call a format conversion tool to convert the dialogue flow test data to obtain a dialogue flow display diagram, so as to display the dialogue flow display diagram on a target display page;

[0153] The modification module is used to modify the dialog flow display diagram and the dialog flow test data in association with each other in response to the modification operation on the target display page to obtain new dialog flow test data.

[0154] According to an embodiment of the present disclosure, the data format includes at least one of the following: an image format, a log format, a code format, and a text format.

[0155] According to an embodiment of the present disclosure, the recognition fusion module 720 includes:

[0156] The recognition and fusion unit is used to use the target multimodal model to perform recognition and fusion processing on at least one dialogue flow file corresponding to at least one format of image format, log format, code format and text format to obtain dialogue flow test data.

[0157] According to an embodiment of the present disclosure, the dialog flow test data includes multiple flow paths and flow nodes on each flow path, wherein the flow nodes include conditional judgment nodes and multiple conditional classification nodes corresponding to the conditional judgment nodes.

[0158] According to an embodiment of the present disclosure, the data enhancement module 730 includes:

[0159] A data enhancement unit, configured to perform data enhancement processing on each conditional classification node using the first language model to obtain multiple conditional example information;

[0160] An adding unit, used for adding the conditional example information to the conditional classification node for each conditional example information to obtain a complete node;

[0161] The first generating unit is used to generate a dialogue test case according to the complete node, the conditional judgment node corresponding to the complete node, and the process path associated with the conditional judgment node.

[0162] According to an embodiment of the present disclosure, the expected classification result is generated in the following manner:

[0163] An obtaining unit is used for processing each dialogue test case by using a second language model to obtain an initial classification result corresponding to the dialogue test case;

[0164] An analysis unit, used for analyzing the initial classification result based on the expected classification result to obtain an analysis result;

[0165] A vectorization unit, used for respectively performing vectorization processing on the dialogue test case and the plurality of preset classification results to obtain a dialogue test vector and a plurality of preset classification vectors when the analysis result indicates that the initial classification result is different from the expected classification result;

[0166] A calculation unit, used for calculating the vector similarity between the dialogue test vector and each preset classification vector;

[0167] The determination unit is used to determine the preset classification result corresponding to the maximum value among the multiple vector similarities as the expected classification result of the dialogue test case.

[0168] According to an embodiment of the present disclosure, the target multimodal model is trained in the following manner:

[0169] An acquisition module, configured to acquire a training sample set, wherein the training sample set includes a plurality of training samples and dialog flow identification data corresponding to the training samples, and the training samples include at least two of dialog flow training images, dialog flow training logs, flow code training data, and flow text training information;

[0170] A processing module, for processing each training sample using the initial multimodal model to obtain predicted dialogue flow data;

[0171] A generation module, used for generating a target loss value according to the predicted dialogue flow data and the dialogue flow identification data corresponding to the predicted dialogue flow data;

[0172] The adjustment module is used to iteratively adjust the model parameters of the initial multimodal model based on the target loss value to obtain the target multimodal model.

[0173] According to an embodiment of the present disclosure, the test module 740 includes:

[0174] The testing unit is used to test the conversational robot using the conversational test case for each conversational test case to obtain a prediction classification result;

[0175] The second generating unit is used to generate a test report according to the predicted classification results and expected classification results of the multiple dialogue test cases.

[0176] According to an embodiment of the present disclosure, the predicted classification result includes the predicted result of each execution node, the expected classification result includes the expected result of each execution node, and the test flow in the dialogue test case includes multiple execution nodes.

[0177] According to an embodiment of the present disclosure, the second generating unit includes:

[0178] A first generating subunit, configured to generate a test result for each execution node according to a predicted result and an expected result of the execution node;

[0179] The second generating subunit is used to generate a test report according to the test results of multiple execution nodes of each dialogue test case.

[0180] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0181] For example, any multiple of the display module 710, the identification fusion module 720, the data enhancement module 730, and the test module 740 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the display module 710, the identification fusion module 720, the data enhancement module 730, and the test module 740 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a suitable combination of any of them. Alternatively, at least one of the display module 710, the recognition fusion module 720, the data enhancement module 730, and the test module 740 may be at least partially implemented as a computer program module, and when the computer program module is executed, the corresponding function may be executed.

[0182] It should be noted that the testing device part of the conversational robot in the embodiment of the present disclosure corresponds to the testing method part of the conversational robot in the embodiment of the present disclosure. The description of the testing device part of the conversational robot specifically refers to the testing method part of the conversational robot, which will not be repeated here.

[0183] Figure 8 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0184] like Figure 8 As shown, the electronic device 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 to a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0185] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0186] According to an embodiment of the present disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the input / output (I / O) interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that the computer program read therefrom is installed into the storage portion 808 as needed.

[0187] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0188] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0189] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0190] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 802 and / or the RAM 803 described above and / or one or more memories other than the ROM 802 and the RAM 803 .

[0191] The embodiments of the present disclosure also include a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiments of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the method provided by the embodiments of the present disclosure.

[0192] When the computer program is executed by the processor 801, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0193] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0194] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0195] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0196] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for testing a conversational robot, characterized in that: include: Displaying a dialog test data upload page, wherein the dialog test data upload page includes a plurality of upload components, each of the upload components being configured to upload a dialog flow file in a data format corresponding to the upload component; In response to the test instruction, using the target multimodal model to perform recognition and fusion processing on at least one dialog flow file uploaded by the uploading component to obtain complete dialog flow test data; Performing data enhancement processing on the dialogue flow test data using the first language model to obtain multiple dialogue test cases; The conversation robot is tested based on the multiple conversation test cases and the expected classification results corresponding to each conversation test case to obtain a test report.

2. The method according to claim 1, characterized in that: Before data enhancement, it also includes: Calling a format conversion tool to convert the dialog flow test data to obtain a dialog flow display diagram, so as to display the dialog flow display diagram on a target display page; In response to the modification operation on the target display page, the dialog flow display diagram and the dialog flow test data are modified in association to obtain new dialog flow test data.

3. The method according to claim 1, characterized in that The data format includes at least one of the following: an image format, a log format, a code format, and a text format; The target multimodal model is used to identify and fuse at least one dialog flow file uploaded by the upload component to obtain complete dialog flow test data, including: The target multimodal model is used to perform identification and fusion processing on at least one of the dialog flow files corresponding to at least one of the image format, the log format, the code format and the text format to obtain the dialog flow test data.

4. The method according to claim 1, characterized in that The dialog flow test data includes a plurality of flow paths and flow nodes on each of the flow paths, wherein the flow nodes include a conditional judgment node and a plurality of conditional classification nodes corresponding to the conditional judgment node; The first language model is used to perform data enhancement processing on the dialogue flow test data to obtain multiple dialogue test cases, including: For each of the conditional classification nodes, using the first language model to perform data enhancement processing on the conditional classification node to obtain multiple conditional example information; For each of the conditional example information, add the conditional example information to the conditional classification node to obtain a complete node; A dialog test case is generated according to the complete node, the conditional judgment node corresponding to the complete node, and the process path associated with the conditional judgment node.

5. The method according to claim 1 or 4, characterized in that: The expected classification result is generated in the following way: For each of the dialogue test cases, the dialogue test case is processed using a second language model to obtain an initial classification result corresponding to the dialogue test case; Analyze the initial classification result based on the expected classification result to obtain an analysis result; When the analysis result indicates that the initial classification result is different from the expected classification result, vectorization is performed on the dialogue test case and the plurality of preset classification results to obtain a dialogue test vector and a plurality of preset classification vectors; Calculating the vector similarity between the dialogue test vector and each of the preset classification vectors; A preset classification result corresponding to a maximum value among the plurality of vector similarities is determined as the expected classification result of the dialogue test case.

6. The method according to claim 1, characterized in that The target multimodal model is trained as follows: Acquire a training sample set, wherein the training sample set includes a plurality of training samples and dialog flow identification data corresponding to the training samples, and the training samples include at least two of dialog flow training images, dialog flow training logs, flow code training data, and flow text training information; For each of the training samples, the initial multimodal model is used to process the training sample to obtain predicted dialogue flow data; generating a target loss value according to the predicted dialogue flow data and the dialogue flow identification data corresponding to the predicted dialogue flow data; The model parameters of the initial multimodal model are iteratively adjusted based on the target loss value to obtain the target multimodal model.

7. The method according to claim 1, characterized in that The conversation robot is tested based on the plurality of conversation test cases and the expected classification results corresponding to each of the conversation test cases to obtain a test report, including: For each of the dialogue test cases, the dialogue robot is tested using the dialogue test case to obtain a prediction classification result; The test report is generated according to the predicted classification results and the expected classification results of the plurality of dialogue test cases.

8. The method according to claim 7, characterized in that The predicted classification result includes the predicted result of each execution node, the expected classification result includes the expected result of each execution node, and the test flow in the dialogue test case includes a plurality of the execution nodes; Wherein, generating the test report according to the predicted classification results and the expected classification results of the plurality of the dialogue test cases includes: For each of the execution nodes, generating a test result according to the prediction result of the execution node and the expected result; The test report is generated according to the test results of the multiple execution nodes of each of the dialogue test cases.

9. A testing device for a conversational robot, comprising: A display module, used for displaying a dialog test data upload page, wherein the dialog test data upload page includes a plurality of upload components, each of which is configured to upload a dialog flow file in a data format corresponding to the upload component; an identification and fusion module, configured to, in response to a test instruction, use a target multimodal model to perform identification and fusion processing on at least one dialog flow file uploaded by the upload component to obtain complete dialog flow test data; A data enhancement module, configured to perform data enhancement processing on the dialogue flow test data using the first language model to obtain a plurality of dialogue test cases; The testing module is used to test the conversational robot based on a plurality of the conversational test cases and an expected classification result corresponding to each of the conversational test cases to obtain a test report.

10. An electronic device comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images