Testing method, device, electronic device and storage medium for conversational robot system

Abstract

The present disclosure relates to a test method, device, electronic device, storage medium and computer program product for a conversational robot system. The method comprises: obtaining a test case corresponding to a response test task for a conversational robot system; generating a response test request corresponding to the test case according to a test method corresponding to the test case; sending the response test request to the conversational robot system, the conversational robot system being used to respond to the conversation content carried in the response test request and output the corresponding actual response content; receiving the actual response content returned by the conversational robot system, and determining the test result corresponding to the response test task according to the actual response content. In the present disclosure, after obtaining the test case, a response test request can be generated according to the test method corresponding to the test case and the conversational robot system can be triggered to respond, thereby avoiding manual input of query information for testing the conversational robot system one by one, and effectively improving the test efficiency for the conversational robot system.

Description

Technical Field

[0001] The present disclosure relates to the field of automated testing, and in particular to a testing method, device, electronic device, storage medium, and computer program product for a conversational robot system. Background Art

[0002] With the development of computer technology, the interaction between users and conversational robot systems is increasing. During the interaction process, after receiving the query information input by the user, the client can generate a request and send it to the conversational robot system, triggering the conversational robot system to return the corresponding reply information according to the received request, so as to achieve the effect of timely response. In order to ensure that users can get the correct reply information, it is often necessary to test the conversational robot system.

[0003] In the related art, the tester can input the conversation content for testing on the client and send it to the conversation robot system, and obtain the test result according to the reply information of the conversation robot system.

[0004] However, in the above-mentioned testing method, the tester needs to manually input multiple conversation contents one by one to conduct multi-faceted tests on the conversational robot system, which results in low testing efficiency. Summary of the invention

[0005] The present disclosure provides a test method, device, electronic device, storage medium and computer program product for a conversational robot system, so as to at least solve the problem of low test efficiency of the conversational robot system in the related art. The technical solution of the present disclosure is as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, a method for testing a conversational robot system is provided, comprising:

[0007] Obtaining a test case corresponding to a response test task for the conversational robot system;

[0008] Generate a response test request corresponding to the test case according to the test method corresponding to the test case;

[0009] The response test request is sent to the conversational robot system, and the conversational robot system is used to respond to the conversation content carried in the response test request and output the corresponding actual response content;

[0010] Receive the actual response content returned by the conversational robot system, and determine the test result corresponding to the response test task according to the actual response content.

[0011] In an exemplary embodiment, generating a response test request corresponding to the test case according to a test method corresponding to the test case includes:

[0012] Importing the test case parameters corresponding to the test case into the test method corresponding to the test case;

[0013] According to the test request generation mode indicated by the test method, a response test request corresponding to the test case is constructed based on the test case parameters, and a request sending condition corresponding to the response test request is determined.

[0014] In an exemplary embodiment, the test case parameters are multiple groups of test case parameters, the test methods are multiple test methods for different types of session content, and the step of importing the test case parameters corresponding to the test case into the test method corresponding to the test case includes:

[0015] Obtain parameter type labels for each group of test case parameters;

[0016] For each of the test methods, a target test case parameter is obtained from the multiple groups of test case parameters, and the target test case parameter is imported into the test method; the parameter type label of the target test case parameter matches the session content type of the test method.

[0017] In an exemplary embodiment, sending the response test request to the conversational robot system includes:

[0018] In response to satisfying the request sending conditions corresponding to the response test request, the preset virtual client is called to send the response test request to the remote procedure call interface of the conversational robot system; the remote procedure call interface is used to call the semantic recognition service in response to the response test request to identify the conversation content carried by the response test request, and output the corresponding actual response content.

[0019] In an exemplary embodiment, the receiving the actual response content returned by the conversational robot system includes:

[0020] Receiving a response message returned by the conversational robot system in response to the response test request;

[0021] The response message is parsed according to the transmission protocol corresponding to the response test request to obtain the actual response content carried by the response message.

[0022] In an exemplary embodiment, determining the test result corresponding to the response test task according to the actual response content includes:

[0023] Obtaining the expected response content corresponding to the test case;

[0024] Based on the difference between the expected response content and the actual response content, the test result of the conversational robot system is determined.

[0025] In an exemplary embodiment, after determining the test result corresponding to the response test task according to the actual response content, the method further includes:

[0026] Obtain a preset test report template, and generate a test report according to the test results and the test report template.

[0027] According to a second aspect of an embodiment of the present disclosure, a testing device for a conversational robot system is provided, comprising:

[0028] A test case acquisition unit, configured to execute and acquire a test case corresponding to a response test task for the conversational robot system;

[0029] A request generating unit, configured to execute a test method corresponding to the test case and generate a response test request corresponding to the test case;

[0030] A request sending unit, configured to execute sending the response test request to the conversational robot system, wherein the conversational robot system is used to output corresponding actual response content in response to the conversation content carried by the response test request;

[0031] The test result acquisition unit is configured to receive the actual response content returned by the conversational robot system, and determine the test result corresponding to the response test task according to the actual response content.

[0032] In an exemplary embodiment, the request generating unit includes:

[0033] A parameter import module is configured to import the test case parameters corresponding to the test case into the test method corresponding to the test case;

[0034] The request construction module is configured to execute the test request generation method indicated by the test method, construct a response test request corresponding to the test case based on the test case parameters, and determine the request sending condition corresponding to the response test request.

[0035] In an exemplary embodiment, the test case parameters are multiple groups of test case parameters, the test methods are multiple test methods for different types of session content, and the parameter import module is specifically configured to execute:

[0036] Obtain parameter type labels for each group of test case parameters;

[0037] For each of the test methods, a target test case parameter is obtained from the multiple groups of test case parameters, and the target test case parameter is imported into the test method; the parameter type label of the target test case parameter matches the session content type of the test method.

[0038] In an exemplary embodiment, the request sending unit is specifically configured to execute:

[0039] In response to satisfying the request sending conditions corresponding to the response test request, the preset virtual client is called to send the response test request to the remote procedure call interface of the conversational robot system; the remote procedure call interface is used to call the semantic recognition service in response to the response test request to identify the conversation content carried by the response test request, and output the corresponding actual response content.

[0040] In an exemplary embodiment, the test result acquisition unit is specifically configured to execute:

[0041] Receiving a response message returned by the conversational robot system in response to the response test request;

[0042] The response message is parsed according to the transmission protocol corresponding to the response test request to obtain the actual response content carried by the response message.

[0043] In an exemplary embodiment, the test result acquisition unit is specifically configured to execute:

[0044] Obtaining the expected response content corresponding to the test case;

[0045] Based on the difference between the expected response content and the actual response content, the test result of the conversational robot system is determined.

[0046] In an exemplary embodiment, the apparatus further comprises:

[0047] The test report generating unit is configured to execute obtaining a preset test report template and generate a test report according to the test result and the test report template.

[0048] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0049] processor;

[0050] a memory for storing instructions executable by the processor;

[0051] Wherein, the processor is configured to execute the instructions to implement the testing method of the conversational robot system as described in any one of the above items.

[0052] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute a testing method for a conversational robot system as described in any one of the above items.

[0053] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, wherein the computer program product includes instructions, and when the instructions are executed by a processor of an electronic device, the electronic device is enabled to execute a testing method for a conversational robot system as described in any one of the above items.

[0054] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0055] The terminal can obtain the test case corresponding to the response test task for the conversational robot system, generate the response test request corresponding to the test case according to the test method corresponding to the test case, and then send the response test request to the conversational robot system. The conversational robot system can respond to the conversation content carried in the response test request and output the corresponding actual response content. The terminal can then receive the actual response content returned by the conversational robot system, and determine the test result corresponding to the response test task according to the actual response content. In the present disclosure, after obtaining the test case, it is possible to automatically generate a response test request and trigger the conversational robot system to respond according to the test method corresponding to the test case, thereby avoiding manual input of query information for testing the conversational robot system one by one, and effectively improving the testing efficiency of the conversational robot system.

[0056] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0058] Figure 1 It is an application environment diagram of a testing method for a conversational robot system according to an exemplary embodiment.

[0059] Figure 2 The present invention is a flowchart of a method for testing a conversational robot system according to an exemplary embodiment.

[0060] Figure 3 The present invention is a flowchart showing steps of generating a response test request according to an exemplary embodiment.

[0061] Figure 4The figure is a flowchart showing the steps of obtaining actual response content according to an exemplary embodiment.

[0062] Figure 5 The present invention is a flowchart of another method for testing a conversational robot system according to an exemplary embodiment.

[0063] Figure 6 It is a block diagram of a testing device for a conversational robot system according to an exemplary embodiment.

[0064] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0065] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0066] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0067] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0068] The present disclosure provides a method for testing a conversational robot system, which can be applied to Figure 1 In the application environment shown, the application environment may include a terminal and a conversational robot system, and the terminal may communicate with the conversational robot system through a network. The conversational robot system may perform semantic recognition on the information sent by the user and return the corresponding response content, simulating a conversation between a human and a user. The conversational robot system may have a corresponding data storage system, which may store data that the conversational robot system needs to process and call. In actual applications, the data storage system may be integrated in the conversational robot system, or may be placed on the cloud or other network servers.

[0069] In the present disclosure, the terminal can obtain the test case corresponding to the response test task for the conversational robot system, and generate the response test request corresponding to the test case according to the test method corresponding to the test case, and then send the response test request to the conversational robot system. After receiving the response test request, the conversational robot system can respond to the conversation content carried by the response test request and output the corresponding actual response content. After receiving the actual response content returned by the conversational robot system, the terminal can determine the test result corresponding to the response test task according to the actual response content. Among them, the terminal can be but not limited to various personal computers, laptops, smart phones and tablets. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The conversational robot system can be implemented with an independent server or a server cluster consisting of multiple servers.

[0070] Figure 2 is a flowchart of a method for testing a conversational robot system according to an exemplary embodiment. Figure 2 As shown, the method is used in a terminal as an example for explanation, and may include the following steps.

[0071] In step S210, a test case corresponding to a response test task for a conversational robot system is obtained.

[0072] As an example, a response test task may be a task for testing the performance of a conversational robot system in interacting with a user in response, such as testing how the conversational robot system responds to received conversation content.

[0073] A test case may be a task description for a response test task, and may reflect the test content or strategy in the response test task. Exemplarily, the content of a test case may include at least one of the following: test case parameters, test objectives, test steps, and expected results.

[0074] In actual applications, corresponding test cases can be constructed in advance for the response test tasks of the conversational robot system, and then when the terminal performs a response test on the conversational robot, the test cases corresponding to the response test tasks can be obtained.

[0075] In step S220, a response test request corresponding to the test case is generated according to the test method corresponding to the test case.

[0076] As an example, the test method may be used to indicate how a response test request is generated. For example, the test method may be a request generation logic for generating a response test request of a specified type, or may be a model or function for generating a response test request.

[0077] The response test request may be information for requesting the conversational robot system to respond to the specified conversation content. In other words, the response test request may trigger the conversational robot system to respond to the specified conversation content, wherein the specified conversation content may be determined based on the test case.

[0078] After obtaining the test case corresponding to the response test task, the test method corresponding to the test case can be determined. Each test case can be a test case set for a specified session content, such as a session content of text or a picture, or can include both text and pictures. Test cases used to test different types of session content can have corresponding test methods, and then after obtaining the test case, the test method corresponding to the test case can be obtained, and the response test request corresponding to the test case can be generated according to the test method.

[0079] In step S230, the response test request is sent to the conversational robot system, and the conversational robot system is used to respond to the conversation content carried in the response test request and output the corresponding actual response content.

[0080] After generating the response test request, the request can be sent to the conversational robot system. The conversational robot system can respond to the received response test request, parse the request, and thereby obtain the session content (the session content can also be called a test query) carried in the response test request, and obtain and output the actual response content for the session content.

[0081] In step S240, the actual response content returned by the conversational robot system is received, and the test result corresponding to the response test task is determined according to the actual response content.

[0082] In a specific implementation, after outputting the actual response content, the conversational robot system can return the actual response content to the terminal. After receiving the actual response content, the terminal can determine the test result of the response test task according to the actual response content.

[0083] In the test method of the above-mentioned conversational robot system, the terminal can obtain the test case corresponding to the response test task for the conversational robot system, generate the response test request corresponding to the test case according to the test method corresponding to the test case, and then send the response test request to the conversational robot system. The conversational robot system can respond to the conversation content carried in the response test request and output the corresponding actual response content. The terminal can receive the actual response content returned by the conversational robot system, and determine the test result corresponding to the response test task according to the actual response content. In the present disclosure, after obtaining the test case, according to the test method corresponding to the test case, the response test request can be automatically generated and the conversational robot system can be triggered to respond, avoiding manual input of query information for testing the conversational robot system one by one, and effectively improving the testing efficiency of the conversational robot system.

[0084] In an exemplary embodiment, if Figure 3 As shown, in step S220, generating a response test request corresponding to the test case according to the test method corresponding to the test case may include:

[0085] In step S310, the test case parameters corresponding to the test case are imported into the test method corresponding to the test case.

[0086] As an example, the test case parameters may be parameter information used to describe the test case content, and the test case parameters may include at least one of the following information: conversation content, answer test account, answer test request type. Among them, the conversation content may include textual conversation content and / or picture-based conversation content. For textual conversation content, specific text information may be set through the test case parameters; or, the conversation content may be set through the test case parameters to be generated by clicking on preset information. For example, in actual applications, after the conversation robot returns the answer content to the information sent by the user client, it may provide "solved" and "unsolved" options accordingly, or may provide related question options for the questions sent by the user. The user may trigger the user client to send the corresponding text message by clicking on any option, without the need for manual input by the user. In this embodiment, the text information may also be set through the test case parameters to be triggered by clicking on preset options, thereby simulating various dialogue scenarios between the user and the conversation robot system during the actual answering process.

[0087] In actual applications, after obtaining the test case, the test case parameters can be obtained and imported into the test method corresponding to the test case, so that the test method can receive the relevant parameters for generating a response test request. Exemplarily, the test case parameters can be recorded in a table file, such as pre-storing the test case parameters through an Excel table. When the terminal obtains the test case parameters of the test case, it can obtain them through a table data reading tool, such as reading through the xlrd tool, which is an extension tool in the Python language for reading Excel table information.

[0088] In step S320, according to the test request generation mode indicated by the test method, a response test request corresponding to the test case is constructed based on the test case parameters, and a request sending condition corresponding to the response test request is determined.

[0089] After the test case parameters are imported into the test method, the response test request corresponding to the test case can be constructed according to the test request generation method indicated by the test method and combined with the test case parameters, and the request sending conditions corresponding to the constructed response test request can be determined based on the test case parameters.

[0090] Exemplarily, when executing a test method, one or more test methods can be pre-stored in a record file of the test method, and the terminal can import the test case parameters into a preset test framework, and execute each test in the record file one by one through the test framework, wherein the test framework can also be referred to as an automated test framework, which can provide a reusable basic automated test module. For example, the preset test framework can be a pytest test framework, which is a test framework for the Python language. When the pytest test framework executes a test case, the test method starting with test in the record file in the test_*.py format or *_test.py format in the preset directory can be executed, and the test case parameters can be passed into the test method during execution.

[0091] In this embodiment, by importing the test case parameters corresponding to the test case into the test method corresponding to the test case, constructing the response test request corresponding to the test case based on the test case parameters in accordance with the test request generation method indicated by the test method, and determining the request sending conditions corresponding to the response test request, the corresponding test request can be quickly constructed based on the test case parameters, avoiding the test personnel from generating response test requests by writing a large amount of coding, thereby effectively improving the testing efficiency of the conversational robot system.

[0092] In an exemplary embodiment, the test case may include multiple test cases, each test case has a corresponding set of test case parameters, and accordingly, the test case parameters are multiple sets of test case parameters, and the test method may be multiple test methods for different types of session content. In step S310, importing the test case parameters corresponding to the test case into the test method corresponding to the test case may include:

[0093] Obtain parameter type labels of each group of test case parameters; for each test method, obtain target test case parameters from multiple groups of test case parameters, and import the target test case parameters into the test method; the parameter type labels of the target test case parameters match the session content type of the test method.

[0094] As an example, a parameter type tag may indicate the type of session content that the test case parameter tests.

[0095] In a specific implementation, parameter type tags may be set in advance for the test case parameters corresponding to each group of test cases. After reading multiple groups of test case parameters, multiple test methods may be obtained and the parameter type tags of each group of test case parameters may be read. Then, for each test method, the target test case parameters corresponding to the test method may be obtained from the multiple groups of test cases, and then the target test case parameters may be imported into the test method. In other words, in this embodiment, multiple groups of test case parameters may be screened by parameter type tags. For example, after receiving a test instruction, the type of session content to be tested indicated by the test instruction may be determined. Then, for the session content type, matching test case parameters may be obtained from the multiple groups of test case parameters that have been read according to the parameter type tags, and the test case parameters may be imported into the corresponding test method.

[0096] In this embodiment, by obtaining the parameter type labels of each group of test case parameters, for each test method, the target test case parameters are obtained from multiple groups of test case parameters according to the parameter type labels, and the target test case parameters are imported into the test method. The test case parameters can be correctly imported into the matching test method, thereby ensuring the accuracy of the generated response test request.

[0097] In an exemplary embodiment, in step S230, sending the response test request to the conversational robot system may include the following steps:

[0098] In response to satisfying the request sending conditions corresponding to the answer test request, the preset virtual client is called to send the answer test request to the remote procedure call interface of the conversational robot system; the remote procedure call interface is used to call the semantic recognition service in response to the answer test request to identify the conversation content carried by the answer test request, and output the corresponding actual answer content.

[0099] In a specific implementation, a virtual client can be pre-packaged. The client is also called a test client. It is created by simulating a published client (such as a client App installed on a user's mobile terminal). It can be used as an object for sending response test requests to the conversational robot system during the conversational robot system testing process.

[0100] After the test case parameters corresponding to the test case are imported into the test method corresponding to the test case, a response test request can be constructed and the request sending conditions corresponding to the request can be determined. When it is detected that the request sending conditions are met, the terminal can call the virtual client to send the response test request to the remote procedure call (rpc, Remote Procedure Call) interface of the conversational robot system. For example, the terminal can construct an rpc request (such as a grpc request), and when the request sending conditions are met, the virtual client is called, and the virtual client sends the rpc request to the rpc interface (such as the grpc interface) of the conversational robot system.

[0101] After the conversational robot system receives a response test request through the remote procedure call interface, the remote procedure call interface can respond to the response test request, call the pre-built semantic recognition server to identify the conversation content carried in the response test request, and output the corresponding actual response content according to the recognition result. For example, after receiving the response test request, the remote procedure call interface can pre-process the response test request through the control center of the conversational robot system. The control center can also configure rules, and then call the semantic recognition service to identify the pre-processed response test request, obtain the conversation content carried therein, and match the actual response content whose confidence meets the preset conditions in the preset knowledge base through vector retrieval, algorithm model and other means.

[0102] In this embodiment, in response to satisfying the request sending conditions corresponding to the response test request, a preset virtual client can be called to send the response test request to the remote procedure call interface of the conversational robot system. The remote procedure call interface is used to respond to the response test request. The semantic recognition service can be called to identify the conversation content carried by the response test request, and output the corresponding actual response content. In this scheme, the terminal can bypass the published and installed user client through the virtual client and directly access the remote procedure call service of the conversational robot system, avoiding sending the response test request on the published and installed user client, ensuring that external factors affect the test results, and improving the accuracy and reliability of the test results.

[0103] In an exemplary embodiment, if Figure 4 As shown, in step S240, receiving the actual response content returned by the conversational robot system may include:

[0104] In step S41O, a response message returned by the conversational robot system in response to the test request is received.

[0105] In a specific implementation, when a terminal communicates with a conversational robot system, communication can be performed through a preset transmission protocol. The preset transmission protocol can use a platform-independent, language-independent, scalable, lightweight and efficient serialized data structure protocol that can be used for network communication and data storage. For example, when the response test request is a grpc request, data transmission between the terminal and the conversational robot system can be performed through the ProtoBuf (Google Protocol Buffer) data protocol, which can be used to define a serial structured data format for data exchange. When the conversational robot system communicates based on a preset transmission protocol, if it responds to the response test request and outputs the actual response content, it can generate a response message based on the actual response content, and the response message can be a data object.

[0106] In step S42O, the response message is parsed according to the transmission protocol corresponding to the response test request to obtain the actual response content carried by the response message.

[0107] After receiving the response message returned by the conversational robot system, the response message can be parsed according to the transmission protocol corresponding to the response test request to obtain the actual response content carried by the response message. For example, when the response test request is a grpc request, the terminal and the conversational robot system communicate through the ProtoBuf protocol, then the response message returned by the conversational robot system can be a pb (ProtoBuf) data object. When the terminal executes the test method through a preset test framework (such as the pytest test framework, which is an open source software test framework for the Python language), the test framework cannot directly process the data object. Then, the response message can be parsed based on the ProtoBuf protocol first, and the actual response content that the test framework can assert, such as a string, can be obtained based on the parsing result.

[0108] In this embodiment, a response message returned by the conversational robot system in response to a test request can be received, and the response message can be parsed according to the transmission protocol corresponding to the test request to obtain the actual response content carried by the response message, and the actual response content returned by the conversational robot system can be accurately identified, providing a basis for subsequently obtaining reliable test results.

[0109] In an exemplary embodiment, in step S240, determining a test result corresponding to the response test task according to the actual response content includes:

[0110] Obtain expected response content corresponding to the test case; and determine the test result of the conversational robot system based on the difference between the expected response content and the actual response content.

[0111] Specifically, for each test case, the corresponding expected response content can be pre-set for the conversation content of the test case, and the expected response content can be the correct reply to the conversation content. After receiving the actual response content returned by the conversational robot system, the terminal can obtain the expected response content corresponding to the test case, and determine the test result of the conversational robot system based on the difference between the expected response content and the actual response content. For example, if the degree of difference between the actual response content and the expected response content is less than or equal to a preset threshold, it can be determined that the test has passed. If the degree of difference between the actual response content and the expected response content is greater than the preset threshold, it can be determined that the test has failed.

[0112] In actual applications, assertions can also be made based on the expected response content and the actual response content (assertion is a first-order logic in a program), and the assertion results can be used as test results.

[0113] In this embodiment, the expected response content corresponding to the test case can be obtained, and based on the difference between the expected response content and the actual response content, the test result of the conversational robot system can be determined, so that an accurate and reliable test report can be quickly obtained.

[0114] In an exemplary embodiment, after step S240, the method may further include the following steps:

[0115] Obtain a preset test report template and generate a test report based on the test results and the test report template.

[0116] In a specific implementation, a test report template may be pre-set. For example, a plurality of section contents may be pre-set in the test report template, and the display style of the section contents may be pre-defined for each section content. After obtaining the test results, a preset test report template may be obtained, and a test report may be generated according to the test results and the test report template. For example, the test results may be imported into the test report template to obtain a test report in HTML format.

[0117] In this embodiment, a test report can be generated based on the test results and a preset test report template, which can not only avoid the uneven quality of test reports caused by manual writing of test reports and unify the reporting standards, but also eliminate the need for testers to manually organize test reports for response test tasks, thereby improving test efficiency.

[0118] In order to enable those skilled in the art to better understand the above steps, the embodiment of the present disclosure is illustrated below by using an example, but it should be understood that the embodiment of the present disclosure is not limited thereto.

[0119] In actual applications, the user can pre-download and install a mobile client on his mobile phone. When he needs to query relevant information from the conversational robot system, the user can enter questions in the mobile client and send the questions to the client's backend through the mobile client, and then call the remote procedure call service (such as grpc service) of the conversational robot system through the client's backend.

[0120] In some ways of testing a conversational robot system, the tester can repeat the process of the user using the conversational robot system, and enter multiple questions for testing one by one on the mobile phone; or, the tester can also use the test tool of a third-party platform (such as jmeter) to test. During the test, the interface test function of the test tool can be used to define the channel for sending remote procedure call requests, and write relevant code according to the channel definition content to construct the corresponding request. After the request is constructed, the request is sent to the conversational robot system interface, and the test result is obtained according to the information returned by the conversational robot system. However, the former needs to be manually input one by one, and the latter also requires the tester to write a large amount of code to generate the requested resource package, such as a jar package, and after the test is completed, the tester needs to manually write a test report, which leads to the problem of low efficiency in conversational robot system testing.

[0121] Based on this, the present disclosure provides a testing method for a conversational robot system, such as Figure 5As shown, the tester can pre-configure the test case set through the table file and set the applicable test method. The test case set records the test case parameters corresponding to multiple test cases. During the test, the terminal can call xlrd to read the test case parameters recorded in the table file and pass them into the pytest test framework. Then, when the test framework executes the test method, it can import the test case parameters into the corresponding test method, generate a response test request according to the test method, and call the pre-packaged public virtual client to send the response test request to the grpc interface of the conversational robot system. For the received response test request, the conversational robot system can call the semantic recognition service to identify it, obtain the actual response content for the recognition result, and generate a pb data object according to the transmission protocol ProtoBuf corresponding to the response test request and return it to the virtual client. The virtual client forwards the data object to the terminal. After calling the method for parsing the pb data object, the terminal can obtain the actual response content that can be used as an assertion, and then the actual response content can be passed into the pytest test framework. The pytest test framework can obtain the corresponding expected response content from the test case set, and make assertions based on the expected response content and the actual response content, and generate a test report based on the assertion results.

[0122] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0123] It can be understood that the same / similar parts between the various embodiments of the above method in this specification can refer to each other, and each embodiment focuses on the differences from other embodiments. For related points, please refer to the description of other method embodiments.

[0124] Based on the same inventive concept, the disclosed embodiment also provides a testing device for a conversational robot system for implementing the above-mentioned testing method for a conversational robot system.

[0125] Figure 6 is a block diagram of a test device for a conversational robot system according to an exemplary embodiment. Figure 6The device includes a test case acquisition unit 601, a request generation unit 602, a request sending unit 603 and a test result acquisition unit 604.

[0126] A test case acquisition unit 601 is configured to execute and acquire a test case corresponding to a response test task for the conversational robot system;

[0127] The request generating unit 602 is configured to execute a test method corresponding to the test case and generate a response test request corresponding to the test case;

[0128] The request sending unit 603 is configured to execute sending the response test request to the conversational robot system, and the conversational robot system is used to respond to the conversation content carried by the response test request and output the corresponding actual response content;

[0129] The test result acquisition unit 604 is configured to receive the actual response content returned by the conversational robot system, and determine the test result corresponding to the response test task according to the actual response content.

[0130] In an exemplary embodiment, the request generating unit includes:

[0131] A parameter import module is configured to import the test case parameters corresponding to the test case into the test method corresponding to the test case;

[0132] The request construction module is configured to execute the test request generation method indicated by the test method, construct a response test request corresponding to the test case based on the test case parameters, and determine the request sending condition corresponding to the response test request.

[0133] In an exemplary embodiment, the test case parameters are multiple groups of test case parameters, the test methods are multiple test methods for different types of session content, and the parameter import module is specifically configured to execute:

[0134] Obtain parameter type labels for each group of test case parameters;

[0135] For each of the test methods, a target test case parameter is obtained from the multiple groups of test case parameters, and the target test case parameter is imported into the test method; the parameter type label of the target test case parameter matches the session content type of the test method.

[0136] In an exemplary embodiment, the request sending unit is specifically configured to execute:

[0137] In response to satisfying the request sending conditions corresponding to the response test request, the preset virtual client is called to send the response test request to the remote procedure call interface of the conversational robot system; the remote procedure call interface is used to call the semantic recognition service in response to the response test request to identify the conversation content carried by the response test request, and output the corresponding actual response content.

[0138] In an exemplary embodiment, the test result acquisition unit is specifically configured to execute:

[0139] Receiving a response message returned by the conversational robot system in response to the response test request;

[0140] The response message is parsed according to the transmission protocol corresponding to the response test request to obtain the actual response content carried by the response message.

[0141] In an exemplary embodiment, the test result acquisition unit is specifically configured to execute:

[0142] Obtaining the expected response content corresponding to the test case;

[0143] Based on the difference between the expected response content and the actual response content, the test result of the conversational robot system is determined.

[0144] In an exemplary embodiment, the apparatus further comprises:

[0145] The test report generating unit is configured to execute obtaining a preset test report template and generate a test report according to the test result and the test report template.

[0146] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0147] Each module in the test device of the above conversational robot system can be implemented in whole or in part by software, hardware and their combination. Each module can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0148] Figure 7 1 is a block diagram of an electronic device 700 for implementing a method for testing a conversational robot system according to an exemplary embodiment. For example, the electronic device 700 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0149] Reference Figure 7 , the electronic device 700 may include one or more of the following components: a processing component 702 , a memory 704 , a power component 706 , a multimedia component 708 , an audio component 710 , an input / output (I / O) interface 712 , a sensor component 714 , and a communication component 716 .

[0150] The processing component 702 generally controls the overall operation of the electronic device 700, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 702 may include one or more modules to facilitate the interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0151] The memory 704 is configured to store various types of data to support operations on the electronic device 700. Examples of such data include instructions for any application or method operating on the electronic device 700, contact data, phone book data, messages, pictures, videos, etc. The memory 704 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, an optical disk, or a graphene memory.

[0152] The power supply component 706 provides power to the various components of the electronic device 700. The power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 700.

[0153] The multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0154] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), and when the electronic device 700 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 704 or sent via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting an audio signal.

[0155] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0156] The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the electronic device 700. For example, the sensor assembly 714 can detect the open / closed state of the electronic device 700, the relative positioning of the components, such as the display and keypad of the electronic device 700, and the sensor assembly 714 can also detect the position change of the electronic device 700 or the components of the electronic device 700, the presence or absence of contact between the user and the electronic device 700, the orientation or acceleration / deceleration of the device 700, and the temperature change of the electronic device 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0157] The communication component 716 is configured to facilitate wired or wireless communication between the electronic device 700 and other devices. The electronic device 700 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0158] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0159] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, and the above instructions can be executed by the processor 720 of the electronic device 700 to perform the above method. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0160] In an exemplary embodiment, a computer program product is further provided. The computer program product includes instructions. The instructions can be executed by the processor 720 of the electronic device 700 to complete the above method.

[0161] It should be noted that the above-mentioned devices, electronic devices, computer-readable storage media, computer program products, etc. may also include other implementation methods according to the description of the method embodiments. The specific implementation methods can refer to the description of the relevant method embodiments, which will not be described one by one here.

[0162] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0163] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for testing a conversational robot system, characterized in that: include: Obtain multiple test cases corresponding to the response test task for the conversational robot system; For each of the test cases, import the test case parameters corresponding to the test case into the test method corresponding to the test case; The session content type corresponding to the session content included in the test case parameter includes text-form session content and / or image-form session content, and the session content type targeted by the test method corresponding to the test case matches the session content type of the test case parameter; According to the test request generation mode indicated by the test method, a response test request corresponding to the test case is constructed based on the test case parameters; The response test request is information for requesting the conversational robot system to respond to the conversation content; Sending a plurality of the response test requests to the conversational robot system, the conversational robot system being configured to output corresponding actual response content in response to the conversation content carried in the response test request; Receive the actual response content returned by the conversational robot system, and determine the test result corresponding to the response test task according to the actual response content.

2. The method according to claim 1, characterized in that The test case parameters are multiple groups of test case parameters, the test methods are multiple test methods for different types of session content, and the importing of the test case parameters corresponding to the test case into the test method corresponding to the test case includes: Obtain parameter type labels for each group of test case parameters; For each of the test methods, a target test case parameter is obtained from the multiple groups of test case parameters, and the target test case parameter is imported into the test method; the parameter type label of the target test case parameter matches the session content type of the test method.

3. The method according to claim 1, characterized in that After generating a response test request corresponding to the test case according to the test method corresponding to the test case, the method further includes: Determine a request sending condition corresponding to the response test request.

4. The method according to claim 3, characterized in that The step of sending the response test request to the conversational robot system comprises: In response to satisfying the request sending conditions corresponding to the response test request, the preset virtual client is called to send the response test request to the remote procedure call interface of the conversational robot system; the remote procedure call interface is used to call the semantic recognition service in response to the response test request to identify the conversation content carried by the response test request, and output the corresponding actual response content.

5. The method according to claim 1, characterized in that The receiving the actual response content returned by the conversational robot system includes: Receiving a response message returned by the conversational robot system in response to the response test request; The response message is parsed according to the transmission protocol corresponding to the response test request to obtain the actual response content carried by the response message.

6. The method according to claim 1, characterized in that The determining, according to the actual response content, a test result corresponding to the response test task includes: Obtaining the expected response content corresponding to the test case; Based on the difference between the expected response content and the actual response content, the test result of the conversational robot system is determined.

7. The method according to claim 1, characterized in that After determining the test result corresponding to the response test task according to the actual response content, the method further includes: Obtain a preset test report template, and generate a test report according to the test results and the test report template.

8. A testing device for a conversational robot system, characterized in that: include: A test case acquisition unit, configured to execute and acquire a plurality of test cases corresponding to a response test task for the conversational robot system; A request generating unit is configured to execute, for each of the test cases, import the test case parameters corresponding to the test case into the test method corresponding to the test case, and construct a response test request corresponding to the test case based on the test case parameters according to the test request generating mode indicated by the test method; The conversation content type corresponding to the conversation content included in the test case parameters includes text-based conversation content and / or picture-based conversation content. The conversation content type targeted by the test method corresponding to the test case matches the conversation content type of the test case parameters. The response test request is information for requesting the conversational robot system to respond to the conversation content. A request sending unit, configured to execute sending a plurality of the response test requests to the conversational robot system, wherein the conversational robot system is used to output corresponding actual response content in response to the conversation content carried by the response test request; The test result acquisition unit is configured to receive the actual response content returned by the conversational robot system, and determine the test result corresponding to the response test task according to the actual response content.

9. The device according to claim 8, characterized in that The test case parameters are multiple groups of test case parameters, the test methods are multiple test methods for different types of session content, and the request generation unit is specifically configured to execute: Obtain parameter type labels for each group of test case parameters; For each of the test methods, obtaining target test case parameters from the multiple groups of test case parameters, and importing the target test case parameters into the test method; The parameter type tag of the target test case parameter matches the session content type of the test method.

10. The device according to claim 8, characterized in that The request generation unit is further configured to execute: Determine a request sending condition corresponding to the response test request.

11. The device according to claim 10, characterized in that The request sending unit is specifically configured to execute: In response to satisfying a request sending condition corresponding to the response test request, calling a preset virtual client to send the response test request to a remote procedure call interface of the conversational robot system; The remote procedure call interface is used to call the semantic recognition service in response to the response test request to identify the session content carried by the response test request, and output the corresponding actual response content.

12. The device according to claim 8, characterized in that The test result acquisition unit is specifically configured to execute: Receiving a response message returned by the conversational robot system in response to the response test request; The response message is parsed according to the transmission protocol corresponding to the response test request to obtain the actual response content carried by the response message.

13. The device according to claim 8, characterized in that The test result acquisition unit is specifically configured to execute: Obtaining the expected response content corresponding to the test case; Based on the difference between the expected response content and the actual response content, the test result of the conversational robot system is determined.

14. The device according to claim 8, characterized in that Also includes: The test report generating unit is configured to execute obtaining a preset test report template and generate a test report according to the test result and the test report template.

15. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method as claimed in any one of claims 1 to 7.

17. A computer program product, comprising instructions, characterized in that: When the instructions are executed by a processor of an electronic device, the electronic device is enabled to perform the method according to any one of claims 1 to 7.

Images