Testing method, device, equipment and storage medium of voice robot

Abstract

The present invention relates to the field of artificial intelligence, and discloses a voice robot testing method, device, equipment and storage medium for improving the testing efficiency of the voice robot. The voice robot testing method includes: preprocessing the initial voice data to obtain preprocessed voice data, verifying the preprocessed voice data to obtain target voice data; creating a target test case set based on the target voice data and multiple node information; sending the target test case set to the voice robot to be tested, receiving multiple voice interaction results returned by the voice robot to be tested, comparing the multiple voice interaction results with a preset standard intent information table, and obtaining a voice test result; classifying the voice test results according to preset classification rules, creating a voice test report based on the classification results, and obtaining a target voice test report. In addition, the present invention also relates to blockchain technology, and the target voice test report can be stored in the blockchain.

Description

Technical Field

[0001] The present invention relates to the field of long short-term memory network models, and in particular to a testing method, device, equipment and storage medium for a voice robot. Background Art

[0002] After the voice robot process is built, a delivery test is required. Only after the test results are evaluated as qualified can it be put into production for outbound calls. Therefore, text robot testing in robot process management is a very important part of it.

[0003] The text testing phase requires manual input of dialogue content to interact with the robot in order to observe the flow of the process. Since the waiting time of the text robot is short, the process is often completed before manual input is allowed. The process under test needs to be tested repeatedly later, and in order to ensure the quality of the robot, each branch process needs to be covered for testing. However, if the existing text robot test needs to cover all branches of the process, it may take up to 3-5 days. The recognition results also need to be queried and proofread in the background of the robot process management system interface, resulting in low testing efficiency of the voice robot. Summary of the invention

[0004] The present invention provides a voice robot testing method, apparatus, equipment and storage medium, which are used to create a target test case set based on target voice data and multiple node information, send the target test case set to the voice robot to be tested, receive multiple voice interaction results returned by the voice robot to be tested, compare the multiple voice interaction results with a preset standard intent information table, obtain voice test results, and improve the testing efficiency of the voice robot.

[0005] The first aspect of the present invention provides a testing method for a voice robot, comprising: obtaining initial voice data, preprocessing the initial voice data to obtain preprocessed voice data, verifying the preprocessed voice data to obtain target voice data, wherein the target voice data includes multiple node information; creating a target test case set based on the target voice data and multiple node information in the target voice data, wherein the target test case set corresponds to a voice robot; sending the target test case set to a voice robot to be tested, receiving multiple voice interaction results returned by the voice robot to be tested, comparing the multiple voice interaction results with a preset standard intent information table to obtain a voice test result; classifying the voice test result according to a preset classification rule to obtain a classification result, creating a voice test report based on the classification result to obtain a target voice test report.

[0006] Optionally, in a first implementation method of the first aspect of the present invention, the initial voice data is obtained, the initial voice data is preprocessed to obtain preprocessed voice data, the preprocessed voice data is verified to obtain target voice data, and the target voice data includes multiple node information including: obtaining initial voice data, calling a preset voice endpoint detection algorithm, segmenting the initial voice data to obtain voice segmentation segments, filtering the voice segmentation segments and deleting invalid segments to obtain preprocessed voice data, wherein the invalid segments are voice segments containing noise signals and silent segments; reading global variable parameters in the preprocessed voice data, determining whether the global variable parameters meet preset verification conditions, obtaining verification results, extracting global variable parameters that meet the preset verification conditions from the verification results, obtaining target parameters, and determining the preprocessed voice data corresponding to the target parameters as target voice data, wherein the target voice data includes multiple node information.

[0007] Optionally, in a second implementation method of the first aspect of the present invention, a target test case set is created based on the target voice data and multiple node information in the target voice data, and the target test case set corresponds to a voice robot: the business line identification number corresponding to the target voice data is obtained, and the target voice data is classified based on a preset classification rule and the business line identification number to obtain multiple test case data sets, and one test case data set corresponds to one test case; based on a preset case naming rule and multiple node information in the target voice data, multiple case names are generated, and one case name corresponds to one test case; a preset snowflake algorithm is called to create a preset number of business serial numbers to obtain multiple business serial numbers, and one business serial number corresponds to one test case; the multiple test case data sets, the multiple case names and the multiple business serial numbers are loaded into a preset case set template to obtain multiple test cases, and the multiple test cases are determined as a target test case set, and the target test case set corresponds to a voice robot, and each test case corresponds to a process node of the voice robot.

[0008] Optionally, in a third implementation method of the first aspect of the present invention, sending the target test case set to the voice robot to be tested, receiving multiple voice interaction results returned by the voice robot to be tested, and comparing the multiple voice interaction results with a preset standard intent information table to obtain a voice test result includes: sending the target test case set to the voice robot to be tested, receiving multiple voice interaction results returned by the voice robot to be tested after voice interaction based on the target test case set; obtaining standard intent information corresponding to the target test case set from a preset standard intent information table, calculating and sorting the multiple voice interaction results with the standard intent information to obtain a voice test result.

[0009] Optionally, in a fourth implementation method of the first aspect of the present invention, obtaining standard intent information corresponding to the target test case set from a preset standard intent information table, calculating and sorting the similarity between the multiple voice interaction results and the standard intent information, and obtaining a voice test result includes: obtaining the standard intent information corresponding to the target test case set from a preset standard intent information table, calling a preset similarity model, calculating the similarity between the multiple voice interaction results and the standard intent information, and obtaining multiple similarity scores; calling a preset bubble sort algorithm, sorting the multiple similarity scores in order from large to small, obtaining a similarity sorting result, and determining the similarity sorting result and the corresponding multiple voice interaction results as the voice test result.

[0010] Optionally, in a fifth implementation manner of the first aspect of the present invention, the speech test results are classified according to preset classification rules to obtain classification results, and a speech test report is created based on the classification results to obtain a target speech test report, including: obtaining a similarity score threshold, and classifying the speech test results based on the similarity score threshold and preset classification rules to obtain classification results, wherein the classification results include test success cases and test failure cases; loading the classification results into a preset speech test report template to obtain a target speech test report, wherein the target speech test report includes case success rate, case failure rate and case test time.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, before obtaining initial voice data, preprocessing the initial voice data to obtain preprocessed voice data, verifying the preprocessed voice data, and obtaining target voice data, the voice robot testing method also includes: obtaining historical test data, preprocessing the historical test data to obtain preprocessed historical data, the historical test data including historical voice data and historical case test results corresponding to the historical voice data; loading the preprocessed historical data into a preset intent information template to obtain a standard intent information table.

[0012] The second aspect of the present invention provides a testing device for a voice robot, comprising: an acquisition module, used to acquire initial voice data, preprocess the initial voice data to obtain preprocessed voice data, verify the preprocessed voice data to obtain target voice data, wherein the target voice data includes multiple node information; a creation module, used to create a target test case set based on the target voice data and multiple node information in the target voice data, wherein the target test case set includes multiple test cases corresponding to a voice robot; a comparison module, used to send the target test case set to the voice robot to be tested, receive multiple voice interaction results returned by the voice robot to be tested, compare the multiple voice interaction results with a preset standard intent information table, and obtain a voice test result; a classification module, used to classify the voice test result according to a preset classification rule to obtain a classification result, create a voice test report based on the classification result, and obtain a target voice test report.

[0013] Optionally, in a first implementation method of the second aspect of the present invention, the acquisition module includes: a segmentation unit, used to acquire initial voice data, call a preset voice endpoint detection algorithm, segment the initial voice data, obtain voice segmentation segments, filter the voice segmentation segments and delete invalid segments to obtain preprocessed voice data, wherein the invalid segments are voice segments containing noise signals and silent segments; a judgment unit, used to read global variable parameters in the preprocessed voice data, judge whether the global variable parameters meet preset verification conditions, obtain verification results, extract global variable parameters that meet the preset verification conditions from the verification results, obtain target parameters, and determine the preprocessed voice data corresponding to the target parameters as target voice data, wherein the target voice data includes multiple node information.

[0014] Optionally, in a second implementation of the second aspect of the present invention, the creation module includes: an acquisition unit, used to acquire the business line identification number corresponding to the target voice data, and classify the target voice data based on a preset classification rule and the business line identification number to obtain multiple test case data sets, one test case data set corresponding to one test case; a determination unit, used to generate multiple case names based on a preset case naming rule and multiple node information in the target voice data, one case name corresponding to one test case; a matching unit, used to call a preset snowflake algorithm to create a preset number of business serial numbers to obtain multiple business serial numbers, one business serial number corresponding to one test case; a first generation unit, used to load the multiple test case data sets, the multiple case names and the multiple business serial numbers into a preset case set template to obtain multiple test cases, and determine the multiple test cases as a target test case set, the target test case set corresponds to a voice robot, and each test case corresponds to a process node of the voice robot.

[0015] Optionally, in a third implementation method of the second aspect of the present invention, the comparison module includes: a sending unit, used to send the target test case set to the voice robot to be tested, and receive multiple voice interaction results returned by the voice robot to be tested after voice interaction based on the target test case set; a similarity calculation unit, used to obtain standard intent information corresponding to the target test case set from a preset standard intent information table, calculate and sort the similarity of the multiple voice interaction results with the standard intent information, and obtain a voice test result.

[0016] Optionally, in a fourth implementation method of the second aspect of the present invention, the similarity calculation unit can be specifically used to: obtain standard intent information corresponding to the target test case set from a preset standard intent information table, call a preset similarity model, perform similarity calculation on the multiple voice interaction results and the standard intent information, and obtain multiple similarity scores; call a preset bubble sort algorithm, sort the multiple similarity scores in order from large to small, obtain a similarity sorting result, and determine the similarity sorting result and the corresponding multiple voice interaction results as voice test results.

[0017] Optionally, in a fifth implementation of the second aspect of the present invention, the classification module includes: a classification unit, used to obtain a similarity score threshold, and classify the speech test results based on the similarity score threshold and a preset classification rule to obtain classification results, wherein the classification results include test success cases and test failure cases; a second generation unit, used to load the classification results into a preset speech test report template to obtain a target speech test report, wherein the target speech test report includes a case success rate, a case failure rate and a case test time.

[0018] Optionally, in a sixth implementation method of the second aspect of the present invention, before the acquisition module, the testing device of the voice robot also includes a construction module, and the construction module includes: a preprocessing unit, used to acquire historical test data, preprocess the historical test data, and obtain preprocessed historical data, wherein the historical test data includes historical voice data and historical case test results corresponding to the historical voice data; a loading unit, used to load the preprocessed historical data into a preset intent information template to obtain a standard intent information table.

[0019] The third aspect of the present invention provides a testing device for a voice robot, comprising: a memory and at least one processor, wherein a computer program is stored in the memory; the at least one processor calls the computer program in the memory so that the testing device for the voice robot executes the above-mentioned testing method for the voice robot.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned voice robot testing method.

[0021] In the technical solution provided by the present invention, initial voice data is obtained, the initial voice data is preprocessed to obtain preprocessed voice data, the preprocessed voice data is verified to obtain target voice data, and the target voice data includes multiple node information; a target test case set is created based on the target voice data and multiple node information in the target voice data, and the target test case set corresponds to a voice robot; the target test case set is sent to the voice robot to be tested, multiple voice interaction results returned by the voice robot to be tested are received, and the multiple voice interaction results are compared with the preset standard intent information table to obtain a voice test result; the voice test result is classified according to the preset classification rules to obtain a classification result, and a voice test report is created based on the classification result to obtain a target voice test report. In an embodiment of the present invention, a target test case set is created based on the target voice data and multiple node information, the target test case set is sent to the voice robot to be tested, multiple voice interaction results returned by the voice robot to be tested are received, and the multiple voice interaction results are compared with the preset standard intent information table to obtain a voice test result, thereby improving the test efficiency of the voice robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of an embodiment of a method for testing a voice robot in an embodiment of the present invention;

[0023] Figure 2 Schematic diagram of another embodiment of the method for testing a voice robot in an embodiment of the present invention;

[0024] Figure 3 Schematic diagram of an embodiment of a testing device for a voice robot in an embodiment of the present invention;

[0025] Figure 4 It is a schematic diagram of another embodiment of a testing device for a voice robot in an embodiment of the present invention;

[0026] Figure 5 Schematic diagram of an embodiment of a testing device for a voice robot in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The embodiments of the present invention provide a voice robot testing method, apparatus, equipment and storage medium, which are used to create a target test case set based on target voice data and multiple node information, send the target test case set to the voice robot to be tested, receive multiple voice interaction results returned by the voice robot to be tested, compare the multiple voice interaction results with a preset standard intent information table, obtain voice test results, and improve the testing efficiency of the voice robot.

[0028] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the testing method of the voice robot in the embodiment of the present invention includes:

[0030] 101. Acquire initial voice data, preprocess the initial voice data to obtain preprocessed voice data, verify the preprocessed voice data to obtain target voice data, wherein the target voice data includes a plurality of node information.

[0031] It is understandable that the execution subject of the present invention can be a test device for a voice robot, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking the server as the execution subject as an example. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0032] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0033] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0034] The server obtains the initial voice data, preprocesses the initial voice data, obtains the preprocessed voice data, verifies the preprocessed voice data, and obtains the target voice data, which includes multiple node information. The process of the server preprocessing the initial voice data mainly involves calling the Voice Activity Detection (VAD) algorithm to segment the initial voice data, obtain voice segmentation segments, filter the voice segmentation segments and delete invalid segments. The Voice Activity Detection (VAD) algorithm is to separate the effective voice signal from the useless voice signal or noise signal. It is necessary to find the start and end points of the voice part from the input signal and extract the signal features required for voice emotion recognition. After generating the preprocessed voice data, the server reads the global variable parameters in the preprocessed voice data, determines whether the global variable parameters meet the preset verification conditions, and determines the preprocessed voice data that meets the preset verification conditions in the verification results as the target voice data. The preset verification condition is that the global variable parameters meet the corresponding preset fixed values.

[0035] 102. Create a target test case set based on the target voice data and multiple node information in the target voice data, where the target test case set corresponds to a voice robot.

[0036] The server creates a target test case set based on the target voice data and multiple node information in the target voice data, and the target test case set corresponds to a voice robot. After generating the target voice data, the server classifies the target voice data according to the preset classification rules. The target voice data contains the business line identification number to which the corresponding scene belongs. The preset classification rule is to classify the target voice data with the same business line identification number into one category, and obtain multiple test case data sets. One test case data set corresponds to one test case. At the same time, multiple test cases are named, and a business serial number corresponding to each test case is created. Multiple test case data sets, multiple case names, and multiple business serial numbers are loaded into a preset case set template. The preset case set template contains multiple files, such as case name file, business serial number file, case specific content file, remark information, etc. After the server loads multiple test case data sets, multiple case names, and multiple business serial numbers into the case set template, it will automatically load different information into different files and finally obtain the target test case set. Multiple test cases can be filled in a case set template.

[0037] 103. Send the target test case set to the voice robot to be tested, receive multiple voice interaction results returned by the voice robot to be tested, compare the multiple voice interaction results with the preset standard intent information table, and obtain the voice test results.

[0038] The server sends the target test case set to the voice robot to be tested, receives multiple voice interaction results returned by the voice robot to be tested, compares the multiple voice interaction results with the preset standard intent information table, and obtains the voice test results. The server sends the target test case set to the voice robot to be tested so that the voice robot to be tested performs voice interaction, randomly selects a test case for voice interaction test each time, until all test cases in the target test case are tested, receives multiple voice interaction results returned by the voice robot to be tested, obtains the standard intent information corresponding to the target test case from the preset intent table, and obtains the voice test results after similarity matching. The number of voice robots to be tested is one, and one voice robot to be tested corresponds to one target test case set.

[0039] 104. Classify the speech test results according to preset classification rules to obtain classification results, create a speech test report based on the classification results, and obtain a target speech test report.

[0040] The server classifies the speech test results according to the preset classification rules to obtain the classification results, creates a speech test report based on the classification results, and obtains the target speech test report. The server obtains the similarity score threshold, classifies the test cases corresponding to the similarity scores greater than or equal to the similarity score threshold in the speech test results as successful cases, and classifies the test cases corresponding to the similarity scores less than the similarity score threshold in the speech test results as failed cases, and obtains the classification results. By loading the classification results into the preset speech test report template, the case success rate and case failure rate can be automatically calculated to obtain the target speech test report.

[0041] In an embodiment of the present invention, a target test case set is created based on target voice data and multiple node information, the target test case set is sent to the voice robot to be tested, multiple voice interaction results returned by the voice robot to be tested are received, and the multiple voice interaction results are compared with a preset standard intent information table to obtain a voice test result, thereby improving the testing efficiency of the voice robot.

[0042] See also Figure 2 Another embodiment of the method for testing a voice robot in the embodiment of the present invention includes:

[0043] 201. Acquire initial voice data, pre-process the initial voice data to obtain pre-processed voice data, verify the pre-processed voice data to obtain target voice data, wherein the target voice data includes a plurality of node information.

[0044] The server obtains initial voice data, preprocesses the initial voice data to obtain preprocessed voice data, verifies the preprocessed voice data, and obtains target voice data, which includes multiple node information. Specifically, the server obtains initial voice data, calls a preset voice endpoint detection algorithm, segments the initial voice data, obtains voice segmentation segments, filters the voice segmentation segments and deletes invalid segments to obtain preprocessed voice data, where invalid segments are voice segments containing noise signals and silent segments; the server reads global variable parameters in the preprocessed voice data, determines whether the global variable parameters meet preset verification conditions, obtains verification results, extracts global variable parameters that meet the preset verification conditions from the verification results, obtains target parameters, and determines the preprocessed voice data corresponding to the target parameters as target voice data, which includes multiple node information.

[0045] In this embodiment, the VAD algorithm is called to segment the initial voice data, separate and filter out invalid segments, and obtain pre-processed voice data. The server reads the global variable parameters in the pre-processed voice data, determines whether the global variable parameters meet the preset verification conditions, and obtains the verification results, wherein the global variable parameters include the customer number, account number, complete card number, customer contact number, etc. The preset verification condition is that the global variable parameters meet the corresponding preset fixed values. If not, an error message will be returned. The server extracts the global variable parameters that meet the preset fixed values ​​in the verification results to obtain the target parameters, and determines the pre-processed voice data corresponding to the target parameters as the target voice data. The target voice data includes multiple node information. Information of multiple nodes includes the speech number, small intention, interface variable parameters, global variable parameters, jump speech number, number of repeated nodes, uploaded extraction information and interface identification number of each node. For example, a certain flow link includes R01 identity verification (Are you Mr. / Ms. XX?), and the customer answers "I am" at this node (the corresponding intention is "I am myself"), then enters the next node R02 for product introduction, where R01 is the speech number of the previous node, the small intention is "I am myself", the jump speech number is R02, the global variable parameters are the customer identity information in the R01 speech, and the interface variable parameters refer to the need to retrieve the bill interface parameters in real time when the speech contains the customer's bill amount.

[0046] 202. Create a target test case set based on the target voice data and multiple node information in the target voice data, where the target test case set corresponds to a voice robot.

[0047] The server creates a target test case set based on the target voice data and multiple node information in the target voice data, and the target test case set corresponds to a voice robot. Specifically, the server obtains the business line identification number corresponding to the target voice data, and classifies the target voice data based on the preset classification rules and the business line identification number to obtain multiple test case data sets, and one test case data set corresponds to one test case; the server generates multiple case names based on the preset case naming rules and multiple node information in the target voice data, and one case name corresponds to one test case; the server calls the preset snowflake algorithm to create a preset number of business serial numbers, and obtains multiple business serial numbers, and one business serial number corresponds to one test case; the server loads multiple test case data sets, multiple case names, and multiple business serial numbers into the preset case set template, and obtains multiple test cases, and determines the multiple test cases as the target test case set, and the target test case set corresponds to a voice robot, and each test case corresponds to a process node of the voice robot.

[0048] After generating the target voice data, the server classifies the target voice data according to the preset classification rules. The target voice data includes the business line identification number of the corresponding scenario. The preset classification rule is to classify the target voice data with the same business line identification number into one category. After the target voice data is classified, it is written into the preset test case template to obtain multiple test case data sets, wherein one test case data set corresponds to one test case. The server determines the case names corresponding to the multiple test cases based on the preset case naming rule, and one case name corresponds to one test case. The preset case naming rule is the speech number + jump speech number. Each case needs to be associated with a small intention, interface variable parameters and global variable parameters. If a re-listening node is encountered (that is, when the customer's words are recognized as the "not heard clearly" intention, it will return to the previous node to broadcast the preset auxiliary speech), the current node needs to be generated according to the number of repetitions, and the repeated node is used as the end node. The re-listening node is also added to the target test case set. For example: Template: I01-Re-listening-3, generated: I01-Re-listening-I01-Re-listening-I01-Re-listening-End. The snowflake algorithm is an open source distributed coding generation algorithm. In this embodiment, the snowflake algorithm is used to obtain a 64-bit long type business serial number, where the long type is a basic data type. The snowflake algorithm consists of a 1-bit identification bit, a 41-bit timestamp bit, a 10-bit storage machine code, and a 12-bit random number. The server splits the 10-bit storage machine code into a 2-bit data center bit and an 8-bit machine id bit. It is necessary to pre-acquire the 2-bit data center ID in the preset configuration file to fill the 2-bit data center bit. The generation rule of the 41-bit timestamp of the snowflake algorithm is the current timestamp minus the set start timestamp. The number of created business serial numbers is consistent with the number of test cases, and one business serial number corresponds to one test case. The preset case set template contains multiple sub-files, such as case name file, business serial number sub-file, case specific content sub-file, remark information, etc. After the server loads multiple test case data sets, multiple case names, and multiple business serial numbers into the case set template, it will automatically load different information into different sub-files to finally obtain the target test case set. Multiple test cases can be filled in a case set template.

[0049] 203. Send the target test case set to the voice robot to be tested, and receive multiple voice interaction results returned by the voice robot to be tested after performing voice interaction based on the target test case set.

[0050] The server sends the target test case set to the voice robot to be tested, and receives multiple voice interaction results returned by the voice robot to be tested after voice interaction based on the target test case set. The server sends the target test case set to the voice robot to be tested so that the voice robot to be tested can perform voice interaction. A test case is randomly selected each time for voice interaction testing until all test cases in the target test case are tested. Before the voice interaction test, it also includes the identification and judgment of the business serial number. If the unique business serial number is empty, an error message is returned (the test result is recorded as failure). If there is a business serial number corresponding to the test case, the test interface is called normally for voice interaction testing.

[0051] 204. Obtain standard intent information corresponding to the target test case set from a preset standard intent information table, calculate and sort the similarity between multiple voice interaction results and the standard intent information, and obtain a voice test result.

[0052] The server obtains the standard intent information corresponding to the target test case set from the preset standard intent information table, calculates and sorts the similarity between multiple voice interaction results and the standard intent information, and obtains the voice test result. Specifically, the server obtains the standard intent information corresponding to the target test case set from the preset standard intent information table, calls the preset similarity model, calculates the similarity between multiple voice interaction results and the standard intent information, and obtains multiple similarity scores; the server calls the preset bubble sort algorithm, sorts the multiple similarity scores in order from large to small, obtains the similarity sorting result, and determines the similarity sorting result and the corresponding multiple voice interaction results as the voice test result. After the voice interaction test is completed, the server receives multiple voice interaction results returned by the voice robot to be tested, obtains the standard intent information corresponding to the target test case from the preset standard intent information table, and uses the preset similarity model to calculate the similarity between each voice interaction result and the standard intent information to obtain the similarity score corresponding to each voice interaction result, thereby obtaining the similarity scores corresponding to the multiple voice interaction results, that is, multiple similarity scores, wherein the standard intent information table can be constructed in advance by obtaining historical voice data and historical case test results corresponding to the historical voice data, and loading them into a preset intent information template, the similarity model can be a long short term memory network (long short term memory, LSTM) model, a bidirectional long short term memory network (bi-directional long short term memory, Bi-LSTM) model or an enhanced sequential inference model (enhanced sequential inference model, ESIM) model, the server calls a bubble sort algorithm, sorts the multiple similarity scores in order from large to small, obtains a similarity sorting result, and determines the similarity sorting result and the corresponding multiple voice interaction results as the voice test result.

[0053] 205. Classify the speech test results according to preset classification rules to obtain classification results, create a speech test report based on the classification results, and obtain a target speech test report.

[0054] The server classifies the speech test results according to the preset classification rules to obtain the classification results, creates a speech test report based on the classification results, and obtains the target speech test report. Specifically, the server obtains the similarity score threshold, classifies the speech test results based on the similarity score threshold and the preset classification rules, and obtains the classification results, which include test success cases and test failure cases; the server loads the classification results into the preset speech test report template to obtain the target speech test report, which includes the case success rate, case failure rate, and case test time.

[0055] The server obtains a similarity score threshold, classifies the test cases corresponding to similarity scores greater than or equal to the similarity score threshold in the speech test results as successful cases, and classifies the test cases corresponding to similarity scores less than the similarity score threshold in the speech test results as failed cases, to obtain a classification result. By loading the classification result into a preset speech test report template, the case success rate and the case failure rate can be automatically calculated to obtain a target speech test report. For example, the number of test cases corresponding to the similarity scores in the speech test results is 100, of which 60 test cases are greater than the similarity score threshold, and the case success rate is 60 / 100=60%. The target speech test report contains multiple test information, and the multiple test information can be case success rate, number of successful cases, case failure rate, number of failed cases, case test start time, case test end time, etc.

[0056] In an embodiment of the present invention, a target test case set is created based on target voice data and multiple node information, the target test case set is sent to the voice robot to be tested, multiple voice interaction results returned by the voice robot to be tested are received, and the multiple voice interaction results are compared with a preset standard intent information table to obtain a voice test result, thereby improving the testing efficiency of the voice robot.

[0057] The above describes the testing method of the voice robot in the embodiment of the present invention. The following describes the testing device of the voice robot in the embodiment of the present invention. Figure 3 , an embodiment of a testing device for a voice robot in an embodiment of the present invention includes:

[0058] The acquisition module 301 is used to acquire initial voice data, pre-process the initial voice data to obtain pre-processed voice data, verify the pre-processed voice data to obtain target voice data, and the target voice data includes a plurality of node information;

[0059] A creation module 302, used to create a target test case set based on the target voice data and multiple node information in the target voice data, where the target test case set corresponds to a voice robot;

[0060] The comparison module 303 is used to send the target test case set to the voice robot to be tested, receive multiple voice interaction results returned by the voice robot to be tested, compare the multiple voice interaction results with the preset standard intent information table, and obtain the voice test result;

[0061] The classification module 304 is used to classify the speech test results according to preset classification rules to obtain classification results, and to create a speech test report based on the classification results to obtain a target speech test report.

[0062] In an embodiment of the present invention, a target test case set is created based on target voice data and multiple node information, the target test case set is sent to the voice robot to be tested, multiple voice interaction results returned by the voice robot to be tested are received, and the multiple voice interaction results are compared with a preset standard intent information table to obtain a voice test result, thereby improving the testing efficiency of the voice robot.

[0063] See also Figure 4 Another embodiment of the testing device of the voice robot in the embodiment of the present invention includes:

[0064] The acquisition module 301 is used to acquire initial voice data, pre-process the initial voice data to obtain pre-processed voice data, verify the pre-processed voice data to obtain target voice data, and the target voice data includes a plurality of node information;

[0065] A creation module 302, used to create a target test case set based on the target voice data and multiple node information in the target voice data, where the target test case set corresponds to a voice robot;

[0066] The comparison module 303 is used to send the target test case set to the voice robot to be tested, receive multiple voice interaction results returned by the voice robot to be tested, compare the multiple voice interaction results with the preset standard intent information table, and obtain the voice test result;

[0067] The comparison module 303 specifically includes:

[0068] The sending unit 3031 is used to send the target test case set to the voice robot to be tested, and receive multiple voice interaction results returned by the voice robot to be tested after performing voice interaction based on the target test case set;

[0069] A similarity calculation unit 3032 is used to obtain standard intent information corresponding to the target test case set from a preset standard intent information table, perform similarity calculation and sorting on multiple voice interaction results and the standard intent information, and obtain a voice test result;

[0070] The classification module 304 is used to classify the speech test results according to preset classification rules to obtain classification results, and to create a speech test report based on the classification results to obtain a target speech test report.

[0071] Optionally, the acquisition module 301 includes:

[0072] The segmentation unit 3011 is used to obtain initial voice data, call a preset voice endpoint detection algorithm, segment the initial voice data to obtain voice segmentation segments, filter the voice segmentation segments and delete invalid segments to obtain pre-processed voice data, where the invalid segments are voice segments containing noise signals and silent segments;

[0073] The judgment unit 3012 is used to read the global variable parameters in the preprocessed voice data, judge whether the global variable parameters meet the preset verification conditions, obtain the verification results, determine the global variable parameters that meet the preset verification conditions in the verification results as target parameters, and determine the preprocessed voice data corresponding to the target parameters as target voice data, and the target voice data includes multiple node information.

[0074] Optionally, the creation module 302 includes:

[0075] The acquisition unit 3021 is used to acquire the business line identification number corresponding to the target voice data, and classify the target voice data based on the preset classification rule and the business line identification number to obtain multiple test case data sets, where one test case data set corresponds to one test case;

[0076] A determination unit 3022, configured to generate multiple case names based on a preset case naming rule and multiple node information in the target voice data, where one case name corresponds to one test case;

[0077] The matching unit 3023 is used to call a preset snowflake algorithm to create a preset number of business serial numbers to obtain multiple business serial numbers, where one business serial number corresponds to one test case;

[0078] The first generation unit 3024 is used to load multiple test case data sets, multiple case names and multiple business serial numbers into a preset case set template to obtain multiple test cases, and determine the multiple test cases as a target test case set. The target test case set corresponds to a voice robot, and each test case corresponds to a process node of the voice robot.

[0079] Optionally, the similarity calculation unit 3032 may be specifically used for:

[0080] Obtain the standard intent information corresponding to the target test case set from the preset standard intent information table, call the preset similarity model, calculate the similarity between multiple voice interaction results and the standard intent information, and obtain multiple similarity scores; call the preset bubble sort algorithm, sort the multiple similarity scores in descending order, obtain the similarity sorting result, and determine the similarity sorting result and the corresponding multiple voice interaction results as the voice test result.

[0081] Optionally, the classification module 304 includes:

[0082] The classification unit 3041 is used to obtain a similarity score threshold, and classify the speech test results based on the similarity score threshold and a preset classification rule to obtain a classification result, where the classification result includes a test success case and a test failure case;

[0083] The second generating unit 3042 is used to load the classification result into a preset voice test report template to obtain a target voice test report, which includes a case success rate, a case failure rate and a case test time.

[0084] Optionally, before acquiring module 301, the voice robot testing device further includes a construction module 305, including:

[0085] A preprocessing unit 3051 is used to obtain historical test data, preprocess the historical test data, and obtain preprocessed historical data, where the historical test data includes historical voice data and historical case test results corresponding to the historical voice data;

[0086] The loading unit 3052 is used to load the pre-processed historical data into a preset intent information template to obtain a standard intent information table.

[0087] In an embodiment of the present invention, a target test case set is created based on target voice data and multiple node information, the target test case set is sent to the voice robot to be tested, multiple voice interaction results returned by the voice robot to be tested are received, and the multiple voice interaction results are compared with a preset standard intent information table to obtain a voice test result, thereby improving the testing efficiency of the voice robot.

[0088] above Figure 3 and Figure 4 The testing device of the voice robot in the embodiment of the present invention is described in detail from the perspective of modular functional entities, and the testing equipment of the voice robot in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0089] Figure 51 is a schematic diagram of the structure of a voice robot test device provided by an embodiment of the present invention. The voice robot test device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 510 (for example, one or more processors) and a memory 520, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 533 or data 532. Among them, the memory 520 and the storage medium 530 can be short-term storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of computer program operations in the voice robot test device 500. Furthermore, the processor 510 can be configured to communicate with the storage medium 530 to execute a series of computer program operations in the storage medium 530 on the voice robot test device 500.

[0090] The voice robot test device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input and output interfaces 560, and / or one or more operating systems 531, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 5 The structure of the test equipment for the voice robot shown does not constitute a limitation on the test equipment for the voice robot, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0091] The present application also provides a voice robot testing device, comprising: a memory and at least one processor, wherein a computer program is stored in the memory, and the memory and the at least one processor are interconnected via a line; the at least one processor calls the computer program in the memory so that the voice robot testing device executes the steps in the above-mentioned voice robot testing method.

[0092] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. A computer program is stored in the computer-readable storage medium. When the computer program runs on a computer, the computer executes the steps of the voice robot testing method.

[0093] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0094] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0095] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0096] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several computer programs to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0097] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for testing a voice robot, characterized in that: The testing method of the voice robot includes: Acquire initial voice data, preprocess the initial voice data to obtain preprocessed voice data, verify the preprocessed voice data to obtain target voice data, wherein the target voice data includes a plurality of node information; Creating a target test case set based on the target voice data and multiple node information in the target voice data, wherein the target test case set corresponds to a voice robot; Sending the target test case set to the voice robot to be tested, receiving multiple voice interaction results returned by the voice robot to be tested, and comparing the multiple voice interaction results with a preset standard intent information table to obtain a voice test result, wherein the standard intent information table is constructed in advance by obtaining historical voice data and historical case test results corresponding to the historical voice data, and loading them into a preset intent information template; Classifying the speech test results according to preset classification rules to obtain classification results, creating a speech test report based on the classification results to obtain a target speech test report; The method of sending the target test case set to the voice robot to be tested, receiving multiple voice interaction results returned by the voice robot to be tested, and comparing the multiple voice interaction results with a preset standard intent information table to obtain a voice test result includes: sending the target test case set to the voice robot to be tested, receiving multiple voice interaction results returned by the voice robot to be tested after voice interaction based on the target test case set; obtaining standard intent information corresponding to the target test case set from a preset standard intent information table, calculating and sorting the similarity between the multiple voice interaction results and the standard intent information, and obtaining a voice test result.

2. The voice robot testing method according to claim 1, characterized in that: The initial voice data is obtained, the initial voice data is preprocessed to obtain preprocessed voice data, the preprocessed voice data is verified to obtain target voice data, and the target voice data includes a plurality of node information including: Acquire initial voice data, call a preset voice endpoint detection algorithm, segment the initial voice data to obtain voice segmentation segments, filter the voice segmentation segments and delete invalid segments to obtain preprocessed voice data, wherein the invalid segments are voice segments containing noise signals and silent segments; Read the global variable parameters in the preprocessed voice data, determine whether the global variable parameters meet the preset verification conditions, obtain the verification results, extract the global variable parameters that meet the preset verification conditions from the verification results, obtain the target parameters, and determine the preprocessed voice data corresponding to the target parameters as the target voice data, wherein the target voice data includes multiple node information.

3. The voice robot testing method according to claim 1, characterized in that: The target test case set is created based on the target voice data and multiple node information in the target voice data, and the target test case set corresponds to a voice robot: Obtaining a business line identification number corresponding to the target voice data, and classifying the target voice data based on a preset classification rule and the business line identification number to obtain multiple test case data sets, where one test case data set corresponds to one test case; Based on a preset case naming rule and multiple node information in the target voice data, multiple case names are generated, where one case name corresponds to one test case; Call the preset snowflake algorithm to create a preset number of business serial numbers to obtain multiple business serial numbers, one business serial number corresponds to one test case; The multiple test case data sets, the multiple case names and the multiple business serial numbers are loaded into a preset case set template to obtain multiple test cases, and the multiple test cases are determined as a target test case set, wherein the target test case set corresponds to a voice robot, and each test case corresponds to a process node of the voice robot.

4. The voice robot testing method according to claim 1, characterized in that: The step of acquiring standard intent information corresponding to the target test case set from a preset standard intent information table, calculating and sorting the similarity between the multiple voice interaction results and the standard intent information, and obtaining the voice test results includes: Obtaining standard intent information corresponding to the target test case set from a preset standard intent information table, calling a preset similarity model, performing similarity calculation on the multiple voice interaction results and the standard intent information, and obtaining multiple similarity scores; A preset bubble sorting algorithm is called to sort the multiple similarity scores in descending order to obtain a similarity sorting result, and the similarity sorting result and multiple voice interaction results corresponding to the similarity sorting result are determined as voice test results.

5. The voice robot testing method according to claim 1, characterized in that: The method of classifying the speech test results according to preset classification rules to obtain classification results, creating a speech test report based on the classification results, and obtaining a target speech test report includes: Obtaining a similarity score threshold, and classifying the speech test result based on the similarity score threshold and a preset classification rule to obtain a classification result, wherein the classification result includes a test success case and a test failure case; The classification result is loaded into a preset voice test report template to obtain a target voice test report, wherein the target voice test report includes a case success rate, a case failure rate and a case test time.

6. The method for testing a voice robot according to any one of claims 1 to 5, characterized in that: Before obtaining the initial voice data, preprocessing the initial voice data to obtain the preprocessed voice data, and verifying the preprocessed voice data to obtain the target voice data, the voice robot testing method further includes: Acquire historical test data, preprocess the historical test data to obtain preprocessed historical data, the historical test data including historical voice data and historical case test results corresponding to the historical voice data; The preprocessed historical data is loaded into a preset intent information template to obtain a standard intent information table.

7. A voice robot testing device, characterized in that: The testing device of the voice robot comprises: An acquisition module, used to acquire initial voice data, pre-process the initial voice data to obtain pre-processed voice data, verify the pre-processed voice data to obtain target voice data, wherein the target voice data includes a plurality of node information; A creation module, used to create a target test case set based on the target voice data and multiple node information in the target voice data, wherein the target test case set includes multiple test cases corresponding to a voice robot; A comparison module is used to send the target test case set to the voice robot to be tested, receive multiple voice interaction results returned by the voice robot to be tested, and compare the multiple voice interaction results with a preset standard intent information table to obtain a voice test result, wherein the standard intent information table is constructed in advance by obtaining historical voice data and historical case test results corresponding to the historical voice data, and loading them into a preset intent information template; A classification module, used to classify the speech test results according to preset classification rules to obtain classification results, and create a speech test report based on the classification results to obtain a target speech test report; The comparison module includes: a sending unit, used to send the target test case set to the voice robot to be tested, and receive multiple voice interaction results returned by the voice robot to be tested after voice interaction based on the target test case set; a similarity calculation unit, used to obtain standard intent information corresponding to the target test case set from a preset standard intent information table, calculate and sort the similarity of the multiple voice interaction results with the standard intent information, and obtain a voice test result.

8. A voice robot testing device, characterized in that: The voice robot test device comprises: a memory and at least one processor, wherein the memory stores a computer program; The at least one processor calls the computer program in the memory so that the voice robot testing device executes the voice robot testing method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for testing the voice robot as described in any one of claims 1 to 6 is implemented.

Images