A customer service system testing method, device and electronic equipment

By automatically obtaining and processing test configuration data, and using neural network models and monitoring data to judge abnormalities, efficient testing of customer service systems is achieved, solving the problem of inefficiency in traditional testing methods.

CN112835805BActive Publication Date: 2024-11-22BANK OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110217708.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-02-26
Publication Date
2024-11-22
Estimated Expiration
2041-02-26

AI Technical Summary

Technical Problem

Traditional customer service system testing methods require testers to monitor each test node, resulting in low testing efficiency.

Method used

By obtaining the target test data containing the target test nodes and testing them according to the order of nodes, using neural network models to predict test data and historical data to generate test configurations, automate the test process, monitor the data and judge abnormalities and perform corresponding operations.

Benefits of technology

No testers are required to monitor each test node, which significantly improves the testing efficiency of the customer service system and realizes automated testing of multiple test nodes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112835805B_ABST
    Figure CN112835805B_ABST
Patent Text Reader

Abstract

The present application discloses a testing method, device and electronic device for a customer service system, the method comprising: obtaining test configuration data; the test configuration data comprising target test data of at least one target test node, and the target test nodes having a node sequence; inputting the test configuration data into a customer service system to be tested, so that the customer service system tests the target test node according to the node sequence and the target test data of the target test node, so as to obtain output content of the customer service system on the target test node; comparing the output content of the customer service system on the target test node with the corresponding preset standard content on the target test node, so as to obtain a test result of the customer service system on the target test node.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent customer service technology, and in particular to a testing method, device and electronic equipment for a customer service system. Background Art

[0002] In order to reduce labor costs in various service companies, intelligent customer service systems are used to answer users' questions.

[0003] In order to ensure the normal operation of the intelligent customer service system, the built intelligent customer service system is usually tested before it goes online.

[0004] However, the traditional testing method requires testers to monitor each test node of the intelligent customer service system, such as requiring testers to input test data or select input options at each test node, which results in low testing efficiency of the intelligent customer service system. Summary of the invention

[0005] In view of this, the present application provides a customer service system testing method, device and electronic equipment to solve the technical problem of low customer service system testing efficiency.

[0006] The present application provides a method for testing a customer service system, the method comprising:

[0007] Obtaining test configuration data; the test configuration data includes target test data of at least one target test node, and the target test nodes have a node order;

[0008] Inputting the test configuration data into the customer service system to be tested, so that the customer service system tests the target test node according to the target test data of the target test node in the node order, so as to obtain the output content of the customer service system on the target test node;

[0009] The output content of the customer service system on the target test node is compared with the corresponding preset standard content on the target test node to obtain the test result of the customer service system on the target test node.

[0010] The above method, preferably, obtains the test configuration data, including:

[0011] Outputting a configuration interface, wherein the configuration interface includes a plurality of test nodes to be selected, each of the test nodes to be selected corresponds to test data to be selected, and the test data to be selected is obtained according to historical test data;

[0012] Receiving input operations for the test nodes to be selected and the test data to be selected in the configuration interface;

[0013] According to the input operation, at least one target test node and target test data of the target test node are obtained, and there is a node sequence between the target test nodes.

[0014] The above method, preferably, obtains the test configuration data, including:

[0015] Obtaining a current test case, wherein the current test case at least includes a plurality of test nodes to be selected;

[0016] Inputting the current test case into a neural network model to obtain at least one target test node output by the neural network model and test data of the target test node;

[0017] The neural network model is trained using multiple historical test cases, wherein the historical test cases include multiple initial test nodes and at least one historical test node, and the historical test node includes historical test data. The neural network model is trained using at least the initial test node as an input sample and the historical test node and the historical test data as output samples.

[0018] In the above method, preferably, the neural network model uses the initial test node, the adjacent test nodes of the initial test node, and the historical test results of the initial test node as input samples;

[0019] The current test case also includes adjacent test nodes of the candidate test node and historical test results of the candidate test node.

[0020] The above method preferably further comprises:

[0021] Output the output content of the customer service system on the target test node.

[0022] In the above method, preferably, the node order between the target test nodes is generated based on a jump parameter between the target test nodes, and the jump parameter is used to characterize the jump adjacency relationship between the target test nodes.

[0023] The above method preferably further comprises:

[0024] Obtaining monitoring data during the test of the target test node by the customer service system;

[0025] According to the monitoring data, determining whether the customer service system has a test anomaly on the target test node;

[0026] In the case that a test exception exists in the customer service system on the target test node, a target exception type of the customer service system on the target test node is obtained, and an operation corresponding to the target exception type is performed.

[0027] In the above method, preferably, the monitoring data includes data on multiple monitoring dimensions;

[0028] According to the monitoring data, determining whether the customer service system has a test anomaly on the target test node includes:

[0029] Obtaining, respectively according to the data on each of the monitoring dimensions in the monitoring data, an initial probability value of at least one abnormal type during the process of the customer service system testing the target test node;

[0030] For each of the abnormality types, the initial probability value corresponding to each of the monitoring dimensions is counted to obtain an abnormality probability value for each of the abnormality types;

[0031] It is judged whether the abnormal probability value is greater than or equal to the abnormal threshold corresponding to the abnormal type to determine whether the customer service system has a test abnormality on the target test node.

[0032] The present application also provides a testing device for a customer service system, the device comprising:

[0033] A configuration obtaining unit, used to obtain test configuration data; the test configuration data includes target test data of at least one target test node, and there is a node order between the target test nodes;

[0034] A node testing unit, configured to input the test configuration data into the customer service system to be tested, so that the customer service system tests the target test node according to the target test data of the target test node in the node order, so as to obtain the output content of the customer service system on the target test node;

[0035] The result comparison unit is used to compare the output content of the customer service system on the target test node with the corresponding preset standard content on the target test node to obtain the test result of the customer service system on the target test node.

[0036] The present application also provides an electronic device, including:

[0037] A memory, used to store applications and data generated by the operation of the applications;

[0038] A processor is used to execute the application to achieve: obtaining test configuration data; the test configuration data includes target test data of at least one target test node, and there is a node order between the target test nodes; inputting the test configuration data into a customer service system to be tested, so that the customer service system tests the target test node according to the target test data of the target test node in the node order to obtain output content of the customer service system on the target test node; comparing the output content of the customer service system on the target test node with the corresponding preset standard content on the target test node to obtain a test result of the customer service system on the target test node.

[0039] It can be seen from the above scheme that in a test method, device and electronic device for a customer service system provided by the present application, after obtaining the target test data containing at least one target test node and the test configuration data having a node order between the target test nodes, the test configuration data is input into the customer service system to be tested, so that the customer service system can test the target test node according to the target test data of the target test node in the node order, and then obtain the output content of the customer service system on the target test node. Based on this, the output content of the customer service system on the target test node can be compared with the corresponding preset standard content on the target test node to obtain the test result of the customer service system on the target test node. It can be seen that in the present application, in the process of implementing the test of the customer service system, there is no need for the test personnel to monitor each test node of the intelligent customer service system, and the test on multiple test nodes can be implemented, thereby speeding up the test progress and improving the test efficiency of the customer service system. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 A flowchart of a method for testing a customer service system provided in Example 1 of the present application;

[0042] Figure 2 This is an example diagram of the configuration interface in the embodiment of the present application;

[0043] Figure 3 This is a test example diagram of the customer service system in the embodiment of the present application;

[0044] Figure 4 Another flow chart of a method for testing a customer service system provided in Embodiment 1 of the present application;

[0045] Figure 5 A partial flow chart of a method for testing a customer service system provided in Example 1 of the present application;

[0046] Figure 6 A schematic diagram of the structure of a test device for a customer service system provided in Embodiment 2 of the present application;

[0047] Figure 7 Another structural schematic diagram of a test device for a customer service system provided in Embodiment 2 of the present application;

[0048] Figure 8 A schematic diagram of the structure of an electronic device provided in Example 3 of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] refer to Figure 1 , is a flowchart of a method for testing a customer service system provided in the first embodiment of the present application, which can be applied to a computer or server that can establish a data connection with the customer service system and can perform data processing. The technical solution in this embodiment is mainly used to improve the test efficiency of the customer service system.

[0051] Specifically, the method in this embodiment may include the following steps:

[0052] Step 101: Obtain test configuration data.

[0053] The test configuration data includes target test data of at least one target test node, and there is a node sequence between the target test nodes, and the node sequence between the target test nodes represents the test sequence in which the target test nodes are tested.

[0054] It should be noted that the target test node here can be a part of the test nodes of all the test nodes included in a complete test case of the customer service system, and the node order between the target test nodes can be configured according to the needs. For example, the node order between the target test nodes is generated based on the jump parameters between the target test nodes, and these jump parameters are used to characterize the jump adjacent relationship between the target test nodes. The jump parameters can be configured by the user in the corresponding configuration interface, or can be obtained according to the initial node position of these target test nodes in the complete test case. For example, the user uses a line to connect each target test node on the configuration interface to generate the jump parameters between each target test node, so as to characterize the jump adjacent relationship between the target test nodes at both ends of the line, and then generate the node order between the target test nodes based on this, which can also be understood as the tested order between the target test nodes. For another example, after obtaining the target test node, the initial node position of each target test node is obtained in the complete test case, and the target test nodes adjacent to the initial node position are automatically connected with a line according to the order between the initial node positions, so as to generate the jump parameters between each target test node, so as to characterize the jump adjacent relationship between the target test nodes at both ends of the line, and then generate the node order between the target test nodes based on this.

[0055] It should be noted that the initial node positions of the target test nodes in the complete test case may be adjacent or non-adjacent.

[0056] In one implementation, when obtaining the test configuration data in step 101, a configuration interface may be output to the user and then implemented based on the user's input operation on the configuration interface, as follows:

[0057] First, output the configuration interface, such as Figure 2As shown in , the configuration interface may include multiple test nodes to be selected, and these test nodes to be selected may include all the test nodes in the complete test case, or may only include some test nodes, and each test node to be selected corresponds to one or more test data to be selected, and each test data to be selected is obtained according to historical test data. Based on this, the user can select or determine the test nodes to be selected in the configuration interface, and connect the selected or determined test nodes to be selected with lines, and select or determine the test data to be selected for the test nodes to be selected, and finally perform a determination operation; thereafter, receive input operations for the test nodes to be selected and the test data to be selected in the configuration interface, such as the user's selection or determination operations on the test nodes to be selected and the test data to be selected and the connection operations between the test nodes, and the user's determination operations, and so on; finally, according to the input operations, obtain at least one target test node and the target test data of the target test node, and there is a node order between the target test nodes. For example, the candidate test node selected or determined by the user in the input operation is used as the target test node, and the node order between the target test nodes is obtained according to the lines connected by the user between the target test nodes. In addition, the candidate test data selected or determined by the user is used as the target test data of the corresponding target test node.

[0058] It should be noted that the candidate test nodes in the configuration interface can be obtained based on historical test cases, and the candidate test data of the candidate test nodes can be obtained based on historical test data.

[0059] Specifically, in this embodiment, multiple historical test cases can be used in advance to train the constructed neural network model. Specifically, the initial test nodes in the historical test cases are used as input samples and the historical test nodes and the historical test data of the historical test nodes are used as output samples to train the neural network model. The neural network model thus trained can process the initial test nodes in the current test case, and then output one or more current test nodes and the test data of the current test nodes corresponding to the current test case. At this time, these current test nodes are used as test nodes to be selected in the configuration interface for the user to select or confirm, and the test data of these current test nodes are used as test data to be selected for the test nodes to be selected in the configuration interface for the user to select or confirm.

[0060] Furthermore, in the implementation scheme for obtaining the test nodes to be selected and the test data to be selected for the configuration interface, the neural network model can also use the initial test node, the adjacent test nodes of the initial test node, and the historical test results of the initial test node in the historical test case as input samples, and use the historical test nodes and historical test data in the historical test case as output samples for training. After multiple trainings, the trained neural network model can process the initial test node, the adjacent test nodes of the initial test node, and the historical test results of the initial test node in the current test case, and then output one or more current test nodes and test data of the current test node corresponding to the current test case.

[0061] It should be noted that the current test case mentioned in this embodiment refers to the test case used for testing the customer service system this time. Specifically, the current test case can be a preset complete test case, which includes all test nodes, or the current test case can be a test case edited by the test node, which includes some test nodes.

[0062] In another implementation, when obtaining the test configuration data in step 101, the test configuration data may be automatically obtained without user participation, as follows:

[0063] First, obtain the current test case, which contains at least multiple test nodes to be selected. The test case to be selected here can be all the test nodes in the complete test case or part of the test nodes; then, input the current test case into the neural network model to obtain at least one target test node output by the neural network model and the target test data of the target test node.

[0064] Among them, the neural network model is trained using multiple historical test cases, the historical test cases include multiple initial test nodes and at least one historical test node, the initial test node refers to all test nodes initially included in the historical test case that have not yet been selected or confirmed as target test nodes, the historical test node refers to the test node selected as the target test node in the historical test case, the historical test node includes historical test data, and the neural network model is trained using at least the initial test node as an input sample and the historical test node and the historical test data as output samples.

[0065] Furthermore, the neural network model can also take the initial test node in the historical test case, the adjacent test nodes of the initial test node such as the test nodes before and after the initial test node, and the historical test results of the initial test node such as the results of test success or test failure as input samples. Based on this, the current test case also includes the adjacent test nodes of the test node to be selected and the historical test results of the test node to be selected. After the current test case is input into the neural network model, the neural network model can output at least one target test node and the target test data of the target test node.

[0066] Step 102: input the test configuration data into the customer service system to be tested, so that the customer service system tests the target test node according to the target test data of the target test node in the node order, so as to obtain the output content of the customer service system on the target test node.

[0067] like Figure 3 As shown in , in this embodiment, the target test node in the test configuration data and the target test data of the target test node are input into the customer service system, and the customer service system processes the target test data of each target test node in sequence according to the front-to-back adjacent order between the target test nodes in the node sequence, so that the customer service system can obtain output content for the target test data of each target test node. For example, the customer service system performs voice recognition on the input voice data at each target test node, and then outputs a text reply or voice reply for the voice data, thereby completing the test on the target test node.

[0068] It should be noted that the test cases involved in this application can be understood as test items for the corresponding functions of the customer service system, the test nodes therein can be understood as the test links of the customer service system on this function, and the test data of the test nodes can be understood as the test input data of the customer service system, such as voice input data or character input data.

[0069] Step 103: Compare the output content of the customer service system on the target test node with the corresponding preset standard content on the target test node to obtain the test result of the customer service system on the target test node.

[0070] Among them, the corresponding preset standard content on the target test node can be understood as the preset standard reply content of the customer service system on the target test node. In this embodiment, the output content of the customer service system on the target test node is compared with the corresponding preset standard content to obtain a test result that characterizes whether the customer service system responds accurately on the target test node.

[0071] It can be seen from the above scheme that in a test method for a customer service system provided in the first embodiment of the present application, after obtaining the target test data containing at least one target test node and the test configuration data having a node order between the target test nodes, the test configuration data is input into the customer service system to be tested, so that the customer service system can test the target test node according to the target test data of the target test node in the node order, and then obtain the output content of the customer service system on the target test node. Based on this, the output content of the customer service system on the target test node can be compared with the corresponding preset standard content on the target test node to obtain the test result of the customer service system on the target test node. It can be seen that in the present embodiment, in the process of implementing the test of the customer service system, there is no need for the test personnel to monitor each test node of the intelligent customer service system, and the test on multiple test nodes can be implemented, thereby speeding up the test progress and improving the test efficiency of the customer service system.

[0072] In one implementation, after obtaining the output content of the customer service system on the target test node in step 102, the following steps may be performed: Figure 4 As shown in:

[0073] Step 104: Output the output content of the customer service system on the target test node.

[0074] Specifically, in this embodiment, the output content of the customer service system on the target test node can be output to the tester's terminal through the data connection between the tester's terminal, so as to prompt the tester to perform manual inspection and other processing in time.

[0075] In one implementation, in this embodiment, during the process of the customer service system testing the target test node, the following steps may also be included: Figure 5 As shown in:

[0076] Step 501: Obtain monitoring data of the customer service system during the test of the target test node.

[0077] The monitoring data may include data on multiple monitoring dimensions, and the monitoring dimensions here can be set according to needs. For example, the monitoring data may include: the number of interactions of the customer service system on the target test node, the interaction duration of the customer service system on the target test node, the reply content of the customer service system on the target test node, etc.

[0078] Step 502: Based on the monitoring data, determine whether there is a test anomaly in the customer service system on the target test node. If there is a test anomaly in the customer service system on the target test node, execute step 503.

[0079] Among them, in this embodiment, the monitoring data in each monitoring dimension can be analyzed, such as numerical judgment and other processing, and then the customer service system can be judged whether there is a test anomaly on the target test node based on the analysis results such as numerical judgment results.

[0080] In one implementation, step 502 may be implemented in the following manner:

[0081] According to the data on each monitoring dimension in the monitoring data, respectively, an initial probability value of at least one abnormal type in the process of the customer service system testing the target test node is obtained. For example, the data on each monitoring dimension in the monitoring data is judged with the dimension threshold corresponding to each abnormal type on the corresponding monitoring dimension. Then, when the data on the monitoring dimension is greater than or equal to the dimension threshold corresponding to each abnormal type on the corresponding monitoring dimension, the dimension probability value corresponding to the dimension threshold is used as the initial probability value corresponding to the abnormal type on the corresponding monitoring dimension.

[0082] Afterwards, for each abnormality type, the corresponding initial probability value on each monitoring dimension is counted to obtain the abnormality probability value for each abnormality type; for example, for each abnormality type, the corresponding initial probability on each monitoring dimension is added to obtain the abnormality probability value for each abnormality type;

[0083] Finally, it is determined whether the abnormal probability value is greater than or equal to the abnormal threshold corresponding to the abnormal type to determine whether the customer service system has a test abnormality on the target test node. For example, if the abnormal probability value is greater than or equal to the abnormal threshold corresponding to the abnormal type, then it can be determined that the customer service system has a test abnormality on the target test node, and further, it can be determined that the customer service system has a target abnormality type corresponding to the abnormal threshold on the target test node.

[0084] For example, on the same target test node, if the number of interactions between the test systems implemented by the customer service system in this application exceeds 5 times, then the probability of an infinite loop exception type exceeds 30%; if the interaction time between the test systems implemented by the customer service system in this application exceeds 5 minutes, then the probability of an infinite loop exception type reaches 20%, and so on. Based on this, when the overall probability of adding up all probabilities exceeds 82%, it is determined that an infinite loop exception has occurred.

[0085] Step 503: Obtain a target exception type of the customer service system on the target test node, and perform an operation corresponding to the target exception type.

[0086] For example, when the target exception type is an infinite loop exception type, an operation to jump out of the infinite loop is performed, such as controlling the customer service system to stop testing, or controlling the customer service system to test the next target test node that has not been tested according to the node sequence.

[0087] Based on the above implementation, in this embodiment, the test process of the customer service system can also be controlled by judging whether the output content of the customer service system on the target test node is the reply content for the target test data. For example, the output content of the customer service system on the target test node is analyzed by the function modules such as the inference engine in the customer service system to see whether it is the reply content for the target test data. When the output content of the customer service system on the target test node is not the reply content for the test data, in this embodiment, the customer service system can be controlled to re-test the target test node according to the target test data of the target test node, that is, to re-interact with the customer service system until the end condition is met, so that the customer service system tests the next target test node that has not been tested in the node order;

[0088] The termination conditions include: the number of tests for the target test node is greater than or equal to the number threshold, or the output content of the customer service system on the target test node is the reply content for the target test data.

[0089] refer to Figure 6 , is a schematic diagram of the structure of a test device for a customer service system provided in Embodiment 2 of the present application, which can be configured in a computer or server that can establish a data connection with the customer service system and can perform data processing. The technical solution in this embodiment is mainly used to improve the test efficiency of the customer service system.

[0090] Specifically, the device in this embodiment may include the following units:

[0091] The configuration obtaining unit 601 is used to obtain test configuration data; the test configuration data includes target test data of at least one target test node, and there is a node order between the target test nodes;

[0092] The node testing unit 602 is used to input the test configuration data into the customer service system to be tested, so that the customer service system tests the target test node according to the target test data of the target test node in the node order, so as to obtain the output content of the customer service system on the target test node;

[0093] The result comparison unit 603 is used to compare the output content of the customer service system on the target test node with the corresponding preset standard content on the target test node to obtain the test result of the customer service system on the target test node.

[0094] It can be seen from the above scheme that in a test device for a customer service system provided in the second embodiment of the present application, after obtaining the target test data including at least one target test node and the test configuration data having a node order between the target test nodes, the test configuration data is input into the customer service system to be tested, so that the customer service system can test the target test node according to the target test data of the target test node in the node order, and then obtain the output content of the customer service system on the target test node. Based on this, the output content of the customer service system on the target test node can be compared with the corresponding preset standard content on the target test node to obtain the test result of the customer service system on the target test node. It can be seen that in the present embodiment, in the process of implementing the test of the customer service system, there is no need for the test personnel to monitor each test node of the intelligent customer service system, and the test on multiple test nodes can be implemented, thereby speeding up the test progress and improving the test efficiency of the customer service system.

[0095] In one implementation, the configuration acquisition unit 601 is specifically used to: output a configuration interface, wherein the configuration interface includes multiple test nodes to be selected, each of the test nodes to be selected corresponds to test data to be selected, and the test data to be selected is obtained based on historical test data; receive input operations for the test nodes to be selected and the test data to be selected in the configuration interface; and obtain at least one target test node and target test data of the target test node based on the input operations, wherein there is a node order between the target test nodes.

[0096] In one implementation, the configuration acquisition unit 601 is specifically used to: obtain a current test case, wherein the current test case at least includes a plurality of test nodes to be selected; input the current test case into a neural network model to obtain at least one target test node output by the neural network model and test data of the target test node; wherein the neural network model is trained using a plurality of historical test cases, wherein the historical test cases include a plurality of initial test nodes and at least one historical test node, wherein the historical test node includes historical test data, and the neural network model is trained using at least the initial test node as an input sample and the historical test node and the historical test data as output samples.

[0097] Optionally, the neural network model uses the initial test node, adjacent test nodes of the initial test node, and historical test results of the initial test node as input samples;

[0098] The current test case also includes adjacent test nodes of the candidate test node and historical test results of the candidate test node.

[0099] In one implementation, the node testing unit 602 is further configured to:

[0100] Output the output content of the customer service system on the target test node.

[0101] In one implementation, the node order between the target test nodes is generated based on a jump parameter between the target test nodes, where the jump parameter is used to characterize a jump adjacency relationship between the target test nodes.

[0102] In one implementation, the device in this embodiment may also include the following structure: Figure 7 As shown in:

[0103] The test control unit 604 is used to obtain monitoring data during the test of the target test node by the customer service system; determine whether there is a test anomaly of the customer service system on the target test node based on the monitoring data; and when there is a test anomaly of the customer service system on the target test node, obtain the target anomaly type of the customer service system on the target test node and perform operations corresponding to the target anomaly type.

[0104] Optionally, the monitoring data includes data on multiple monitoring dimensions;

[0105] Among them, when the test control unit 604 determines whether there is a test anomaly of the customer service system on the target test node based on the monitoring data, it is specifically used to: obtain the initial probability value of at least one abnormality type in the process of the customer service system testing the target test node according to the data on each of the monitoring dimensions in the monitoring data; for each of the abnormality types, respectively, count the corresponding initial probability values ​​on each of the monitoring dimensions to obtain the abnormal probability value for each of the abnormality types; and judge whether the abnormal probability value is greater than or equal to the abnormal threshold corresponding to the abnormality type to determine whether there is a test anomaly of the customer service system on the target test node.

[0106] refer to Figure 8 , is a schematic diagram of the structure of an electronic device provided in the third embodiment of the present application, and the electronic device may be a computer or server capable of establishing a data connection with the customer service system and performing data processing. The technical solution in this embodiment is mainly used to improve the test efficiency of the customer service system.

[0107] Specifically, the electronic device in this embodiment may include the following structure:

[0108] The memory 801 is used to store the application and the data generated by the running of the application;

[0109] Processor 802 is used to execute the application to achieve: obtaining test configuration data; the test configuration data includes target test data of at least one target test node, and there is a node order between the target test nodes; inputting the test configuration data into the customer service system to be tested, so that the customer service system tests the target test node according to the node order and the target test data of the target test node to obtain the output content of the customer service system on the target test node; comparing the output content of the customer service system on the target test node with the corresponding preset standard content on the target test node to obtain the test result of the customer service system on the target test node.

[0110] It can be seen from the above scheme that in an electronic device provided in the third embodiment of the present application, after obtaining the target test data including at least one target test node and the test configuration data having a node order between the target test nodes, the test configuration data is input into the customer service system to be tested, so that the customer service system can test the target test node according to the target test data of the target test node in the node order, and then obtain the output content of the customer service system on the target test node. Based on this, the output content of the customer service system on the target test node can be compared with the corresponding preset standard content on the target test node to obtain the test result of the customer service system on the target test node. It can be seen that in the present embodiment, in the process of implementing the test of the customer service system, there is no need for the test personnel to monitor each test node of the intelligent customer service system, and the test on multiple test nodes can be implemented, thereby speeding up the test progress and improving the test efficiency of the customer service system.

[0111] Taking the customer service system of a bank in the financial industry as an example, the inventor of this application proposes to establish a bank customer service intelligent testing system for the bank customer service system, a bank customer service intelligent testing system based on process jumping, in response to the technical problem of low existing testing efficiency. For example, a customer service intelligent testing module is developed in mobile banking. When the tester needs to test the customer service system, the process template can be entered into the test system, and the jump parameters can be set to directly jump to the corresponding node, saving testing time. At the same time, the test system can make intelligent responses based on the feedback of the customer service system. Based on this, the tester enters his own test options and test voice or jump parameters in the mobile banking test module. The test system can directly jump to the corresponding module according to the tester's settings, saving testing time; moreover, the test system collects all possible responses of the customer service system and makes intelligent responses based on the responses, improving the intelligence level of the test system.

[0112] Specifically, the test system implemented in this application may include the following modules:

[0113] Test system jump module: The tester enters his / her own test options and test voice or jump parameters in the mobile banking test module. The test system can directly jump to the corresponding module according to the tester's settings, saving test time.

[0114] Intelligent response module: The test system collects all possible responses from the customer service system and makes intelligent responses based on the responses, thereby improving the intelligence level of the system.

[0115] The bank customer service intelligent testing system based on process jump implemented in this application can solve the problem of low test efficiency at present. First, a customer service testing module is developed in the mobile banking. The module can directly import the usage flowchart of the customer service system into the module and present the flowchart to the tester in a vivid way. The tester only needs to enter the digital selection or voice that he needs to enter at each node of the flowchart.

[0116] Testers can also set up their own flow charts, select the corresponding components in the mobile banking test module to complete the connection, and click Submit, and the test system can test itself;

[0117] The test system will record the relevant test data of the tester during use. If the tester's test task is urgent and heavy, the process jump module can be used. The tester can automatically jump to the corresponding node by connecting the adjusted starting point and end point together. The specific method is that the background system automatically fills the test with the information before the node in the process link. After the task flows to the current node set by the tester, the customer will be reminded, which is equivalent to going directly to this node. The customer only needs to enter the test data they want at this node.

[0118] The selection of node data before is data selected by analyzing historical data with the help of artificial intelligence algorithms. The specific method is to analyze historical test data to obtain the number of branches covered by case data in historical data, branch specific information and historical error information, use this information as model input, and use the selected test data as model output. The neural network structure is determined according to the number of input and output, and then the number of parameters that need to be optimized in the genetic algorithm is determined. In addition, in this application, the prediction accuracy of the model can be verified by test samples to obtain an effective model. Which test data is selected through model prediction can improve the effectiveness of the test data. The test system automatically fills the historical test data into the corresponding process node and goes to the node set by the tester to wait for the tester, so that the customer service system can achieve the goal of process jump without modification, thereby improving the test efficiency.

[0119] At the same time, the test system connects to the expert system in the background, collects the dialogue information between the test system and the customer service system, inputs the dialogue information into the expert system, and uses the system to determine whether there is any abnormal information in the current test process and the response method for the abnormality. The expert system consists of a knowledge base and an inference engine. The knowledge base contains knowledge rules obtained through data analysis such as induction and summary, including whether the test system has entered an infinite loop and the corresponding response method.

[0120] For example, Rule 1: If the number of interactions between the test system and the customer service system at the same node exceeds 5 times, the probability of an infinite loop exceeds 30%; if the interaction time exceeds 5 minutes, the probability of an infinite loop reaches 20%, and so on. If the overall probability exceeds 82%, it is an infinite loop. The corresponding method to deal with the infinite loop is to first notify the tester and end the case at the same time, and then test the problematic case after giving priority to other cases.

[0121] In addition, the inference engine is built using machine learning methods. By analyzing historical data, a usable machine learning model is obtained. Then, based on the characteristics of this test case, it is determined which rules in the knowledge base are met. The rules are integrated together to determine the overall probability information and whether it is an exception. If it is an exception, the corresponding response method is queried and the corresponding response is automatically performed to prevent the test process from being delayed due to failures. At the same time, the correct answer method of the customer service system is integrated into the knowledge base. The answer method is matched by the inference engine. For example, if the customer service system feedback is not heard clearly, please repeat it or return to the previous level, etc., the test system can repeat the voice information or return to the previous level as required, improving the intelligence level and usefulness of the test system and reducing labor costs.

[0122] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0123] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0124] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0125] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for testing a customer service system, characterized in that: The method comprises: Obtaining test configuration data; the test configuration data includes target test data of at least one target test node, and the target test nodes have a node order; Inputting the test configuration data into the customer service system to be tested, so that the customer service system tests the target test node according to the target test data of the target test node in the node order, so as to obtain the output content of the customer service system on the target test node; Comparing the output content of the customer service system on the target test node with the corresponding preset standard content on the target test node to obtain the test result of the customer service system on the target test node; The test configuration data is obtained, including: Obtaining a current test case, wherein the current test case at least includes a plurality of test nodes to be selected; Inputting the current test case into a neural network model to obtain at least one target test node output by the neural network model and test data of the target test node; The neural network model is trained using multiple historical test cases, wherein the historical test cases include multiple initial test nodes and at least one historical test node, and the historical test node includes historical test data. The neural network model is trained using at least the initial test node as an input sample and the historical test node and the historical test data as output samples.

2. The method according to claim 1, characterized in that: Get test configuration data, including: Outputting a configuration interface, wherein the configuration interface includes a plurality of test nodes to be selected, each of the test nodes to be selected corresponds to test data to be selected, and the test data to be selected is obtained according to historical test data; Receiving input operations for the test nodes to be selected and the test data to be selected in the configuration interface; According to the input operation, at least one target test node and target test data of the target test node are obtained, and there is a node sequence between the target test nodes.

3. The method according to claim 1, characterized in that: The neural network model uses the initial test node, the adjacent test nodes of the initial test node, and the historical test results of the initial test node as input samples; The current test case also includes adjacent test nodes of the candidate test node and historical test results of the candidate test node.

4. The method according to claim 1, characterized in that: Also includes: Output the output content of the customer service system on the target test node.

5. The method according to claim 1, characterized in that The node order between the target test nodes is generated based on a jump parameter between the target test nodes, where the jump parameter is used to characterize a jump adjacency relationship between the target test nodes.

6. The method according to claim 1, characterized in that Also includes: Obtaining monitoring data during the test of the target test node by the customer service system; According to the monitoring data, determining whether the customer service system has a test anomaly on the target test node; In the case that a test exception exists in the customer service system on the target test node, a target exception type of the customer service system on the target test node is obtained, and an operation corresponding to the target exception type is performed.

7. The method according to claim 6, characterized in that The monitoring data includes data on multiple monitoring dimensions; According to the monitoring data, determining whether the customer service system has a test anomaly on the target test node includes: Obtaining, respectively according to the data on each of the monitoring dimensions in the monitoring data, an initial probability value of at least one abnormal type during the process of the customer service system testing the target test node; For each of the abnormality types, the initial probability value corresponding to each of the monitoring dimensions is counted to obtain an abnormality probability value for each of the abnormality types; It is judged whether the abnormal probability value is greater than or equal to the abnormal threshold corresponding to the abnormal type to determine whether the customer service system has a test abnormality on the target test node.

8. A testing device for a customer service system, characterized in that: The device comprises: A configuration obtaining unit, used to obtain test configuration data; the test configuration data includes target test data of at least one target test node, and there is a node order between the target test nodes; A node testing unit, configured to input the test configuration data into the customer service system to be tested, so that the customer service system tests the target test node according to the target test data of the target test node in the node order, so as to obtain the output content of the customer service system on the target test node; A result comparison unit, used to compare the output content of the customer service system on the target test node with the corresponding preset standard content on the target test node to obtain the test result of the customer service system on the target test node; The test configuration data is obtained, including: Obtaining a current test case, wherein the current test case at least includes a plurality of test nodes to be selected; Inputting the current test case into a neural network model to obtain at least one target test node output by the neural network model and test data of the target test node; The neural network model is trained using multiple historical test cases, wherein the historical test cases include multiple initial test nodes and at least one historical test node, and the historical test node includes historical test data. The neural network model is trained using at least the initial test node as an input sample and the historical test node and the historical test data as output samples.

9. An electronic device, characterized in that: include: A memory, used to store applications and data generated by the operation of the applications; A processor is used to execute the application program to achieve: obtaining test configuration data; the test configuration data includes target test data of at least one target test node, and the target test nodes have a node sequence; inputting the test configuration data into a customer service system to be tested, so that the customer service system tests the target test node according to the target test data of the target test node in the node sequence, so as to obtain output content of the customer service system on the target test node; Comparing the output content of the customer service system on the target test node with the corresponding preset standard content on the target test node to obtain the test result of the customer service system on the target test node; The test configuration data is obtained, including: Obtaining a current test case, wherein the current test case at least includes a plurality of test nodes to be selected; Inputting the current test case into a neural network model to obtain at least one target test node output by the neural network model and test data of the target test node; The neural network model is trained using multiple historical test cases, wherein the historical test cases include multiple initial test nodes and at least one historical test node, and the historical test node includes historical test data. The neural network model is trained using at least the initial test node as an input sample and the historical test node and the historical test data as output samples.