Testing Method, Device, Equipment and Computer Storage Medium

Abstract

The present invention relates to the technical field of financial technology (Fintech), and discloses a testing method, which includes: determining a data input source according to the input business scenario type, and obtaining a first preset number of target label information from the un-trained label information in the knowledge graph according to the data input source; obtaining evaluation indicators corresponding to each target label information, and obtaining index influence factors corresponding to each evaluation indicator; calculating result weight values corresponding to each target label information according to each evaluation indicator and each index influence factor, and correcting the recommended score values corresponding to each target label information according to each result weight value; determining a second preset number of target recommended score values based on the corrected recommended score values, and using the to-be-evaluated solutions corresponding to each target recommended score value as test cases for testing. The present invention also discloses a testing device, equipment and a computer storage medium. The present invention improves the effectiveness of automated testing.

Description

Technical Field

[0001] The present invention relates to the field of testing technology of financial technology (Fintech), and in particular to testing methods, devices, equipment and computer storage media. Background Art

[0002] With the development of computer technology, more and more technologies (big data, distributed, artificial intelligence, etc.) are applied in the financial field. The traditional financial industry is gradually transforming to Fintech. However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology. At present, when testers conduct various automated tests (such as use case regression testing), they generally execute test cases manually based on their own experience, which is easily affected by the testers' own subjective consciousness. There are problems such as scattered test focus, no specificity, easy to ignore key points, insufficient coverage of key business scenarios, etc., which leads to low test effectiveness. Therefore, how to improve the effectiveness of automated testing has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The main purpose of the present invention is to provide a testing method, device, equipment and computer storage medium, aiming to solve the technical problem of how to improve the effectiveness of automated testing.

[0004] To achieve the above object, the present invention provides a testing method, which comprises the following steps:

[0005] Determine a data input source according to an input business scenario type, and obtain a first preset number of target label information from untrained label information in the knowledge graph according to the data input source;

[0006] Obtaining the evaluation index corresponding to each target label information, and obtaining the index impact factor corresponding to each evaluation index;

[0007] Calculate the result weight value corresponding to each target label information according to each evaluation indicator and each indicator influencing factor, and correct the recommendation score value corresponding to each target label information according to each result weight value;

[0008] A second preset number of target recommendation score values ​​are determined based on the corrected recommendation score values, and the to-be-evaluated solutions corresponding to the target recommendation score values ​​are tested as test cases.

[0009] Optionally, the step of calculating the result weight value corresponding to each target tag information according to each evaluation indicator and each indicator influencing factor includes:

[0010] Calculate the total corrected index value of the solution to be evaluated corresponding to each of the target label information according to each of the evaluation indexes and each of the index impact factors;

[0011] Calculate the result weight value corresponding to each of the target label information according to each of the total corrected index values and each of the evaluation indexes.

[0012] Optionally, the step of correcting the recommended score value of the solution to be evaluated corresponding to each of the target label information according to each of the result weight values includes:

[0013] Obtain the recommended score value of the solution to be evaluated corresponding to each of the target label information, traverse each of the recommended score values in turn, and determine the current result weight value corresponding to the current recommended score value being traversed among each of the result weight values;

[0014] Perform calculations on the current recommended score value and the current result weight value based on a pre-designed calculation formula, and correct the current recommended score value according to the calculation result until each of the recommended score values has been traversed.

[0015] Optionally, the step of obtaining the recommended score value of the solution to be evaluated corresponding to each of the target label information includes:

[0016] Traverse the solutions to be evaluated corresponding to each of the target label information in turn, determine the target library of the current solution to be evaluated being traversed in the knowledge graph, and obtain the default score value of the target library, and use the default score value as the recommended score value of the current solution to be evaluated until each of the solutions to be evaluated has been traversed.

[0017] Optionally, the step of obtaining the evaluation indexes corresponding to each of the target label information includes:

[0018] Traverse each of the target label information in turn, and determine multiple different types of index values corresponding to the current target label information being traversed, and use each of the index values as the evaluation index corresponding to the current target label information until each of the target label information has been traversed.

[0019] Optionally, the step of obtaining the index impact factors corresponding to each of the evaluation indexes includes:

[0020] Traverse each of the evaluation indexes in turn, determine all the index values in the current evaluation index being traversed, and determine the index impact factor corresponding to the current evaluation index according to each of the index values until each of the evaluation indexes has been traversed.

[0021] Optionally, the step of obtaining the first preset number of target label information from the unlabeled information in the knowledge graph according to the data input source includes:

[0022] Obtain multiple un-trained label information in the knowledge graph, and sort each of the un-trained label information according to the data input source;

[0023] Obtain a first preset number of target label information from each of the un-trained label information according to the sorting result of the sorting.

[0024] In addition, to achieve the above object, the present invention also provides a testing device, the testing device includes:

[0025] A determination module, configured to determine a data input source according to the input business scenario type, and obtain a first preset number of target label information from the un-trained label information in the knowledge graph according to the data input source;

[0026] An acquisition module, configured to acquire evaluation indicators corresponding to each of the target label information, and acquire index impact factors corresponding to each of the evaluation indicators;

[0027] A correction module, configured to calculate result weight values corresponding to each of the target label information according to each of the evaluation indicators and each of the index impact factors, and correct the recommended score values corresponding to each of the target label information according to each of the result weight values;

[0028] A testing module, configured to determine a second preset number of target recommended score values based on the corrected recommended score values, and use the evaluation schemes corresponding to each of the target recommended score values as test cases for testing.

[0029] In addition, to achieve the above object, the present invention also provides a testing device, the testing device includes: a memory, a processor, and a testing program stored on the memory and executable on the processor, and when the testing program is executed by the processor, the steps of the above-mentioned testing method are implemented.

[0030] In addition, to achieve the above object, the present invention also provides a computer storage medium, on which a testing program is stored, and when the testing program is executed by a processor, the steps of the above-mentioned testing method are implemented.

[0031] The present invention determines a data input source according to the input service scenario type, and obtains a first preset number of target label information from the un-trained label information of the knowledge graph according to the data input source; obtains evaluation indicators corresponding to each of the target label information, and obtains index influence factors corresponding to each of the evaluation indicators; calculates result weight values corresponding to each of the target label information according to each of the evaluation indicators and each of the index influence factors, and corrects the recommended score values corresponding to each of the target label information according to each of the result weight values; determines a second preset number of target recommended score values based on the corrected recommended score values, and uses the solution to be evaluated corresponding to each of the target recommended score values as a test case for testing. By determining the data input source according to the service scenario type, obtaining target label information in the knowledge graph, then calculating the result weight values according to the evaluation indicators and index influence factors corresponding to the target label information, correcting the recommended score values corresponding to the target label information, and then determining the test case for testing according to the corrected recommended score values, it is possible to effectively select test cases that conform to the service scenario type for testing, avoiding the phenomenon in the prior art that the effectiveness of testing is reduced by manually selecting test cases for testing, and improving the effectiveness and accuracy of automated testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a schematic structural diagram of a test device for the hardware operating environment involved in the embodiment solution of the present invention;

[0033] Figure 2 is a schematic flowchart of the first embodiment of the test method of the present invention;

[0034] Figure 3 is a schematic diagram of the device modules of the test device of the present invention;

[0035] Figure 4 is a schematic diagram of the recommended scores of the knowledge graph in the test method of the present invention;

[0036] Figure 5 is a schematic flowchart of the test process in the test method of the present invention;

[0037] Figure 6 is a schematic diagram of the construction of the knowledge graph in the test method of the present invention.

[0038] The implementation, functional features and advantages of the object of the present invention will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0040] As Figure 1 shownFigure 1 It is a schematic diagram of the test device structure of the hardware operating environment involved in the embodiment of the present invention.

[0041] The test device in the embodiment of the present invention can be a PC or a server device, on which a Java virtual machine is running.

[0042] As shown in Figure 1 the figure, the test device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0043] Those skilled in the art can understand that Figure 1 the test device structure shown in

[0044] does not constitute a limitation to the device, and may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements. Figure 1 As shown in

[0045] In Figure 1 the test device shown in the figure, the network interface 1004 is mainly used to connect to the background server and perform data communication with the background server; the user interface 1003 is mainly used to connect to the client (user side) and perform data communication with the client; and the processor 1001 can be used to call the test program stored in the memory 1005 and execute the operations in the following test method.

[0046] Based on the above hardware structure, an embodiment of the test method of the present invention is proposed.

[0047] Referring to Figure 2 , Figure 2 is a schematic flowchart of the first embodiment of the test method of the present invention, and the method includes:

[0048] Step S10, determine the data input source according to the input business scenario type, and obtain the first preset number of target label information from the un-trained label information of the knowledge graph according to the data input source;

[0049] In this embodiment, during the test version management by using GIT (Distributed Version Control System), it is integrated into JENKINS (open source software project). Then call the CI (Continuous Integration) interface of the preset quality middle platform (which can be represented by TctpTest for version automatic deployment, that is, first determine the business scenario type according to the information input by developers or testers, and request from the knowledge graph to obtain the recommended use case label information under this business scenario type. After obtaining the feedback information fed back by the knowledge graph, determine all test cases to be tested according to the feedback information, and then directly execute all test cases through the test execution engine. And after the test execution engine finishes the execution, it will output the corresponding execution results and store them in the database, so that the TctpTest platform can perform historical backtracking through the data reports with execution results in the database, making the test more efficient and reducing the missed test rate.

[0050] Among them, the knowledge graph includes three parts of functions. The first part is to generate an efficient and valuable benchmark label case library through multi-dimensional modeling. The second part is to perform interface interaction with the TctpTest platform, and its interfaces can include: requesting recommended labels, querying more labels (using paging and creating indexes for query fields in the DB (database) table to avoid too much label information, reduce bandwidth usage, improve query performance, support fuzzy matching, and quickly find the label information desired by the executor), feeding back label information (push interface), executing specified label test cases, etc. The third part is that the label case information executed this time can be used as a new data input source to participate in the modeling algorithm again to obtain more accurate label weights.

[0051] Therefore, in this embodiment, the input business scenario type can be obtained first, and the parent function type and the sub-function type associated with this business scenario type can be obtained in the database to form a data input source. That is, the data input source includes the business scenario type, the parent function type, and the sub-function type. Among them, the parent function type can be the large requirement of the test case. And the sub-function type can be the small requirement under the large requirement. For example, opening an account (account) is the parent function type, and the card information (card information table) under opening an account can be the sub-function type.

[0052] Therefore, when the data input source is obtained, M pieces of training data with a flag of 0 (i.e., a certain number of untrained training data) can be obtained from the knowledge graph according to this data input source, and the training case label information (i.e., untrained label information) under the business scenario type can be determined based on these training data. Then, the untrained label information is sorted with the usage times of the parent function type + child function type in the data input source as the keyword, and the untrained label information corresponding to the top N (i.e., the first preset quantity) maximum usage times is selected from the sorting result and used as the target label information. That is, it is determined that the score (recommended score value) values corresponding to these target label information need to be modified.

[0053] Step S20: Obtain the evaluation indicators corresponding to each of the target label information, and obtain the index impact factors corresponding to each of the evaluation indicators;

[0054] After obtaining each target label information, the cases corresponding to these target label information can be used as the to-be-evaluated solutions, and the library where the cases of these to-be-evaluated solutions are located, the business scenario, the test expert experience score, and the ranking times of the actual usage times on the TctpTest platform are used as the evaluation indicators, that is:

[0055] where, a ij represents the value of the jth index in the ith test case.

[0056] And in this embodiment, the index impact factor is given according to the importance occupied by each evaluation indicator, that is:

[0057] where, b i represents the impact factor of the ith evaluation indicator.

[0058] And since the knowledge graph includes a knowledge common library, a BUG experience library, a specific business scenario library, and a training database. Among them, such as Figure 4As shown in the figure, the knowledge common library includes the most basic use case scenarios, and the general default main process must be executed in this business scenario. Therefore, the score (recommended score value) of the knowledge common library can be set to the maximum value MAX. Since the BUG experience library is only used in very few scenarios, the score of the BUG experience library can be set to the median value MID. And the specific business scenario library is only for certain specific business scenarios. Therefore, the score of the specific business scenario library can be set to the minimum value MIN. Moreover, the training database only affects the score values of the BUG experience library and the specific business scenario library, and does not affect the score value of the knowledge common library. And in this embodiment, the recommended label cases for each test execution depend on the size of the score. The larger the score, the more likely it is to be recommended. Therefore, when calculating the evaluation metrics, it is necessary to detect the library where the cases of the solution to be evaluated are located, so as to determine the evaluation metrics based on the score.

[0059] Step S30: Calculate the result weight value corresponding to each target label information according to each of the evaluation metrics and each of the metric impact factors, and correct the recommended score value corresponding to each target label information according to each of the result weight values.

[0060] After obtaining each evaluation metric and each metric impact factor, the result weight value corresponding to each target label information can be calculated, and then the recommended score value of the solution to be evaluated corresponding to each target label information is corrected according to each result weight value. It should be noted that one result weight value only corrects the recommended score value of the solution to be evaluated associated with this result weight value, and does not change other recommended score values.

[0061] That is, first obtain a new matrix:

[0062]

[0063] where c i represents the total corrected metric value of the i-th test case, and the formula for calculating the result weight value in the i-th test case is:

[0064]

[0065] When obtaining each result weight value and modifying the recommended score value according to the result weight value, the corrected recommended score value can be determined according to the formula score = score * (1 + p). Where score is the recommended score value, and the corrected recommended score value cannot exceed the default maximum score value in the knowledge graph. And after correcting the recommended score value, the flag of the training data of the target label information corresponding to this corrected recommended score value is set to 1 (that is, it is determined that the processing has been performed).

[0066] Step S40: Determine a second preset number of target recommendation score values based on the corrected recommendation score values, and use the to-be-evaluated solutions corresponding to the target recommendation score values as test cases for testing.

[0067] After obtaining the corrected recommendation score values, these corrected recommendation score values can be sorted in descending order, and a second preset number of recommendation score values (i.e., target recommendation score values) greater than a certain value can be selected from them. Then, the to-be-evaluated solutions corresponding to the second preset number of recommendation score values are used as test cases and placed in the test execution engine for testing. After the testing is completed, the test results will be actively stored in the database.

[0068] In addition, to assist in understanding the test process in this embodiment, an example is given below for illustration.

[0069] For example, as Figure 5 shown, developers or testers determine the business scenario type in the Tctp Test platform, and request recommended tags from the knowledge graph according to this business scenario type. The knowledge graph then models and filters out appropriate recommended tags and returns the recommended tags to the Tctp Test platform. The Tctp Test platform then executes the cases corresponding to the recommended tags returned by the knowledge graph in the test execution engine, that is, the test execution engine requests to execute the corresponding cases (i.e., test cases), and at the same time, the Tctp Test platform feeds back information (i.e., the case execution status) to the knowledge graph. After the test execution engine finishes executing the test cases, it will output the results and store the results in the DB (database) for the knowledge graph to call.

[0070] Among them, the architecture composition of the knowledge graph can be as Figure 6As shown in the figure, it includes a knowledge common library, a BUG experience library, a business scenario library, and a training database. Among them, the business scenario library includes a scenario type "type", a parent function type, a child function type, and a score value. And the modeling in the knowledge graph is based on the SCORE value of the business scenario library, that is, the top N with the highest actual usage are selected from the recent M training trees (about M * 100 test cases), and different SCORE weights correspond to different usage rates in different intervals. And when modeling in the knowledge graph, data input sources (including type, parent type, and child type) are obtained from the training database for modeling. The knowledge common library includes the most basic use cases, such as account opening, login, etc. The BUG experience library is a case library formed by summarizing and precipitating representative BUGs found in production. The business scenario library is a collection library of test cases for scenarios involved in specific projects. The training database is a repository for storing the case label information finally determined by the executor each time on the Tctp Test platform. And the knowledge common library, the BUG experience library, and the business scenario library are a collection of case libraries directly facing and provided to test executors, which can be regularly maintained and updated by testers. The training database is the underlying engine library, which is used to input the information of actual test execution labels for modeling training, and feedback the obtained test guidance labels to the other three libraries to form a closed loop of the knowledge graph. And knowledge fusion will be carried out in the knowledge graph, that is, for the case failure flag, duplicate cases will be automatically identified and excluded. And knowledge storage can be carried out in the knowledge graph, that is, a key index based on the business scenario type + parent function type can be established to speed up the query.

[0071] In this embodiment, by determining the data input source according to the input business scenario type, and obtaining the first preset number of target label information from the untrained label information in the knowledge graph according to the data input source; obtaining the evaluation indicators corresponding to each of the target label information, and obtaining the index influence factors corresponding to each of the evaluation indicators; calculating the result weight value corresponding to each of the target label information according to each of the evaluation indicators and each of the index influence factors, and correcting the recommended score value corresponding to each of the target label information according to each of the result weight values; determining the second preset number of target recommended score values based on the corrected recommended score values, and using the evaluation schemes corresponding to each of the target recommended score values as test cases for testing. By determining the data input source according to the business scenario type, obtaining the target label information in the knowledge graph, then calculating the result weight value according to the evaluation indicators and index influence factors corresponding to the target label information, correcting the recommended score value corresponding to the target label information, and then determining the test cases according to the corrected recommended score values for testing, it is possible to effectively select the test cases that meet the business scenario type for testing, avoiding the phenomenon that the effectiveness of testing is reduced due to manually selecting test cases for testing in the prior art, and improving the effectiveness and accuracy of automated testing.

[0072] Further, based on the first embodiment of the test method of the present invention, a second embodiment of the test method of the present invention is proposed. This embodiment is a refinement of step S30 of the first embodiment of the present invention, which is the step of calculating the result weight value corresponding to each target label information according to each evaluation index and each index influencing factor, and includes:

[0073] Step a, calculating the total value of the corrected index of the to-be-evaluated solution corresponding to each target label information according to each evaluation index and each index influencing factor;

[0074] In this embodiment, when each evaluation index and the index influencing factor corresponding to each evaluation index are obtained, the total value of the corrected index of the to-be-evaluated solution corresponding to each target label information can be calculated through a pre-set calculation formula. The calculation formula can be:

[0075]

[0076] where c i represents the total value of the corrected index of the i-th test case (i.e., the total value of the corrected index).

[0077] Step b, calculating the result weight value corresponding to each target label information according to each total value of the corrected index and each evaluation index.

[0078] When the total value of the corrected index of each to-be-evaluated solution is obtained, the result weight value corresponding to each target label information can be calculated in turn according to each total value of the corrected index and each evaluation index. The calculation formula can be:

[0079]

[0080] In this embodiment, by calculating the total value of the corrected index according to each evaluation index and each index influencing factor, and calculating the result weight value according to the total value of the corrected index, the accuracy of the calculated result weight value is ensured.

[0081] Further, the step of correcting the recommended score value of the to-be-evaluated solution corresponding to each target label information according to each result weight value includes:

[0082] Step c, obtaining the recommended score value of the to-be-evaluated solution corresponding to each target label information, and sequentially traversing each recommended score value to determine the current result weight value corresponding to the current recommended score value in each result weight value;

[0083] In this embodiment, after obtaining the respective result weight values, it is also necessary to obtain the recommended score values of the solutions to be evaluated corresponding to each target tag information in the knowledge graph, traverse the respective recommended score values in sequence to determine the current recommended score value being traversed, and then obtain the result weight value corresponding to the current recommended score value (i.e., the current result weight value) among the respective result weight values.

[0084] Step d: Calculate the current recommended score value and the current result weight value based on a pre-designed calculation formula, and correct the current recommended score value according to the calculation result until all the recommended score values are traversed.

[0085] After obtaining the current recommended score value and the current result weight value, the current recommended score value can be corrected according to the pre-designed calculation formula. Among them, the pre-designed calculation formula can be score = score * (1 + p), where score is the current recommended score value and p is the current result weight value. And it should be noted that each recommended score value is corrected in the same way until all the recommended score values are traversed.

[0086] In this embodiment, by determining the current result weight value from the respective result weight values according to the current recommended score value, calculating based on the current recommended score value and the current result weight value, and correcting the current recommended score value based on the calculation result, the accuracy of correcting the current recommended score value is ensured.

[0087] Specifically, the step of obtaining the recommended score values of the solutions to be evaluated corresponding to each target tag information includes:

[0088] Step e: Traverse the solutions to be evaluated corresponding to each target tag information in sequence, determine the target library of the current solution to be evaluated being traversed in the knowledge graph, obtain the default score value of the target library, and use the default score value as the recommended score value of the current solution to be evaluated until all the solutions to be evaluated are traversed.

[0089] When obtaining the respective recommended score values, the solutions to be evaluated corresponding to each target tag information can be traversed first, and the target library of the current solution to be evaluated being traversed in the knowledge graph can be determined. Among them, the target library can be any one of the knowledge common library, BUG experience library, and specific business scenario library in the knowledge graph. And a default recommended score value is set for each library in the knowledge graph. Therefore, after determining the target library, the default score value of this target library (i.e., the default recommended score value) can be obtained and used as the recommended score value of the current solution to be evaluated. Until all the solutions to be evaluated are traversed, that is, each solution to be evaluated has a determined recommended score value.

[0090] In this embodiment, by traversing each solution to be evaluated, determining the target library in the knowledge graph where the current solution to be evaluated is placed, and using the default score value of the target library as the recommended score value of the current solution to be evaluated until all solutions to be evaluated are traversed. Thus, the accuracy of the recommended score values of the solutions to be evaluated obtained is ensured.

[0091] Further, the step of obtaining the evaluation indicators corresponding to each of the target label information includes:

[0092] Step f: Traverse each of the target label information in sequence, determine that the current target label information currently traversed corresponds to multiple different types of metric values, and use each of the metric values as the evaluation indicator corresponding to the current target label information until all of the target label information is traversed.

[0093] In this embodiment, when obtaining the evaluation indicators, it is necessary to traverse each of the target label information in sequence, determine the current target label information currently traversed, and it is necessary to determine multiple different types of metric values corresponding to the current target label information (such as the library where the case is located, the business scenario, the expert experience score of the test, the ranking of the actual usage times on the Tctp Test platform, etc.), and use these metric values together as the evaluation indicators corresponding to the current target label information until all of the target label information is traversed, that is, the evaluation indicators corresponding to each of the target label information are obtained.

[0094] In this embodiment, by traversing each of the target label information in sequence and using the multiple metric values corresponding to the current target label information currently traversed as the evaluation indicators corresponding to the current target label information until all of the target label information is traversed, the accuracy of the obtained evaluation indicators is ensured.

[0095] Further, the step of obtaining the index impact factors corresponding to each of the evaluation indicators includes:

[0096] Step h: Traverse each of the evaluation indicators in sequence, determine all of the metric values in the current evaluation indicator currently traversed, and determine the index impact factor corresponding to the current evaluation indicator according to each of the metric values until all of the evaluation indicators are traversed.

[0097] After obtaining the evaluation indicators corresponding to the target label information, traverse each of the evaluation indicators in sequence, determine all of the metric values carried in the current evaluation indicator currently traversed, and determine the index impact factor corresponding to the current evaluation indicator according to the importance degree of each of the metric values to the current evaluation indicator. Until all of the evaluation indicators are traversed.

[0098] In this embodiment, by traversing each evaluation index and determining all the index values in the current evaluation index to determine the index influence factor until all the evaluation indexes are traversed, the accuracy of the obtained index influence factor is ensured.

[0099] Further, the step of obtaining the first preset number of target label information from the unlabeled information in the knowledge graph according to the data input source includes:

[0100] Step m, obtaining a plurality of unlabeled information in the knowledge graph and sorting each of the unlabeled information according to the data input source;

[0101] In this embodiment, it is necessary to take the training data (about M * 100) with the flag of (untrained) in the knowledge graph for the last M times to obtain the training case label information (i.e., unlabeled information) under the business scenario type, and sort each unlabeled information with the usage times of the parent function type + child function type in the data input source as the keyword.

[0102] Step n, obtaining the first preset number of target label information from each of the unlabeled information according to the sorting result of the sorting.

[0103] Then, according to the sorting result of the sorting, obtain the first preset number of target label information with more usage times from each of the unlabeled information.

[0104] In this embodiment, by sorting a plurality of unlabeled information in the knowledge graph according to the data input source and obtaining the first preset number of target label information according to the sorting result, the accuracy of the obtained target label information is ensured.

[0105] The present invention also provides a testing device, refer to Figure 3 , the testing device includes:

[0106] Determination module A10, configured to determine a data input source according to the input business scenario type, and obtain the first preset number of target label information from the unlabeled information in the knowledge graph according to the data input source;

[0107] Obtaining module A20, configured to obtain the evaluation indexes corresponding to each of the target label information, and obtain the index influence factors corresponding to each of the evaluation indexes;

[0108] Calibration module A30, configured to calculate the result weight values corresponding to each of the target label information according to each of the evaluation indexes and each of the index influence factors, and calibrate the recommended score values corresponding to each of the target label information according to each of the result weight values;

[0109] The test module A40 is used to determine a second preset number of target recommendation score values based on each of the corrected recommendation score values, and use the solution to be evaluated corresponding to each of the target recommendation score values as a test case for testing.

[0110] Optionally, the correction module A30 is further configured to:

[0111] Calculate the total corrected index value of the solution to be evaluated corresponding to each of the target label information according to each of the evaluation indexes and each of the index influence factors;

[0112] Calculate the result weight value corresponding to each of the target label information according to each of the total corrected index values and each of the evaluation indexes.

[0113] Optionally, the correction module A30 is further configured to:

[0114] Obtain the recommendation score value of the solution to be evaluated corresponding to each of the target label information, traverse each of the recommendation score values in sequence, and determine the current result weight value corresponding to the current recommendation score value being traversed among each of the result weight values;

[0115] Calculate the current recommendation score value and the current result weight value based on a pre-designed calculation formula, and correct the current recommendation score value according to the calculation result until each of the recommendation score values is traversed.

[0116] Optionally, the correction module A30 is further configured to:

[0117] Traverse each of the solutions to be evaluated corresponding to the target label information in sequence, determine the target library of the current solution to be evaluated being traversed in the knowledge graph, and obtain the default score value of the target library, and use the default score value as the recommendation score value of the current solution to be evaluated until each of the solutions to be evaluated is traversed.

[0118] Optionally, the acquisition module A20 is further configured to:

[0119] Traverse each of the target label information in sequence, and determine multiple different types of index quantity values corresponding to the current target label information being traversed, and use each of the index quantity values as the evaluation index corresponding to the current target label information until each of the target label information is traversed.

[0120] Optionally, the acquisition module A20 is further configured to:

[0121] Traverse each of the evaluation indexes in sequence, determine all the index quantity values in the current evaluation index being traversed, and determine the index influence factor corresponding to the current evaluation index according to each of the index quantity values until each of the evaluation indexes is traversed.

[0122] Optionally, the determining module A10 is further configured to:

[0123] Obtain multiple untrained label information in the knowledge graph, and sort each of the untrained label information according to the data input source;

[0124] Obtain a first preset number of target label information from each of the untrained label information according to the sorting result of the sorting.

[0125] The methods executed by the above program units can refer to the respective embodiments of the test method of the present invention, which will not be elaborated here.

[0126] The present invention also provides a computer storage medium.

[0127] A test program is stored on the computer storage medium of the present invention, and when the test program is executed by a processor, the steps of the test method described above are implemented.

[0128] Among them, the method implemented when the test program running on the processor is executed can refer to the respective embodiments of the test method of the present invention, which will not be elaborated here.

[0129] It should be noted that in this article, the terms "including", "comprising" or any other variant thereof are intended to cover a non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or system. Without more limitations, the element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or system including the element.

[0130] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0131] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions for causing a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the respective embodiments of the present invention.

[0132] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be similarly included in the patent protection scope of the present invention.

Claims

1. A testing method, characterized in that, the testing method includes the following steps: Determine a data input source according to the input business scenario type, and obtain a first preset number of target label information from the un-trained label information in the knowledge graph; wherein, the data input source includes the business scenario type, as well as the parent function type and the child function type associated with the business scenario type, the parent function type is the major requirement of the test case, and the child function type is the minor requirement under the major requirement; the step of obtaining a first preset number of target label information from the un-trained label information in the knowledge graph according to the data input source includes: obtaining a plurality of un-trained label information in the knowledge graph, and using the usage times of the parent function type and the child function type in the data input source as keywords to sort each of the un-trained label information; obtaining a first preset number of target label information from each of the un-trained label information according to the sorting result of the sorting. Obtain the evaluation index corresponding to each of the target label information, and obtain the index influence factor corresponding to each of the evaluation indexes. Calculate the result weight value corresponding to each of the target label information according to each of the evaluation indexes and each of the index influence factors, and correct the recommended score value corresponding to each of the target label information according to each of the result weight values. Determine a second preset number of target recommended score values based on the corrected recommended score values, and use the solution to be evaluated corresponding to each of the target recommended score values as a test case for testing.

2. The testing method according to claim 1, characterized in that, the step of calculating the result weight value corresponding to each of the target label information according to each of the evaluation indexes and each of the index influence factors includes: Calculate the total corrected index value of the solution to be evaluated corresponding to each of the target label information according to each of the evaluation indexes and each of the index influence factors. Calculate the result weight value corresponding to each of the target label information according to each of the total corrected index values and each of the evaluation indexes.

3. The testing method according to claim 1, characterized in that, the step of correcting the recommended score value of the solution to be evaluated corresponding to each of the target label information according to each of the result weight values includes: Obtain the recommended score value of the solution to be evaluated corresponding to each of the target label information, traverse each of the recommended score values in turn, and determine the current result weight value corresponding to the current recommended score value currently traversed among each of the result weight values. Calculate the current recommended score value and the current result weight value based on a pre-designed calculation formula, and correct the current recommended score value according to the calculation result until each of the recommended score values is traversed.

4. The testing method according to claim 3, characterized in that, the step of obtaining the recommended score value of the solution to be evaluated corresponding to each of the target label information includes: Traverse the solution to be evaluated corresponding to each of the target tag information in sequence, determine the target library of the current solution to be evaluated being traversed in the knowledge graph, and obtain the default score value of the target library. Use the default score value as the recommended score value of the current solution to be evaluated until all the solutions to be evaluated are traversed.

5. The testing method according to claim 1, wherein, the step of obtaining the evaluation index corresponding to each of the target tag information includes: Traverse each of the target tag information in sequence, and determine that the current target tag information being traversed corresponds to a plurality of different types of index values. Use each of the index values as the evaluation index corresponding to the current target tag information until all the target tag information is traversed.

6. The testing method according to claim 1, wherein, the step of obtaining the index influence factor corresponding to each of the evaluation indexes includes: Traverse each of the evaluation indexes in sequence, determine all the index values in the current evaluation index being traversed, and determine the index influence factor corresponding to the current evaluation index according to each of the index values until all the evaluation indexes are traversed.

7. A testing device, wherein, the testing device includes: a determination module, configured to determine a data input source according to the input business scenario type, and obtain a first preset number of target tag information from the untrained tag information in the knowledge graph; wherein, the data input source includes a business scenario type, as well as a parent function type and a child function type associated with the business scenario type, the parent function type is the large requirement of the test case, and the child function type is the small requirement under the large requirement; the step of obtaining a first preset number of target tag information from the untrained tag information in the knowledge graph according to the data input source includes: obtaining a plurality of untrained tag information in the knowledge graph, and sorting each of the untrained tag information with the usage times of the parent function type and the child function type in the data input source as keywords; obtaining a first preset number of target tag information from each of the untrained tag information according to the sorting result of the sorting; an acquisition module, configured to obtain the evaluation index corresponding to each of the target tag information, and obtain the index influence factor corresponding to each of the evaluation indexes; a correction module, configured to calculate the result weight value corresponding to each of the target tag information according to each of the evaluation indexes and each of the index influence factors, and correct the recommended score value corresponding to each of the target tag information according to each of the result weight values; a testing module, configured to determine a second preset number of target recommended score values based on the corrected recommended score values, and use the solution to be evaluated corresponding to each of the target recommended score values as a test case for testing.

8. A testing device, wherein, the testing device includes: a memory, a processor, and a testing program stored on the memory and executable on the processor. When the testing program is executed by the processor, the steps of the testing method according to any one of claims 1 to 6 are implemented.

9. A computer storage medium, characterized in that, a test program is stored on the computer storage medium, and when the test program is executed by a processor, the steps of the test method described in any one of claims 1 to 6 are implemented.

Images