Test case optimization method for automatic test industry

Abstract

The invention discloses an automatic test industry-oriented test case optimization method, which is based on a graph theory and a solution algorithm of a traveling salesman problem (TSP), can analyze an optimal test sequence of complex test cases, and removes invalid operation of a measuring instrument in a test process. The method comprises the following steps: (1) calibrating operation time required by all instrument operations to form a time cost configuration file; (2) defining limitation and association conditions among the operation of each instrument, and making a set of judgment rules capable of judging the effectiveness of the operation of the instrument; (3) converting the use case information into a relation graph based on related knowledge of a graph theory; (4) on the basis of a TSP problem algorithm, calculating a shortest path capable of traversing each node in a non-repeated manner according to the relation graph; and (5) reconstructing the test case based on the shortest path. The method has the following beneficial effects: (1) the test efficiency of the automatic test platform is improved; and (2) invalid operation of a measuring instrument is reduced, and instrument loss is reduced.

Description

Technical Field

[0001] The invention relates to a test case optimization method for the automatic testing industry. Background Art

[0002] Test cases are an important concept in the automatic testing industry. In the automatic testing industry, it is often necessary to control the test instruments through program control to automatically complete the test result collection work, and the test case is the bridge connecting the upper-level test plan and the underlying device driver. The test case is essentially a task list issued by the host computer to the test client. It must use simple and easy-to-understand natural language to express the test object, test method, and result evaluation criteria, which is convenient for testers to read and edit; it must also be written in a fixed format and keywords, and can be parsed by the test client as clear instrument instructions.

[0003] In a large-scale production test environment, test time is a factor that cannot be ignored. Even under program control, it still takes time for the instrument to execute program control instructions, especially for some mechanically actuated equipment such as relay fixtures, sorting machines, conveyor belts, etc., whose actuation delay will be more obvious. However, in the process of writing test cases, engineers often pay more attention to whether the test cases can correctly obtain test results, and tend to ignore the test time. When manually writing test cases, invalid instrument repetitive operations may be added, and it is impossible to analyze the impact of the test sequence on the test time from a global perspective.

[0004] A test case that can complete the test task does not mean that it is reasonable. An unreasonable test case will lead to frequent actuation of the test instrument, which not only increases the test time but also accelerates the wear and tear of the test instrument. Although engineers will classify and integrate some instrument operations, this manual optimization method usually cannot achieve the best optimization effect, so it is very necessary to optimize the test case in a systematic and scientific way.

[0005] There are two main factors that affect the test time. One is the existence of invalid repetitive operations in the test cases, such as repeated setting of the same instrument parameters, repeated switching of instruments, etc.; the other is the test order of the test items. Different test orders will cause corresponding changes in instrument operation. A reasonable test order can make full use of the current instrument status to complete multiple tests. An unreasonable order often mixes test items that require different measurement parameters together, resulting in frequent changes in instrument status and increased test time.

[0006] In order to distinguish valid operations from invalid operations, we need to formulate discrimination rules based on actual conditions; as for how to obtain the optimal test sequence, the essence of this problem is the classic traveling salesman problem (TSP), and we can refer to the relevant algorithms of the TSP problem to solve the optimal sequence. Summary of the invention

[0007] The present invention provides a test case optimization method to solve the problems of invalid test operations and unreasonable test sequence that may exist in test cases in the automatic testing industry. The present invention is based on graph theory and a solution of the traveling salesman problem (TSP), attempts to optimize complex test cases, and achieves the removal of invalid operations in the test cases and the search for the optimal test sequence, thereby reducing test time consumption and improving test efficiency.

[0008] The test case optimization method can be applied to the automatic test platform and play a role in the preliminary review of the test case. The implementation steps of the method are as follows: first, the operation list of all test instruments is listed, and the operation time is calibrated for each operation according to the actual situation to form a time cost configuration file in a fixed format. Then, the restrictions between the operations of each instrument are clarified, and a set of discrimination rules that can judge the effectiveness of the instrument operation are formulated, and the discrimination rule configuration file is written in a fixed format. After that, the test case information is converted into a relationship graph and stored in the form of an adjacency matrix. Finally, based on the dynamic programming algorithm and annealing algorithm of the TSP problem, the shortest path that can traverse each node without repetition is calculated according to the test case relationship graph, and the traversed node order is the optimal test order. Under the optimal test order, the effective operations are extracted item by item from the test case relationship graph and recombined into a complete test case. The test case is the optimized test case, which corresponds to the lowest test time cost and the highest test efficiency.

[0009] The characteristics of the present invention are:

[0010] (1) Good versatility: Test cases, time cost configuration files, and discrimination rule configuration files can be edited using a browser interface. They are applicable to different test objects and test instruments, can be adjusted in a timely manner according to actual conditions, and can be easily integrated into the test platform.

[0011] (2) Fully shorten the test time: Based on the judgment rules and the test cases optimized by the TSP algorithm, the test tasks can be completed with the least operations in the best test sequence, thus shortening the test time.

[0012] (3) Evaluation indicators are quantifiable: The total test time of the original case and the optimized case can be calculated based on the calibrated time cost. The number of invalid operations in the original case can be obtained based on the judgment rules. By comparing the two indicators of the number of invalid operations and the total test time, the quantitative evaluation of the test case optimization effect can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 Flowchart of the optimization method.

[0014] Figure 2 Example of discrimination rule description.

[0015] Figure 3Test case relationship diagram.

[0016] Figure 4 Adjacency matrix storage structure.

[0017] Figure 5 Typical data samples.

[0018] Figure 6 Test process build. DETAILED DESCRIPTION

[0019] A test case optimization method for the automatic testing industry provided by the present invention is described in detail below with reference to the accompanying drawings.

[0020] This method is based on graph theory and the solution of the traveling salesman problem (TSP). It can remove invalid operations in test cases and optimize the test sequence, thereby reducing the test time cost and improving the test efficiency. The overall method implementation is divided into five steps, such as Figure 1 As shown, the implementation steps are described in detail below.

[0021] Step 1: Calibrate the time required for the instrument operation required for the test, that is, measure the time required for the instrument to perform a certain operation. Actual measurements are performed for instrument operations that are obviously time-consuming. For operations that are extremely short or inconvenient to measure directly, reasonable calibration can be performed based on experience. The calibrated data is compiled into an operation time configuration file. This configuration file is the data basis for solving the optimal path through the TSP algorithm in the future, so the actual measurement data should be used as much as possible during the calibration process to be close to the actual situation.

[0022] Step 2: Clarify the restrictions and related conditions between the operations of each instrument, formulate a set of discrimination rules that can judge the effectiveness of instrument operations, and form a discrimination rule configuration file in a fixed format. During the test process, there are many instrument states that have inheritance and restriction relationships. Figure 2 An example is provided. In this example, the protection voltage of the power supply has been set in test item 1, so it is not necessary to set it again in test item 2. The power supply is also needed in test item 2, so there is no need to turn off the power supply in test item 1, and there is no need to turn on the power supply again in test item 2, which saves a lot of repeated operations. Therefore, the judgment rules for the power supply are: the power switch state can be inherited, the protection voltage setting parameters of the power supply can be inherited, and if a certain (group) power supply operation does not change the original state of the power supply, the operation of this (group) is judged as an invalid operation. The judgment logic of other instruments is similar to that of the power supply.

[0023] Step 3: Convert the use case information into a relationship graph. Each test item is a vertex of the graph. The paths between vertices have weights. The weights are set according to the time taken by the test operation. The longer the time taken, the larger the corresponding weight. If there is no need to change the state of the test instrument between two vertices, then the weight on the path between the two vertices is 0. Since the test items are relatively independent and there is no binding of the test sequence, there is a path between any two vertices in the graph. Finally, a weighted completely directed graph can be constructed, such as Figure 3 shown.

[0024] The adjacency matrix method is used to store the graph. Since the test case relationship graph is a complete graph, only the diagonal elements of its adjacency matrix are 0, and the rest of the elements are non-zero elements. The resources in the matrix can be fully utilized. The entire test process starts from the initial state of the system. After the test task is completed, all devices need to be shut down to return the system to the initial state, so an initial item must be added to the test item sequence as a starting point. The adjacency matrix structure is as follows Figure 4 As shown, the initial item corresponds to the first row and the first column of the matrix. The data in the first row represents the transition from the initial state to other test items at the beginning of the test, and the data in the first column represents the return from other test items to the initial state at the end of the test.

[0025] Each element of the adjacency matrix stores a dictionary data, such as Figure 5 As shown in the figure, it includes three items: "weight", "operation" and "identification". "Weight" is the weight generated according to the time cost of the operation, which plays a role in the subsequent TSP algorithm. "Operation" records the effective operation to complete the target item test. "Identification" is generated according to the discrimination rules and is used to judge necessary operations and non-necessary operations.

[0026] After the adjacency matrix is ​​constructed, the operation sequence required for the target item can be read from the matrix according to the row and column indexes according to the given test order, and then multiple target items are spliced ​​in sequence until a complete test case is formed. The process is as follows: Figure 6 For example, given the test sequence of [0, 2, 4, 1, 3, 5, 6, 7, 0], where item 0 represents the initial item, under this order, a test case that removes invalid operations can be completely restored based on the adjacency matrix that stores the operation information.

[0027] Step 4: Based on the dynamic programming algorithm and annealing algorithm of the TSP problem, the "weight" in the adjacency matrix elements is used as the path length to calculate the shortest path that can traverse each node without repetition. The order of traversed nodes is the optimal test order.

[0028] The mathematical model of TSP is described as formula (1), where d ij is the distance from point i to point j, and V is the set of all vertices.

[0029]

[0030] The TSP problem is an NPC problem. If all possible results are traversed, the number of traversals will increase in the form of n!, so a special algorithm is needed to efficiently solve the TSP problem. Currently, common algorithms for solving TSP include genetic algorithms, annealing algorithms, ant colony algorithms, taboo search methods, dynamic programming algorithms, etc. This invention combines dynamic programming and annealing algorithms to perform segmented calculations on test cases with different data volumes, and ultimately obtains the shortest path. The order in which the path traverses the nodes is the optimal test order.

[0031] Step 5: According to the optimal test sequence solved by the algorithm, reconstruct the optimized test case according to the adjacency matrix. According to the optimal test item sequence, locate the corresponding element in the adjacency matrix through the row and column index, obtain the effective operations to complete the target test item from the element information, and combine these effective operations in turn to finally reconstruct the optimized test case. This case consists of only effective operations, with the least test time and the highest test efficiency. At this point, the entire test case optimization process is completed. The evaluation of the optimization effect can be quantified into the following two indicators:

[0032]

[0033]

[0034] By examining the indicators, we can obtain the test performance of the optimization case and provide a reliable and specific basis for further design and improvement of the optimization method.

Claims

1. A test case optimization method for the automatic testing industry, characterized in that: The following steps are involved: Calibrate the time required for all instrument operations to form a time cost profile; Clarify the restrictions and related conditions between the operations of each instrument, formulate a set of discrimination rules that can judge the effectiveness of instrument operations, and write them in a fixed format to form a discrimination rule configuration file; Convert use case information into a relationship graph, and store the graph information in the form of an adjacency matrix; Based on the dynamic programming algorithm and annealing algorithm of the TSP problem, the "weight" in the adjacency matrix element is used as the path length to calculate the shortest path that can traverse each node without repetition. The order of traversed nodes is the optimal test order. According to the optimal test sequence solved by the algorithm, the optimized test cases are reconstructed according to the adjacency matrix.

2. According to claim 1, the test case optimization method for the testing industry is characterized in that: The time required for the calibration instrument operation specifically includes: The operation time of all measuring instruments used is calibrated, and the operation time calibration results are converted into an operation time configuration file in a fixed format.

3. According to claim 1, the test case optimization method for the testing industry is characterized in that: Develop a set of rules to determine the effectiveness of instrument operation, including: A set of discrimination rules for judging the effective operation of the instrument is formulated according to the use specifications of the test instrument. The discrimination rules need to be specifically combined with the safe use of the instrument, specially formulated according to the inheritance and restriction relationship of the instrument status, and form a discrimination rule configuration file in a fixed format.

4. According to claim 1, the test case optimization method for the testing industry is characterized in that: Convert use case information into a relationship diagram, including: Analyze the instrument status changes between each test item, obtain valid operations according to the discrimination rules, calculate the total operation time of the test items, and build a test item relationship graph based on this and store the graph in the form of an adjacency matrix. The nodes in the graph represent test items, and the edges represent the relationship between two test items. The edge information includes: target test item name, test valid operations, total operation time, and status flag. For any given test item sequence, a test case containing only valid operations can be reconstructed through the test item relationship graph.

5. According to claim 1, the test case optimization method for the testing industry is characterized in that: Based on the dynamic programming algorithm and annealing algorithm of the TSP problem, the optimal test sequence is calculated, including: Based on the test item relationship graph and following the solution to the TSP problem, the dynamic programming algorithm and annealing algorithm are combined to calculate the order of test items with the shortest total operation time.

6. According to claim 1, the test case optimization method for the testing industry is characterized in that: Refactor the test cases according to the optimal test item sorting, including: According to the optimal test item sequence, the corresponding elements in the adjacency matrix are located by row and column indexes, and effective operations to complete the target test items are obtained from the element information. These effective operations are spliced ​​and combined in sequence, and finally an optimized test case is reconstructed.

Images