Collection terminal test case priority ranking method based on multi-feature fusion

Abstract

The invention discloses an acquisition terminal test case priority ranking method based on multi-feature fusion, and aims to solve the problems of low case ranking accuracy and unreasonable test resource allocation in the existing regression test from a version control system, a defect management system and a test execution history. Extracting three types of features including the program change point correlation degree, the historical defect detection rate and the use case execution efficiency; after quantization and normalization processing, constructing a feature vector of each test case; and establishing a priority evaluation model by utilizing a decision tree algorithm, calculating a comprehensive priority score of each test case according to the feature vectors, and generating a test sequence according to descending sort of the scores. According to the method, the regression test defect detection efficiency can be improved by 30%, the test resource utilization rate can be improved by 50%, and the method is suitable for collection terminal software testing with frequent version iteration.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, specifically to a test case priority ranking method for smart grid data acquisition terminal software, and particularly to a dynamic evaluation method for test case priority based on multi-feature fusion. Background Technology

[0002] In the software development and maintenance of smart grid data acquisition terminals, software versions iterate frequently, resulting in heavy regression testing tasks. The contradiction between limited testing resources and a large test case set is becoming increasingly prominent. Traditional test case prioritization methods often rely on the personal experience of testers or are based on a single criterion (such as code coverage), which leads to problems such as strong subjectivity, insufficient accuracy, and inability to dynamically adapt to code changes.

[0003] In existing technologies, some methods only focus on code changes but ignore the historical value (such as the ability to find defects) and execution cost of the test cases themselves. This may result in high-value test cases not being prioritized or inefficient execution. Therefore, there is a need for a method that can comprehensively consider multiple dimensions such as code changes, defect history, and execution efficiency to achieve scientific, automatic, and accurate ranking, in order to optimize test resource allocation and improve the efficiency and defect detection capability of regression testing. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a method for prioritizing test cases for data acquisition terminals based on multi-feature fusion. This method can comprehensively evaluate the correlation between test cases and current software changes, historical defect detection capabilities, and execution costs, thereby achieving high-precision and high-efficiency prioritization.

[0005] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0006] A method for prioritizing test cases of a data acquisition terminal based on multi-feature fusion, characterized by the following steps:

[0007] S1: Feature extraction stage, extract program change point features, historical defect features, and test case execution efficiency features from the version control system, defect management system, and test management platform, respectively;

[0008] S2: Feature quantization and normalization stage, the extracted multi-dimensional features are quantized and normalized to the same dimension to form the feature vector of each test case;

[0009] S3: Priority evaluation model construction stage. Based on the decision tree algorithm, an evaluation model is constructed with multiple feature vectors as input and priority scores as output.

[0010] S4: Priority calculation and sorting stage. Input the set of test cases to be sorted into the evaluation model, calculate the comprehensive priority score of each test case, and sort them in descending order according to the score.

[0011] S5: Test execution and feedback phase, execute high-priority test cases, and optimize the model based on test results.

[0012] Furthermore, the program change point characteristics are quantified by calculating the intersection degree between the code coverage set of the test cases and the current code change set. The calculation formula is as follows:

[0013]

[0014] Among them, Cov(TC) i ) indicates test case TC i The set of code to be covered, where ΔCode represents the set of code to be changed in this instance.

[0015] Furthermore, the historical defect characteristics are quantified by the historical defect detection rate of test cases, calculated using the following formula:

[0016]

[0017] Where, N defect (TC i ) represents the historical TC i The total number of defects detected, N execution (TC i ) represents the historical TC i The total number of times it is executed.

[0018] Furthermore, the test case execution efficiency characteristic is quantified by the reciprocal of the average execution time of the test cases, so as to achieve the effect of higher efficiency and higher score. The calculation formula is as follows:

[0019]

[0020] Among them, T avg (TC i ) is the test case TC i The average execution time.

[0021] Furthermore, the decision tree model employs the CART algorithm, the core of which is to select the optimal feature for node splitting using the Gini index. For each node, feature A and split point a are selected such that the sum of the Gini indices of the resulting subsets is minimized. The formula for calculating the Gini index is:

[0022]

[0023] Where D is the sample set of the current node, K is the number of categories (e.g., high priority / low priority), and C... k This is a subset of samples belonging to class k in D. The Gini gain of feature A at the split point a is:

[0024]

[0025] Here, D1 and D2 are two subsets partitioned based on whether feature A is less than or equal to a. The model constructs a tree structure through recursive splitting, ultimately mapping test cases to different leaf nodes, with each leaf node corresponding to a priority score.

[0026] Technical effect

[0027] This invention achieves intelligent and dynamic prioritization of test cases by combining multi-feature fusion with a decision tree model, and has the following technical effects:

[0028] (1) Improved sorting accuracy: Compared with the traditional single-criteria method, the defect detection rate of this invention is improved by more than 30%;

[0029] (2) Test efficiency optimization: Prioritize the execution of the top 30% of high-priority test cases to cover more than 80% of defects and improve test resource utilization by 50%;

[0030] (3) The model is highly adaptable: After three rounds of feedback optimization, the model ranking accuracy is stable at over 90%, and it can continuously adapt to the evolution of software versions. Attached Figure Description

[0031] Figure 1 This is a flowchart illustrating the overall process of the test case priority ranking method described in this invention.

[0032] Figure 2 This is a schematic diagram of the multi-feature fusion and decision tree evaluation model in this invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0034] Example:

[0035] like Figure 1 As shown, the specific implementation process of the present invention is as follows:

[0036] 1. Data Acquisition and Feature Extraction (S1):

[0037] Change point characteristics: Obtain the list of code modifications (ΔCode, i.e., modified files and methods) for this commit from the Git version control system. Simultaneously, obtain the test case (TC) for each test case from a code coverage repository (such as JaCoCo). i Code coverage set Cov(TC) i ). Calculate the change association for each use case. change (TC i ).

[0038] Historical Defect Characteristics: From defect management systems such as JIRA, query the number of defects associated with each test case in historical regression testing. From test management platforms such as TestRail, query the historical execution count of each test case. Calculate the defect value for each test case. defect (TC i ).

[0039] Test case execution efficiency characteristics: Obtain the average execution time T of each test case across multiple executions from the test management platform. avg (TC i ), and calculate its efficiency (TC) i ).

[0040] 2. Feature Quantization and Normalization (S2):

[0041] The above three eigenvalues ​​(Association) change (TC i Value defect (TC i Efficiency (TC) i Minimum-maximum normalization is performed to scale the data to the [0,1] interval, eliminating the influence of dimensions. The normalization formula is:

[0042]

[0043] Ultimately, each test case TC i Represented as a three-dimensional feature vector: V i = (x1, x2, x3).

[0044] 3. Priority evaluation model construction (S3):

[0045] Model training: Historical test data (containing feature vectors of numerous test cases and their final labels indicating whether a defect was found) is used as the training set. The CART decision tree algorithm is employed for training.

[0046] Model workflow: as attached Figure 2As shown, the trained decision tree model is a binary tree structure. For a new use case, the feature vector V... i Starting from the root node:

[0047] Determine if x1 (change correlation degree) is ≤ 0.6. If yes, proceed to the left subtree; otherwise, proceed to the right subtree.

[0048] In the left subtree, continue to determine if x2 (defect value) is ≤ 0.3?

[0049] This process is repeated recursively until a leaf node is reached. This leaf node contains the average priority score calculated based on the training data (e.g., the probability that all training use cases in this node find a defect).

[0050] In this way, the model can learn the complex nonlinear relationships between the three features (e.g., "even if the correlation between changes is not high, if the historical defect detection rate is extremely high, it should still be given high priority").

[0051] 4. Priority Calculation and Sorting (S4):

[0052] The feature vectors of all test cases to be tested are input into the trained decision tree model, and the model outputs a comprehensive priority score S for each test case. i According to score S i All test cases are sorted from high to low to form the final priority test sequence.

[0053] 5. Test Execution and Feedback (S5):

[0054] Testers execute test cases in the ordered sequence. The results of this test (especially which test cases discovered new defects) are recorded and fed back into the training set as new labeled data. This data is used to periodically retrain the decision tree model, enabling the model to continuously optimize and adapt to project evolution.

[0055] Through the above implementation methods, the present invention achieves multi-dimensional, data-driven intelligent evaluation of test case priorities, significantly improving the accuracy and efficiency of regression testing of data acquisition terminal software.

Claims

1. A method for prioritizing test cases of a data acquisition terminal based on multi-feature fusion, characterized in that, Includes the following steps: Step 1: Multi-dimensional feature extraction: Extract three types of features for each test case from the software version control system, defect management system, and test management platform, including: program change point correlation degree reflecting the degree of correlation between the test case and the current code modification, historical defect value degree reflecting the test case's ability to discover historical defects, and execution efficiency degree reflecting the execution cost of the test case. Step 2: Feature normalization process: Normalize the values ​​of the above three types of features to the same numerical range to eliminate the influence of units and form a standardized feature vector for each test case; Step 3: Priority evaluation model construction steps: Based on the decision tree algorithm, a classification or regression model is trained using historical test data and its feature vectors. This model can output a comprehensive priority score based on the input standardized feature vectors. Step 4: Priority sorting and test execution steps: Input the feature vector of the current test case set to be sorted into the model to obtain the priority score of each test case, and sort them from high to low according to the score to generate a test execution sequence; Step 5: Model Feedback Optimization Step: Based on the test execution results, the data from this test is added to the training set as new samples, and the priority evaluation model is periodically retrained to achieve adaptive optimization of the model.

2. The method as described in claim 1, characterized in that, The correlation of the program change point is measured by calculating the proportion of the intersection between the code coverage of the test case and the scope of the current code change relative to the scope of the current code change.

3. The method as described in claim 1, characterized in that, The historical defect value is measured by calculating the average number of defects that the test case can find per execution throughout its history.

4. The method as described in claim 1, characterized in that, The execution efficiency is inversely proportional to the average execution time of the test case; that is, the shorter the execution time, the higher the efficiency.

5. The method as described in claim 1, characterized in that, The normalization process in step 1 uses the minimum-maximum normalization method to linearly transform the original feature values ​​to the interval [0,1].

6. The method as described in claim 1, characterized in that, The decision tree algorithm in step 3 uses the CART algorithm; this algorithm selects the optimal splitting feature and splitting point by calculating the Gini index gain, and the formula for calculating the Gini index gain is as follows: The model constructs a decision tree by recursively selecting the splitting method that maximizes the Gini index gain.

7. The method as described in claim 1, characterized in that, The feedback optimization in step (5) is specifically implemented as follows: the execution results of the test cases are marked, the test cases that successfully detect defects are used as high-priority positive samples, and the test cases that do not detect defects are used as low-priority samples for updating the model.

Images