A method and system for automatically generating test cases based on MoMuUTML

Abstract

The application discloses a test case automatic generation method and system based on MoMuTUML, reduces and deletes original function overlapping mutation operators, and adds three new mutation operators which are not functionally overlapped with the original mutation operators, so that the generation of a syntax equivalent mutation model is reduced, and the test case generation time cost is reduced; test cases are generated according to a mutation model set and a model under test, and equivalent mutation models are deleted according to the test cases, so that effective mutation models are obtained, the problem of too many equivalent mutation models in MoMuTUML is avoided, and the test case generation efficiency is improved; through a feedback-based mutation method, the mutation operators corresponding to the effective mutation models are scored, an optimal mutation operator set is constructed, and the mutation operators in the set are selected to perform mutation operations on the model under test, so that the problem of low efficiency of the MoMuTUML mutation method is solved, and the test case generation efficiency is further improved.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and specifically to a method and system for automatically generating test cases based on MoMuTUML. Background Technology

[0002] MoMuTUML employs a model-based fault-based test case generation method, relying on the model of the system under test (SUT) to create test cases; it is a black-box testing approach. After inserting a fault into the SUT model, it generates test cases that reveal that fault. This makes model-based test case generation one of the most powerful and versatile available methods, as it creates test cases that guarantee the disclosure of certain faults. Testing with these generated test cases can both prove the absence of certain faults and reveal behavioral information under fault conditions. However, MoMuTUML currently has the following limitations:

[0003] (1) The existing mutation operators in MoMuTUML may have overlapping functions, which can easily lead to syntactically equivalent mutation models, increasing the overhead of test case generation and resulting in low test case generation efficiency.

[0004] (2) The current mutation method of MoMuTUML is relatively inefficient and cannot select the most suitable set of mutation operators for the model under test, resulting in low efficiency in test case generation.

[0005] (3) The overall process design of MoMuTUML is inflexible. It is executed sequentially and cannot receive feedback on the test case generation results, nor can it adjust the mutation process based on the feedback, resulting in low test case generation efficiency. Summary of the Invention

[0006] In view of this, the present invention provides a method and system for automatically generating test cases based on MoMuTUML, which can efficiently generate test cases and feed the test case generation results back to the mutation process to adjust the mutation process and improve the efficiency of test case generation.

[0007] The specific technical solution adopted in this invention is as follows:

[0008] A method for automatically generating test cases based on MoMuTUML, comprising:

[0009] Step 1: Establish the model under test, and obtain the initial global state coverage statistics table and the initial global transformation coverage statistics table based on the model under test;

[0010] Step 2: Perform mutation operations on the tested model according to each mutation operator in the mutation operator set to construct the corresponding mutated model set;

[0011] Step 3: Generate test cases based on the mutation model set and the tested model, delete equivalent mutation models in the mutation model set according to the test cases, obtain effective mutation models, and update the initial global state coverage statistical table and the initial global transition coverage statistical table according to the effective mutation models to obtain the global state coverage statistical table and the global transition coverage statistical table;

[0012] Step 4: Score each mutation operator according to the global state coverage statistical table, the global transition coverage statistical table and the effective mutation models, and construct the optimal mutation operator set of the tested model;

[0013] Step 5: Count the number of rounds n in which no new test cases are generated continuously. If n < N, replace the mutation operator set in Step 2 with the optimal mutation operator set in Step 4, and repeat Steps 2 to 4; if n ≥ N, stop the test case generation process and obtain the generated test case set.

[0014] Further, in Step 1, obtaining the initial global state coverage statistical table and the initial global transition coverage statistical table according to the tested model includes:

[0015] Perform statistical analysis on the tested model according to predicate logic, and establish the initial global state coverage statistical table and the initial global transition coverage statistical table according to the statistical analysis results.

[0016] Further, in Step 2, performing mutation operations on the tested model according to each mutation operator in the mutation operator set includes:

[0017] Add three mutation operators, namely: the mutation operator for changing the source state of the transition, the mutation operator for changing the target state of the transition, and the mutation operator for removing the random transition of a certain state;

[0018] Delete four mutation operators, namely: setting the guard condition to false, setting the guard condition to true, setting the Boolean type OCL sub-expression to false, and setting the Boolean type OCL sub-expression to true;

[0019] Perform mutation operations on the tested model according to the modified mutation operators.

[0020] Further, in Step 2, performing mutation operations on the tested model according to each mutation operator in the mutation operator set further includes:

[0021] Group the modified mutation operators, and use the mutation operator for changing the source state of the transition and the mutation operator for changing the target state of the transition as the state coverage gain group, and use other mutation operators as the transition coverage gain group.

[0022] Further, in step three, the step of generating test cases based on the set of mutated models and the model under test, deleting equivalent mutated models from the set of mutated models based on the test cases to obtain effective mutated models, and updating the initial global state coverage statistics table and the initial global transformation coverage statistics table based on the effective mutated models to obtain the global state coverage statistics table and the global transformation coverage statistics table includes:

[0023] Select a variant model from the set of variant models and perform a consistency check with the model under test. Search for behavioral differences between the variant model and the model under test. If differences exist, generate test cases based on the behavioral differences, designate the variant model as a valid variant model, and mark the mutation operator corresponding to the valid variant model. If no differences exist, continue to select a variant model from the set of variant models until a variant model with behavioral differences from the model under test is selected or the set of variant models has been traversed.

[0024] The states and transitions of the tested model covered by the effective mutation model are marked and updated in the initial global state coverage statistics table and the initial global transition coverage statistics table to obtain the global state coverage statistics table and the global transition coverage statistics table.

[0025] Further, in step four, based on the global state coverage statistics table, the global transformation coverage statistics table, and the score given to each mutation operator by the effective mutation model, the optimal mutation operator set for the tested model is constructed, including:

[0026] Calculate the state coverage and transition coverage of the generated test cases for the model under test based on the global state coverage statistics table and the global transition coverage statistics table:

[0027] State coverage:

[0028]

[0029] Where |S′| is the number of covered states, and |S| is the total number of states of the tested model;

[0030] Conversion coverage:

[0031]

[0032] Where |T′| is the number of transformations that have been covered, and |T| is the total number of transformations of the tested model;

[0033] The mutation operators are scored based on the state coverage, the transition coverage, and the proportion of effective mutation models. The top b mutation operators (b < count(i), where i refers to the mutation operator) are selected to construct the optimal mutation operator set for the tested model.

[0034] Further, the scoring of the mutation operator based on state coverage, transition coverage, and the proportion of effective mutation models includes:

[0035] For the mutation operator in the state coverage gain group, first calculate the state coverage gain based on the state coverage rate:

[0036]

[0037] The mutation operator is then scored based on the state coverage gain and the proportion of effective mutation models.

[0038]

[0039] Among them, F i For the score of the mutation operator numbered i, |m i |M represents the number of effective mutation models derived from the mutation operator numbered i. i | represents the number of all mutation models derived from the mutation operator numbered i. This represents the proportion of effective mutation models generated by the mutation operator numbered i, where α and β are weighting factors, α > β, and StateCovGain is the state coverage gain.

[0040] For the mutation operator in the conversion coverage gain group, first calculate the conversion coverage gain based on the conversion coverage rate:

[0041]

[0042] The mutation operator is then scored based on the transformation coverage gain and the proportion of effective mutation models.

[0043]

[0044] Where Fj is the score of the mutation operator numbered j, |m j | represents the number of effective mutation models derived from the mutation operator numbered j, |M j | represents the number of all mutation models derived from the mutation operator with ID j. The proportion of effective mutation models generated by the mutation operator numbered j is represented by Y and δ, where Y > δ, and TransCovGain is the transformation coverage gain.

[0045] A test case automatic generation system based on MoMuTUML includes:

[0046] The modeling module is used to build the model under test and obtain the initial global state coverage statistics table and the initial global transformation coverage statistics table based on the model under test.

[0047] The mutation module is used to perform mutation operations on the tested model according to each mutation operator in the mutation operator set, and construct a corresponding mutated model set;

[0048] The test case generation module is used to generate test cases based on the set of variant models and the model under test, delete equivalent variant models in the set of variant models based on the test cases, obtain effective variant models, update the initial global state coverage statistics table and the initial global transformation coverage statistics table based on the effective variant models, and obtain the global state coverage statistics table and the global transformation coverage statistics table.

[0049] The feedback and coverage statistics module is used to score each mutation operator based on the effective mutation model, the global state coverage statistics table, and the global transformation coverage statistics table, and to construct the optimal mutation operator set for the tested model; it is also used to count the number of rounds n in which no new test cases are generated consecutively. If n < N, the optimal mutation operator set is returned to the mutation module to replace the mutation operator set, and the mutation operation is performed again; if n ≥ N, the test case generation process is stopped, and the generated test case set is obtained.

[0050] Furthermore, in the mutation module, mutation operations are performed on the tested model according to each mutation operator, including:

[0051] Three mutation operators are added: a mutation operator that changes the source state of the transformation, a mutation operator that changes the target state of the transformation, and a mutation operator that removes a random transformation of a certain state; four mutation operators are deleted: setting the warning condition to false, setting the warning condition to true, setting the Boolean type OCL subexpression to false, and setting the Boolean type OCL subexpression to true.

[0052] The tested model is mutated according to the modified mutation operator.

[0053] Furthermore, in the mutation module, the modified mutation operators are grouped, with mutation operators that change the source state of the transformation and mutation operators that change the target state of the transformation forming a state coverage gain group, and other mutation operators forming a transformation coverage gain group.

[0054] Beneficial effects:

[0055] (1) A test case automatic generation method based on MoMuTUML generates test cases based on the set of mutation models and the model under test, and deletes equivalent mutation models based on the test cases to obtain effective mutation models, which avoids the problem of too many effective mutation models in MoMuTUML and improves the generation efficiency of test cases; through a feedback-based mutation method, the mutation operators are scored according to the effective mutation models and the coverage of states and transitions, and the optimal mutation operator set is constructed. The mutation operators in the optimal mutation operator set are selected to perform mutation operations on the model under test, which solves the problem of low efficiency of the MoMuTUML mutation method and improves the generation efficiency of test cases.

[0056] (2) The original functional overlap mutation operators were reduced and three new mutation operators with non-overlapping functions were added, which reduced the generation of syntactic equivalent mutation models, more efficiently simulated possible errors in the modeling process, and reduced time costs.

[0057] (3) The modified mutation operators are grouped and scored, and state coverage, transition coverage, and effective mutation models are used as scoring criteria. This allows for dynamic scoring and feedback of each mutation operator based on the results and coverage of the generated test cases. Furthermore, by considering the proportion of effective mutation models generated by each mutation operator and the priority of coverage, the weights in the scoring formula are adjusted accordingly. This allows for more accurate scoring of each mutation operator, selection of the optimal set of mutation operators for the tested model, optimization of the mutation process, and improvement of test case generation efficiency.

[0058] (4) By modifying the original test case generation process, a new feedback and coverage statistics module was added. The scoring results are fed back to the mutation module, and the mutation module adjusts the mutation process according to the feedback to improve the efficiency of test case generation. Attached Figure Description

[0059] Figure 1 This is a basic flowchart of MoMuTUML in the existing technology;

[0060] Figure 2 This is a flowchart of the test case automatic generation method according to an embodiment of the present invention;

[0061] Figure 3 This is a structural block diagram of a test case automatic generation system according to an embodiment of the present invention. Detailed Implementation

[0062] Fault-based testing, also known as mutation testing, originated in 1971, with the work of DeMillo et al. in 1978 considered pioneering research in the field. As a defect-based software analysis technique, mutation testing assesses the defect detection capability of test data by injecting specific types of defects into the program under test. The basic idea is as follows: First, based on certain rules, minor modifications are made to the program under test to generate a new program that still meets the grammatical requirements; this new program is called a variant. Then, given the same test data, the program under test and the variant are run. If their final outputs are the same, it means that the mutation was not detected, and the test case failed to detect the defect; otherwise, it is considered that the test data detected the mutation, indicating that the test case detected the defect.

[0063] MoMuTUML, developed by the Austrian Institute of Technology (ETH Zurich), is a mutation testing tool based on the Unified Modeling Language (UML) model. It's a black-box testing tool that derives test cases from the model of the system under test (SUT) without relying on the SUT's source code. Unlike most other test case generation tools, MoMuTUML focuses on fault-based testing. It applies mutation operators to the model under test to derive a series of mutated models. By identifying behavioral differences between the mutated models and the original model, test cases are generated based on these differences. The generated test cases guarantee the detection of models containing fault types from existing mutation operators. In model-based Mutation Testing for UML (MoMuTUML), these erroneous models are called mutated models; therefore, the test case generation strategy is called "mutation-driven test case generation" or "model-based mutation test case generation."

[0064] Figure 1 This is a basic flowchart of MoMuTUML in existing technology, such as Figure 1As shown, the main steps are as follows: 1. Create and verify the input / output behavior model of the system under test; 2. Apply mutation operators (modify, reverse, etc.) to the model to derive some mutation models. Among them, the supported mutation operators include: (1) change the integer literal in the action, (2) change the integer literal in the Object Constraint Language (OCL) expression, (3) change the time trigger, (4) reverse the transformation expression with the change trigger, (5) reverse the transformation with the warning condition, (6) reverse the Boolean type OCL subexpression, (7) remove the action statement, (8) delete the trigger in the transformation with the trigger, (9) remove the result in the transformation with the result, (10) delete the entry action, (11) delete the exit action, (12) delete the signal trigger, (13) delete the time trigger, (14) replace the transformation result, (15) replace the entry action. (16) Replace the exit action, (17) Replace the enumeration in the OCL expression, (18) Change the OCL operator, (19) Change the signal event, (20) Set the warning condition to false, (21) Set the warning condition to true, (22) Set the Boolean type OCL subexpression to false, (23) Set the Boolean type OCL subexpression to true; 3. Sequentially obtain the original model and a variant model, perform a consistency check, and determine whether there is a difference between the input / output sequence of the variant model and the original model. If there is, generate test cases that can reveal the difference based on the difference in the sequence.

[0065] MoMuTUML uses model-based mutation testing technology to automate test case generation, but it currently has the following problems:

[0066] (1) The functions of the 23 existing mutation operators in MoMuTUML may overlap, which can easily generate syntactically equivalent mutation models. For example, when the mutation operator "6 Invert Boolean type OCL subexpression" and the mutation operators "22 Set Boolean type OCL subexpression to false" and "23 Set Boolean type OCL subexpression to true" are applied to the model under test at the same time, there is functional overlap, which will generate syntactically equivalent mutation models. These mutation models cannot generate valuable test cases and increase the overhead of test case generation.

[0067] (2) MoMuTUML’s current mutation method is inefficient and cannot select the most suitable set of mutation operators for the model under test. During the test case generation process, since MoMuTUML does not select a suitable mutation strategy for the model under test, MoMuTUML applies all mutation operators to the model under test, generating a large number of mutated models. Then, it compares the behavioral differences between the original model and the mutated model one by one to generate test cases. Among these mutated models, there are many invalid mutated models, which will consume a lot of time and result in low test case generation efficiency.

[0068] (3) The overall process design of MoMuTUML is inflexible. The mutation module applies all mutation operators to the model under test to derive a large number of mutated models before generating test cases. It cannot receive feedback based on the test case generation results and cannot adjust the mutation process based on the feedback results, resulting in low test case generation efficiency.

[0069] To address the above shortcomings, the present invention proposes the following improvement method:

[0070] To address the limitation (1), this invention reduces the number of existing mutation operators in MoMuTUML and adds three new mutation operators whose functions do not overlap with the original mutation operators, thereby reducing the generation of syntactically equivalent mutation models and lowering the time cost.

[0071] To address limitation (2), this invention proposes a feedback-based mutation method. Based on the feedback mutation operator scores, the mutation operators with higher scores are selected as the optimal mutation operator set for the tested model. The optimal mutation operator set is then used to perform mutation operations on the tested model, improving the efficiency of test case generation. To score the mutation operators, this invention proposes two new coverage standards: state coverage and transition coverage. Each mutation operator is dynamically scored based on the test case generation results and coverage status, and the scores are fed back into the mutation process.

[0072] To address the limitation (3), this invention modifies the original test case generation process by adding a feedback and coverage statistics module, which feeds back the scoring results to the mutation module. The mutation module adjusts the mutation process based on the feedback, thereby improving the efficiency of test case generation.

[0073] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0074] First, the technical terms used in the embodiments of this invention will be explained:

[0075] Test Case: A test case is a description of a specific software product testing task, reflecting the test plan, methods, techniques, and strategies. Its content includes test objectives, test environment, input data, test steps, expected results, and test scripts. Simply put, a test case is a set of test inputs, execution conditions, and expected results compiled for a specific goal to verify whether a particular software requirement is met.

[0076] MoMuTUML: MoMuTUML is an automated model-based test case generation tool, belonging to the black-box testing tool category. It derives test cases from the model of the system under test (SUT) and does not depend on the SUT's source code. Unlike most other test case generation tools, MoMuTUML focuses on model-based mutation testing, meaning the generated test cases guarantee the ability to detect models containing fault types with existing mutation operators.

[0077] UML: UML is a visual modeling language primarily used for writing software system specifications, system construction, and documentation. It is a third-generation modeling and specification language. UML is an object-oriented design modeling tool, independent of any specific programming language.

[0078] Model-based mutation testing: Mutation testing is a method for evaluating software quality by injecting errors into code and seeing if tests detect them. With the advancement of large-scale computing power, mutation testing has seen new developments, including mutation analysis for object-oriented programming languages, non-procedural languages ​​(such as XML), finite state machines, and other emerging software testing methods. Model-based mutation testing involves applying specific mutation operators to a UML model to generate a series of model "variants," and then comparing the behavioral differences between the original model and the "variants" to perform software testing.

[0079] Mutation operators: In the process of model-based mutation testing, a series of syntactic minor modifications need to be made to the original model to simulate errors that may occur during the modeling process. These syntactic minor modifications to the model are called mutation operators.

[0080] Mutation model: A series of erroneous models generated by applying the mutation operator to the original model are called mutation models.

[0081] State diagrams describe all possible states of an object of a class, and the conditions under which states transition when events occur. They can capture the lifecycle of objects, subsystems, and the system itself. They show the states an object can have and how events (such as receiving messages, the passage of time, errors, conditions becoming true, etc.) affect these states over time. A state diagram should connect all classes with clearly identifiable states and complex behaviors; the diagram can determine the behavior of a class and how that behavior changes according to the current state, and it can also show which events will change the state of objects of the class.

[0082] This invention provides a method for automatically generating test cases based on MoMuTUML. Figure 2 This is a flowchart of the test case automatic generation method according to an embodiment of the present invention, such as... Figure 2 As shown, it includes the following steps:

[0083] Step 1: Establish the model under test, and obtain the initial global state coverage statistics table and the initial global transformation coverage statistics table based on the model under test;

[0084] In one specific embodiment, obtaining an initial global state coverage statistics table and an initial global transformation coverage statistics table based on the model under test includes: performing statistical analysis on the model under test according to predicate logic, and establishing an initial global state coverage statistics table and an initial global transformation coverage statistics table based on the statistical analysis results.

[0085] Step 2: Perform mutation operations on the tested model according to each mutation operator in the mutation operator set to construct the corresponding mutated model set;

[0086] In one specific embodiment, the test model is mutated according to each mutation operator in the mutation operator set, including:

[0087] Three mutation operators are added: a mutation operator that changes the source state of the transformation, a mutation operator that changes the target state of the transformation, and a mutation operator that removes a random transformation of a certain state. Four mutation operators are deleted: a mutation operator that sets a warning condition to false, a mutation operator that sets a warning condition to true, a mutation operator that sets a Boolean type OCL subexpression to false, and a mutation operator that sets a Boolean type OCL subexpression to true. At the same time, the modified mutation operators are grouped, with the mutation operators that change the source state of the transformation and the mutation operators that change the target state of the transformation forming a state coverage gain group, and the other mutation operators forming a transformation coverage gain group.

[0088] The tested model is mutated using the modified mutation operator.

[0089] Step 3: Generate test cases based on the set of variant models and the model under test, delete equivalent variant models in the set of variant models based on the test cases, obtain effective variant models, update the initial global state coverage statistics table and the initial global transformation coverage statistics table based on the effective variant models, and obtain the global state coverage statistics table and the global transformation coverage statistics table.

[0090] In one specific embodiment, step three above specifically includes:

[0091] Select a variant model from the set of variant models and perform a consistency check with the model under test. Search for behavioral differences between the variant model and the model under test. If differences exist, generate test cases based on the behavioral differences, designate the variant model as a valid variant model, and mark the mutation operator corresponding to the valid variant model. If no differences exist, continue to select a variant model from the set of variant models until a variant model with behavioral differences from the model under test is selected or the set of variant models has been traversed.

[0092] The states and transitions of the tested model covered by the effective mutation model are marked and updated in the initial global state coverage statistics table and the initial global transition coverage statistics table to obtain the global state coverage statistics table and the global transition coverage statistics table.

[0093] Step 4: Based on the effective mutation model, the global state coverage statistics table, and the global transformation coverage statistics table, score each mutation operator to construct the optimal mutation operator set for the tested model.

[0094] In one specific embodiment, step four above specifically includes:

[0095] Calculate the state coverage and transition coverage of the generated test cases for the model under test based on the global state coverage statistics table and the global transition coverage statistics table:

[0096] State coverage:

[0097]

[0098] Where |S′| is the number of covered states, and |S| is the total number of states of the tested model;

[0099] Conversion coverage:

[0100]

[0101] Where |T′| is the number of transformations that have been covered, and |T| is the total number of transformations of the tested model;

[0102] The mutation operators are scored based on the state coverage, the transition coverage, and the proportion of effective mutation models. The top b mutation operators (b < count(i), where i refers to the mutation operator) are selected to construct the optimal mutation operator set for the tested model.

[0103] In one specific embodiment, the mutation operator is scored based on the state coverage, transition coverage, and the proportion of effective mutation models, including:

[0104] For the mutation operator in the state coverage gain group, first calculate the state coverage gain based on the state coverage rate:

[0105]

[0106] The mutation operator is then scored based on the state coverage gain and the proportion of effective mutation models.

[0107]

[0108] Among them, F i For the score of the mutation operator numbered i, |m i |M represents the number of effective mutation models derived from the mutation operator numbered i. i | represents the number of all mutation models derived from the mutation operator numbered i. This represents the proportion of effective mutated models generated by the mutation operator numbered i. α and β are weighting factors, with α > β. StateCovGain is the state coverage gain. In actual implementation, considering that generating more effective mutated models has a greater impact than increasing coverage in the overall test case generation process, α > β is chosen.

[0109] For the mutation operator in the conversion coverage gain group, first calculate the conversion coverage gain based on the conversion coverage rate:

[0110]

[0111] The mutation operator is then scored based on the transformation coverage gain and the proportion of effective mutation models.

[0112]

[0113] Where Fj is the score of the mutation operator numbered j, |m j | represents the number of valid mutation models derived from the mutation operator with number j, and |Mj| represents the total number of mutation models derived from the mutation operator with number j. γ represents the proportion of effective mutated models generated by the mutation operator with number j, Y and δ are weighting factors, Y > δ, and TransCovGain is the transformation coverage gain. In actual implementation, considering that generating more effective mutated models has a greater impact than increasing coverage in the overall test case generation process, γ > δ is chosen.

[0114] Step 5: Count the number of rounds in which no new test cases are generated consecutively, n. If n < N, replace the mutation operator set in Step 2 with the optimal mutation operator set in Step 4, and repeat Step 2 to Step 4. If n ≥ N, stop the test case generation process and obtain the generated test case set.

[0115] N is the threshold for stopping the test case generation process. If no new test cases are generated in N consecutive rounds of generation, the mutation process is considered to be unable to generate new test cases and the generation process can be terminated. N is a positive integer.

[0116] This invention also provides an automatic test case generation system based on MoMuTUML. Figure 3 This is a structural block diagram of the test case automatic generation system constructed according to an embodiment of the present invention, such as... Figure 3 As shown, it includes:

[0117] The modeling module is used to build the model under test and obtain the initial global state coverage statistics table and the initial global transformation coverage statistics table based on the model under test.

[0118] In actual implementation, the modeling module performs analysis and statistics on various elements of the model under test, which facilitates subsequent statistics such as coverage.

[0119] In actual implementation, the workflow of the modeling module is as follows:

[0120] Step (1): Using the Papyrus plugin tool for UML models, based on OCL constraints and the Object Management Group (OMG) standard, obtain the UML model file in XML format.

[0121] Step (II): Analyze the UML model file under test according to predicate logic, and represent it in the following form:

[0122] SUTMode1 = {S, P} entry P exit A, B, T

[0123] Where S is a finite set of states; P entry To enter the node; P exitA is the exit node; A is a finite set of input actions; B is a finite set of output actions; T is a finite set of transitions of the form (q, p, c, g, e, r), where q represents the original source state, p represents the target state of the transition, c represents the action accompanying the transition, g represents the warning condition of the transition, e represents the result after the transition, and r represents the transition trigger.

[0124] Step (3): Establish an initial global state coverage statistics table to record whether all states s∈S are covered.

[0125] Step (4): Establish an initial global transformation coverage statistics table to record whether all transformations t∈T are covered.

[0126] Steps (ii), (iii), and (iv) in the modeling module are all new steps added based on the original MoMuTUML tool. These steps collect information such as the state and transformation of the tested system model and establish a coverage statistics table to support the subsequent module work.

[0127] The mutation module is used to perform mutation operations on the tested model according to each mutation operator in the mutation operator set, and construct the corresponding mutated model set.

[0128] In one specific embodiment, in the mutation module, the tested model is mutated according to each mutation operator in the mutation operator set, including: adding three mutation operators: a mutation operator that changes the source state of the transformation, a mutation operator that changes the target state of the transformation, and a mutation operator that removes a random transformation of a certain state; deleting four mutation operators: setting the warning condition to false, setting the warning condition to true, setting the Boolean type OCL subexpression to false, and setting the Boolean type OCL subexpression to true; and performing mutation operations on the tested model according to the modified mutation operators.

[0129] In one specific embodiment, in the mutation module, the modified mutation operators are grouped into a state coverage gain group, with the mutation operators that change the source state of the transformation and the mutation operators that change the target state of the transformation being grouped into a state coverage gain group, and the other mutation operators being grouped into a transformation coverage gain group.

[0130] In actual implementation, the mutation module is used to receive feedback and guidance from the coverage statistics module. It applies the mutation operators in the optimal mutation operator set constructed for the model under test to the model under test, generating a series of mutated models.

[0131] In actual implementation, the workflow of the mutation module includes the following steps:

[0132] Step (1): In the first round of mutation, the modified mutation operator is applied to the original model. Each mutation operator i derives a mutated models (the value of a can be adjusted according to the complexity of the model being tested; the more complex the model, the larger the value of a) to form a set M of mutated models for mutation operator i. i ={m i1 m i2 , ..., m ia The set of variant models is sent to the generation module to generate test cases.

[0133] The present invention modifies the mutation operator as follows:

[0134] (1) Add a mutation operator to change the source state of the transformation, number 24. The source state can be replaced with any other state except the original source state and the target state.

[0135] (2) Add a mutation operator to change the target state of the transformation, number 25. The target state can be replaced with any other state except the original target state and the source state.

[0136] (3) Add a mutation operator to remove random transitions of a certain state, numbered 26, which randomly selects any transition of a certain state.

[0137] (4) Delete the mutation operators “20 Set warning condition to false”, “21 Set warning condition to true”, “22 Set Boolean type OCL subexpression to false”, and “23 Set Boolean type OCL subexpression to true”.

[0138] (5) The mutation operators are divided into two categories: mutation operators numbered 24 and 25 are state coverage gain groups, and the remaining mutation operators are transformation coverage gain groups.

[0139] Step (2): In each round of mutation after the first round of mutation is completed, the mutation operators in the optimal mutation operator set are selected to perform mutation operations on the model under test. Each mutation operator still derives a mutated models, forming the mutation model set for that round. The mutation model set is then sent to the generation module to generate test cases.

[0140] The test case generation module is used to generate test cases based on the set of variant models and the model under test, delete equivalent variant models in the set of variant models based on the test cases, obtain effective variant models, update the initial global state coverage statistics table and the initial global transformation coverage statistics table based on the effective variant models, and obtain the global state coverage statistics table and the global transformation coverage statistics table.

[0141] In actual implementation, the test case generation module generates test cases by comparing the behavioral differences between the original model and the mutated model, and applies the generated test cases to the remaining mutated models in the mutated model set to filter the mutated models.

[0142] In actual implementation, the process in the test case generation module includes the following steps:

[0143] Step (1): In each round of generation, receive a set of mutation models from the mutation module.

[0144] Step (II): Select a variant model from a set of variant models, perform a consistency check against the original model, and search for behavioral differences between the original model and the variant model. Generate test cases based on these behavioral differences. If a test case is successfully generated, the variant model that generated it is considered a valid variant model, and the number of valid variant models generated by the corresponding mutation operator is incremented by 1. Mark the states and transitions of the tested model covered by the valid variant model corresponding to the test case in the global state coverage statistics table and the global transition coverage statistics table, and update the global state coverage statistics table and the global transition coverage statistics table.

[0145] Step (3): Apply the generated test cases to the remaining variant models in the variant model set. Variant models that can be detected by the test cases as having behavioral differences from the original model will be filtered out, reducing the number of equivalent variant models.

[0146] Step (IV): Repeat steps (I), (II), and (III) until the set of mutated models is empty. An empty set of mutated models means that each mutated model in the set of mutated models for each mutation operator has been tested once.

[0147] The feedback and coverage statistics module is used to score each mutation operator based on the effective mutation model, the global state coverage statistics table, and the global transformation coverage statistics table, and to construct the optimal mutation operator set for the tested model; it is also used to count the number of rounds n in which no new test cases are generated consecutively. If n < N, the optimal mutation operator set is returned to the mutation module to replace the mutation operator set, and the mutation operation is performed again; if n ≥ N, the test case generation process is stopped, and the generated test case set is obtained.

[0148] In one specific embodiment, in the feedback and coverage statistics module, each mutation operator is scored according to the effective mutation model and the global state coverage statistics table and global transformation coverage statistics table, including:

[0149] For mutation operators in the state coverage gain group, first calculate the state coverage gain based on the state coverage rate, and then score the mutation operator based on the state coverage gain and the proportion of effective mutation models:

[0150]

[0151] Among them, F i For the score of the mutation operator numbered i, |m i |M represents the number of effective mutation models derived from the mutation operator numbered i. i | represents the number of all mutation models derived from the mutation operator numbered i. This represents the proportion of effective mutation models generated by the mutation operator numbered i, where α and β are weighting factors, and StateCovGain is the state coverage gain.

[0152] For the mutation operators in the transformation coverage gain group, first calculate the transformation coverage gain based on the transformation coverage rate, and then score the mutation operators based on the transformation coverage gain and the proportion of effective mutation models:

[0153]

[0154] Where Fj is the score of the mutation operator numbered j, |m j | represents the number of effective mutation models derived from the mutation operator numbered j, |M j | represents the number of all mutation models derived from the mutation operator with ID j. This represents the proportion of effective mutation models generated by the mutation operator numbered j, where Y and δ are weighting factors, and TransCovGain is the transformation coverage gain.

[0155] In actual implementation, the feedback and coverage statistics module counts the number of effective mutated models and coverage information generated by each mutation operator, scores each mutation operator according to the formula, constructs the optimal mutation operator set for the tested model, and calculates the number of rounds n in which no new test cases are generated. If n < N, the optimal mutation operator set is sent to the mutation module to continue the test case generation process; otherwise, the generation process is stopped, and the generated test case set is obtained. Here, n and N are both positive integers.

[0156] In actual implementation, the process in the feedback and coverage statistics module includes the following steps:

[0157] Since there was no information related to test case generation during the first round of test case generation, this module will start executing from the second round of test case generation.

[0158] Step (1): Apply the test cases generated by the generation module to the original model, and calculate the state coverage and transformation coverage of the existing test case set to the model through the global state coverage statistics table and the global transformation coverage statistics table.

[0159] State coverage calculation formula:

[0160]

[0161] Where |S′| is the number of states that have been covered, and |S| is the total number of states of the model under test.

[0162] Conversion coverage calculation formula:

[0163]

[0164] Where |T′| represents the number of transformations that have been covered, and |T| represents the total number of transformations of the model under test.

[0165] Step (2): Score each mutation operator.

[0166] For mutation operators in the state coverage gain group, the following formula is used to score the mutation operators:

[0167]

[0168] Among them, F i For the score of the mutation operator numbered i, |m i |M represents the number of effective mutation models derived from the mutation operator numbered i. i | represents the number of all mutated models derived from the mutation operator numbered i. α and β are weighting factors; considering that generating more effective mutated models has a greater impact than increasing coverage in the overall test case generation process, α > β. StateCovGain is the state coverage gain; considering that the state coverage gain should decrease as coverage increases, its calculation formula is:

[0169]

[0170] For mutation operators in the transformation coverage gain group, the mutation operator is scored using the following formula:

[0171]

[0172] Where Fj is the score of the mutation operator numbered j, |m j | represents the number of effective mutation models derived from the mutation operator numbered j, |M j|The number of all mutation models derived for the mutation operator numbered j. Y and δ are weight factors. Considering that in the overall test case generation process, the impact of generating more effective mutation models is greater than the impact of increasing coverage, so Y > δ. TransCovGain is the transformation coverage gain. Considering that the transformation coverage gain should decrease as the coverage rate increases, its calculation formula is:

[0173]

[0174] Step (3): Select the top b mutation operators (b < count(i), where i refers to the mutation operator. In this embodiment, count(i) = 22, so b < 22) with the highest scores as the optimal mutation operator set for the model under test, and this set is dynamically updated according to the scoring situation of each round of mutation operators.

[0175] Step (4): Calculate the number of consecutive rounds n in which no new test cases are generated. If n < N, send the optimal mutation operator set to the mutation module to continue the test case generation process; otherwise, stop the generation process and obtain the set of generated test cases, where N is a positive integer.

[0176] In summary, in view of the problems of the existing MoMuTUML tool flow being inflexible, mutation operator functions overlapping, and mutation methods being inefficient, the present invention provides a method and system for automatically generating test cases based on MoMuTUML. The present invention improves the mutation operator set, proposes a feedback-based mutation method, designs a mutation operator scoring mechanism, proposes state and transformation coverage measurement criteria, and modifies the execution process of MoMuTUML by adding a feedback and coverage statistics module.

[0177] The present invention mainly focuses on the processing of derived mutation models in MoMuTUML and the improvement of the overall process, and proposes: (1) new mutation operators, mutation methods, and coverage criteria, and redesigns the test case generation process. Deleting the original mutation operators and adding new mutation operators can effectively reduce the number of syntactically equivalent mutation models, more efficiently simulate the errors that may occur in the modeling process, and improve the test case generation efficiency; (2) a feedback mechanism-based mutation method can select the optimal mutation operator set suitable for the model under test, quickly locate the positions in the model where defects are more likely to occur, and improve the test case generation efficiency; (3) the reconstruction of the test case generation process changes the design of the original process that cannot flexibly adjust the mutation process according to the feedback of the test case generation results, making the test case generation more efficient.

[0178] Although similar technical solutions propose a method for generating variant test cases based on model detection technology, their proposed mutation operators, coverage criteria, etc., are significantly different from those of this invention. Furthermore, they do not design a feedback strategy, and the model of the system under test they target is also different from that of this invention. The mutation operators, mutation methods, coverage criteria, and test case generation process proposed in this invention are all different from other methods.

[0179] This invention reduces the number of existing mutation operators in MoMuTUML and adds three new mutation operators whose functions do not overlap with the original mutation operators, thus reducing the number of syntactically equivalent mutation models and improving test case generation efficiency. The proposed feedback-based mutation method selects the highest-scoring mutation operators as the optimal mutation operator set for the tested model based on the feedback mutation operator scores, prioritizing the selection of the optimal mutation operator set for mutation operations, thereby improving test case generation efficiency. For scoring mutation operators, this invention proposes two new coverage standards: state coverage and transition coverage. Each mutation operator is dynamically scored and feedback is provided based on the test case generation results and coverage status. This invention modifies the original test case generation process, adding a feedback and coverage statistics module, and integrating improvements to mutation operators and mutation methods. This allows for flexible adjustments to the mutation process based on the feedback of test case generation results, further improving test case generation efficiency.

[0180] The key points of this invention are: (1) adding a feedback and coverage statistics module to the MoMuTUML process; (2) modifying the mutation operator in MoMuTUML; (3) designing a mutation operator scoring method in MoMuTUML; (4) designing a feedback-based mutation method in MoMuTUML; and (5) proposing state coverage and transition coverage in MoMuTUML.

[0181] The specific embodiments described above only illustrate the design principles of the present invention. The shapes and names of the components in this description may differ and are not limited. Therefore, those skilled in the art can modify or make equivalent substitutions to the technical solutions described in the foregoing embodiments; and these modifications and substitutions do not depart from the inventive spirit and technical solutions of the present invention, and should all fall within the protection scope of the present invention.

Claims

1. A method for automatically generating test cases based on MoMuTUML, characterized in that, Including: Step 1: Establish a model under test, and obtain an initial global state coverage statistical table and an initial global transition coverage statistical table according to the model under test; Step 2: Perform mutation operations on the model under test according to each mutation operator in the mutation operator set, and construct a corresponding set of mutant models; Add three mutation operators, namely: a mutation operator for changing the source state of a transition, a mutation operator for changing the target state of a transition, and a mutation operator for removing a random transition of a state; Delete four mutation operators, namely: set the guard condition to false, set the guard condition to true, set the boolean type OCL sub-expression to false, and set the boolean type OCL sub-expression to true; Perform mutation operations on the model under test according to the modified mutation operators; Group the modified mutation operators, take the mutation operator for changing the source state of a transition and the mutation operator for changing the target state of a transition as the state coverage gain group, and take other mutation operators as the transition coverage gain group; Step 3: Generate test cases according to the set of mutant models and the model under test, delete equivalent mutant models in the set of mutant models according to the test cases, obtain effective mutant models, update the initial global state coverage statistical table and the initial global transition coverage statistical table according to the effective mutant models, and obtain a global state coverage statistical table and a global transition coverage statistical table; including: Select a mutant model from the set of mutant models, perform a consistency check with the model under test, search for the behavioral differences between the mutant model and the model under test. If there are differences, generate test cases according to the behavioral differences, take the mutant model as an effective mutant model, and mark the mutation operator corresponding to the effective mutant model; If there are no differences, continue to select mutant models from the set of mutant models until a mutant model with behavioral differences from the model under test is selected or the set of mutant models is traversed; Mark and update the states and transitions of the model under test covered by the effective mutant models in the initial global state coverage statistical table and the initial global transition coverage statistical table to obtain a global state coverage statistical table and a global transition coverage statistical table; Step 4: Score each mutation operator according to the global state coverage statistical table, the global transition coverage statistical table and the effective mutant models, and construct an optimal set of mutation operators for the model under test; Step 5: Count the number of rounds n of continuously generating no new test cases. If n < N, replace the set of mutation operators in Step 2 with the optimal set of mutation operators in Step 4, and repeat Steps 2 to 4; if n ≥ N, stop the test case generation process and obtain the generated set of test cases; both n and N are positive integers.

2. The method as described in claim 1, characterized in that, In Step 1, obtaining the initial global state coverage statistical table and the initial global transition coverage statistical table according to the model under test includes: Perform statistical analysis on the model under test according to predicate logic, and establish an initial global state coverage statistical table and an initial global transition coverage statistical table according to the statistical analysis results.

3. The method as described in claim 1, characterized in that, In step four, based on the global state coverage statistics table, the global transformation coverage statistics table, and the score given to each mutation operator by the effective mutation model, the optimal mutation operator set for the tested model is constructed, including: Calculate the state coverage and transition coverage of the model under test in step three based on the global state coverage statistics table and the global transition coverage statistics table: State coverage: Where |S′| is the number of covered states, and |S| is the total number of states of the tested model; Conversion coverage: Where |T′| is the number of transformations that have been covered, and |T| is the total number of transformations of the tested model; The mutation operators are scored based on the state coverage, the transition coverage, and the proportion of effective mutation models. The top b mutation operators (b < count(i), where i refers to the mutation operator) are selected to construct the optimal mutation operator set for the tested model.

4. The method as described in claim 3, characterized in that, The scoring of the mutation operator based on the state coverage, the transition coverage, and the proportion of effective mutation models includes: For the mutation operator in the state coverage gain group, first calculate the state coverage gain based on the state coverage rate: The mutation operator is then scored based on the state coverage gain and the proportion of effective mutation models. Among them, F i For the score of the mutation operator numbered i, |m i |M represents the number of effective mutation models derived from the mutation operator numbered i. i | represents the number of all mutation models derived from the mutation operator numbered i. This represents the proportion of effective mutation models generated by the mutation operator numbered i, where α and β are weighting factors, α > β, and StateCovGain is the state coverage gain. For the mutation operator in the conversion coverage gain group, first calculate the conversion coverage gain based on the conversion coverage rate: The mutation operator is then scored based on the transformation coverage gain and the proportion of effective mutation models. Among them, F j For the score of the mutation operator numbered j, |m j | represents the number of effective mutation models derived from the mutation operator numbered j, |M j | represents the number of all mutation models derived from the mutation operator with ID j. denoted by j, representing the proportion of effective mutation models generated by the mutation operator with number j, where γ and δ are weighting factors, γ > δ, and TransCovGain is the transformation coverage gain.

5. A test case automatic generation system based on MoMuTUML, characterized in that, include: The modeling module is used to build the model under test and obtain the initial global state coverage statistics table and the initial global transformation coverage statistics table based on the model under test. The mutation module is used to perform mutation operations on the tested model according to each mutation operator in the mutation operator set, and construct a corresponding mutated model set. Three mutation operators are added: a mutation operator that changes the source state of the transformation, a mutation operator that changes the target state of the transformation, and a mutation operator that removes a random transformation of a certain state. Four mutation operators are deleted: setting the warning condition to false, setting the warning condition to true, setting the Boolean type OCL subexpression to false, and setting the Boolean type OCL subexpression to true. The tested model is mutated according to the modified mutation operator; The modified mutation operators are grouped into a state coverage gain group, with mutation operators that change the source state of the transformation and mutation operators that change the target state of the transformation being grouped into a state coverage gain group, and other mutation operators being grouped into a transformation coverage gain group. The test case generation module is used to generate test cases based on the set of variant models and the model under test, delete equivalent variant models from the set of variant models based on the test cases, obtain valid variant models, and update the initial global state coverage statistics table and the initial global transformation coverage statistics table based on the valid variant models to obtain a global state coverage statistics table and a global transformation coverage statistics table; including: Select a variant model from the set of variant models and perform a consistency check with the model under test. Search for behavioral differences between the variant model and the model under test. If differences exist, generate test cases based on the behavioral differences, designate the variant model as a valid variant model, and mark the mutation operator corresponding to the valid variant model. If no differences exist, continue to select a variant model from the set of variant models until a variant model with behavioral differences from the model under test is selected or the set of variant models has been traversed. The states and transitions of the tested models covered by the effective mutation model are marked and updated in the initial global state coverage statistics table and the initial global transition coverage statistics table to obtain the global state coverage statistics table and the global transition coverage statistics table. The feedback and coverage statistics module is used to score each mutation operator based on the effective mutation model, the global state coverage statistics table, and the global transformation coverage statistics table, and to construct the optimal mutation operator set for the tested model; it is also used to count the number of rounds n in which no new test cases are generated consecutively. If n < N, the optimal mutation operator set is returned to the mutation module to replace the mutation operator set, and the mutation operation is performed again; if n ≥ N, the test case generation process is stopped, and the generated test case set is obtained.

Images