Test assertion evaluation method and device based on metamorphic relation, equipment and medium
By defining two types of metamorphosis relationships and constructing multiple metamorphosis transformation templates, the Java test code is transformed equivalently, which solves the problem of low diversity in test assertion generation and improves the efficiency and applicability of automated testing.
Patent Information
- Application Number
- CN202510640971.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-19
- Publication Date
- 2025-10-17
Smart Images
Figure CN120803904A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of computer, in particular to a test assertion evaluation method and device based on metamorphic relationship, equipment and medium. BACKGROUND
[0002] With the continuous expansion of software development scale and the increase of complexity, the demand for code quality assurance and automation tools is increasing. In the software development life cycle, especially in the testing stage, code behavior prediction and problem positioning are important links to ensure software quality.
[0003] In related technologies, widely used test automation tools can generate test cases based on regression testing techniques, and deep learning methods are introduced into the field of software testing to improve the readability of test cases and the quality of test assertions.
[0004] However, the related technology cannot be applied to complex development scenarios, and the assertion generation diversity is low and the applicability is poor, resulting in low efficiency of automated testing, which needs to be solved urgently. SUMMARY
[0005] The present application provides a test assertion evaluation method and device based on metamorphic relationship, equipment and medium, to solve the problem of low efficiency of automated testing caused by low diversity and poor applicability of assertion generation in related technologies, improve the diversity of assertion generation, and improve the efficiency of automated testing.
[0006] The first aspect of the present application provides a test assertion evaluation method based on metamorphic relationship, comprising the following steps: determining a first metamorphic relationship and a second metamorphic relationship that are semantically equivalent to a test program; based on the first metamorphic relationship and the second metamorphic relationship, constructing a plurality of metamorphic transformation templates, and generating a plurality of code variants corresponding to each metamorphic transformation template; based on a preset assertion generation model, testing each code variant to obtain a test assertion evaluation result, and generating a test assertion evaluation report according to the test assertion evaluation result.
[0007] Further, in some embodiments, based on the first metamorphic relationship and the second metamorphic relationship, a plurality of metamorphic transformation templates are constructed, including: based on the first metamorphic relationship, a plurality of first descriptions are determined, and each first description is added to the test program to obtain a metamorphic transformation template corresponding to each first description; based on the first metamorphic relationship, a plurality of second descriptions are determined, and each second description is added to the test program to obtain a metamorphic transformation template corresponding to each second description; and according to the metamorphic transformation template corresponding to each first description and the metamorphic transformation template corresponding to each second description, a plurality of metamorphic transformation templates are obtained.
[0008] Further, in some embodiments, the testing each code variant based on the preset assertion generation model to obtain a test assertion evaluation result comprises: inputting the each code variant into the preset assertion generation model to obtain a plurality of test assertions, verifying the accuracy of each test assertion, and obtaining the executable result of the each code variant according to the verification result; based on the executable result of the each code variant, obtaining a plurality of test assertions, comparing the generation quality of the plurality of test assertions based on a preset experimental comparison strategy, and obtaining the test assertion evaluation result according to the generation quality of the plurality of test assertions.
[0009] Further, in some embodiments, after generating the plurality of code variants corresponding to the each metamorphic transformation template, the method further comprises: performing a compilation check on each code variant, and filtering out a code variant that does not meet a preset compilation result according to a compilation result, and discarding the code variant that does not meet the preset compilation result.
[0010] According to the test assertion evaluation method based on metamorphic relationship provided by the embodiment of the application, by defining two types of metamorphic relationships, a plurality of metamorphic transformation templates are constructed, Java test code is equivalently transformed, and an assertion generation model is induced to learn diversified test patterns, and each code variant is tested and evaluated by the assertion generation model to obtain a test assertion evaluation result, thereby solving the problem of low efficiency of automatic testing caused by low diversity and poor applicability of assertion generation in related technologies, improving the diversity of assertion generation, and improving the efficiency of automatic testing.
[0011] The second aspect of the embodiment of the application provides a test assertion evaluation device based on metamorphic relationship, the device comprises: an acquisition module configured to determine a first metamorphic relationship and a second metamorphic relationship that are semantically equivalent to a test program; a generation module configured to construct a plurality of metamorphic transformation templates based on the first metamorphic relationship and the second metamorphic relationship, and generate a plurality of code variants corresponding to each metamorphic transformation template; and an evaluation module configured to test each code variant based on a preset assertion generation model to obtain a test assertion evaluation result, and generate a test assertion evaluation report according to the test assertion evaluation result.
[0012] Further, in some embodiments, the generation module is specifically configured to: determine a plurality of first descriptions based on the first metamorphic relationship, and add each first description to the test program to obtain a metamorphic transformation template corresponding to the each first description; determine a plurality of second descriptions based on the first metamorphic relationship, and add each second description to the test program to obtain a metamorphic transformation template corresponding to the each second description; and obtain a plurality of metamorphic transformation templates according to the metamorphic transformation template corresponding to the each first description and the metamorphic transformation template corresponding to the each second description.
[0013] Further, in some embodiments, the evaluation module is specifically configured to: input the each code variant into the preset assertion generation model to obtain a plurality of test assertions, verify the accuracy of each test assertion, and obtain the executability result of the each code variant according to the verification result; based on the executability result of the each code variant, obtain a plurality of test assertions, compare the generation quality of the plurality of test assertions based on a preset experimental comparison strategy, and obtain the test assertion evaluation result according to the generation quality of the plurality of test assertions.
[0014] Further, in some embodiments, after generating the plurality of code variants corresponding to the each metamorphic transformation template, the generation module is further configured to: perform a compilation check on each code variant, filter out a code variant that does not meet a preset compilation result according to a compilation result, and discard the code variant that does not meet the preset compilation result.
[0015] According to the test assertion evaluation device based on metamorphic relationship provided by the embodiment of the application, by defining two types of metamorphic relationships, a plurality of metamorphic transformation templates are constructed, Java test codes are equivalently transformed, and an assertion generation model is induced to learn diversified test patterns. Each code variant is tested and evaluated by the assertion generation model to obtain a test assertion evaluation result, thereby solving the problem of low efficiency of automatic testing caused by low diversity and poor applicability of assertion generation in related technologies, improving the diversity of assertion generation, and improving the efficiency of automatic testing.
[0016] The third aspect of the embodiment of the application provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the test assertion evaluation method based on metamorphic relationship as described in the above embodiments.
[0017] The fourth aspect of the embodiment of the application provides a computer readable storage medium, which stores a computer program executable by a processor to implement the test assertion evaluation method based on metamorphic relationship as described in the above embodiments.
[0018] The fifth aspect of the embodiment of the application provides a computer program product comprising a computer program executable to implement the test assertion evaluation method based on metamorphic relationship as described in the above embodiments.
[0019] Therefore, the application has the following beneficial effects:
[0020] (1) High expansibility: modular design and integrated interface make the tool easy to expand and integrate with other assertion generation model systems;
[0021] (2) Promote the diversity of test assertions: induce the model to generate different forms of assertions through code conversion, improve the coverage and metamorphic score of the test;
[0022] (3) Optimize the generalization ability of test assertions: avoid relying on the assumption of a single dataset (such as ATLAS), enable the model to adapt to more complex test scenarios, and improve applicability in different projects;
[0023] (4) Enhance the effectiveness of automated testing: improve the practical value of automated testing by generating more reasonable assertions to help developers more efficiently find potential defects. BRIEF DESCRIPTION OF DRAWINGS
[0024] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description, taken in conjunction with the following drawings, in which:
[0025] Figure 1 A flowchart of a test assertion evaluation method based on a metamorphic relationship according to an embodiment of the present application is provided.
[0026] Figure 2 A flowchart of a test assertion evaluation method based on a metamorphic relationship according to one specific embodiment of the present application is provided.
[0027] Figure 3 A block diagram of a test assertion evaluation device based on a metamorphic relationship according to an embodiment of the present application is provided.
[0028] Figure 4 A structural schematic diagram of an electronic device according to an embodiment of the present application is provided. DETAILED DESCRIPTION
[0029] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.
[0030] Embodiments of a metamorphic relation based test assertion evaluation method, device, equipment and medium are described below with reference to the accompanying drawings. To solve the problem of low efficiency of automated testing caused by low diversity and poor applicability of assertion generation in the background art, the present application provides a metamorphic relation based test assertion evaluation method. By defining two types of metamorphic relations, a plurality of metamorphic transformation templates are constructed to perform equivalent transformation on Java test code, thereby inducing the assertion generation model to learn diversified test patterns, and performing test evaluation on each code variant through the assertion generation model to obtain test assertion evaluation results, thereby solving the problem of low efficiency of automated testing caused by low diversity and poor applicability of assertion generation in related art, improving the diversity of assertion generation, and improving the efficiency of automated testing.
[0031] Specifically, Figure 1 A flowchart of the metamorphic relation based test assertion evaluation method according to embodiments of the present application is shown in FIG. 1.
[0032] As Figure 1 shown, the metamorphic relation based test assertion evaluation method includes the following steps:
[0033] In step S101, a first metamorphic relation and a second metamorphic relation that are semantically equivalent to a test program are determined.
[0034] The metamorphic relation refers to a relationship in which two program fragments or expressions are semantically equivalent in program analysis and verification, and is represented as a set of multiple structurally different programs but semantically equivalent programs.
[0035] Specifically, given a program P, different but equivalent programs P1 ′ ,…P n ′ are generated through metamorphic relations (MRs). ′ ,…P n ′ constitute the metamorphic relation, wherein the first metamorphic relation is represented as MR-1, i.e. a program variant that is different in syntax but equivalent in semantics can be constructed by adding non-informative code elements (such as code comments, unused variables); the second metamorphic relation is represented as MR-2, i.e. a program variant that is different in syntax tree structure but equivalent in program behavior is constructed by replacing the code elements of the original program with another equivalent code element (such as different variable names, code expansion).
[0036] In step S102, based on the first metamorphic relation and the second metamorphic relation, a plurality of metamorphic transformation templates are constructed, and a plurality of code variants corresponding to each metamorphic transformation template are generated.
[0037] The metamorphic transformation template is a pattern or rule for generating a program variant that is semantically equivalent but syntactically different, and it should be noted that the metamorphic transformation template can have multiple patterns, and through multiple metamorphic transformation templates, a variant semantically equivalent to the test prefix and the code in the method under test can be generated.
[0038] Specifically, in some embodiments, based on the first metamorphic relationship and the second metamorphic relationship, a plurality of metamorphic transformation templates are constructed, including: determining a plurality of first descriptions based on the first metamorphic relationship, and adding each first description to the test program respectively to obtain a metamorphic transformation template corresponding to each first description; determining a plurality of second descriptions based on the first metamorphic relationship, and adding each second description to the test program respectively to obtain a metamorphic transformation template corresponding to each second description; and obtaining a plurality of metamorphic transformation templates according to the metamorphic transformation template corresponding to each first description and the metamorphic transformation template corresponding to each second description.
[0039] The first description is to realize a plurality of construction modes of the corresponding metamorphic transformation template under the constraint of the first metamorphic relationship, and the second description is to realize a plurality of construction modes of the corresponding metamorphic transformation template under the constraint of the second metamorphic relationship.
[0040] Specifically, the embodiments of the present application construct 9 kinds of metamorphic transformation templates through the first metamorphic relationship and the second metamorphic relationship, as shown in Table 1, Table 1 is a metamorphic template diagram constructed by two kinds of metamorphic relationships provided by the embodiments of the present application, wherein the name of each metamorphic template and the description of the specific behavior, through the 9 kinds of metamorphic templates, a variant semantically equivalent to the test prefix and the code in the method under test can be generated, which is used to detect the behavior consistency of the large model under different code structures.
[0041] Table 1: Metamorphic templates constructed by two kinds of metamorphic relationships
[0042]
[0043] It should be noted that the code conversion work can be processed using a static analysis tool, and a code parsing tool (such as Spoon) is used to construct the metamorphic template of Java code, and the technical form is to construct an abstract syntax tree based on a static analysis method, and then map the abstract syntax tree to the code block and rewrite the code block to achieve the goal of converting the code.
[0044] For example, MR-1.Add Unused Variable and MR-2.Add neutral element represent two kinds of metamorphic relations respectively, wherein, the implementation of the Add Unused Variable metamorphic template is to add an unused variable member to the specific method structure or the class structure, an abstract syntax tree corresponding to the method or the class can be obtained through a code parsing tool (such as Spoon), the specific code block area is located through structure traversal and type screening, and a statement of a declared but unused variable is inserted at the specified position, the type, name and initial value of the variable can be generated by a random strategy, and are added to a globally maintained dictionary to prevent secondary declaration of a member variable with the same name; the implementation of the Add neutral element metamorphic template is to perform equivalent replacement on the specific value of the basic type variable through heuristic rules, for example, if it is an int basic type, the original value + 0 is used to make the value equivalent to the original value, and for a String basic type, an empty character is added to the original value to ensure the equivalence of the original value.
[0045] It should be noted that in actual application, if the code to be tested involves third-party dependent libraries during the static analysis process, it is necessary to ensure that these dependent libraries are available, otherwise the code parsing tool (such as Spoon) may fail to parse. Therefore, a complete classpath can be provided to ensure the integrity of code parsing; secondly, based on different metamorphic relations, the conversion rules can include method call transformation, variable renaming, conditional expression modification, exception handling adjustment, etc., to ensure that the model can learn diversified code patterns; finally, under the framework of the code parsing tool, the structural rewriting of the Java code can be realized by traversing the Abstract Syntax Tree (AST) nodes and applying the corresponding conversion rules. For example, the accessor module of the code parsing tool can be used to access the method body, and the target node content can be modified by means of the construction module to generate a new code variant.
[0046] In step S103, each code variant is tested based on the preset assertion generation model, a test assertion evaluation result is obtained, and a test assertion evaluation report is generated according to the test assertion evaluation result.
[0047] The preset assertion generation model is a model capable of automatically generating a statement in a program for checking whether a specific condition is satisfied, and the assertion evaluation result is a result obtained by quantitatively evaluating the generated assertion, including assertion bug discovery rate, assertion accuracy, test case branch coverage, etc.
[0048] In some embodiments, based on the preset assertion generation model, each code variant is tested to obtain a test assertion evaluation result, including: inputting each code variant into the preset assertion generation model to obtain a plurality of test assertions, verifying the accuracy of each test assertion, and obtaining the executability result of each code variant according to the verification result; based on the executability result of each code variant, a plurality of test assertions are obtained, and based on a preset experimental comparison strategy, the generation quality of the plurality of test assertions is compared to obtain the test assertion evaluation result.
[0049] For example, the code variants generated by the embodiments of the present application are input into the existing assertion generation model (such as method TOGA, etc.), and the assertion generation model is induced to generate a variety of test assertions. Through systematic testing and analysis, the effectiveness of the method in improving the quality of test assertions is evaluated. If the generated test assertions can capture the error behaviors missed by the original test in the variant, it indicates that the variant strategy can improve the defect detection capability of the assertion, wherein the proportion of the generated assertions that can discover bugs is quantified, and then the test assertion method is evaluated, and the evaluation can be performed on the standard data set (such as Defects4J, ATLAS, etc.), and the evaluation indexes include bug discovery rate, accuracy of assertion, branch coverage of test case, etc.
[0050] It should be noted that the assertion evaluation result includes functional evaluation, that is, verifying whether the generated test assertion is correct to ensure that it can truly reflect the behavior of the code to be tested; diversity analysis, that is, statistics of the changes of assertions induced by different code variants to evaluate the effect of the method in improving the diversity of test assertions, such as using the metamorphosis score; effectiveness comparison, that is, compared with the traditional method (such as the assertions generated by ATLAS evaluation set, EvoSuite and Randoop or Defects4j data set), the actual effect of the method of the present application in improving the quality of test assertions is analyzed.
[0051] Further, in some embodiments, after generating a plurality of code variants corresponding to each metamorphic transformation template, it further includes: performing a compilation check on each code variant, and filtering out the code variants that do not meet the preset compilation result, and discarding the code variants that do not meet the preset compilation result.
[0052] Specifically, the code variant is subjected to a compilation check to ensure that it meets the Java syntax specification. The original test case is run to verify whether the behavior of the code variant is consistent with the original code. If the variant cannot pass the compilation or the test, it is discarded to avoid affecting the subsequent assertion generation task.
[0053] For the skilled in the art to better understand the test assertion evaluation method based on metamorphic relationship of the embodiments of the present application, the following will be explained and described in conjunction with specific embodiments.
[0054] Figure 2 The flow chart of the test assertion evaluation method based on metamorphic relationship according to one specific embodiment of the present application is provided.
[0055] As shown in Figure 2 , first, the metamorphic transformation template is constructed, and the "code to be tested", "test prefix" and "metamorphic relationship" are received as input information, and the input content is processed by using the "static analysis tool". The syntax tree is constructed by static analysis, and the metamorphic transformation template is generated based on the syntax tree, such as MR-1, MR-2, etc. This step lays the foundation for subsequent generation of code variants, aiming to extract patterns that can be used for transformation from the code structure and semantic level; secondly, the generated metamorphic transformation template is obtained, and the generated metamorphic template is applied to operate the code to generate code variants, and then the compilation checking and test verification are carried out, if the compilation and test verification do not pass, the variant is determined as "discarded variant"; if the verification passes, the subsequent process is continued. In this stage, through the screening of the code variants, it is ensured that the variants entering the next stage are effective in the basic running level; finally, the "assertion generation model" is introduced, and the code variants that pass the verification are processed, and diversified test assertions are finally output. These assertions can be used to more comprehensively detect the correctness and robustness of the code to be tested.
[0056] According to the test assertion evaluation method based on metamorphic relationship provided by the embodiments of the present application, by defining two types of metamorphic relationships, a variety of metamorphic transformation templates are constructed, the Java test code is transformed equivalently, so as to induce the assertion generation model to learn diversified test patterns, and each code variant is tested and evaluated by the assertion generation model, and the test assertion evaluation result is obtained. The problem of low efficiency of automatic test caused by low diversity and poor applicability of assertion generation in related technologies is solved, the diversity of assertion generation is improved, and the efficiency of automatic test is improved.
[0057] Secondly, the test assertion evaluation device based on metamorphic relationship according to the embodiments of the present application is described with reference to the accompanying drawings.
[0058] Figure 3 The block diagram of the test assertion evaluation device based on metamorphic relationship according to the embodiments of the present application is provided.
[0059] As shown in Figure 3 , the test assertion evaluation device 10 based on metamorphic relationship includes an acquisition module 100, a generation module 200 and an evaluation module 300.
[0060] The acquisition module 100 is configured to determine a first metamorphic relationship and a second metamorphic relationship that are semantically equivalent to the test program; the generation module 200 is configured to construct a plurality of metamorphic transformation templates based on the first metamorphic relationship and the second metamorphic relationship, and generate a plurality of code variants corresponding to each metamorphic transformation template; and the evaluation module 300 is configured to test each code variant based on a preset assertion generation model, obtain a test assertion evaluation result, and generate a test assertion evaluation report according to the test assertion evaluation result.
[0061] Further, in some embodiments, the generation module 200 is specifically configured to: determine a plurality of first descriptions based on the first metamorphic relationship, and add each first description to the test program to obtain a metamorphic transformation template corresponding to each first description; determine a plurality of second descriptions based on the first metamorphic relationship, and add each second description to the test program to obtain a metamorphic transformation template corresponding to each second description; and obtain a plurality of metamorphic transformation templates according to the metamorphic transformation template corresponding to each first description and the metamorphic transformation template corresponding to each second description.
[0062] Further, in some embodiments, the evaluation module 300 is specifically configured to: input each code variant into the preset assertion generation model to obtain a plurality of test assertions, verify the accuracy of each test assertion, and obtain an executability result of each code variant according to a verification result; obtain a plurality of test assertions based on the executability result of each code variant, compare a plurality of test assertions based on a preset experimental comparison strategy to obtain a generation quality of the plurality of test assertions, and obtain the test assertion evaluation result according to the generation quality of the plurality of test assertions.
[0063] Further, in some embodiments, after the generation module 200 generates a plurality of code variants corresponding to each metamorphic transformation template, the generation module 200 is further configured to: perform a compilation check on each code variant, filter out a code variant that does not meet a preset compilation result, and discard the code variant that does not meet the preset compilation result.
[0064] It should be noted that the above explanation of the test assertion evaluation method based on metamorphic relationships also applies to the test assertion evaluation device based on metamorphic relationships, which will not be described here.
[0065] The test assertion evaluation device based on metamorphic relationships according to the embodiments of the present application defines two types of metamorphic relationships, constructs a plurality of metamorphic transformation templates, and performs equivalent transformation on Java test code, thereby inducing the assertion generation model to learn diversified test patterns, and testing and evaluating each code variant through the assertion generation model to obtain a test assertion evaluation result, which solves the problem of low efficiency of automatic testing caused by low diversity and poor applicability of assertion generation in related technologies, improves the diversity of assertion generation, and improves the efficiency of automatic testing.
[0066] Figure 4 A structural schematic diagram of an electronic device according to an embodiment of the present application is provided. The electronic device can include:
[0067] The memory 401, the processor 402 and the computer program stored in the memory 401 and executable on the processor 402.
[0068] The processor 402 implements the test assertion evaluation method based on the transmutation relationship provided in the above embodiments when executing the program.
[0069] Further, the electronic device further includes:
[0070] The communication interface 403 is used for communication between the memory 401 and the processor 402.
[0071] The memory 401 is used for storing the computer program executable on the processor 402.
[0072] The memory 401 can include a high-speed RAM memory, and can also include a non-volatile memory, for example, at least one disk memory.
[0073] If the memory 401, the processor 402 and the communication interface 403 are independently implemented, the communication interface 403, the memory 501 and the processor 502 can be connected to each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 In the figure, only one thick line is used to represent, but it does not mean that there is only one bus or one type of bus.
[0074] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can complete communication between each other through an internal interface.
[0075] The processor 402 can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to perform the operations of the embodiments of the application.
[0076] In addition, the embodiments of the present application further provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the test assertion evaluation method based on the transmutation relationship as described above.
[0077] In addition, the embodiments of the present application further provide a computer program product, which comprises a computer program, and the computer program is executed to implement the test assertion evaluation method based on the transmutation relationship as described above.
[0078] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present description, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or N embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present description and the features of the different embodiments or examples, without contradiction.
[0079] In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, for example, two, three, etc., unless otherwise specifically limited.
[0080] Any process or method descriptions in flow charts or described elsewhere herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for performing specific logic functions (or steps) or portions of the application, and the various embodiments of the application can include additional or fewer steps performing the same or equivalent functions. In some embodiments, the steps or functions can be performed out of order. The various embodiments of the application can include components other than those shown in the figures or described herein, and can include multiple instances of components described herein, or other components not specifically described herein. In addition, instances of components described herein can be shared or used by multiple components.
[0081] It should be understood that portions of the present application can be implemented in hardware, software, firmware, or combinations thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As such, if implemented in hardware, and in another embodiment, any of the following technologies, known in the art, or their combinations, can be used to implement: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.
[0082] Those skilled in the art can understand that all or part of the steps carried out by the above-mentioned embodiment methods can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.
Claims
1. A test assertion evaluation method based on metamorphosis relationship, characterized in that: The following steps are included: Determine the first metamorphosis relation and the second metamorphosis relation of semantic equivalence of the test program; Based on the first metamorphosis relationship and the second metamorphosis relationship, construct a plurality of metamorphosis conversion templates, and generate a plurality of code variants corresponding to each metamorphosis conversion template; Based on a preset assertion generation model, each code variant is tested to obtain a test assertion evaluation result, and a test assertion evaluation report is generated according to the test assertion evaluation result.
2. The test assertion evaluation method based on metamorphosis relationship according to claim 1, characterized in that: The step of constructing a plurality of transformation conversion templates based on the first transformation relationship and the second transformation relationship includes: Determining a plurality of first descriptions based on the first transformation relationship, and adding each first description to the test program to obtain a transformation conversion template corresponding to each first description; Determining a plurality of second descriptions based on the first transformation relationship, and adding each second description to the test program to obtain a transformation conversion template corresponding to each second description; A plurality of metamorphosis conversion templates are obtained according to the metamorphosis conversion template corresponding to each first description and the metamorphosis conversion template corresponding to each second description.
3. The test assertion evaluation method based on metamorphosis relationship according to claim 1, characterized in that: The assertion generation model is based on a preset one, and each code variant is tested to obtain a test assertion evaluation result, including: Inputting each code variant into the preset assertion generation model to obtain multiple test assertions, verifying the accuracy of each test assertion, and obtaining an executable result of each code variant based on the verification result; Based on the executable results of each code variant, multiple test assertions are obtained, and based on a preset experimental comparison strategy, the multiple test assertions are compared to obtain the generation quality of the multiple test assertions, and the test assertion evaluation result is obtained according to the generation quality of the multiple test assertions.
4. The test assertion evaluation method based on metamorphosis relationship according to claim 1, characterized in that: After generating a plurality of code variants corresponding to each transformation template, the method further includes: A compilation check is performed on each code variant, and code variants that do not conform to a preset compilation result are screened out according to the compilation result, and the code variants that do not conform to the preset compilation result are discarded.
5. A test assertion evaluation device based on metamorphosis relationship, characterized in that: The device comprises: An acquisition module, configured to determine a first metamorphosis relationship and a second metamorphosis relationship of semantic equivalence of a test program; A generating module, configured to construct a plurality of metamorphic conversion templates based on the first metamorphic relationship and the second metamorphic relationship, and generate a plurality of code variants corresponding to each metamorphic conversion template; The evaluation module is used to test each code variant based on a preset assertion generation model, obtain a test assertion evaluation result, and generate a test assertion evaluation report according to the test assertion evaluation result.
6. The test assertion evaluation device based on metamorphosis relationship according to claim 5, characterized in that: The generation module is specifically used to: Determining a plurality of first descriptions based on the first transformation relationship, and adding each first description to the test program to obtain a transformation conversion template corresponding to each first description; Determining a plurality of second descriptions based on the first transformation relationship, and adding each second description to the test program to obtain a transformation conversion template corresponding to each second description; A plurality of metamorphosis conversion templates are obtained according to the metamorphosis conversion template corresponding to each first description and the metamorphosis conversion template corresponding to each second description.
7. The test assertion evaluation device based on metamorphosis relationship according to claim 5, characterized in that: The evaluation module is specifically used to: Inputting each code variant into the preset assertion generation model to obtain multiple test assertions, verifying the accuracy of each test assertion, and obtaining an executable result of each code variant based on the verification result; Based on the executable results of each code variant, multiple test assertions are obtained, and based on a preset experimental comparison strategy, the multiple test assertions are compared to obtain the generation quality of the multiple test assertions, and the test assertion evaluation result is obtained according to the generation quality of the multiple test assertions.
8. An electronic product, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the test assertion evaluation method based on the metamorphosis relationship according to any one of claims 1 to 4.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the test assertion evaluation method based on the metamorphosis relationship as described in any one of claims 1 to 4.
10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the test assertion evaluation method based on the metamorphosis relationship according to any one of claims 1 to 4 is implemented.