Compiler test oracle generation method and compiler test method

Abstract

The invention discloses a compiler test oracle generation method and a compiler test method, belongs to the technical field of software testing, and effectively detects program potential defects in the process that a compiler compiles program source codes into final executable codes by performing program property consistency verification on the source codes and compiled code intermediate representations (IR). According to the method, the defect detection capability of the compiler can be effectively improved when the compiler processes source codes of complex branches and control flows; meanwhile, the method does not need to construct expected output of single execution, so that the manual workload in the test process is reduced, and the test efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention provides a compiler test oracle generation and compiler test method. Based on the consistency check of program properties, the compiler test oracle is designed and constructed. This method is used to detect defects generated during the process of the compiler compiling the program source code into the final executable code, and belongs to the technical field of software testing. Background Art

[0002] As an important tool for computer software development, the correctness of the compiler itself is crucial for the normal operation of the program. In the field of compiler testing, testers usually rely on the test oracle to judge whether the compiler generates the correct target code. However, for many compilers, it is quite difficult to obtain the test oracle, and testers often cannot directly judge whether the generated target code is correct for a specific input source code.

[0003] Currently, the mainstream testing methods in the field of compiler testing include differential testing and metamorphic testing. The test oracle based on differential testing usually compiles the same input program with different behaviors to compare whether the output results are consistent. This includes cross-compiler type comparison, cross-compiler version comparison, cross-compiler optimization comparison, etc. The test oracle based on metamorphic testing usually changes the input program structure and then observes whether the output results change correspondingly.

[0004] Although the existing test oracles solve the problem of compiler defect detection to a certain extent, they usually only focus on the comparison of output results and often ignore the details during the program execution process. In some cases, even if the output results generated by the compiler are seemingly correct, there may still be potential problems during its execution process. However, these traditional test oracles are difficult to accurately capture and diagnose the deep-seated problems inside the compiler. Summary of the Invention

[0005] Aiming at the deficiencies of the existing technologies in compiler testing, the present invention proposes a compiler test oracle generation method and a compiler test method based on the consistency check of program properties. Through the present invention, the potential defects of the program during the process of the compiler compiling the program source code into the final executable code can be effectively detected by verifying the consistency of program properties between the test program source code and the intermediate representation (IR) of the compiled code.

[0006] In practical applications, these program defects may cause the compiler to fail to correctly generate the intermediate code (IR) that meets the expected behavior, and in some cases, cause the program to crash or generate incorrect target code, thereby affecting the correct execution of the program. Through the detection of this method, the reliability and correctness of the compiler can be effectively improved.

[0007] The present invention uses the test program source code and the code intermediate representation (IR) generated by the compiler to perform program property consistency checking. The input of the method is a test program with multi-path characteristics. The test program has a complex branch and path structure and can cover different execution paths. Specifically, when implemented, the test program includes a program generated by an existing random program generation tool or constructed manually.

[0008] The specific steps of the method of the present invention include test program path generation, source code to code intermediate representation (IR) mapping, source code and code intermediate representation (IR) program property consistency analysis, etc. The specific technical solutions are as follows:

[0009] A compiler test oracle generation method based on program property consistency checking includes the following steps:

[0010] A. Obtain a compiler test program with multi-path characteristics, generate test program paths, and obtain a path set;

[0011] Input a test program with multi-path characteristics, label the key variables related to the branches of the test program, and combine with a symbolic execution tool to generate a complete path set of the compiler test program as the basis for subsequent consistency analysis.

[0012] Use a symbolic analysis tool (such as KLEE) to symbolically process the labeled variables, explore multiple possible execution paths of the test program, and generate a source code path set Path P ={Path p1 ,Path p2 ,…,Path pm}, covering all execution paths in the test program.

[0013] B. Source code to code intermediate representation (IR) mapping;

[0014] After compiling the test program into intermediate code (IR) through a compiler, identify and extract the debugging information in the IR file, and use the debugging information to accurately map the instructions in the IR to the line numbers of the source code, thereby establishing a mapping relationship between the source code path and the IR path.

[0015] B1. For the test program P, use the compiler to compile it into the code intermediate representation IR, and this process is described as P→IR;

[0016] B2. Identify and extract the debugging information in the code intermediate representation IR file, and use the association information between the source code and the code intermediate representation (IR) in the debugging information to accurately map the instructions in the code intermediate representation (IR) code to the source code, and establish a mapping relationship between the source code line number l and the IR instruction f, such as expressed as Through these mapping relationships, the source code path set PathP and the IR path set Path IR Establish a two-way association, denoted as

[0017] C. Program property consistency analysis between source code and intermediate representation (IR) of code;

[0018] Combined with the generated path set, perform reachability analysis on the paths in the source code to determine whether a specific code line (i.e., the key code line) can be executed; check whether the corresponding IR code line is also executed; thus, realize the generation of compiler test oracles based on the consistency check of program execution paths. Including:

[0019] C1. Utilize the generated source code path set Path P , perform reachability analysis on specific code lines in the source code, and determine whether each source code path in the source code path set can be executed;

[0020] C2. Combined with the IR path set {Path IR} generated by the symbolic analysis tool, check whether the corresponding IR code path is also executed;

[0021] C3. If it is found that the source code path is executed, but the corresponding IR path is not executed, then it is identified that there is a defect in the compiler when generating IR;

[0022] The method of the present invention does not compare the entire order of the source code and IR, but only compares some special error-prone code lines, that is, selects special code lines to improve efficiency. For the selection of special code lines, specifically, perform static analysis on the branch structures (such as loop statements and conditional judgment statements) in the source code of the test program, and identify the variables that have an impact on the execution of the branch structure. By comparing the occurrence frequencies of these variables in branch decisions (i.e., the occurrence frequencies in loop statements or conditional judgment statements), select the variable with the highest occurrence times and label it as the key variable that has the greatest impact on branch selection. Set the branch code lines affected by the key variable as special code lines (key code lines).

[0023] In summary, the compiler test oracle of the present invention is generated based on the consistency check of program execution paths. First, the possible execution paths of the test program are explored based on symbolic execution, and the mapping relationship between the source code and the IR code is constructed using debugging information. Subsequently, by performing path consistency analysis to compare the execution of the source code path and the IR path, it is determined whether there are missing paths or incorrect mappings, so as to detect potential defects in the process of the compiler compiling the program source code into the final executable code. This test oracle does not rely on manually expected outputs, but automatically constructs test oracles based on the nature of the program itself, and precisely verifies the behavior of the compiler from the execution level. Among them, the core lies in the proposed path consistency analysis method, which can accurately identify potential errors of the compiler in compiling the program source code into the final executable code by differentiating and comparing the program consistency between the source code and the IR code, which is difficult to achieve by existing testing techniques.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] The present invention provides a method for generating a compiler test oracle and a compiler test method based on the consistency check of program properties. The designed and generated compiler test oracle can effectively improve the defect detection ability of the compiler when processing source code with complex branches and control flows. At the same time, this method does not need to construct the expected output of a single execution, reduces the manual workload in the testing process, and improves the testing efficiency and accuracy. Compared with the test oracles generated by existing traditional test oracle generation methods, using the compiler test oracle generated by the present invention for compiler testing takes a more in-depth consideration of the execution process of the program and can more accurately locate potential defects in the process of the compiler compiling the program source code into the final executable code. Brief Description of the Drawings

[0026] Figure 1 It is a flowchart of the method of the present invention. Detailed Embodiments

[0027] The present invention realizes a method for designing and constructing a compiler test oracle based on the consistency check of program properties, which is used to detect potential defects in the compilation process of the compiler. By testing the compiler with the technical solution proposed by the present invention, it can efficiently discover the defects existing in the compiler in dealing with complex control flows, especially solve the limitation that the traditional method cannot capture execution process errors only through input and output result verification, and improve the detection efficiency and accuracy of compiler defects.

[0028] Through the following steps, the present invention constructs a complete test framework:

[0029] Input: A multi-path test program;

[0030] Output: Judging the correctness of the compilation result; if an error is detected, providing the code location where compilation defects may exist.

[0031] The present invention takes a test program containing multi-path characteristics as input, and the test program can be generated by existing tools or constructed manually. These test programs contain complex control flow structures (such as branches, loops) and multiple execution paths.

[0032] 1. Generating test program paths;

[0033] Using a symbolic execution tool to symbolically process the annotated variables, obtaining multiple execution paths of the test program, and generating a path set.

[0034] 2. Establishing a mapping between the source code and the intermediate representation (IR) of the code, so that each source code corresponds to the IR;

[0035] Using a compilation tool to compile the generated source code test program and convert it into an intermediate code representation (IR). In addition, the present invention compiles with different optimization levels to test the influence of different optimization levels on the generated intermediate code representation of the code. When compiling the source code into intermediate code (IR), it supports compilation with different optimization levels (-O0, -O1, -O2, -O3), tests the influence of different optimization levels on the intermediate code, and ensures that the control flow and semantic information of the source code during the optimization process are correctly retained in the IR.

[0036] By extracting the debugging information in the IR file, establishing an accurate mapping relationship between the source code line numbers and the IR instructions, and ensuring that each source code path can correspond to the IR path.

[0037] 3. Analyzing the consistency of the properties of the source code and the intermediate representation (IR) program to identify whether there are defects in the compiler when generating the IR;

[0038] Combined with the path set generated by the symbolic execution tool, the present invention checks the execution consistency between the source code path and the IR path. Selecting a specific path and variable values given by the symbolic analysis tool, and using these variable values to execute the source code and the IR program. If it is found that a certain path is executed in the source code but the corresponding IR path is not triggered, it can be inferred that there may be defects in the compiler during compilation.

[0039] In order to improve the efficiency and accuracy of the analysis, the present invention conducts consistency analysis on special code lines. By deeply analyzing the key statements in the branch structure, variables and code lines that have a greater impact on the execution path can be accurately found. Conducting consistency analysis on these special code lines can improve the efficiency and accuracy of path consistency checking.

[0040] The following uses a specific example to illustrate the compiler test oracle method based on program property consistency checking provided by the present invention to capture compiler defects where errors occur during execution but the results are correct.

[0041] 1. Test scenario

[0042] A certain tester needs to verify whether the compiler correctly processes branch conditions. The test program is as follows:

[0043]

[0044] The expected behavior of this program is:

[0045] · When x > 0, return x + y.

[0046] · When x < 0, return x - y.

[0047] · When x = 0, return y × y - y.

[0048] 2. Potential problems

[0049] During the syntax analysis or intermediate code generation and optimization process of the compiler, if the branch condition logic is not correctly processed, it may lead to the execution of incorrect paths. For example:

[0050] · Source code path set:

[0051] ο

[0052] ο

[0053] ο

[0054] · IR path set (with errors):

[0055] ο

[0056] ο

[0057] When the input is x = 0, y = 2, the IR executes the incorrect path Path IR , but since the results of 0 + 2 = 2 and 2 × 2 - 2 = 2 are the same, traditional methods that only verify the results cannot detect the problem.

[0058] 3. The test and verification process of the present invention;

[0059] 1) Path generation and mapping

[0060] Symbolic analysis can use the KLEE symbolic execution tool to generate the source code path set:

[0061] ο

[0062] ο

[0063] ο

[0064] Correspondingly generate the IR path set:

[0065] ο

[0066] ο

[0067] 2) Consistency analysis

[0068] Comparison path set: {Path P} ≠ {Path IR}

[0069] When the input is x = 0 and y = 2, the IR executes the wrong path. However, since the results are the same, the traditional method can only verify the results and cannot find the problem. Using the present invention to detect the source code path is executed, but the corresponding path in the IR is not executed, and wrongly enters It is found that there is a compilation error in the compiler.

[0070] It should be noted that the purpose of publishing the embodiments is to help further understand the present invention. However, those skilled in the art can understand that various substitutions and modifications are possible without departing from the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the scope of protection required by the present invention is subject to the scope defined by the claims.

Claims

1. A compiler test oracle generation and compiler testing method, characterized in that: By verifying the consistency of program properties between the source code and the compiled code intermediate representation IR, there is no need to construct the expected output of a single execution, and effectively detect potential program defects in the process of the compiler compiling the program source code into the final executable code; including the following steps: 1) Obtain a test program with multi-path characteristics and generate a complete path set of the test program; 2) After compiling the test program into the intermediate code IR, identify and extract the debugging information in the IR file, and use the debugging information to accurately map the instructions in the IR to the line numbers of the source code, thereby establishing a mapping relationship between the source code path and the IR path; 3) Based on the generated path set, perform reachability analysis on the source code path to determine whether the key code lines can be executed; Check if the corresponding IR code line is also executed; include: 31) Using the generated source code path set, perform reachability analysis on key code lines in the source code to determine whether each source code path in the source code path set can be executed; 32) According to the generated IR path set, check whether the corresponding IR code path is also executed; 33) If the source code path is executed but the corresponding IR path is not executed, it is recognized that the compiler has a defect when generating IR; That is, the consistency check based on the program execution path is realized to generate compiler test predictions and compiler tests.

2. The compiler test oracle generation and compiler testing method according to claim 1, wherein: In step 3), the branch structure in the source code is statically analyzed, the variables that appear most frequently in the execution of the branch structure are identified as key variables, and the branch code lines affected by the key variables are identified as key code lines.

3. The compiler test oracle generation and compiler testing method according to claim 2, wherein: The branch structure in the source code includes the loop body and the judgment statement.

4. The compiler test oracle generation and compiler testing method according to claim 1, wherein: When compiling source code into intermediate code IR, it supports compilation at different optimization levels, tests the impact of different optimization levels on the intermediate code, and correctly retains the control flow and semantic information of the source code during the optimization process in the IR.

5. The compiler test oracle generation and compiler testing method according to claim 4, characterized in that: Different optimization levels include: -O0, -O1, -O2, -O3.

Images