A compiler test oracle generation and compiler testing method

Abstract

This invention discloses a compiler test oracle generation and compiler testing method, belonging to the field of software testing technology. By verifying the consistency of program properties between the source code and the intermediate representation (IR) of the compiled code, it effectively detects potential program defects in the compiler's process of compiling the source code into the final executable code. This method effectively improves the compiler's defect detection capability when handling source code with complex branches and control flow; simultaneously, it eliminates the need to construct the expected output for a single execution, reducing manual workload and improving testing efficiency and accuracy.

Description

Technical Field

[0001] This invention provides a compiler test oracle generation and compiler testing method. The compiler test oracle is designed and constructed based on program property consistency checks. This method is used to detect defects generated by the compiler in the process of compiling program source code into final executable code, and belongs to the field of software testing technology. Background Technology

[0002] As a crucial tool in computer software development, the correctness of the compiler is essential for the proper functioning of programs. In the field of compiler testing, testers typically rely on test oracles to determine whether the compiler generates correct target code. However, for many compilers, obtaining test oracles is quite difficult, and testers often cannot directly determine whether the generated target code is correct under specific input source code.

[0003] Currently, the mainstream testing methods in the compiler testing field include differential testing and metamorphic testing. Differential testing typically involves compiling the same input program with different behaviors to compare the consistency of the output results. This includes cross-compiler type comparison, cross-compiler version comparison, and cross-compiler optimization comparison. Metamorphic testing, on the other hand, usually involves changing the structure of the input program and then observing whether the output results change accordingly.

[0004] While existing test precepts have addressed compiler defect detection to some extent, they typically focus only on comparing output results, often neglecting details of the program execution process. In some cases, even if the compiler-generated output appears correct, underlying problems may still exist during execution. Furthermore, these traditional test precepts struggle to accurately capture and diagnose deep-seated problems within the compiler. Summary of the Invention

[0005] This invention addresses the shortcomings of existing compiler testing techniques by proposing a compiler test oracle generation method and a compiler testing method based on program property consistency checks. Through this invention, program property consistency can be verified by comparing the test program source code with the compiled intermediate representation (IR), thereby effectively detecting potential program defects in the compiler's process of compiling the source code into the final executable code.

[0006] In practical applications, these program defects may cause the compiler to fail to generate intermediate code (IR) that behaves as expected, leading to program crashes or the generation of incorrect target code in some cases, thus affecting the correct execution of the program. The detection method described in this paper can effectively improve the reliability and correctness of the compiler.

[0007] This invention utilizes the intermediate representation (IR) of the test program source code and the compiler-generated code to perform consistency checks on program properties. The method's input is a test program with multi-path characteristics, featuring complex branching and path structures capable of covering different execution paths. In specific implementations, the test program can be generated by existing random program generation tools or manually constructed programs.

[0008] The specific steps of the method of this invention include test program path generation, source code to intermediate representation (IR) mapping, and consistency analysis of the properties of source code and intermediate representation (IR) programs. The specific technical solution is as follows:

[0009] A compiler test oracle generation method based on program property consistency checking includes the following steps:

[0010] A. Obtain compiler test programs with multi-path characteristics, generate test program paths, and obtain a path set;

[0011] Input a test program with multi-path characteristics, annotate the key variables related to the branches of the test program, and use symbolic execution tools to generate a complete set of paths for the compiler test program, which serves as the basis for subsequent consistency analysis.

[0012] Using symbolic analysis tools (such as KLEE), the labeled variables are symbolized to explore multiple possible execution paths of the test program, generating a set of source code paths called Path. P ={Path p1 ,Path p2 ,…,Path pm This covers all execution paths in the test program.

[0013] B. Source code to intermediate representation (IR) mapping;

[0014] After the test program is compiled into intermediate code (IR) by the compiler, the debugging information in the IR file is identified and extracted. The debugging information is used to accurately map the instructions in the IR to the line numbers of the source code, thereby establishing a mapping relationship between the source code path and the IR path.

[0015] B1. For the test program P, use a compiler to compile it into the intermediate code representation IR. This process is described as P→IR.

[0016] B2. Identify and extract debugging information from the Intermediate Code Representation (IR) file. Utilize the association information between the source code and the IR file within the debugging information to precisely map instructions in the IR code to the source code, establishing a mapping relationship between source code line number l and IR instruction f, as shown below. Through these mapping relationships, the source code path collection PathP and IR path set Path IR Establishing a bidirectional association is represented as

[0017] C. Consistency analysis of program properties between source code and intermediate representation (IR);

[0018] By combining the generated path set, reachability analysis is performed on the paths in the source code to determine whether specific lines of code (i.e., critical lines of code) can be executed; the corresponding IR lines of code are also checked for execution; thus, a compiler test oracle is generated based on the consistency check of the program execution path. This includes:

[0019] C1. Utilize the generated source code path set Path P Perform reachability analysis on specific lines of code in the source code to determine the reachability of each source code path in the set of source code paths. Whether it can be executed;

[0020] C2. IR path set {Path} generated using symbolic analysis tools IR Check the corresponding IR code path. Is it also executed?

[0021] C3. If the source code path is found Executed, but corresponding IR path If it is not executed, it is identified as a defect in the compiler when generating the IR;

[0022] This invention does not compare the entire sequence of source code and IR, but only compares certain error-prone lines of code, i.e., selects specific lines of code to improve efficiency. Specifically, the selection of specific lines of code involves static analysis of the branching structures (such as loop statements and conditional statements) in the test program's source code to identify variables that influence the execution of these branching structures. By comparing the frequency of these variables in branch decisions (i.e., their frequency of occurrence in loop statements or conditional statements), the variable with the highest frequency is selected and marked as the key variable with the greatest influence on branch selection. The lines of code affected by these key variables are designated as specific lines of code (key lines of code).

[0023] In summary, the compiler test oracle of this invention is generated based on consistency checks of program execution paths. First, it explores possible execution paths of the test program using symbolic execution and constructs a mapping relationship between the source code and the IR code using debugging information. Then, it compares the execution of the source code path and the IR path through path consistency analysis to determine if there are any missing paths or incorrect mappings, thereby detecting potential defects in the compiler's process of compiling the program source code into the final executable code. This test oracle does not rely on manually expected output but automatically constructs test oracles based on the program's inherent properties, accurately verifying the compiler's behavior at the execution level. The core of this invention lies in the proposed path consistency analysis method. This method, through differential comparison of the program consistency between the source code and the IR code, can accurately identify potential errors in the compiler's compilation of the program source code into the final executable code, which is difficult to achieve with existing testing techniques.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] This invention provides a compiler test oracle generation method and a compiler testing method based on program property consistency checks. The generated compiler test oracle effectively improves the compiler's defect detection capability when processing source code with complex branches and control flow. Simultaneously, this method eliminates the need to construct the expected output for a single execution, reducing manual workload and improving testing efficiency and accuracy. Compared to test oracle generation methods, the compiler test oracle generated by this invention provides a deeper consideration of the program execution process, enabling more accurate location of potential defects in the compiler's process of compiling the program source code into the final executable code. Attached Figure Description

[0026] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0027] This invention implements a compiler test oracle design and construction method based on program property consistency checks, used to detect potential defects in the compiler during the compilation process. By testing the compiler using the proposed technical solution, defects in complex control flow processing can be efficiently discovered. In particular, it overcomes the limitation of traditional methods that rely solely on input / output verification to capture execution errors, thus improving the efficiency and accuracy of compiler defect detection.

[0028] Through the following steps, this invention constructs a complete testing framework:

[0029] Input: Multipath test program;

[0030] Output: A judgment on the correctness of the compilation result; if an error is detected, the location of the code that may contain a compilation defect is provided.

[0031] This invention takes as input a test program that incorporates multi-path characteristics. The test program can be generated by existing tools or constructed manually. These test programs contain complex control flow structures (such as branches and loops) and multiple execution paths.

[0032] 1. Generate the test program path;

[0033] Symbolic execution tools are used to symbolize the annotated variables, obtain multiple execution paths of the test program, and generate a path set.

[0034] 2. Establish a mapping between source code and intermediate representation (IR) so that each line of source code corresponds to an IR;

[0035] The generated source code test program is compiled using a compiler tool and converted into an intermediate code representation (IR). Furthermore, this invention employs different optimization levels for compilation to test the impact of different optimization levels on the generated intermediate code representation. When compiling the source code into intermediate code (IR), different optimization levels (-O0, -O1, -O2, -O3) are supported to test the impact of different optimization levels on the intermediate code and ensure that the control flow and semantic information of the source code are correctly preserved in the IR during the optimization process.

[0036] By extracting debugging information from the IR file, a precise mapping relationship between source code line numbers and IR instructions is established, ensuring that each source code path corresponds to an IR path.

[0037] 3. Analyze the consistency between source code and intermediate representation (IR) program properties to identify any defects in the compiler when generating the IR;

[0038] By combining the path set generated by the symbolic execution tool, this invention checks the execution consistency between the source code path and the IR path. A specific path and variable value provided by the symbolic analysis tool are selected, and the source code and IR program are executed using that variable value. If a path is found to be executed in the source code, but the corresponding IR path is not triggered, it can be inferred that there may be a defect in the compiler during compilation.

[0039] To improve the efficiency and accuracy of the analysis, this invention performs consistency analysis on specific lines of code. By deeply analyzing key statements in the branching structure, variables and lines of code that have a significant impact on the execution path can be accurately identified. Performing consistency analysis on these specific lines of code can improve the efficiency and accuracy of path consistency checks.

[0040] The following specific example illustrates how the compiler test prediction method based on program property consistency checks provided by this invention can capture compiler defects that result in correct execution despite errors during execution.

[0041] 1. Test Scenario

[0042] A tester needs to verify whether the compiler handles branch conditions correctly. The test program is as follows:

[0043]

[0044] The expected behavior of this program is:

[0045] • When x > 0, return x + y.

[0046] • When x < 0, return xy.

[0047] When x = 0, return y × yy.

[0048] 2. Potential problems

[0049] During the compiler's syntax analysis or intermediate code generation and optimization process, if branch conditional logic is not handled correctly, it may lead to incorrect path execution. For example:

[0050] • Source code path set:

[0051] ο

[0052] ο

[0053] ο

[0054] • IR path set (error occurred):

[0055] ο

[0056] ο

[0057] When the input x = 0, y = 2, IR executed the wrong path. IR However, since the results of 0+2=2 and 2×2-2=2 are the same, the traditional method can only verify the results and cannot find the problem.

[0058] 3. The testing and verification process of this invention;

[0059] 1) Path generation and mapping

[0060] Symbolic analysis can use the KLEE symbolic execution tool to generate a set of source code paths:

[0061] ο

[0062] ο

[0063] ο

[0064] The corresponding IR path set is generated as follows:

[0065] ο

[0066] ο

[0067] 2) Consistency Analysis

[0068] Compare the set of paths: {Path P}≠{Path IR}

[0069] When x = 0 and y = 2 is input, IR executes an incorrect path. However, because the results are consistent, traditional methods that only verify the results cannot detect the problem. This invention is used to detect source code paths. It was executed, but the corresponding path in the IR was not. It was not executed and entered incorrectly. A compilation error was found in the compiler.

[0070] It should be noted that the purpose of disclosing the embodiments is to help further understand the present invention. However, those skilled in the art will understand that various substitutions and modifications are possible without departing from the scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the scope of protection of the present invention is defined by the scope of the claims.

Claims

1. A compiler test oracle generation and compiler testing method, characterized in that, By verifying the consistency of program properties between the source code and the intermediate representation (IR) of the compiled code, without constructing the expected output for a single execution, potential defects in the program can be effectively detected during the compiler's process of compiling the source code into the final executable code. This includes the following steps: 1) Obtain test programs with multi-path characteristics and generate a complete set of paths for the test programs; 2) After compiling the test program into intermediate code (IR), identify and extract the debugging information in the IR file, and use the debugging information to accurately map the instructions in the IR to the line numbers of the source code, thereby establishing a mapping relationship between the source code path and the IR path; 3) Based on the generated path set, perform reachability analysis on the source code paths to determine whether key lines of code can be executed; Check if the corresponding IR code line has also been executed; include: 31) Using the generated set of source code paths, perform reachability analysis on the key lines of code in the source code to determine whether each source code path in the set of source code paths can be executed; 32) Based on the generated IR path set, check whether the corresponding IR code path is also executed; 33) If the source code path is executed, but the corresponding IR path is not executed, it is identified as a defect in the compiler when generating the IR; This means implementing consistency checks based on program execution paths to generate compiler test predictions and compiler tests.

2. The compiler test oracle generation and compiler testing method as described in claim 1, characterized in that, In step 3), a static analysis is performed on the branch structure in the source code to identify the variable that appears most frequently in the execution of the branch structure as the key variable, and the branch code line affected by the key variable is identified as the key code line.

3. The compiler test oracle generation and compiler testing method as described in claim 2, characterized in that, The branching structure in the source code includes loop bodies and conditional statements.

4. The compiler test oracle generation and compiler testing method as described in claim 1, characterized in that, When compiling source code into intermediate code (IR), it supports compilation with different optimization levels, tests the impact of different optimization levels on intermediate code, and correctly preserves the control flow and semantic information of the source code during the optimization process in IR.

5. The compiler test oracle generation and compiler testing method as described in claim 4, characterized in that, Different optimization levels include: -O0, -O1, -O2, and -O3.

Images