Aero-engine compiler fuzz testing method and system based on large language model

Abstract

This invention discloses a fuzzy testing method and system for aero-engine compilers based on a large language model. In the mutation hint generation stage, a seed program library is constructed using embedded programs with various structures and compiler test suite programs. Mutation operators are selected through diversified guidance strategies, and mutation hints are constructed based on hint templates. In the test generation stage, variant programs with complex data and control flows are generated using the large language model to enhance compiler test coverage. Test outputs are inserted into the program to monitor global and local variables, enabling the program to effectively detect the most harmful silent compilation errors. Furthermore, a front-end error feedback strategy is designed to improve the effectiveness of test programs generated by the large language model. In the differential testing stage, random differential testing and testing under different optimization levels are combined to detect back-end defects in the compiler. This invention can improve compiler test coverage and detect silent compilation errors.

Description

Technical Field

[0001] This invention relates to the fields of software testing and compiler testing, and mainly to backend fuzz testing in an embedded C compiler for aero-engines. Background Technology

[0002] Compilers are among the most crucial foundational software components. In embedded systems for aero-engines, the stability and correctness of the compiler are paramount to the overall safety of the system. Therefore, when embedded systems perform critical tasks, any compiler error can lead to unpredictable system failures and even threaten aviation safety. Thus, rigorous and comprehensive testing of the compiler is an indispensable and critical step in ensuring the high reliability and safety of embedded systems. This not only helps verify the compiler's performance but also allows for the early identification of potential vulnerabilities and errors, thereby avoiding catastrophic consequences in real-world applications.

[0003] Fuzzing is the most widely used compiler testing method. The general process of fuzzing involves generating a large number of syntactically correct test codes and inputting them into different compilers. Any unexpected or inconsistent behavior during compilation indicates an error in one of the compilers. Fuzzing can be divided into generation-based and mutation-based methods. Generation-based methods design a series of syntax rules or train a deep neural network using a large amount of syntactically correct code to generate test code; mutation-based methods design various mutation operators to locally mutate an existing seed program to generate test code.

[0004] With the development of Large Language Models (LLMs), researchers have begun to explore their application in compiler fuzzing. Large language models, especially code generation models, are trained on large amounts of natural language and code data, possessing powerful and intelligent code generation capabilities. They can generate complex code structures for compiler testing based on prompts.

[0005] While both traditional methods and large language model-based methods have proven effective at detecting compiler errors, they still have limitations. First, traditional fuzzing methods lack diversity in the test code they generate. Generation-based methods like Csmith and Yarpgen generate grammatically correct test code, but their diversity is limited by manually designed generation rules. The state-of-the-art mutation-based method, GrayC, accumulates minor mutations in a seed program, generating a large amount of code that doesn't trigger compiler errors. Second, almost all fuzzing methods struggle to detect the most serious miscompilation errors, only detecting less serious compiler crashes. Finally, there is currently no fuzzing method specifically designed for C compilers based on large language models. Fuzz4all is a method based on both large language models and generation / mutation. This method uses a large natural language model (ChatGPT-4) to automatically extract and summarize the features of the programming language input by the user, generating input suggestions that prompt a large code generation model (StarCoder) to generate test programs for new features of the programming language. Although Fuzz4All has found many bugs in compilers for programming languages ​​that have been actively updated and have new features in recent years (such as C++ and Go), it has not found any bugs in the C language compiler (i.e., GCC), which prioritizes stability and security and is not updated frequently. Summary of the Invention

[0006] Purpose of the invention: To address the problem that existing technologies cannot effectively detect backend miscompilation errors in C compilers for aero-engines, especially the most harmful silent miscompilation errors, this invention provides a fuzzy testing method and system for aero-engine compilers based on a large language model.

[0007] Technical solution: To achieve the above objectives, the technical solution adopted by this invention is as follows:

[0008] A fuzz testing method for aero-engine compilers based on large language models includes the following steps:

[0009] Collect embedded code for various aero-engine structures and C compiler test suite programs as a seed program library;

[0010] Based on the control structure and data flow characteristics of aero-engine codes, various mutation operators and their corresponding mutation prompt templates are designed, including insertion mutation operators, deletion mutation operators, and replacement mutation operators. The mutation prompt template initially contains three template parts, which respectively indicate the mutation task, the specific instructions, and the seed program to be mutated. The insertion mutation prompt template requires the large language model to be inserted at the most complex position in the data flow and reuses existing variables to enhance the complexity of the data flow.

[0011] A diversity-guided mutation operator selection algorithm is adopted to mutate the seed program in a more diverse direction. In the algorithm, a score is assigned to each mutation operator, and the score of the mutation operator is determined by the diversity of the programs it generates, i.e., the average inter-program distance generated when generating program variants. The algorithm uses the Markov chain Monte Carlo method to sample mutation operators.

[0012] To enable variant programs to detect the most harmful compiler errors, namely silent compilation errors, a large language model is used to insert output statements into the variant programs to monitor local and global variables. Variant programs with output statements become test cases. The prompt template for inserting output statements includes the task, specific instruction example code, and the variant program that needs to have the output statements inserted.

[0013] To ensure the effectiveness of test cases, test cases that fail the compiler front-end and their error messages are used as feedback to update the mutation prompt template of the corresponding mutation operator;

[0014] Test cases are used to test the aero-engine compiler under test and detect backend errors in the compiler.

[0015] Furthermore, the various mutation operators include, among which, insertion mutation includes inserting branch structures, inserting loop structures, inserting control statements, inserting structures, inserting expressions, and inserting dead code; among which, replacement mutation operators include replacing certain constants, replacing certain variables, replacing certain operators, and replacing the rvalues ​​of assignment statements; and among which, deletion mutation operators include removing quantifiers, removing modifiers, and removing certain unary operators.

[0016] Furthermore, in the prompt template for inserting mutations, the specific instructions include: instruction [1] requiring the reuse of existing variables or expressions in the seed program in the branch conditions and code blocks when constructing branch statements; instruction [2] requiring the insertion to be performed at positions with high control flow complexity; instruction [3] requiring the definition of new variables for use in the branch statement code blocks when there are insufficient existing variables in the seed program.

[0017] Furthermore, for the insertion mutation operator, the [Instructions] in the mutation prompt template include: [1] Utilize existing variables or expressions for <insertion structure>. [2] It is best to insert at a location with complex control flow. [3] If necessary, declare and initialize new variables. [4] The response should contain only code.

[0018] For the replacement mutation operator, the [Instructions] in the mutation prompt template include: [1] If there are no <replacement object> to replace, respond with the original code I provide. [2] The response should contain only code.

[0019] For the deletion mutation operator, the [Instructions] in the mutation prompt template include: [1] If there are no <object to be deleted> to remove, respond with the original code I provide. [2] The response should contain only code.

[0020] The <insertion structure>, <replacement object>, and <deletion object> correspond to the description strings in the operators.

[0021] Furthermore, the distance Dist between two programs P1 and P2 is the Jaccard distance between them, calculated using the following formula:

[0022]

[0023] Among them, Stmt P1 and Stmt P2 Let P1 and P2 represent the line-level code sets of programs P1 and P2, respectively. These sets are formed by dividing the program into lines, and each element in the set is a line of code.

[0024] Furthermore, the algorithm is based on the Markov chain Monte Carlo algorithm, selecting the mutation operator according to its score ranking; the formula for calculating the mutation operator score (·) is as follows:

[0025]

[0026] Where Mut represents the mutation operator, n is the number of program variants obtained by Mut mutation, and Dist(P) i ,P i-1 ) represents the inter-program distance generated by the i-th mutation. When i=1, P i-1 This is the seed program; Rate(Mut) represents the probability that the Mut mutation will produce a valid variant program, calculated as follows:

[0027]

[0028] Among them, #FrontPass Mut This indicates the number of variant programs generated using Mut that pass the compiler frontend, #All Mut This indicates the total number of variant programs generated using Mut.

[0029] Furthermore, if a test case encounters a syntax error during compilation, i.e. fails to pass the compiler front-end, a new part will be added to the mutation hint template. This new part will be filled with the test case and the error information generated during compilation, thereby prompting the large language model to avoid generating the same type of error.

[0030] Furthermore, the test cases are used to test the C compiler under test, including random difference testing and different optimization level testing. Random difference testing belongs to cross-compiler scenario testing, while different optimization level testing belongs to cross-optimization level scenario testing. Random difference testing is used to find errors in the compiler at the -O0 optimization level, while different optimization level testing is used to find errors in the compiler at other optimization levels besides -O0.

[0031] A fuzz testing system for an aero-engine compiler based on a large language model, comprising:

[0032] The program mutation module collects embedded code from various aero-engine structures and C compiler test suites as a seed program library. Based on the control structure and data flow characteristics of the aero-engine code, it designs various mutation operators and their corresponding mutation hint templates, including insertion mutation operators, deletion mutation operators, and replacement mutation operators. The mutation hint template initially contains three parts, indicating the mutation task, specific instructions, and the seed program to be mutated. The insertion mutation hint template requires the large language model to be inserted at the most complex position in the data flow and reuses existing variables to enhance data flow complexity. A diversity-guided mutation operator selection algorithm is employed to mutate the seed program in more diverse directions. In this algorithm, each mutation operator is assigned a score, determined by the diversity of the programs it generates, i.e., the average inter-program distance generated when generating program variants. The algorithm uses a Markov chain Monte Carlo method to sample mutation operators. Test cases that fail the compiler front-end and their error messages are used as feedback to update the mutation hint template of the corresponding mutation operator.

[0033] The output statement insertion module is used to insert output statements into variant programs using a large language model to monitor local and global variables. Variant programs with output statements become test cases. The prompt template for inserting output statements includes the task, specific instruction example code, and the variant program that needs to have the output statements inserted.

[0034] The differential testing module is used to test the aero-engine compiler under test using test cases and to detect backend errors in the compiler.

[0035] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned fuzz testing method for an aero-engine compiler based on a large language model.

[0036] Beneficial effects: Compared with existing methods, the fuzz testing method and system for aero-engine compilers based on a large language model provided by this invention addresses the problem that the test programs generated by previous methods have low complexity and require the accumulation of a large number of small mutations. It designs a variety of mutation operators and their mutation hint templates, and selects mutation operators with diversity as a guide, so that the large language model can generate variant programs with more complex data flow and control flow. It also addresses the problem that previous methods lack the ability to detect compiler miscompilation errors. It uses the large language model to insert output statements into the variant programs to monitor variables, which helps to discover the most harmful miscompilation errors, especially silent miscompilation errors. Attached Figure Description

[0037] Figure 1 This is a flowchart of the fuzz testing method for an aero-engine compiler based on a large language model provided in an embodiment of the present invention. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0039] like Figure 1 As shown, this embodiment of the invention provides a fuzz testing method for an aero-engine compiler based on a large language model, mainly including program mutation, output statement insertion, and differential testing. The specific implementation of each step is as follows:

[0040] Step S1, Program Mutation

[0041] Step S1.1: Collect embedded code for various aero-engine architectures and C compiler test suite programs as a seed program library. Each seed program in the seed program library can be successfully compiled and run.

[0042] Step S1.2: Based on the control structure and data flow characteristics of the aero-engine code, design a variety of mutation operators and their corresponding mutation prompt templates.

[0043] This embodiment designs 13 program mutation operators and their corresponding mutation prompt templates. Specific descriptions of the mutation operators are shown in Table 1:

[0044] Table 1. Mutation Operators and Their [Task] Settings

[0045]

[0046]

[0047] Each mutation operator has a corresponding mutation hint template, which initially contains three parts: [Task], [Instructions], and [Code], indicating the mutation task, specific instructions, and the seed program to be mutated, respectively. [Task] summarizes the mutation task to be executed, [Instructions] describes several more detailed instructions, and [Code] is the program to be mutated. The insertion mutation hint template will require the large language model to insert code at the most complex location in the control flow within the [Instructions] section, and reuse existing variables to enhance data flow complexity.

[0048] In the insertion mutation prompt template, the first three instructions in [Instructions] are as follows: Instruction [1] requires that when constructing branch statements, the variables or expressions already existing in the seed program be reused in the branch conditions and code blocks. This will link the data flow in the original seed program to the branch statements, thereby achieving the effect of data flow enhancement. Instruction [2] requires that the insertion be performed at a position with high control flow complexity, which can also enhance the data flow. Moreover, when the insertion has a new control flow structure (such as loop statements and branch statements), the control flow complexity of the program will also be enhanced. Instruction [3] ensures that new variables can be defined for use in the branch statement code blocks when there are insufficient variables in the seed program.

[0049] In this embodiment, for the insertion mutation operator, the [Instructions] in the mutation prompt template include: [1] Utilize existing variables or expressions for <insertion structure>. [2] It is best to insert at a location with complex control flow. [3] If necessary, declare and initialize new variables. [4] The response should contain only code. Among them, <insertion structure> includes "the conditions and code blocks within the branching statements", "the conditions and code blocks within the loop statements", etc., which respectively represent the insertion of branching structures and loop structures.

[0050] For the replacement mutation operator, the [Instructions] in the mutation prompt template include: [1] If there are no <replacement object> to replace, respond me with the original code I provide. [2] The response should contain only code. The <replacement structure> includes "constants", "types", etc., which respectively represent replacement constants and data types.

[0051] For the deletion mutation operator, the [Instructions] in the mutation prompt template include: [1]If there are no <delete object> to remove, respond me with the original code I provide. [2]The response should contain only code. Among them, <delete object> includes "qualifiers" and "modifiers", which represent deletion qualifiers and modifiers, respectively.

[0052] Below is an example of a mutation hint template for inserting a branch structure:

[0053] [Task]

[0054] Insert a branch statement such as `if`, `if-else`, `switch`, or other conditional constructs into the provided C code.

[0055] [Instructions]

[0056] [1]Utilize existing variables or expressions for the conditions and code blocks.

[0057] [2]It is best to insert at a location with complex control flow.

[0058] [3]If necessary, declare and initialize new variables.

[0059] [4]The response should contain only code.

[0060] [Code]

[0061] / / Seed program

[0062] The [Instructions] section specifies concrete instructions. Instruction 1 requires reusing existing variables or expressions to construct the conditions and code blocks of conditional statements, in order to enhance the data flow complexity of the variant program. Instruction 2 requires that insertions be made preferably at locations with complex control flow, in order to enhance the control flow complexity of the variant program. Instruction 3 requires that new variables be declared and initialized if necessary, in order to prevent the number of existing variables or expressions from being too small to construct a complete conditional statement code block. Instruction 4 requires that the response of the large language model contain only code, in order to reduce the overhead caused by the number of tokens and facilitate the extraction of code from the response.

[0063] Step S1.3: Employ a diversity-guided mutation operator selection algorithm to mutate the seed program in a more diverse direction. In this algorithm, each mutation operator is assigned a score, which is determined by the diversity of the programs it generates, i.e., the average inter-program distance produced when generating program variants. The algorithm uses the Markov chain Monte Carlo method to sample mutation operators.

[0064] The distance Dist between two programs P1 and P2 is the Jaccard distance between them, calculated using the following formula:

[0065]

[0066] Among them, Stmt P1 and Stmt P2 Let P1 and P2 represent the line-level code sets of programs P1 and P2, respectively. These sets are formed by dividing the program into lines, and each element in the set is a line of code.

[0067] The algorithm uses the Markov chain Monte Carlo method to sample mutation operators based on their scores. The formula for calculating the mutation operator score, Score(·), is as follows:

[0068]

[0069] Where Mut represents the mutation operator, n is the number of program variants obtained by Mut mutation, and Dist(P) i ,P i-1 ) represents the inter-program distance generated by the i-th mutation. Note that P here... i-1 It could be a seed program or a variant program, because here we are iterating and mutating a seed program multiple times. Rate(Mut) represents the probability that the Mut mutation will produce a valid variant program.

[0070] Step S1.4: Update the mutation prompt template based on error message feedback.

[0071] To ensure the effectiveness of test cases, test cases that fail the compiler frontend and their error messages are used as feedback to update the mutation suggestion template for the corresponding mutation operator. If a test case encounters a syntax error during compilation, i.e. fails the compiler frontend, a new section, "[example to avoid syntax error]", will be added to the mutation suggestion template. This section will be filled with the test case and the error message generated during compilation, thereby prompting the large language model to avoid generating the same type of error.

[0072] Step S2: Insert output statement

[0073] To enable variant programs to detect the most harmful compiler errors, namely silent compilation errors, a large language model is used to insert output statements into the variant programs to monitor local and global variables. Variant programs with output statements become test cases. The specific process is as follows: For a variant program generated by the large language model, a prompt for inserting an output statement is generated. This prompt template consists of four parts: [Task], [Instruction], [Example], and [Code]. [Example] is a sample code snippet used to teach the large language model how to insert output statements into the variant program, and [Code] is the variant program for which the output statement needs to be inserted. An example of the prompt template for inserting an output statement is shown below:

[0074]

[0075]

[0076] }

[0077] [Code]

[0078] / / Variant program

[0079] Step S3, Differential Test

[0080] Test cases are used to test the aero-engine compiler under test to detect backend errors. In this embodiment, test cases are used to perform random difference testing and different optimization level testing on the C compiler under test. Random difference testing belongs to cross-compiler scenario testing, while different optimization level testing belongs to cross-optimization level scenario testing. Here, random difference testing is generally used to find errors at the compiler's -O0 optimization level, while different optimization level testing is generally used to find errors at other optimization levels besides -O0.

[0081] During the implementation of this invention, five types of miscompilation errors were discovered in a cross-compiler for an aero-engine. Among them, three of these errors are the most harmful silent miscompilation errors. The discovery of these three errors is attributed to the role of the insert output statement step.

[0082] Based on the same inventive concept, embodiments of the present invention also provide a fuzzy testing system for an aero-engine compiler based on a large language model, comprising:

[0083] The program mutation module collects embedded code from various aero-engine structures and C compiler test suites as a seed program library. Based on the control structure and data flow characteristics of the aero-engine code, it designs various mutation operators and their corresponding mutation hint templates, including insertion mutation operators, deletion mutation operators, and replacement mutation operators. The mutation hint template initially contains three parts, indicating the mutation task, specific instructions, and the seed program to be mutated. The insertion mutation hint template requires the large language model to be inserted at the most complex position in the data flow and reuses existing variables to enhance data flow complexity. A diversity-guided mutation operator selection algorithm is employed to mutate the seed program in more diverse directions. In this algorithm, each mutation operator is assigned a score, determined by the diversity of the programs it generates, i.e., the average inter-program distance generated when generating program variants. The algorithm uses a Markov chain Monte Carlo method to sample mutation operators. Test cases that fail the compiler front-end and their error messages are used as feedback to update the mutation hint template of the corresponding mutation operator.

[0084] The output statement insertion module is used to insert output statements into variant programs using a large language model to monitor local and global variables. Variant programs with output statements become test cases. The prompt template for inserting output statements includes the task, specific instruction example code, and the variant program that needs to have the output statements inserted.

[0085] The differential testing module is used to test the aero-engine compiler under test using test cases and to detect backend errors in the compiler.

[0086] This invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned fuzz testing method for an aero-engine compiler based on a large language model. The program / instruction code for implementing the method of this invention can be written in any combination of one or more programming languages. This program / instruction code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program / instruction code causes the steps of the method of this invention to be implemented. The program / instruction code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a standalone software package, or entirely on a remote machine or server. All aspects not detailed in this invention are well-known to those skilled in the art.

[0087] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A fuzz testing method for an aero-engine compiler based on a large language model, characterized in that, Includes the following steps: Collect embedded code for various aero-engine structures and C compiler test suite programs as a seed program library; Based on the control structure and data flow characteristics of aero-engine codes, various mutation operators and their corresponding mutation prompt templates are designed, including insertion mutation operators, deletion mutation operators, and replacement mutation operators. The mutation prompt template initially contains three template parts, which respectively indicate the mutation task, the specific instructions, and the seed program to be mutated. The insertion mutation prompt template requires the large language model to be inserted at the most complex position in the data flow and reuses existing variables to enhance the complexity of the data flow. A diversity-guided mutation operator selection algorithm is adopted to mutate the seed program in a more diverse direction. In the algorithm, a score is assigned to each mutation operator, and the score of the mutation operator is determined by the diversity of the programs it generates, i.e., the average inter-program distance generated when generating program variants. The algorithm uses the Markov chain Monte Carlo method to sample mutation operators. By using a large language model to insert output statements into variant programs to monitor local and global variables, variant programs with output statements become test cases. The prompt template for inserting output statements includes the task, specific instruction example code, and variant programs that require the insertion of output statements; The test cases that fail the compiler front-end and their error messages are used as feedback to update the mutation prompt template of the corresponding mutation operator; Test cases are used to test the aero-engine compiler under test and detect backend errors in the compiler.

2. The fuzz testing method for an aero-engine compiler based on a large language model according to claim 1, characterized in that, The various mutation operators include insertion mutations such as insertion of branch structures, insertion of loop structures, insertion of control statements, insertion of structures, insertion of expressions, and insertion of dead code; The replacement mutation operators include replacing certain constants, certain variables, certain operators, and the rvalues ​​of assignment statements; the deletion mutation operators include removing quantifiers, removing modifiers, and removing certain unary operators.

3. The fuzzy testing method for an aero-engine compiler based on a large language model according to claim 1, characterized in that, The specific instructions in the insertion mutation prompt template include: Instruction [1] requires that when constructing branch statements, the variables or expressions already existing in the seed program be reused in the branch conditions and code blocks; Instruction [2] requires that the insertion be performed at the position with high control flow complexity; Instruction [3] requires that new variables be defined for use in the branch statement code blocks when there are insufficient variables already existing in the seed program.

4. The fuzz testing method for an aero-engine compiler based on a large language model according to claim 3, characterized in that, For the insertion mutation operator, the [Instructions] in the mutation prompt template include: [1] Utilize existing variables or expressions for <insertion structure>; [2] It is best to insert atalocation with complex control flow; [3] If necessary, declare and initialize new variables; [4] The response should contain only code; For the replacement mutation operator, the [Instructions] in the mutation prompt template include: [1] If there are no <replacement object> to replace, respond with the original code I provide; [2] The response should contain only code; For the deletion mutation operator, the [Instructions] in the mutation prompt template include: [1] If there are no <object to be deleted> to remove, respond with the original code I provide; [2] The responses should contain only code; The <insertion structure>, <replacement object>, and <deletion object> correspond to the description strings in the operators.

5. The fuzzy testing method for an aero-engine compiler based on a large language model according to claim 1, characterized in that, The distance Dist between two programs P1 and P2 is the Jaccard distance between them, calculated using the following formula: Among them, Stmt P1 and Stmt P2 Let P1 and P2 represent the line-level code sets of programs P1 and P2, respectively. These sets are formed by dividing the program into lines, and each element in the set is a line of code.

6. The fuzz testing method for an aero-engine compiler based on a large language model according to claim 1, characterized in that, The algorithm is based on the Markov chain Monte Carlo algorithm, which selects the mutation operator according to its score ranking; the formula for calculating the mutation operator score (·) is as follows: Where Mut represents the mutation operator, n is the number of program variants obtained by Mut mutation, and Dist(P) i ,P i-1 ) represents the inter-program distance generated by the i-th mutation. When i=1, P i-1 This is the seed program; Rate(Mut) represents the probability that the Mut mutation will produce a valid variant program, calculated as follows: Among them, #FrontPass Mut This indicates the number of variant programs generated using Mut that pass the compiler frontend, #All Mut This indicates the total number of variant programs generated using Mut.

7. The fuzz testing method for an aero-engine compiler based on a large language model according to claim 1, characterized in that, If a test case encounters a syntax error during compilation, meaning it fails to pass the compiler front-end, a new section will be added to the mutation warning template. This new section will contain the test case and the error message generated during compilation, thus prompting the large language model to avoid generating the same type of error.

8. The fuzz testing method for an aero-engine compiler based on a large language model according to claim 1, characterized in that, Test cases are used to test the C compiler under test, including random difference testing and different optimization level testing. Random difference testing is a test across compiler scenarios, while different optimization level testing is a test across optimization level scenarios. Random difference testing is used to find errors in the compiler at the -O0 optimization level, while different optimization level testing is used to find errors in the compiler at other optimization levels besides -O0.

9. A fuzzy testing system for an aero-engine compiler based on a large language model, characterized in that, include: The program mutation module is used to collect embedded code for various aero-engine structures and C compiler test suite programs as a seed program library; Based on the control structure and data flow characteristics of aero-engine codes, various mutation operators and their corresponding mutation prompt templates are designed, including insertion mutation operators, deletion mutation operators, and replacement mutation operators. The mutation prompt template initially contains three template parts, which respectively indicate the mutation task, the specific instructions, and the seed program to be mutated. The insertion mutation prompt template requires the large language model to be inserted at the most complex position in the data flow and reuses existing variables to enhance the complexity of the data flow. The algorithm employs a diversity-guided mutation operator selection algorithm to mutate seed programs in a more diverse direction. In this algorithm, a score is assigned to each mutation operator, and the score of the mutation operator is determined by the diversity of the programs it generates, i.e., the average inter-program distance generated when generating program variants. The algorithm uses the Markov chain Monte Carlo method to sample mutation operators. The test cases that fail the compiler front-end and their error messages are used as feedback to update the mutation prompt template of the corresponding mutation operator; The output statement insertion module is used to insert output statements into variant programs using a large language model to monitor local and global variables. Variant programs with output statements become test cases. The prompt template for inserting output statements includes the task, specific instruction example code, and variant programs that require the insertion of output statements; The differential testing module is used to test the aero-engine compiler under test using test cases and to detect backend errors in the compiler.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the fuzz testing method for an aero-engine compiler based on a large language model as described in any one of claims 1-8.

Images