Historical data generalization driven compiler system test program generation method and system

Abstract

The application discloses a historical data generalization driven compiling system test program generation method and system, and relates to the fields of software engineering and software testing. The method first acquires historical defect data containing defect reports, repair patches and historical test programs; then jointly analyzes the historical defect data by using a large language model to generate a compiling defect semantic language instance containing key feature metadata and a causal chain, realizes a leap from a historical syntax level to a semantic level; then generates a compilable code segment based on the instance; finally, adapts the segment to a context with a seed program, and inserts the segment into the seed program according to a syntax structure and a scope rule of the seed program to generate a synthetic test program. The application supports migration of defect triggering knowledge across compiling systems, can efficiently generate diversified test programs with error injection capability, and significantly improves defect detection efficiency and coverage of the compiling system.

Description

Technical Field

[0001] This invention relates to the fields of software engineering and software testing, and in particular to an automatic synthesis method and system for test programs of compilation systems (including compilers and Java Virtual Machines). Background Technology

[0002] The compiler system is a core infrastructure of modern software systems, responsible for converting high-level programs into executable code and performing complex performance optimizations. The compiler system encompasses static compilers (such as GCC and Clang) and dynamic runtime environments (such as the Java Virtual Machine, or JVM). The static compiler directly compiles the source code into machine code for the target platform; while the JVM uses a front-end compiler (such as javac) to compile Java source code into bytecode files, and then a back-end compiler (such as the JIT compiler) within the JVM compiles the bytecode into native code at runtime. During this process, the compiler system introduces numerous optimization methods, including loop unrolling, partial redundancy elimination, method inlining, constant propagation, and peephole optimization. These optimizations significantly improve program execution efficiency, but also make the internal logic of the compiler system extremely complex, making it prone to introducing defects. Defects in the compiler system can lead to serious consequences such as crashes, silent miscompilation, or incorrect execution results. Since the compiler system is responsible for the correct execution of all upper-level applications, its reliability is crucial to the entire software ecosystem. Therefore, ensuring the correctness and robustness of the compiler system is a key challenge.

[0003] To thoroughly test compiler systems, researchers have proposed numerous automated testing techniques. Based on different methodologies, existing techniques can be mainly categorized as follows: The first category is generation-based methods. These methods construct test programs based on syntax rules. For example, Csmith and YarpGen design generation strategies based on C / C++ syntax rules to randomly construct test programs that conform to syntax constraints for compiler testing; JavaFuzzer generates source code-level test programs based on Java syntax rules. However, rule-based generation methods have limited coverage of program patterns, and rule design costs are high.

[0004] The second category is mutation-based methods. These methods generate new test programs by mutating existing seed programs. For example, EMI (Equivalence ModuloInputs) constructs equivalent programs by deleting or modifying input-irrelevant code; classfuzz mutates bytecode files by setting numerous syntax mutation rules; and classming achieves semantic mutation by changing the data flow or control flow of the seed file. However, the mutation rules designed by these methods are relatively small, resulting in insufficient diversity of the generated test programs.

[0005] The third category is methods based on Large Language Models (LLM). In recent years, LLM has been introduced into the field of compiler system testing. For example, Fuzz4All uses auto-completion technology to generate multi-language test programs; WhiteFox extracts test requirements from compiler source code to guide the generation process. However, test programs directly generated by LLM have a low compilation success rate, and the generation process is computationally expensive.

[0006] The fourth category is history-driven methods. These methods guide the generation of new test programs by reusing information from historical defect-triggered test programs. For example, JavaTailor extracts code snippets from historical test programs and inserts them into seed programs; VECT vectorizes code snippets from historical programs and then matches and combines them; Jetris further abstracts code snippets into control flow / data flow patterns and instantiates them. History-driven methods demonstrate good performance in defect detection because they utilize experience from triggering real defects.

[0007] However, during the research process, the inventors discovered the following defects and shortcomings in existing history-driven methods: 1) Existing methods reuse historical defect information only at the syntactic level. They primarily extract surface-level code structures, such as control flow patterns, data flow patterns, or specific code snippets, from historically triggered test programs. However, the root causes of compiler system defects are semantic, involving incorrect assumptions about program behavior, improper optimization conditions, or subtle dependency inconsistencies. Simply reusing syntactic structures cannot capture the core semantic features that make test programs capable of causing errors.

[0008] 2) Syntactic reuse is tightly coupled to specific programming languages, making it impossible to transfer knowledge triggered by defects across compilation systems. For example, although the try-catch exception handling in Java and the setjmp non-local jump in C++ have completely different syntaxes, from the perspective of compiler optimization, both can introduce exception control flow edges, thus triggering similar optimization defects. Existing methods cannot identify and utilize this cross-language semantic equivalence.

[0009] Existing methods do not fully utilize the root cause information contained in defect reports and fixes. Defect reports contain diagnostic information from developers, while fixes expose intermediate representation nodes and optimization processes within the build system that involve failures. Relying solely on the code structure of the test program itself, ignoring these valuable root cause clues, limits the error-causing capabilities of the generated test programs. Summary of the Invention

[0010] To address the limitations of existing history-driven methods in terms of syntax reuse, cross-language transfer difficulties, and insufficient root cause utilization, this invention proposes a historical data-driven method and system for generating compiler test programs. By acquiring historical defect data, a large language model is used to analyze and generate compiler defect semantic language instances containing metadata and causal chains. Code snippets are then generated, adapted, and inserted into seed programs according to syntax rules to generate synthetic test programs. This achieves the goals of semantic reuse, cross-system transfer, and efficient test program generation.

[0011] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, the present invention implements a method for generating test programs for a compiler system driven by historical data generalization, comprising the following steps: Step 1: Obtain historical defect data, which includes at least defect report text, repair patches, and historical trigger test programs corresponding to the defect report text and repair patches; Step 2: Utilize a large language model to perform joint analysis on the historical defect data to generate a compilation defect semantic language instance; wherein, the compilation defect semantic language instance is a transferable semantic representation used to uniformly express the root cause of defect triggering, which includes at least metadata for characterizing key features of defect triggering, and a causal chain for describing how the key features interact to trigger the defect. Step 3: Based on the compiled defect semantic language instance, use a large language model to generate at least one compilable code fragment corresponding to the root cause of the defect triggering; Step 4: Adapt the code snippet to the preset seed program, and insert the adapted code snippet into the seed program according to the syntax structure and scope rules of the seed program to generate a synthetic test program for testing the compilation system.

[0012] Furthermore, in step 2, the joint analysis of the historical defect data using a large language model includes: The defect report text and the corresponding repair patch are concatenated into structured prompt words, which are then input into a pre-trained large language model. The model is guided by the thinking chain technique to extract key features and construct causal chains to generate the compilation defect semantic language instance.

[0013] Furthermore, in step 2, the compiler defect semantic language instance also includes alignment information, which is used to map key features in the metadata to the equivalent construction and instantiation strategy of the target compiler system, so as to realize defect-triggered knowledge transfer across compiler systems.

[0014] Furthermore, step 3 further includes: Sub-step 31: Input the compiled defect semantic language instance into the large language model to generate multiple candidate code fragments; Sub-step 32: Compile and verify the multiple candidate code segments, and store the valid code segments that pass the compilation verification into the code segment pool.

[0015] Furthermore, step 4 further includes: Sub-step 41: Randomly select a seed program from the seed program pool and randomly select a verified code fragment from the code fragment pool; Sub-step 42: Perform abstract syntax tree analysis on the seed program, enumerate all legal insertion points in the seed program, and randomly select an insertion point; Sub-step 43: In the context of the insertion point, perform literal substitution and / or definition write-back on the code fragment to bind the code fragment to the runtime state of the seed program, thereby completing context adaptation; Sub-step 44: Based on the syntax structure and scope rules of the seed program, insert the code segments that have completed context adaptation into the corresponding positions of the seed program according to their scopes to generate the synthetic test program.

[0016] Furthermore, in sub-step 43, the literal replacement includes: replacing the literal in the code segment with a type-compatible declared variable in the context of the seed program insertion point with a preset probability; the definition write-back includes: appending an assignment statement to the end of the code segment with a preset probability, and writing back the value of the newly defined variable in the code segment to the type-compatible variable in the seed program.

[0017] Furthermore, step 4 is followed by: Step 5: Execute the synthetic test program on multiple different compiler system implementations, and determine whether the synthetic test program exposes compiler system defects according to the preset inconsistency judgment rules.

[0018] Furthermore, step 5 is followed by: Step 6: For synthetic test programs that do not expose defects, put them back into the seed program pool for use in subsequent test iterations.

[0019] Between steps 5 and 6, there is also a step of analyzing the difference test results using a large language model to generate compilation defect semantic language instances.

[0020] Secondly, this invention also implements a compiler test program generation system driven by historical data generalization, comprising: Data acquisition module: used to acquire historical defect data, which includes at least defect report text, repair patches, and historical trigger test programs corresponding to the defect report text and repair patches; Semantic generalization module: used to perform joint analysis on the historical defect data using a large language model to generate compilation defect semantic language instances; wherein, the compilation defect semantic language instance is a transferable semantic representation used to uniformly express the root cause of defect triggering, which includes at least metadata for characterizing key features of defect triggering, and a causal chain for describing how the key features interact to trigger defects. Code instantiation module: used to generate at least one compilable code fragment corresponding to the root cause of the defect based on the compiled defect semantic language instance using a large language model; Program synthesis module: used to adapt the code snippet to a preset seed program, and insert the adapted code snippet into the seed program according to the syntax structure and scope rules of the seed program, so as to generate a synthetic test program for testing the compilation system.

[0021] Compared with existing technologies, the present invention achieves the following significant technical effects: By jointly analyzing defect reports, patch notes, and original test programs, the root causes and deep semantics of defect triggering are accurately extracted. Based on this generalization result, this invention can generate diverse and effective test programs offline and reorganize the triggering semantics of historical defects during iteration to generate test samples with high-order complexity. This mechanism breaks through simple structural variations, approximating complex edge scenarios in the real world, and significantly improving the depth and relevance of testing. By abstracting the underlying logic of historical defects into reusable semantic knowledge, this invention successfully expands the reuse boundaries of historical test assets. This not only significantly reduces the cost of reconstructing massive test cases for new compilation systems, but also significantly enriches the test corpus for cross-language compilation systems, making it possible to build a universal compilation quality assurance system.

[0022] By adopting an offline-online separation architecture, complex semantic generalization and seed selection are completed using historical data in the offline phase, effectively reducing the real-time computing pressure on the online testing environment. This ensures that high-intensity differential testing can be performed at extremely low cost in the online testing phase, while maintaining the defect triggering capability and diversity of the generated program. Ultimately, while ensuring a strong ability to probe the deep logic of the compiler (such as exception control flow handling in the optimization phase), a significant improvement in overall testing efficiency is achieved. Attached Figure Description

[0023] Figure 1This is an overall flowchart of the historical data generalization-driven compilation system test program generation method proposed in Embodiment 1 of the present invention.

[0024] Figure 2 This is a technical roadmap for the method in Example 1.

[0025] Figure 3 A system module diagram is generated for the test program of the compilation system driven by historical data generalization proposed in Embodiment 2 of the present invention.

[0026] Figure 4 Example diagram defining the abstract syntax for Compiler Defect Semantic Language (CBSL).

[0027] Figure 5 This is a CBSL example diagram illustrating a historical defect in JDK-8272562.

[0028] Figure 6 This is an example image of a Java history test program (JDK-8272562).

[0029] Figure 7 This is a simplified example diagram of a defect report and patch for JDK-8272562.

[0030] Figure 8 This is an example diagram of a C++ code snippet instantiated from a large language model.

[0031] Figure 9 This is a diagram of a C++ seed program example.

[0032] Figure 10 This is a sample image of the synthesized C++ test program. Detailed Implementation

[0033] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. These embodiments are implemented based on the technical solution of the present invention, providing detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0034] Example 1: As Figure 1 The diagram shows the overall flow of the historical data generalization-driven compiler test program generation method proposed in Embodiment 1 of the present invention. The method specifically includes the following steps: Step 1: Obtain historical defect data, which includes at least defect report text, repair patches, and historical trigger test programs corresponding to the defect report text and repair patches; This example selects a historical defect numbered JDK-8272562 from OpenJDK Jira. This defect has been fixed and includes a complete defect trigger test procedure, defect report text, and patch.

[0035] Step 2: Utilize a large language model to perform joint analysis on the historical defect data, generating a compiler defect semantic language instance. This instance is a transferable semantic representation used to uniformly express the root causes of defect triggering. It includes at least metadata characterizing key features of defect triggering, and a causal chain describing how these key features interact to trigger the defect. Specifically, it includes: 2.1: Defect Trigger Semantic Generalization. This invention utilizes a large language model to jointly analyze the aforementioned defect reports, patch remediation, and triggering test programs, generating Compiled Defect Semantic Language (CBSL) instances, such as... Figure 4 The image shows an example of the abstract syntax definition for the Compiler Defect Semantic Language (CBSL). This instance serves as a transferable, unified semantic representation used to standardize the expression of the root causes of defect triggers. It comprises at least two parts: metadata characterizing the key features of the defect, and causal chains describing the interactions between these features.

[0036] 2.2: Code Snippet Instantiation. Based on the CBSL example above, this invention utilizes a large language model to generate specific code snippets for the C++ target, guided by causal chains and mapped by alignment information.

[0037] 2.3: Code Snippet Verification. This invention uses GCC and Clang to compile and verify the generated C++ code snippets. Code snippets that pass the compilation check will be retained in a dedicated C++ code snippet pool for later use.

[0038] Specifically, in step 2, the joint analysis of the historical defect data using a large language model includes: The defect report text and the corresponding repair patch are concatenated into structured prompt words, which are then input into a pre-trained large language model. The model is guided by the thinking chain technique to extract key features and construct causal chains to generate the compilation defect semantic language instance.

[0039] The aforementioned thought chain and causal chain have different technical implications: The aforementioned thought chain is a prompt word engineering technology that guides a large language model to perform step-by-step reasoning, and belongs to the model's input processing strategy; The causal chain is a structured data component that constitutes a Compiler Defect Semantic Language (CBSL) instance, output after reasoning by a large language model, and is an output product of the model.

[0040] The relationship between the two is as follows: This invention uses the thinking chain technology to guide the large language model to reason, thereby generating CBSL instances containing causal chains.

[0041] The thought chain prompts are configured to guide the model in performing a reasoning process that includes input feature extraction, logical causal inference, and structured output. In step 2, to effectively guide the large language model to accurately extract the deep semantics triggered by defects and construct CBSL instances, the design of the thought chain prompts includes the following three core logical parts: Input parsing module: The prompts must clearly instruct the model to perform structured parsing of the input defect reports, patch files, and original test programs, extract key entities (such as defect IDs, affected function names, and line numbers) and their surface relationships, and establish an initial analysis context.

[0042] Logical reasoning module: The prompt words need to guide the model to make causal inferences based on the extracted surface features, simulate the compiler's execution process, identify the root cause of the defect, and sort out the complete logical evolution chain from the normal execution path to the defect triggering state.

[0043] Results formatting module: The prompt words must strictly define the format of the model output content. The model is required to convert the root cause and causal chain derived from the reasoning into a code representation that conforms to the CBSL syntax specification, ensuring that the output CBSL instance can accurately carry the triggering semantics of the original defect.

[0044] Step 3: Based on the compiled defect semantic language instance, use a large language model to generate at least one compilable code fragment corresponding to the root cause of the defect triggering; Specifically, step 3 further includes: Sub-step 31: Input the compiled defect semantic language instance into the large language model to generate multiple candidate code fragments; Sub-step 32: Compile and verify the multiple candidate code segments, and store the valid code segments that pass the compilation verification into the code segment pool.

[0045] Step 4: Adapt the code snippet to the preset seed program, and insert the adapted code snippet into the seed program according to the syntax structure and scope rules of the seed program to generate a synthetic test program for testing the compilation system.

[0046] 4.1: Seed Program Selection. This invention randomly selects a seed program from the GCC regression test suite, such as... Figure 7 As shown. The seed program contains a main function, which declares an integer variable n and a for loop. This invention uses ast-grep to analyze the seed program, enumerates the legal insertion points, and randomly selects an insertion point located within the for loop body of the main function.

[0047] 4.2: Context Adaptation. This invention adapts the execution context of code snippets as follows: Literal replacement: The array size in the original code snippet is 1. This method replaces it with an existing variable n of the same type in the seed program, binding the calculation of the code snippet to the runtime state of the seed program.

[0048] Write-back is defined as follows: the variable k generated in the code snippet is written back to the variable n in the seed program (i.e., the append statement `n= k`), so that the calculation inside the code snippet affects the subsequent loop execution of the seed program.

[0049] Step 4.3: Test Program Synthesis. Insert the adapted code snippets into the corresponding positions in the seed program to generate the final synthesized test program.

[0050] Specifically, in sub-step 43, the literal replacement includes: replacing the literal in the code segment with a type-compatible declared variable in the context of the seed program insertion point with a preset probability; the definition write-back includes: appending an assignment statement to the end of the code segment with a preset probability, and writing back the value of the newly defined variable in the code segment to the type-compatible variable in the seed program.

[0051] Step 4 is followed by: Step 5: Execute the synthetic test program on multiple different compiler system implementations, and determine whether the synthetic test program exposes compiler system defects according to the preset inconsistency judgment rules.

[0052] Step 5 is followed by: Step 6: For synthetic test programs that do not expose defects, put them back into the seed program pool for use in subsequent test iterations.

[0053] The above processing includes further testing, verification, and iterative optimization: I. Difference Test Execution. This invention compiles and executes the synthesized program on both GCC (using the -O2 optimization option) and Clang. Test results show that Clang performs normally, compiling and executing the program correctly, while GCC triggers an internal compiler error (ICE) during compilation. Based on the preset difference test judgment rules, the synthesized program is automatically marked as a potential defect.

[0054] II. Root Cause Analysis. Analysis revealed that the root cause of this GCC defect lies in the Partial Redundancy Removal (PRE) optimization phase: During PRE, GCC incorrectly generates reusable references to previously computed results associated with abnormal control flow, even though these results cannot be treated as ordinary values ​​due to their association with abnormal control flow. This inconsistency corrupts the internal representation and ultimately triggers an internal compiler error. Although this defect and JDK-8272562 involve different internal representation and optimization phases, they share the same fundamental cause: the optimization phase incorrectly handles the interaction between circular dependencies and non-standard control flow.

[0055] III. Vulnerability Reporting and Fixing. This flaw has been lurking in GCC since GCC 8.1 (2018) for many years. This invention submits a streamlined test procedure to the GCC developers, who have confirmed the flaw and fixed it.

[0056] IV. Seed Program Pool Update. During fuzzing iterations, all synthesized programs that pass all difference checks are recycled into the seed program pool. This allows subsequent iterations to combine multiple defect triggering semantics from different historical defects to generate more complex, higher-order test programs.

[0057] I (Difference Test Execution) corresponds to the difference test in step 5; Steps II (root cause analysis of defects) and III (vulnerability reporting and repair) together constitute the core generation and application process of Compiler Defect Semantic Language (CBSL) instances, which is the key link to achieve the purpose of this invention. Its technical essence is already covered in the technical solution of "generating compiler defect semantic language instances using a large language model"; it is executed between steps 5 and 6.

[0058] IV (Seed Pool Update) corresponds to the seed pool update in step 6.

[0059] Example 2: Figure 2 As shown, this invention provides a historical data generalization-driven compilation system test program generation system, which includes a data acquisition module 100, a semantic generalization module 200, a code instantiation module 300, and a program synthesis module 400 connected in sequence.

[0060] Data acquisition module 100: used to acquire historical defect data, the historical defect data including at least defect report text, repair patch, and historical trigger test program corresponding to the defect report text and repair patch; Semantic generalization module 200: used to perform joint analysis on the historical defect data using a large language model to generate a compilation defect semantic language instance; wherein, the compilation defect semantic language instance is a transferable semantic representation used to uniformly express the root cause of defect triggering, which includes at least metadata for characterizing key features of defect triggering, and a causal chain for describing how the key features interact to trigger the defect. Code instantiation module 300: Based on the compiled defect semantic language instance, it uses a large language model to generate at least one compilable code fragment corresponding to the root cause of the defect triggering; Program synthesis module 400: used to perform context adaptation between the code fragment and a preset seed program, and insert the adapted code fragment into the seed program according to the syntax structure and scope rules of the seed program, so as to generate a synthetic test program for testing the compilation system.

[0061] In summary, compared to existing compiler defect detection tools, this invention can learn the semantic-level error-causing logic from historical defects and supports cross-language compiler system migration to discover similar defects. Our team conducted long-term testing on six mainstream compiler implementations (GCC, Clang, Javac, ECJ, HotSpot, and OpenJ9) for two languages ​​(C++ and Java), detecting a total of 143 previously unknown defects, of which 111 have been confirmed by developers and 27 have been fixed. Of the 66 defects in GCC, 22 (33.3%) were marked as P2 or higher priority, with 6 of them being the highest priority P1, indicating that this invention can discover high-impact compiler system defects.

[0062] like Figure 3 The diagram illustrates the overall technical approach of this invention. The main body of this invention is divided into two stages: an offline stage and an online stage. Wherein: The offline phase is responsible for the generalization and instantiation of defect-triggered semantics, including the collection and preprocessing of historical defect data, semantic generalization based on a large language model (generating CBSL instances), instantiation of code snippets, and verification. The online phase is responsible for the synthesis and execution of the test program, including the selection of seed programs and enumeration of insertion points, context adaptation of code snippets, synthesis of the test program, execution of differential tests and defect determination, and updating of the seed program pool. This offline-online separation design places the computationally intensive semantic generalization and instantiation processes in the offline phase, allowing the online phase to perform only lightweight AST analysis operations without calling a large language model, thereby significantly improving the efficiency of fuzz testing. The key technologies in this method will be described in detail below.

[0063] The offline phase includes Issues, Bug Reports, the CBSL Corpus, and the Ingredient Pool. The following processes are implemented through the offline phase: Step 1: Historical defect data collection and preprocessing.

[0064] This invention collects historical defect data from two mainstream defect tracking systems: GCC Bugzilla (for C / C++ compiler defects) and OpenJDK Jira (for Java compiler and JVM defects). For GCC Bugzilla, this invention selects defect reports marked as Internal Compiler Errors (ICE) or wrong codes; for OpenJDK Jira, this invention limits the collection to defects in the HotSpot Compiler component, focusing on JIT compilation-related defects, as such defects are more comparable between the two compilation systems. For both systems, this invention collects defect reports that have been confirmed and fixed by developers.

[0065] For the collected defect reports, this invention requires that each report must contain both the defect-triggered test program and the corresponding fix patch. Defects triggered by invalid input (such as syntax errors) are excluded because their triggering conditions are unstable during instantiation based on a large language model and are difficult to reliably migrate across compilation systems. Specifically, this invention found that both of the aforementioned major compilation systems use Git as a version control system and have strict requirements for the commit information of defect fixes, such as the requirement to include the defect number. Based on this detail, this invention utilizes historical Git information to establish a mapping relationship between defect reports and historical commits, filtering out defect reports without corresponding commits. For each retained defect, this invention aggregates all source code change patches associated with the report and stores the result as a pair of defect report text and fix patches. Due to the large number of candidate defect reports, this invention ultimately retains 500 most recently fixed defect reports from each defect tracking system, totaling 1000 defect reports as raw information for subsequent steps.

[0066] Step 2: Defect triggers semantic generalization.

[0067] This step is the core innovation of this invention. Given a historical defect record (containing defect report text, patch, and trigger test program), this invention uses a large language model to perform joint analysis on it, generalizing it into an instance of Compiled Defect Semantic Language (CBSL).

[0068] To effectively express defect-triggered semantics, this invention designs CBSL as a domain-specific language. CBSL should meet three design requirements: (1) be able to capture the root cause of historical defects rather than the surface structure; (2) be general enough to support migration across different compilation systems; and (3) be friendly to large language models and facilitate subsequent instantiation. Figure 3 It includes the abstract syntax definition of CBSL: CBSL consists of three main components: (1) Metadata: Represented as a set of feature entries, each entry records a symbolic feature identifier and its surface pattern and semantic intent, thereby capturing the key triggering semantic features of historical defects.

[0069] (2) Causal Chain: Represented as an ordered sequence of steps, each step encoding an action description, organizing the defect triggering process into a chain-like reasoning form, from feature declaration to defect triggering. The causal chain is not only used to explain why the defect is triggered, but also to guide the instantiation of subsequent code snippets.

[0070] (3) Alignment: Represented as a set of mappings, each mapping specifies the equivalent construct and corresponding instantiation strategy of a defect triggering feature in the target build system, thereby supporting cross-build system migration. Alignment is an optional component and is only used when cross-build system migration is required.

[0071] Specifically, in the semantic generalization process, this invention configures the large language model as a domain expert in the compiler system, guiding it to perform analysis in a chain-of-thought manner. The large language model first identifies key defect-triggered features from defect reports and patch notes, recording them as feature entries in metadata, including surface patterns and semantic intent. Subsequently, the large language model infers how these features interact to trigger defects, organizing the reasoning process into an ordered causal chain. When cross-compile system migration is required, the large language model further generates alignment information, mapping each feature to an equivalent construction and instantiation strategy in the target system.

[0072] The generated CBSL instances are validated by a Tree-sitter parser based on CBSL grammar, and only grammatically valid instances are retained in the offline semantic corpus.

[0073] Step 3: Instantiate the code snippet.

[0074] This step transforms each CBSL instance in the CBSL semantic corpus into one or more specific compilable code snippets, forming a code snippet pool. This step is also driven by a large language model: this invention uses techniques such as few-shot hints and thought chains to enable the large language model to realize defect triggering features based on metadata, maintain the interaction relationship between features according to causal chains, and instantiate code snippets across systems according to alignment information when needed.

[0075] Each generated code snippet is divided into three scope parts: f_import: File-level import or header file declaration; f_global: Global variable or function declaration; f_block: Executable code block; This partitioning supports independent validation and provides clear insertion boundaries for the online synthesis phase. To enhance diversity, this invention generates 10 candidate code snippets for each CBSL instance and encourages multiple variations of the same feature constructed using different APIs or syntaxes. During the generation of the large language model, a temperature parameter is set to 0.9 to balance diversity and quality.

[0076] Step 4: Code snippet verification and pooling.

[0077] Each generated code snippet is compiled and verified by the target compilation system. For C++ code snippets, GCC and Clang are used for compilation checks; for Java code snippets, Javac and ECJ are used for compilation checks. Only valid code snippets that pass the compilation verification are retained in the reusable code snippet pool. Based on the above process, this invention ultimately obtains a C++ code snippet pool (containing 3826 valid code snippets) and a Java code snippet pool (containing 5061 valid code snippets), which together constitute the code snippet pool.

[0078] The online phase includes potential bugs, adapted code snippets, code snippets (ingredients), test programs, and seed programs. The following processes are achieved through the online phase: Step 5: Seed program selection and insertion point enumeration.

[0079] This invention selects seed programs from two representative regression test suites: the GCC regression test suite (for C++ testing) and the OpenJDK HotSpot test suite (for Java testing). To reduce false positives, this invention first compiles and executes the programs in these test suites, filtering out programs that cannot be compiled or have exposed compilation / execution inconsistencies. After filtering, this invention obtains 3989 C++ seed programs and 783 Java seed programs.

[0080] In each test iteration, this invention randomly selects a seed program from the seed program pool and a verified code snippet from the code snippet pool. Subsequently, this invention uses the ast-grep tool to perform AST-level analysis on the seed program, enumerating all legal insertion points and randomly selecting one. The seed programs used in this invention are all in source code form, and the defined insertion position is within the block scope, such as function blocks, if condition blocks, for loop blocks, etc. This invention collects the block nodes in the seed program and randomly selects child nodes within each block as the insertion position.

[0081] Step Six: Code Snippet Context Adaptation.

[0082] This step is a key technology in the online phase, responsible for adapting the offline-generated code snippets to the selected seed program context. It specifically includes two sub-steps: (1) Literal Replacement: For each literal in the executable code block (f_block) of the code snippet, this invention searches for a type-compatible declared variable in the context of the insertion point of the seed program. If a type-compatible candidate variable exists, a candidate variable is randomly selected with probability p_replace to replace the literal. This replacement binds the computation of the code snippet to the runtime state of the seed program, enhancing the interaction between the code snippet and the seed program.

[0083] (2) Definition Writeback: For each newly defined variable in the executable code block (f_block) of the code segment, this invention searches for a type-compatible existing variable in the context of the insertion point of the seed program. If a match is found, a seed variable is randomly selected with probability p_write, and an assignment statement is appended to the end of the code segment to write back the newly generated value in the code segment to the variable in the seed program. This allows variables inside the code segment to affect the subsequent execution logic of the seed program.

[0084] In the implementation of this invention, the literal substitution probability p_replace and the definition write-back probability p_write are both set to 0.5.

[0085] Step 7: Test program integration.

[0086] Based on the code snippet after context adaptation in step six, this invention inserts it into the corresponding position of the seed program according to its scope: f_import is inserted into the file header (import region), f_global is inserted into the global declaration region, and f_block is inserted into the insertion point selected in step five. This produces a complete synthetic test program.

[0087] It is worth noting that, unlike existing methods, this invention operates on self-contained code snippets that have been verified offline during the online synthesis phase, rather than incomplete or partially valid code snippets. Therefore, online synthesis is simplified to lightweight context adaptation between the code snippet and the seed program. The entire online phase relies solely on lightweight AST analysis, without needing to invoke large language models, making it efficient and robust, applicable to various languages ​​under test (such as C++ and Java, as targeted in this invention), while preserving the semantics of code defect triggering.

[0088] Step 8: Perform difference testing and determine defects.

[0089] The synthesized test program is executed on multiple implementations of the target compilation system to determine whether it exposes defects. Specifically: For C++ test programs: compile and execute them separately on GCC and Clang. For Java test programs: compile on Javac and ECJ respectively, and execute on HotSpot and OpenJ9 respectively. This invention designs the following three inconsistency determination rules: (1) If one compilation system executes normally while another compilation system crashes (including internal compiler errors, assertion failures, segmentation faults, etc.) on the same test program, this invention considers it a potential defect.

[0090] (2) If the compilation results or execution results of different compilation systems are inconsistent on the same test program (such as one compiles successfully while the other fails to compile, or the two produce different running outputs), the present invention will treat the persistent inconsistency as a potential defect after filtering out non-deterministic outputs (such as timestamps, random numbers, thread information, etc.).

[0091] (3) If a compilation system hangs (timeout without termination) on the same test program, this invention will also mark it as a potential defect.

[0092] Test programs marked as potentially defective will be retained for further manual analysis, and after being streamlined to a minimum reproducible test program, they will be submitted to the developers of the corresponding compilation system for verification.

[0093] Step 9: Update the seed program pool.

[0094] For synthesized programs that pass all compilation and execution checks normally (i.e., do not trigger any inconsistent behavior), this invention puts them back into the seed program pool. In this way, subsequent test iterations can further synthesize based on the synthesized programs to generate higher-order test programs that contain multiple defect-triggered semantics from different historical defects, which is beneficial for generating more diverse and more error-prone test programs.

[0095] like Figure 5 The image shown is a simplified CBSL instance illustrating a historical defect in JDK-8272562. In this CBSL instance: The metadata section identified a key defect triggering feature, @DivergentExceptionFlow, whose surface pattern is a try-catch structure and whose semantic intent is "to create a conditionally unreachable path".

[0096] The causal chain organizes the defect triggering process into a four-step reasoning process: defining a loop that includes array access and index arithmetic (S1), wrapping it with exception control flow to introduce a branching path (S2), the compiler propagating value constraints between iterations (S3), and finally crashing due to IR inconsistency on unreachable paths (S4).

[0097] The alignment section specifies that when the target language under test is C++, the equivalent implementation of the @DivergentExceptionFlow feature is setjmp, with the strategy being "as an unreachable path". This is because setjmp introduces an exception control flow edge from the optimizer's perspective, similar to Java's try-catch functionality.

[0098] like Figure 6 The image shows an example of a Java history test program (JDK-8272562).

[0099] The program repeatedly calls the method f() within a loop. Inside f(), it accesses the array arr[j-1] through the loop index j and performs a division k / j. Because division can cause a division-by-zero exception, the code is wrapped in a try-catch block. The program is semantically valid and should terminate normally, but it triggers a crash in the HotSpot C2 JIT compiler.

[0100] like Figure 7 The image shows a simplified defect report and patch example for JDK-8272562. The upper part of the defect report () indicates that the failure manifests as a bad graph error in build_loop_late. The patch () Figure 7The second half shows the fix to prevent certain nodes carrying type information (IR nodes that record value range information) from being incorrectly propagated during loop unrolling. Specifically, array access arr[j-1] introduces boundary checks, based on which the compiler infers the constraint j-1>=0 and records it in the internal type constraint node; at the same time, the case of division operation j=0 introduces an exception branch (although this case is semantically unreachable). During loop unrolling, the compiler incorrectly copies the type constraint node into the exception branch, causing a contradiction between j-1>=0 and j=0, ultimately corrupting the IR graph and triggering a crash.

[0101] like Figure 8 The image shows a C++ code snippet instance after instantiating the large language model. This snippet is divided into three scopes: lines 1-2 are `f_import`, line 3 is `f_global`, and lines 5-10 are `f_block`. This snippet introduces exception control flow (corresponding to try-catch in Java) through `setjmp` on line 8, preserving the scope-constrained data dependency `arr[j-1]` (corresponding to the array access pattern in the original defect), thus reconstructing the same defect triggering condition in the C++ context.

[0102] like Figure 9 The example shown is a C++ seed program. This seed program contains a main function, which declares an integer variable `n` and a for loop. This invention uses ast-grep to analyze the seed program, enumerates valid insertion points, and randomly selects an insertion point located within the for loop body of the main function.

[0103] like Figure 10 The image shown is an example of the synthesized C++ test program, where the shaded area represents the inserted adaptation code snippet.

[0104] In summary, this invention proposes a method and system for generating compiler test programs driven by historical data generalization. First, by acquiring historical defect data and performing joint analysis using a large language model, a compiler defect semantic language instance containing metadata and causal chains is generated to uniformly express the root cause of defect triggering. Then, compilable code snippets are generated based on these instances. Finally, the code snippets are context-adapted with a seed program and inserted according to syntactic structure and scope rules to generate a synthetic test program. Thus, this invention achieves the goals of elevating historical defect reuse from the syntactic level to the semantic level, supporting cross-compile system defect triggering knowledge transfer, and efficiently generating diverse test programs with error-causing capabilities.

[0105] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0106] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

Claims

1. A method for generating test programs for a compiler system driven by historical data generalization, characterized in that, Includes the following steps: Step 1: Obtain historical defect data, which includes at least defect report text, repair patches, and historical trigger test programs corresponding to the defect report text and repair patches; Step 2: Utilize a large language model to perform joint analysis on the historical defect data to generate a compilation defect semantic language instance; wherein, the compilation defect semantic language instance is a transferable semantic representation used to uniformly express the root cause of defect triggering, which includes at least metadata for characterizing key features of defect triggering, and a causal chain for describing how the key features interact to trigger the defect. Step 3: Based on the compiled defect semantic language instance, use a large language model to generate at least one compilable code fragment corresponding to the root cause of the defect triggering; Step 4: Adapt the code snippet to the preset seed program, and insert the adapted code snippet into the seed program according to the syntax structure and scope rules of the seed program to generate a synthetic test program for testing the compilation system.

2. The method for generating test programs for a compiler system driven by historical data generalization according to claim 1, characterized in that, Step 2, which involves jointly analyzing the historical defect data using a large language model, includes: The defect report text and the corresponding repair patch are concatenated into structured prompt words, which are then input into a pre-trained large language model. The model is guided to extract key features and construct causal chains using the mind chain technique to generate the semantic language instance of the compilation defect. The mind chain prompt words are configured to guide the model to perform a reasoning process that includes input feature extraction, logical causal inference, and structured output.

3. The method for generating test programs for a compiler system driven by historical data generalization according to claim 1, characterized in that, In step 2, the compiler defect semantic language instance also includes alignment information. The alignment information is used to map the key features in the metadata to the equivalent construction and instantiation strategy of the target compiler system, so as to realize defect-triggered knowledge transfer across compiler systems.

4. The method for generating test programs for a compiler system driven by historical data generalization according to claim 1, characterized in that, Step 3 further includes: Sub-step 31: Input the compiled defect semantic language instance into the large language model to generate multiple candidate code fragments; Sub-step 32: Compile and verify the multiple candidate code segments, and store the valid code segments that pass the compilation verification into the code segment pool.

5. The method for generating test programs for a compiler system driven by historical data generalization according to claim 4, characterized in that, Step 4 further includes: Sub-step 41: Randomly select a seed program from the seed program pool and randomly select a verified code fragment from the code fragment pool; Sub-step 42: Perform abstract syntax tree analysis on the seed program, enumerate all legal insertion points in the seed program, and randomly select an insertion point; Sub-step 43: In the context of the insertion point, perform literal substitution and / or definition write-back on the code fragment to bind the code fragment to the runtime state of the seed program, thereby completing context adaptation; Sub-step 44: Based on the syntax structure and scope rules of the seed program, insert the code segments that have completed context adaptation into the corresponding positions of the seed program according to their scopes to generate the synthetic test program.

6. The method for generating test programs for a compiler system driven by historical data generalization according to claim 5, characterized in that, In sub-step 43, the literal replacement includes: replacing the literal in the code segment with a type-compatible declared variable in the context of the seed program insertion point with a preset probability; the definition write-back includes: appending an assignment statement to the end of the code segment with a preset probability, and writing back the value of the newly defined variable in the code segment to the type-compatible variable in the seed program.

7. The method for generating test programs for a compiler system driven by historical data generalization according to claim 5, characterized in that, Step 4 is followed by: Step 5: Execute the synthetic test program on multiple different compiler system implementations, and determine whether the synthetic test program exposes compiler system defects according to the preset inconsistency judgment rules.

8. The method for generating test programs for a compiler system driven by historical data generalization according to claim 7, characterized in that, Step 5 is followed by: Step 6: For synthetic test programs that do not expose defects, put them back into the seed program pool for use in subsequent test iterations.

9. The method for generating test programs for a compiler system driven by historical data generalization according to claim 7, characterized in that, Between steps 5 and 6, there is also a step of analyzing the difference test results using a large language model to generate compilation defect semantic language instances.

10. A compiler test program generation system driven by historical data generalization, characterized in that, include: Data acquisition module: used to acquire historical defect data, which includes at least defect report text, repair patches, and historical trigger test programs corresponding to the defect report text and repair patches; Semantic generalization module: used to perform joint analysis on the historical defect data using a large language model to generate compilation defect semantic language instances; wherein, the compilation defect semantic language instance is a transferable semantic representation used to uniformly express the root cause of defect triggering, which includes at least metadata for characterizing key features of defect triggering, and a causal chain for describing how the key features interact to trigger defects. Code instantiation module: used to generate at least one compilable code fragment corresponding to the root cause of the defect based on the compiled defect semantic language instance using a large language model; Program synthesis module: used to adapt the code snippet to a preset seed program, and insert the adapted code snippet into the seed program according to the syntax structure and scope rules of the seed program, so as to generate a synthetic test program for testing the compilation system.

Images