A method and apparatus for automatically generating high-quality micro-benchmark performance tests based on large language models

Abstract

This invention discloses a method and apparatus for automatically generating high-quality micro-benchmark performance tests based on a large language model (JMH). The method includes: parsing the target project and constructing a system dependency graph; analyzing the call chain of the target method using the system dependency graph to guide the large language model in performance classification based on the source code in the call chain; matching the performance classification results with predefined JMH parameter templates; integrating the context information and parameter templates of the target method to effectively guide the large language model in generating high-quality micro-benchmark performance test cases; parsing the successfully generated micro-benchmark performance test code, using a bad practice rule base to detect bad practices in the code, and fixing the bad practice code, ultimately generating efficient and accurate test cases. This invention outperforms traditional methods in terms of test generation accuracy and performance defect detection capabilities, and the generated test cases have significant advantages in understandability and performance measurement accuracy.

Description

Technical Field

[0001] This invention belongs to the field of automatic generation technology of micro-benchmark performance tests, and in particular relates to a method and apparatus for automatic generation of high-quality micro-benchmark performance tests based on a large language model. Background Technology

[0002] In the hierarchy of software testing, performance testing is a crucial component in ensuring the stability and efficient operation of a system under real-world loads. Micro-benchmark performance testing is particularly critical, as it allows for in-depth analysis of the smallest units of code, helping developers identify potential performance issues early on, thereby improving the overall performance and stability of the software. However, writing high-quality micro-benchmark performance test cases is typically a time-consuming and complex process. Developers not only need extensive performance tuning experience but also must dedicate significant time to manually writing and adjusting test cases. This often leads to performance testing being overlooked in actual development.

[0003] To address this challenge, automated micro-benchmark test case generation technology has emerged. This technology aims to automatically generate test cases covering the critical performance paths of a system, ensuring that code performance can be fully evaluated and verified. This automatically generated micro-benchmark can significantly improve test coverage, reduce the burden on developers, and help teams identify performance bottlenecks and potential problems early, thereby greatly improving development efficiency and optimizing the overall performance of the software.

[0004] One common technique for automatically generating microbenchmark performance tests is ju2jmh. By building the ju2jmh framework, microbenchmark test cases can be automatically generated. This framework filters existing JUnit test suites for cases suitable for conversion into performance benchmarks and performs static and dynamic analysis to gain a deep understanding of the code structure and execution logic. Next, ju2jmh converts these JUnit tests into JMH (Java Microbenchmark Harness) benchmark tests, performing necessary optimizations and adjustments to ensure that the generated tests can effectively evaluate system performance.

[0005] As large language models gain increasing prominence in code understanding and generation, automatically generating micro-benchmark performance test cases based on large language models has become a highly promising technical approach. By converting the contextual information of the method under test into cue words and using large language models to generate corresponding micro-benchmark test cases, this method can generate clearer and more accurate performance test code while ensuring high test coverage.

[0006] However, while existing ju2jmh frameworks have achieved a certain degree of automatic conversion from JUnit tests to micro-benchmark performance tests, they have significant shortcomings in key areas such as parameter optimization, test case optimization, and target method context information processing. These limitations not only narrow the application scenarios of existing methods but also negatively impact the accuracy of generated micro-benchmark performance tests and the precision of performance evaluation. Current automatic micro-benchmark performance test generation methods still face the following pressing technical challenges:

[0007] 1. Inability to automatically analyze and generate the most suitable JMH parameters. The ju2jmh framework cannot automatically analyze and generate the most suitable JMH parameters (such as the number of measurements, warm-up time, number of threads, etc.) based on the characteristics of the target code. These parameters are crucial to the accuracy and performance evaluation of micro-benchmarks, but existing methods rely on the default parameters of the JMH framework and lack automated optimization schemes, which may lead to deviations between performance test results and actual conditions, making it difficult to accurately reflect the true performance of the system.

[0008] 2. Insufficient unit tests limit the scope of conversion. The ju2jmh framework relies on existing JUnit unit tests for conversion, but in many codebases, unit test coverage is low, or even completely lacking, making it impossible to achieve automatic generation and conversion of large-scale micro-benchmarks. Furthermore, even if the codebase contains unit tests, these tests are often not designed for performance evaluation and are insufficient to meet the needs of micro-benchmarking, thus limiting the applicability and practical effectiveness of the ju2jmh framework.

[0009] 3. Insufficient optimization of generated micro-benchmarks. While ju2jmh can generate micro-benchmarks, its optimization capabilities are limited, failing to ensure that the generated test cases effectively identify performance bottlenecks and provide accurate performance assessments. Because the framework primarily relies on static and dynamic analysis, it cannot fully consider complex performance influencing factors such as multi-threaded scheduling and caching mechanisms. Therefore, the generated benchmarks may not accurately reflect the system's performance in complex scenarios. Furthermore, the lack of further optimization of test cases can lead to biased test results, limiting the application of ju2jmh in high-precision performance evaluation.

[0010] 4. Lack of contextual information analysis. The ju2jmh framework directly converts JUnit unit tests into micro-benchmarks, but fails to adequately consider the dependencies and contextual information within the test methods. This direct conversion method ignores the actual execution environment of the test cases and their related dependencies, potentially leading to micro-benchmarks that do not accurately reflect the system's performance in real-world usage scenarios. Due to the lack of comprehensive analysis of the test context, the generated micro-benchmarks may not effectively evaluate the system's performance characteristics under complex dependencies, thus limiting the comprehensiveness and accuracy of the tests.

[0011] 5. Performance test biases caused by JMH bad practices. The current ju2jmh framework may not fully identify and avoid bad JMH practices when generating micro-benchmarks, such as incorrect use of return values, loop accumulation, and incorrect fork configuration. These issues may cause the JVM to perform unexpected optimizations, thus affecting the accuracy of benchmarks. Summary of the Invention

[0012] This invention provides a method and apparatus for automatically generating high-quality micro-benchmark performance tests based on a large language model. By analyzing various components of the project and combining in-depth analysis of the impact on the performance of the target code, it automatically generates highly readable and accurate micro-benchmark performance test cases.

[0013] A method for automatically generating high-quality micro-benchmark performance tests based on large language models includes the following steps:

[0014] (1) Perform static analysis on the target project and construct a system dependency graph to identify the context information of the target method;

[0015] (2) Based on the system dependency graph, the call chain of the target method is parsed to obtain the source code of all directly and indirectly called methods in the call chain;

[0016] (3) Using the context information of the target method and the source code in the call chain, guide the large language model to perform performance analysis, identify code that has a significant impact on performance, and classify it according to predefined performance categories;

[0017] (4) Match the performance classification results with the predefined JMH parameter template and select the appropriate test parameter configuration;

[0018] (5) Integrate the contextual information of the target method, performance classification results and test parameter configuration to generate prompt words for input large language model;

[0019] (6) Use the generated prompt words to guide the large language model to generate micro-benchmark performance test cases;

[0020] (7) Verify and fix the generated micro-benchmark performance test cases to ensure their accuracy and operability;

[0021] (8) Pre-set a rule base, which stores a series of code rules for JMH bad practices. Match the micro-benchmark performance test code verified and repaired in step (7) with the rule base to detect bad practices in the test cases;

[0022] (9) Using a large language model combined with the prompts and repair examples corresponding to the pre-defined bad practice categories, the bad practice code detected in the test cases is repaired, and the repaired micro-benchmark test is verified again to ensure its accuracy and operability.

[0023] The specific process of step (1) is as follows;

[0024] (1-1) Use a static parser to read all source code files of the target project for structural analysis. Based on this, generate the corresponding Abstract Syntax Tree (AST) to effectively represent the structure of the code, including elements such as classes, methods, and variables. Next, extract all classes and methods defined in the project from the AST, record the name, location, and methods contained in each class, and record the signature and class of each method.

[0025] (1-2) Analyze the dependencies between classes. By examining inheritance relationships and implemented interfaces, identify the relationships between parent and child classes and construct the class inheritance hierarchy. During the analysis, the system extracts method call relationships, searches for method call nodes in the Abstract Syntax Tree (AST), identifies the call relationships between methods, and records other methods called internally by each method, including methods within the same class, methods in the parent class, implemented interface methods, and methods from external libraries. This process generates a complete call chain, showing the relationship between the target method and other methods. Simultaneously, the system records the construction order of classes and methods to ensure understanding of the initialization order of classes and the call order of methods in the project.

[0026] (1-3) Integrate the identified classes and methods and their dependencies to generate a system dependency graph. The nodes in the graph represent classes and methods, and the edges represent the dependencies between them.

[0027] (1-4) Traverse the system dependency graph constructed in (1-3) and analyze the context information of the target method, including the class construction order, method call hierarchy, inheritance relationship, and interface implementation. To reduce code interference unrelated to the target method, the system deletes AST nodes that are not involved, and the remaining AST nodes are re-parsed into source code fragments.

[0028] The specific process of step (2) is as follows:

[0029] (2-1) Identify all calling nodes directly connected to the target method through the system dependency graph constructed in step (1); these calling nodes correspond to the methods directly called by the target method in its implementation process.

[0030] (2-2) Analyze the call nodes in step (2-1) to extract their related call relationships and form a complete call chain. During this process, the system will recursively analyze each call node to identify other methods it calls, including methods in the same class, methods in the parent class, implemented interface methods, and methods from external libraries. For each called method, the system will further trace its call chain to obtain all directly and indirectly dependent methods, ensuring the completeness and accuracy of the call chain.

[0031] (2-3) Extract the implementation code corresponding to all call nodes related to the target method from the source code based on the call chain.

[0032] The specific process of step (3) is as follows:

[0033] The source code in the call chain obtained in step (2) is input into the large language model to conduct in-depth analysis of the performance characteristics of this code during execution. During the analysis, the large language model identifies code segments that have a significant impact on performance based on the context information of the call chain, including potential performance bottlenecks, memory usage, and I / O operations. Subsequently, the identified code segments are classified according to predefined performance categories, such as CPU-intensive, I / O-intensive, and memory-intensive.

[0034] The specific process of step (4) is as follows:

[0035] Based on the performance categories identified in step (3), predefined corresponding JMH parameter templates are extracted. Different performance categories correspond to different parameter templates, and these templates set the optimal parameter combinations and configurations according to the performance characteristics of the code. For example, for CPU-intensive code, the system uses a predefined template containing the threads and forks parameters to evaluate CPU performance in multi-threaded and multi-process scenarios; for I / O-intensive code, the system uses a template containing the warmup-Iterations and measurement-Iterations parameters to ensure that I / O operations are tested in a stable state.

[0036] The specific process of step (5) is as follows:

[0037] The context information of the target method obtained in step (1) is combined with the JMH parameter template matched in step (4) to construct a prompt word. The prompt word contains the context information of the target method, the performance classification result and the corresponding test parameter configuration, so as to ensure that the large language model can accurately understand the running environment and performance requirements of the method when generating micro-benchmark performance test code, thereby improving the pertinence and effectiveness of the generated test code.

[0038] The specific process of step (6) is as follows:

[0039] The constructed prompts are input into a large language model to guide it in generating micro-benchmark performance test code based on the context information of the target method, the call chain source code, and relevant test parameter configurations. This ensures that the generated test cases effectively reflect the performance characteristics of the target method and include appropriate test parameters for accurate performance evaluation in subsequent verification and optimization phases.

[0040] The specific process of step (7) is as follows:

[0041] First, the generated test cases undergo preliminary syntax checking to ensure the code conforms to syntax standards. After passing the syntax check, the test cases are compiled and tested to verify that the code can be compiled successfully without errors. After successful compilation, the test cases are executed to detect whether there are any errors or exceptions during runtime. For test cases that fail verification, they will enter the repair phase, prioritizing the resolution of syntax, compilation, or runtime errors. During the repair process, heuristic methods are used for automatic repair, or error information is fed back to the large language model for further code adjustments. Through multiple iterations of correction, the generated micro-benchmark performance test cases are ensured to run stably and accurately evaluate the performance of the target method.

[0042] The specific process of step (8) is as follows:

[0043] The system pre-builds a rule base that covers bad practices in different coding patterns, such as ignoring return values, loop accumulation, and incorrect fork configurations, and defines the corresponding detection standards for these bad practices. Subsequently, the system scans and parses test cases, analyzing their code structure and logic, and matching them against the bad practice entries in the rule base to identify the characteristics and categories of bad practices within the test cases.

[0044] The specific process of step (9) is as follows:

[0045] Based on the identified bad practices, the system combines the JMH benchmark code with preset bad practice warning words and remediation examples. Through this combination, the system generates a remediation request for the specific JMH bad practice and inputs it into the large language model to guide the model in generating the corresponding corrective code. After generating the remediation code, the system verifies the improved micro-benchmark code to ensure it meets best practice standards and is operable.

[0046] A high-quality micro-benchmark performance test automatic generation device based on a large language model includes a memory and one or more processors. The memory stores executable code and a rule base for bad practices in micro-benchmark performance testing. When the one or more processors execute the executable code, they are used to implement the above-mentioned high-quality micro-benchmark performance test automatic generation method.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. This invention, through the deep analysis capabilities of large language models, can automatically generate the most suitable JMH parameters based on the characteristics of the target code, such as the number of measurements, warm-up time, and number of threads. This parameter optimization mechanism based on code context overcomes the limitations of existing methods that rely on default parameters, significantly improves the accuracy of micro-benchmark results, and can more realistically reflect the performance of the target code.

[0049] 2. This invention no longer relies on existing unit tests and their coverage in the project. Instead, it generates high-quality micro-benchmark performance test cases through a large language model. Even when unit tests are insufficient or completely missing in the project, this invention can still automatically analyze the code structure and generate micro-benchmark test cases that meet performance evaluation requirements, thereby expanding the applicability of test transformation and overcoming the limitations of existing methods when unit test coverage is insufficient.

[0050] 3. This invention, through a comprehensive analysis of the target code's contextual information, dependencies, and execution environment, generates micro-benchmark performance test cases that more accurately reflect the code's performance characteristics. By introducing a large language model, this invention overcomes the shortcomings of existing methods in contextual information analysis, ensuring that the generated micro-benchmark performance tests can comprehensively evaluate the system's performance under complex dependencies, significantly improving the comprehensiveness and accuracy of the tests.

[0051] 4. This invention effectively improves upon bad practices in generating test code from large models, ensuring the accuracy of benchmark tests, avoiding performance measurement errors caused by JVM optimization, further enhancing the reliability of test results, providing developers with more instructive performance evaluation criteria, and promoting efficient and accurate code optimization and performance tuning. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating an automatic generation method for high-quality micro-benchmark performance tests based on a large language model, according to an embodiment of the present invention.

[0053] Figure 2 This is a flowchart illustrating the process of parsing the target method context information in an embodiment of the present invention.

[0054] Figure 3 This is a flowchart of the target method call chain analysis in an embodiment of the present invention.

[0055] Figure 4 This is a flowchart illustrating the performance classification and JMH parameter template matching process in an embodiment of the present invention.

[0056] Figure 5 This is a flowchart illustrating the generation and optimization process of micro-benchmark performance test cases in an embodiment of the present invention.

[0057] Figure 6 This is a flowchart of the defect detection and repair process in an embodiment of the present invention. Detailed Implementation

[0058] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the embodiments described below are intended to facilitate the understanding of the present invention and do not constitute any limitation thereof.

[0059] like Figure 1 As shown, an automatic generation method for high-quality micro-benchmark performance tests based on a large language model includes the following steps:

[0060] S10: Parse the context information of the target method, the specific process is as follows: Figure 2 As shown.

[0061] S101: Analyze the system dependency graph and extract context information.

[0062] This invention first constructs a system dependency graph to identify the context information of the target method. The system dependency graph displays information such as relationships between classes, dependencies, and method calls within the project. This invention employs a static parser to comprehensively analyze the class containing the target method, its inheritance relationships, implemented interfaces, related member variables, and inner classes. This step aims to fully reveal the execution environment of the target method, ensuring that all relevant context information that may affect its execution is fully identified and extracted. Through this analysis, the information provided to the large language model will be more accurate, thereby helping it understand the overall execution logic of the target method.

[0063] S102: Filter irrelevant context information.

[0064] To improve the accuracy and effectiveness of generating micro-benchmark performance tests, this invention introduces a specific context filtering mechanism to remove context information irrelevant to the execution of the target method. The implementation steps of this filtering mechanism include: first, analyzing the dependencies of the target method to identify and remove classes or methods irrelevant to the testing objective; second, performing further static analysis on calls involving polymorphism to ensure that only context information directly affecting the execution of the target method is retained.

[0065] S103: Generate concise context information.

[0066] After filtering the context information, this invention integrates and simplifies the parsed context information to generate prompts for the input large language model. The generated prompts cover the source code of the target method, information about its class and dependent classes, information about dependent methods, and other necessary context information. This step ensures that the information in the prompts is both concise and comprehensive, aiming to help the large language model accurately call the target method when generating test cases, thereby improving the quality of micro-benchmark performance test generation.

[0067] S20: Analyze the method call chain and extract the source code of the methods in the call chain. The specific process is as follows: Figure 3 As shown.

[0068] S201: Generate a method call graph (CG).

[0069] This invention first performs static analysis on the project based on the system dependency graph parsed in step S10 to generate a call graph of the target method. This call graph shows the call relationships of all methods directly or indirectly called by the target method, including internal method calls and external library method calls. A key step in generating the call graph is parsing the method calls in the source code and tracing the definition location of each called method to accurately determine its dependencies.

[0070] S202: Extract the call chain of the target method.

[0071] By analyzing the generated call graph, the complete call chain of the target method is identified and extracted. The call chain shows all methods that the target method depends on during execution and their call order. This step helps identify relevant methods that may affect the performance of the target method and ensures that the complete call path is accurately recorded.

[0072] S203: Obtain the source code of each method in the call chain.

[0073] After determining the call chain of the target method, this invention uses a static parser to extract the complete source code of all methods in that call chain from the source code. The code of each called method is precisely located in the specific location of the source code file. The extracted source code includes the main implementation of each method, ensuring sufficient contextual information for subsequent performance analysis.

[0074] S204: Integrate the source code of the method in the call chain with the context of the target method.

[0075] To provide a comprehensive analytical context, this invention integrates the source code of each method in the extracted call chain with the source code of the target method. The integrated context includes not only the implementation details of the target method but also the code of all methods in its call chain. This context construction ensures that the performance analysis model can make accurate classification judgments based on the complete code environment.

[0076] S30: Performance classification of code snippets, the specific process is as follows: Figure 4 As shown.

[0077] S301: Performance classification based on call chain context.

[0078] This invention inputs the source code context of the target method and related methods in its call chain, extracted through static analysis, into a conversational large language model. This guides the model to automatically analyze and determine the performance category of the target code. The call chain context information includes the source code of the target method and the source code of related methods in the call chain, ensuring that the large language model can comprehensively analyze all performance characteristics that may be involved in the execution of the target code.

[0079] During the analysis, the large language model identifies the target code and its performance characteristics involved in the call chain based on contextual information, and classifies the code snippets according to performance features. Performance categories include, but are not limited to:

[0080] CPU-intensive code snippets: containing complex calculations, algorithms, or loop operations.

[0081] I / O intensive code snippets: These involve I / O operations such as file operations and network communication.

[0082] Memory-intensive code snippets: These involve frequent memory allocation and deallocation, or operations that manage large data structures.

[0083] This step aims to guide the subsequent generated tests, ensuring that the appropriate test parameter template is selected for the code snippets of each performance characteristic, in order to optimize the accuracy and efficiency of micro-benchmarks.

[0084] S302: Optimize JMH parameter templates based on performance classification.

[0085] This invention provides optimized JMH parameter templates for large language models, targeting code snippets of different performance categories. The parameters in these templates are designed for different performance operations, ensuring the model can generate more targeted micro-benchmark code and appropriately select JMH parameters and their optimal values.

[0086] For I / O-intensive code snippets, the template should preferably include the warmup-Iterations and measurement-Iterations parameters to ensure that I / O operations are tested in a stable state and to obtain accurate performance data.

[0087] For CPU-intensive code snippets, the template includes the `threads` and `forks` parameters to evaluate CPU performance in multi-threaded and multi-process scenarios.

[0088] For memory-intensive code snippets, the template adjusts the warmup-Iterations parameter to ensure the elimination of interference during the warmup phase and accurately reflect the actual performance of memory management operations.

[0089] S303: Integrate classification results with parameter templates and generate prompt words.

[0090] When generating micro-benchmark code, this invention integrates the performance classification information of the code snippets with the corresponding JMH parameter templates into prompt words, which are then input into the conversational large language model. The prompt words not only include source code information about the target method and its context, but also incorporate JMH parameters applicable to different performance characteristics.

[0091] S40: Combining the target code context generated in S10 and the JMH parameter template generated in S30, prompts the generation of test cases for the large language model. The specific process is as follows: Figure 5 As shown.

[0092] S401: Generate prompt words by combining the target method context and parameter template.

[0093] In this step, the present invention first combines the target method context generated in step S10 with the performance classification information and its corresponding JMH parameter template generated in step S30 as prompt words for generating micro-benchmark performance tests. The prompt words contain the target method context information and the JMH parameter template corresponding to the target method performance category, clarifying the parameters that need to be adjusted during the test and their specific configurations.

[0094] S402: Generate micro-benchmark performance test code.

[0095] The generated prompts are input into a conversational large language model. Based on the input context and parameter templates, the model generates responses containing initial micro-benchmark performance test cases. These test cases contain micro-benchmark code suitable for the target method, with parameter configurations based on the classification results and corresponding templates described in S302 above.

[0096] S403: Test Case Optimization and Verification.

[0097] This invention performs syntax checks, compiles, and executes the generated micro-benchmark performance test code. If errors are found during testing, the system will automatically repair and optimize them using heuristic methods or by leveraging a large language model. After multiple iterations of correction, validated micro-benchmark performance test cases are generated.

[0098] S50: Detection and remediation of bad practices, the specific process is as follows: Figure 6 As shown.

[0099] S501: Detecting bad practices in JMH benchmark code.

[0100] This invention constructs a rule base containing various JMH (Junior High Management Machine) bad practices. This rule base covers descriptions of bad practices for different code patterns and corresponding detection standards. Subsequently, the system parses the generated JMH benchmark test code and identifies code features related to bad practices by comparing them with entries in the rule base. If no bad practices are identified, the current micro-benchmark performance test case is considered a final verified micro-benchmark performance test case; if bad practices are identified, the process proceeds to step S502.

[0101] S502: Rectify bad practices.

[0102] After detecting bad practices in the JMH benchmark code, the method of this invention integrates predefined prompts and remediation examples with the JMH benchmark code. Specifically, this method generates remediation requests for the identified bad practice categories and inputs these requests into a large language model to guide the generation of corresponding corrective code. After the micro-benchmark performance test code is generated, the process returns to step S403 for optimization and verification.

[0103] Based on the same inventive principle, this invention also provides an automatic generation device for high-quality micro-benchmark performance tests based on a large language model, including a memory and one or more processors. The memory stores executable code and a rule base for bad practices in micro-benchmark performance testing. When the one or more processors execute the executable code, they are used to implement the high-quality micro-benchmark performance test automatic generation method of the above embodiments.

[0104] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for automatically generating high-quality micro-benchmark performance tests based on a large language model, characterized in that, Includes the following steps: (1) Perform static analysis on the target project and construct a system dependency graph to identify the context information of the target method; (2) Based on the system dependency graph, the call chain of the target method is parsed to obtain the source code of all directly and indirectly called methods in the call chain; (3) Using the context information of the target method and the source code in the call chain, guide the large language model to perform performance analysis, identify code that has a significant impact on performance, and classify it according to predefined performance categories; (4) Match the performance classification results with the predefined JMH parameter template and select the appropriate test parameter configuration; (5) Integrate the contextual information of the target method, performance classification results and test parameter configuration to generate prompt words for input large language model; (6) Use the generated prompt words to guide the large language model to generate micro-benchmark performance test cases; (7) Verify and fix the generated micro-benchmark performance test cases to ensure their accuracy and operability; (8) Pre-set a rule base, which stores a series of code rules for JMH bad practices. Match the micro-benchmark performance test code verified and repaired in step (7) with the rule base to detect bad practices in the test cases; (9) Using a large language model combined with the prompts and repair examples corresponding to the pre-defined bad practice categories, the bad practice code detected in the test cases is repaired, and the repaired micro-benchmark test is verified again to ensure its accuracy and operability.

2. The method for automatically generating high-quality micro-benchmark performance tests based on a large language model according to claim 1, characterized in that, The specific process of step (1) is as follows; (1-1) Use a static parser to read all source code files of the target project, and on this basis, generate the corresponding abstract syntax tree (AST) to effectively represent the structure of the code; Next, extract all classes and methods defined in the project from the Abstract Syntax Tree (AST); (1-2) Analyze the dependencies between classes. By examining the inheritance relationships and implemented interfaces between classes, identify the relationships between parent and child classes and construct the class inheritance hierarchy. (1-3) Integrate the identified classes and methods and their dependencies to generate a system dependency graph. The nodes in the graph represent classes and methods, and the edges represent the dependencies between them. (1-4) Traverse the system dependency graph constructed in (1-3) and analyze the context information of the target method, including the class construction order, method call hierarchy, inheritance relationship and interface implementation.

3. The method for automatically generating high-quality micro-benchmark performance tests based on a large language model according to claim 1, characterized in that, The specific process of step (2) is as follows: (2-1) Using the system dependency graph constructed in step (1), identify all calling nodes directly connected to the target method; these calling nodes correspond to the methods directly called by the target method during its implementation. (2-2) Analyze the calling nodes in step (2-1) to extract their related calling relationships and form a complete calling chain; (2-3) Extract the implementation code corresponding to all call nodes related to the target method from the source code based on the call chain.

4. The method for automatically generating high-quality micro-benchmark performance tests based on a large language model according to claim 1, characterized in that, The specific process of step (3) is as follows: The source code in the call chain obtained in step (2) is input into the large language model to conduct in-depth analysis of the performance characteristics of these codes during execution. During the analysis, the large language model identifies code segments that have a significant impact on performance based on the context information of the call chain. Subsequently, the identified code segments are classified according to predefined performance categories.

5. The method for automatically generating high-quality micro-benchmark performance tests based on a large language model according to claim 1, characterized in that, The specific process of step (4) is as follows: Based on the performance category identified in step (3), the predefined corresponding JMH parameter templates are extracted; different performance categories correspond to different parameter templates, and these templates set the optimal parameter combination and configuration according to the performance characteristics of the code.

6. The method for automatically generating high-quality micro-benchmark performance tests based on a large language model according to claim 1, characterized in that, The specific process of step (5) is as follows: The target method context information obtained in step (1) is combined with the JMH parameter template matched in step (4) to construct a prompt word; the prompt word contains the target method context information, performance classification results and corresponding test parameter configurations.

7. The method for automatically generating high-quality micro-benchmark performance tests based on a large language model according to claim 1, characterized in that, The specific process of step (7) is as follows: First, perform a preliminary syntax check on the generated test cases to ensure that the code conforms to the syntax rules; after passing the syntax check, compile and test the test cases to verify that the code can be compiled successfully and without errors; After successful compilation, execute the test cases to check for errors or exceptions during runtime; For test cases that fail verification, the process will proceed to the repair phase, prioritizing the resolution of syntax, compilation, or runtime errors. repair During the process, heuristic methods are used for automatic repair, or error information is fed back to the large language model for further code adjustments; Through multiple iterations and corrections, we ensure that the generated micro-benchmark performance test cases run stably and accurately evaluate the performance of the target method.

8. An automatic generation device for high-quality micro-benchmark performance tests based on a large language model, characterized in that, The method includes a memory and one or more processors, wherein the memory stores executable code and a base of bad practices rules for micro-benchmark performance testing, and the one or more processors execute the executable code to implement the high-quality micro-benchmark performance test automatic generation method according to any one of claims 1-7.

Images