Efficient annotation-driven hierarchical fault positioning method

Abstract

The invention provides an efficient annotation-driven hierarchical fault positioning method, which comprises the following steps of: guiding a large language model to intelligently analyze and generate semantic annotations of a text based on a positioning algorithm of large model annotations, and performing fine-grained software fault positioning from a file level to a function level and then to a position level by using the annotation-based hierarchical positioning method. And an efficient hierarchical progressive sorting and screening algorithm is used for ensuring the software fault positioning efficiency. According to the method, the defect positioning efficiency can be ensured while the positioning accuracy is ensured. The positioning algorithm based on large model annotation strategically balances the information density, provides enough context clues, and improves the model understanding ability; according to the annotation-based hierarchical positioning method, positioning is divided into file / function / position hierarchies, and annotations are added by using a large model cue word project in sequence, so that the positioning accuracy is improved; and an efficient hierarchical progressive sorting and screening algorithm is provided, so that the optimal balance between the performance and the cost is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of software engineering, and relates to an efficient annotation-driven hierarchical fault localization method. BACKGROUND

[0002] If program repair can realize full-process automatic processing in real scenarios, the development and maintenance costs will be greatly reduced. However, the current large model (LLM) still has the problem of "illusion" (i.e. generating unreasonable content), which makes it necessary to rely solely on the large model to process the problem. Traditional methods such as model fine-tuning are still necessary, but the complexity of the actual application scenario far exceeds the processing of single-line code, which often involves cross-references of multiple files and multiple functions, as well as complex correlations between statements. Therefore, how to improve the fault localization accuracy and code repair accuracy in the case of cross-file and cross-module is crucial, which is also the main difficulty of the current automatic program repair technology. Among them, the fault localization accuracy is particularly critical, because the automatic defect repair framework usually adopts a three-stage process of fault localization, patch generation, and patch validation to achieve efficient processing, and accurate positioning results will significantly improve the quality of subsequent code generation.

[0003] However, the current positioning method still has the problem of low positioning accuracy, which is particularly prominent in complex large-scale code libraries in real scenarios. Since the current advanced method inputs a file name tree to the LLM based on the format of the project file directory, the information that can be used by the LLM for judgment is only the hierarchical structure between files and the names of each file. However, relying solely on "names" as a rough context provides too little information to support the LLM to make accurate judgments. Secondly, in hierarchical positioning, the positioning rate of the previous step affects the positioning effect of the next step, so it is crucial to improve the positioning rate of each part. Finally, the bloated data makes the program processing extremely time-consuming, and a suitable filtering method is needed to balance the positioning rate and time from file to line. SUMMARY

[0004] In view of the problems in the art, the application provides an efficient annotation-driven hierarchical fault localization method to fill the gap in the domestic and foreign fields. To this end, the method uses an annotation-based positioning algorithm based on a large model, uses LLM-supported prompt-based and embedding-based retrieval to perform more accurate fault localization; secondly, the method uses an annotation-based hierarchical positioning method to combine the previous method with hierarchical positioning; finally, in order to achieve an optimized balance between performance and cost, the method designs an efficient hierarchical progressive sorting and filtering algorithm in the algorithm. The application ensures the positioning accuracy while ensuring the efficiency of the positioning defects.

[0005] To achieve the above purpose, the technical scheme of the application is as follows:

[0006] An efficient annotation-driven hierarchical fault localization method, comprising:

[0007] Using a large model annotation-based positioning algorithm, using LLM-supported prompt-based and embedding-based retrieval to guide the large language model to intelligently analyze and generate semantic annotations of text;

[0008] Using an annotation-based hierarchical positioning method, the problem database is divided into file / function / location levels, and annotations are added in turn using carefully designed large model prompts, so that on the basis of generating text annotations, the large file information in the problem database is finely divided, from file-level annotations to function-level annotations, and then to location-level annotations;

[0009] Using an efficient hierarchical progressive ranking algorithm, the file is filtered layer by layer, and finally locked to the location line where the program defect may exist.

[0010] Further, the large model annotation-based positioning algorithm uses LLM-supported prompt-based and embedding-based retrieval to guide the large language model to intelligently analyze and generate semantic annotations of text, which specifically includes:

[0011] (1) When processing data, first input the original problem description and the code library into the system, and build a tree file hierarchy to illustrate the relative position of the file;

[0012] (2) Then, for the problem of lack of detailed source code information in the repository structure format, load the problem dataset, and use the abstract syntax tree to parse the python code;

[0013] (3) Reuse carefully designed prompts to guide the large language model to intelligently analyze and generate semantic annotations of text, thereby improving overall context awareness.

[0014] Further, in the process of generating semantic annotations of text, the following steps are also included: checking whether the output file exists, supporting continuing processing after interruption, ignoring files that cannot be read or parsed, and saving immediately after processing each file to prevent data loss.

[0015] Further, the annotation-based hierarchical positioning method divides the problem database into file / function / location levels, and adds annotations in turn using carefully designed large model prompts, so that on the basis of generating text annotations, the large file information in the problem database is finely divided, from file-level annotations to function-level annotations, and then to location-level annotations, which specifically includes:

[0016] (1) File-level positioning is performed, files are read, python code is parsed using an abstract syntax tree, and a large model is used to supplement additional text annotation information for explaining the function of each file;

[0017] (2) Function-level positioning is performed, and a prompt word is used to guide the large model to analyze each function in detail and generate a summary of the function description;

[0018] (3) Position-level positioning is performed, data flow analysis annotations are added, the scope of the problem is narrowed to position-level annotations, the LLM can track variable dependencies to obtain a deeper contextual understanding, and the LLM is allowed to find further details according to the selected variables, so as to find line-level faults.

[0019] Further, when performing file-level positioning, the annotation generation is dynamically generated during program execution, or pre-generated in batches and cached.

[0020] Further, the efficient hierarchical progressive sorting and screening algorithm is used to screen the files layer by layer, and finally lock to the position line where the program defect may exist, which specifically includes:

[0021] (1) 2K suspicious files are preliminarily screened out at the file level, and after file annotation, irrelevant files are removed through deep semantic analysis, and K key files most matching the problem description are selected for the next step;

[0022] (2) After adding annotations to the K files, M most core problem functions in these files are screened out;

[0023] (3) Only M most critical functions need to be added to the editing level flow annotation, and then the sorting operation is performed, so as to accurately screen out the specific line number that needs to be modified, and the detailed process flow annotation is added to them, and a more fine-grained editing position list is output, so as to accurately modify the code.

[0024] The beneficial effects of the present application are:

[0025] 1. The positioning algorithm based on large model annotation proposed by the present application strategically balances the information density, provides sufficient contextual clues, and improves the understanding ability of the model.

[0026] 2. The hierarchical positioning method based on annotation proposed by the present application divides the positioning into file / function / position levels, and adds annotations using carefully designed large model prompts, thereby solving the challenge of limited analysis ability of complex dependency relationships faced by existing methods, and improving the positioning accuracy.

[0027] 3. The high-efficiency hierarchical progressive sorting screening algorithm realizes the optimization balance of performance and cost, achieves higher hierarchical positioning accuracy and higher resolution rate while maintaining relatively low cost, and such optimization ensures calculation efficiency and preserves key context relationships essential for downstream function level and position level positioning tasks. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 The high-efficiency annotation-driven hierarchical fault positioning method provided by the present application is shown in the flowchart.

[0029] Figure 2 The high-efficiency hierarchical progressive sorting screening algorithm is shown in the diagram. DETAILED DESCRIPTION

[0030] The technical solutions provided by the present application will be described in detail below in conjunction with specific embodiments, and it should be understood that the following specific embodiments are only used to illustrate the present application and not to limit the scope of the present application.

[0031] The high-efficiency annotation-driven hierarchical fault positioning method provided by the present application includes:

[0032] I. Using a positioning algorithm based on large model annotation, using LLM supported prompt-based and embedding-based retrieval to guide the large language model to intelligently analyze and generate semantic annotations of text, thereby performing more accurate fault positioning and improving software fault positioning accuracy. Specifically includes:

[0033] (1) When processing data, first input the original problem description and code library into the system, and construct a tree file hierarchy to illustrate the relative position of the file;

[0034] (2) Then, to address the problem of lack of detailed source code information in the repository structure format, load the problem dataset and use abstract syntax tree to parse python code;

[0035] (3) Reuse carefully designed prompts to guide the large language model to intelligently analyze and generate semantic annotations of text, thereby improving overall context awareness;

[0036] (4) During the entire generation process, it will check whether the output file exists, support continuing processing after interruption, ignore files that cannot be read or parsed, and save immediately after processing each file to prevent data loss.

[0037] Second, an annotation-based hierarchical localization method is used to divide the problem database into file / function / location levels. Carefully designed large-scale model hint words are then used to add annotations sequentially. This allows for fine-grained decomposition of the massive file information in the problem database, building upon the generated text annotations. Annotations are narrowed down from file-level to function-level, and then to location-level. This innovative approach combines semantic annotation with a three-level "file-function-line" localization system, addressing the challenge of limited ability to resolve complex dependencies in existing methods and thus improving localization accuracy. Specifically, this includes:

[0038] (1) Perform file-level location, read the file, parse the Python code using an abstract syntax tree, and use a large model to supplement some additional text annotation information. This information describes the function of each file, and a concise functional description is added to each file in a systematic way, outlining its general responsibilities. Annotation generation can be dynamically generated at runtime or pre-generated in batches and cached to improve efficiency and save API costs. This targeted metadata enables the large language model to more accurately associate potential problems with files related to relevant functions.

[0039] (2) Perform function-level localization, use prompt words to guide the large model to perform a very detailed analysis of each function, and generate a summary of the function description, such as step-by-step analysis of parameter function description, step-by-step logical decomposition, key processing points, return value description, exception handling analysis, etc.

[0040] (3) Perform location-level positioning, add data flow analysis annotations, narrow the scope of the problem to location-level annotations, enable LLM to track variable dependencies to gain a deeper understanding of the context, allow LLM to find further details based on the variables it selects, and thus discover row-level faults.

[0041] Third, an efficient hierarchical sorting and filtering algorithm is used to filter the file layer by layer, ultimately pinpointing the lines where program defects may exist. This process, through its hierarchical filtering mechanism, effectively reduces the computational overhead of processing large models while improving the accuracy of locating critical code. The algorithm flow for this step is as follows: Figure 2 As shown, the specific steps include the following:

[0042] (1) Due to the large number of files, considering the time constraints, 2K suspicious files were initially screened at the file level. To avoid processing too many irrelevant files in subsequent steps, the model was scaled up to generate and embed file-level metadata annotation summaries for these files. After the files were annotated using a localization algorithm based on large model annotation, the algorithm removed irrelevant files through deep semantic analysis and selected the K key files that best matched the problem description (specifically, the semantic similarity was calculated and the top-k files were selected) for the next step.

[0043] (2) After adding comments to K files, the algorithm selects the M core problem functions from these files;

[0044] (3) Only the most critical functions need to be added with edit-level process comments, and then sorted to accurately filter out the specific line numbers that need to be modified. They are then expanded with detailed process flow comments to output a more granular list of edit locations for precise code modification.

[0045] Based on the above scheme, the efficient annotation-driven hierarchical fault location process applied in the example is as follows: Figure 1 As shown, it includes the following steps:

[0046] Step 1: During data processing, the original problem description and code library are first input into the system to construct a tree-like file hierarchy to illustrate the relative positions of the files;

[0047] Step 2: To address the issue of the repository structure lacking detailed source code information, load the problem dataset and use an abstract syntax tree to parse the Python code.

[0048] Step 3: Initially filter out 2,000 suspicious files at the file level;

[0049] Step 4: Repeatedly use carefully designed prompts to guide the large language model to intelligently analyze and generate semantic annotations for the text, thereby improving the overall context awareness capability; throughout the generation process, the existence of the output file is checked, and processing can be resumed after interruption. Files that cannot be read or parsed are ignored, and each file is saved immediately after processing to prevent data loss.

[0050] Step 5: Perform file-level localization, read the file, and use the large model to supplement some additional text annotation information;

[0051] Step 6: Through deep semantic analysis, irrelevant files are removed, and the K key files that best match the problem description are selected for the next step; specifically: semantic similarity is calculated, and the top-k files are selected.

[0052] Step 7: Perform function-level localization, using prompts to guide the large model to perform a very detailed analysis of each function and generate a summary of the function description;

[0053] Step 8: Select the M most critical problem functions from these files;

[0054] Step 9: Add edit-level process comments to these M most critical functions, and then sort them to accurately select the specific line numbers that need to be modified;

[0055] Step 10: Perform location-level positioning by adding data flow analysis annotations to the editing location to narrow down the scope of the problem to location-level annotations;

[0056] Step 11: Locate the suspected fault location based on the original problem description;

[0057] Step 12: Generate and verify patches from the large model, and output the correct patches that have passed verification.

[0058] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. An efficient annotation-driven hierarchical fault localization method, characterized in that, The application relates to a large model annotation-based positioning algorithm using LLM-supported prompt-based and embedding-based retrieval to guide a large language model to intelligently analyze and generate semantic annotations of text. The application relates to a hierarchical positioning method based on annotations, which divides a problem database into file / function / location levels and sequentially adds annotations using carefully designed large model prompts to finely analyze and reduce file information in the problem database from file-level annotations to function-level annotations and then to location-level annotations. The application relates to an efficient hierarchical progressive sorting and screening algorithm, which screens files layer by layer to finally lock to a position line where a program defect may exist. The application relates to a large model annotation-based positioning algorithm using LLM-supported prompt-based and embedding-based retrieval to guide a large language model to intelligently analyze and generate semantic annotations of text.

2. The efficient annotation-driven hierarchical fault localization method of claim 1, wherein, (1) During data processing, the original problem description and the code library are input into the system, and a tree-shaped file hierarchical structure is constructed to indicate the relative position of the files. (2) Then, the problem data set is loaded, and the python code is parsed using an abstract syntax tree to solve the problem of lacking detailed source code information in the repository structure format. (3) The large language model is guided to intelligently analyze and generate semantic annotations of text by repeatedly using carefully designed prompts, thereby improving the overall context awareness capability. During the process of generating semantic annotations of text, the following steps are further included: checking whether the output file exists, supporting the processing after interruption, ignoring files that cannot be read or parsed, and saving immediately after processing each file to prevent data loss.

3. The efficient annotation-driven hierarchical fault localization method of claim 2, wherein, The application relates to a hierarchical positioning method based on annotations, which divides a problem database into file / function / location levels and sequentially adds annotations using carefully designed large model prompts to finely analyze and reduce file information in the problem database from file-level annotations to function-level annotations and then to location-level annotations.

4. The efficient annotation-driven hierarchical fault localization method of claim 1, wherein, (1) File-level positioning is performed, the file is read, the python code is parsed using an abstract syntax tree, and the large model is used to supplement additional text annotation information for explaining the function of each file. (2) Function-level positioning is performed, and the large model is guided to analyze each function in detail using prompts, and a summary of the function description is generated. (3) Position-level positioning is performed, data flow analysis annotations are added, the range of the problem is reduced to position-level annotations, the LLM can track variable dependency relationships to obtain a deeper context understanding, and the LLM is allowed to find further details according to the selected variables to find line-level faults. During file-level positioning, the annotations are generated dynamically during program running or are pre-generated in batches and cached.

5. The efficient annotation-driven hierarchical fault localization method of claim 4, wherein, The application relates to an efficient hierarchical progressive sorting and screening algorithm, which screens files layer by layer to finally lock to a position line where a program defect may exist.

6. The efficient annotation-driven hierarchical fault localization method of claim 1, wherein, ​ (1) In the file level, 2K suspicious files are initially screened out, and after file annotation, irrelevant files are removed through deep semantic analysis, and K key files most matching the problem description are screened out for the next step; (2) Only need to add annotations to K files, and then screen out M most core problem functions in these files; (3) Only need to add edit-level process annotations to these M most critical functions, and then perform sorting operations to accurately screen out specific line numbers that need to be modified, and expand them with detailed process flow annotations, output a more fine-grained edit location list, so as to accurately modify the code.

Images