Code error positioning system and method based on large language model
By constructing code tracing files and error location files in a diff-like format, and training an error locator using a large language model, the problem of inaccurate code error location in traditional methods is solved. This achieves efficient and accurate location of complex code errors, improving development efficiency and code quality.
Patent Information
- Application Number
- CN202511015456.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2025-05-15
- Filing Date
- 2025-07-23
- Publication Date
- 2025-10-31
AI Technical Summary
Traditional code error detection methods struggle to identify complex semantic code errors or deep-seated runtime errors, and cannot dynamically track changes in code execution paths or variable states, leading to inaccurate error localization.
By constructing code tracing files and diff-like error location files, and training an error locator using a large language model, the system can accurately locate potential errors in the code by utilizing tracing and difference information during code execution.
It improves the accuracy and efficiency of code error location, reduces the time cost of manual debugging, enhances development efficiency, and lowers code maintenance costs.
Smart Images
Figure CN120872792A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a code error localization system, which falls under the field of natural language processing technology, and specifically to a code error localization system and method based on a large language model. Background Technology
[0002] As software development becomes increasingly complex, automated code analysis and error localization technologies have become key tools for improving software development efficiency and code quality. Traditional error detection methods, such as rule-based static analysis tools, while capable of identifying code problems in certain specific scenarios, often perform poorly when dealing with complex, dynamic code logic. These methods struggle to identify code errors involving complex semantics or errors occurring during runtime.
[0003] In recent years, deep learning technology, especially the development of large-scale pre-trained language models (such as GPT-3 and Codex), has demonstrated enormous potential in the field of code understanding and generation. These models, through learning from massive amounts of code data, can accurately understand the syntax and structure of code, thereby enabling functions such as code auto-completion, error correction, and code refactoring. Despite these significant advancements in code generation and error correction, locating complex errors in code still faces numerous challenges. Traditional methods cannot dynamically track the execution path or changes in variable states, making it difficult to capture deep-seated errors during program execution. Furthermore, the representation of error location using natural language cannot accurately describe the location of the error. Summary of the Invention
[0004] To address the problems existing in the background technology, this invention provides a code error localization method based on a large language model. This invention designs an efficient code error localization method that accurately locates potential errors in the code through tracing information during code execution. It can train the error localization model by analyzing code discrepancies and further refine the location of errors in complex code using runtime data, thereby improving development efficiency and reducing the cost of error repair.
[0005] The technical solution adopted in this invention is:
[0006] I. A code error localization method based on a large language model:
[0007] 1) Construct a code dataset for code error localization learning. Each piece of code data in the code dataset contains a programming task q and its error code c, a unit test example t, and a correct code s.
[0008] 2) For each piece of code data in the code dataset, obtain tracing information based on the error code c and its unit test examples t, and then construct the code tracing file R for the error code c. t .
[0009] 3) Construct an error location file d in diff format based on the difference information between each error code c and correct code s.
[0010] 4) According to the code tracing file R t After training the Large Language Model (LLM) with the error location file d, an error locator is obtained to identify and locate code errors.
[0011] 5) After performing the same operation as in step 2), input the code data to be located into the error locator for processing. After processing, the error locator generates a file d′ with the error code location of the current code data and displays it on the monitor, thus realizing code error location.
[0012] In step 1), for each piece of code data in the code dataset, there is a programming task q, each programming task q corresponds to several different error codes c, each error code c corresponds to several unit test cases t and one correct code s, each unit test case t of each error code c is run, and one of the unit test cases t that fails to run is added to the code dataset.
[0013] In step 2), the error code c and its unit test case t for each piece of code in the code dataset are traversed. For each error code c and its unit test case t, the unit test case t is input into the error code c and run using a code debugging tool. The resulting error code c with commented trace information is used as the code trace file R. tIn practice, the code debugging tool used is the `traceback` module from the Python standard library. Specifically, for each line of code in the erroneous code `c`, after execution, it obtains all variables whose values changed in the current line of code and the execution order of the current line of code. If all variables in the current line of code did not change, a comment is added at the end of the current line to indicate that they did not change and the execution order. In practice, this can be commented as `NO CHANGE`. If there are three or fewer variables whose values changed in the current line of code, a comment is added at the end of the current line to indicate the final values of all variables whose values changed and the execution order. If there are more than three variables whose values changed in the current line of code, a comment is added at the end of the current line to indicate the final values of the first two and last variables whose values changed and the execution order. The final values of the variables whose values changed in the middle are replaced with ellipses. Finally, the erroneous code `c` with the commented tracing information is used as the code trace file `R`. t .
[0014] In step 3), the error code c and its correct code s for each piece of code data in the code dataset are traversed. For each error code c and its correct code s, the difference information is obtained by executing the diff command through the code version control tool, resulting in an error location file d in a diff-like format. The code version control tool is Git. Specifically, each line of code in the error code c is compared with each line of code in the correct code s. If the current line of code in the error code c is different from each line of code in the correct code s, or if the current line of code in the error code c does not exist, an error symbol is added to the beginning of the current line of code in the error code c and the current line of code is concatenated with a string. If the current line of code in the error code c is the same as one of the lines of code in the correct code s, a whitespace character is added to the beginning of the current line of code in the error code c and the current line of code is concatenated with a string. Finally, the marked error code c is used as the error location file d.
[0015] In step 4), each error code c, correct code s, and code tracing file R are... t The error localization file d is used to construct the error localization learning data {c,s,R}. t The dataset D is constructed by combining the data from the error localization training datasets {c, s, R}. Simultaneously, each error localization training data point {c, s, R} is constructed. t The error location text T, denoted as d, includes explanatory text, programming task q, error code c, and code trace file R. t And the textual representation of the error location file d.
[0016] In step 4), the error localization learning dataset D and its various error localization texts T are input into the Large Language Model (LLM) for training. The LLM's tokenizer transforms the error localization texts T into text embeddings, and the log-likelihood function is selected as the final loss function, i.e., the negative log-likelihood function. Then, backpropagation of errors is used to adjust the learnable parameters of the LLM. During the training process, the set of model parameters corresponding to the minimum final loss function is used as the final model parameters of the LLM, and finally, an error locator for identifying and locating code errors is obtained.
[0017] In step 5), the code data to be located includes the programming task q′ to be located, its error code c′, and unit test cases t′. After performing the same operation as in step 2), the code trace file R to be located is obtained. t The program task q′ to be located, its error code c′, and the code trace file R are included. t The input error locator processes the data, which includes the programming task q' to be located, its error code c', and the code trace file R. t The text embedding Q is converted into a text to be located. The text embedding Q is then input into the error locator for processing, generating a file d' with error code location for the current code data. The file d' with error code location uses error symbols to mark the erroneous lines of code, clearly indicating the potential error locations and parts that need to be corrected in the code. The error symbols are used to indicate the lines of code that need to be modified or deleted, and are displayed on the monitor.
[0018] II. A code error location system:
[0019] The preprocessing module is used to obtain runtime tracing and difference information from the code dataset, thereby constructing the code tracing file R for error code c. t And the error location file d in diff format.
[0020] Error localization learning module, used to trace files in R based on code. t After training the large language model LLM with the error location file d, an error locator is obtained to identify and locate code errors.
[0021] The error location generation module is used to generate a file d′ with the error code location of the current code data after processing it with the code data to be located and its code tracking file using an error locator, and then display it on the screen to achieve code error location.
[0022] 3. An electronic device comprising: a memory and a processor coupled to each other, wherein the memory stores program data, and the processor invokes the program data to execute the method described above.
[0023] IV. A computer-readable storage medium having program data stored thereon, wherein the program data, when executed by a processor, implements the method described above.
[0024] The code error localization method and system based on a large language model of the present invention uses code differences to train a large language model to locate errors, and uses tracing information (such as runtime variable states and execution paths) to enable the large language model to more accurately locate potential errors in the code, especially complex errors that are difficult to detect by traditional static analysis tools. It aims to improve the accuracy and efficiency of error localization, thereby better assisting developers in discovering and fixing errors in the process of software development and maintenance.
[0025] The beneficial effects of this invention are:
[0026] This invention simplifies and clarifies code error location by combining code tracing files and diff-like files. The system uses a diff-like error location file to represent the lines of code that need modification and introduces a self-debugging learning mechanism to train an error locator capable of identifying and locating runtime errors. This locator then generates a file containing error location information, helping developers accurately locate errors in the code. This invention effectively improves the correctness and accuracy of code error location and description, reduces the time cost of manual debugging, thereby increasing development efficiency and reducing code maintenance costs. Attached Figure Description
[0027] Figure 1 This is a flowchart of the method of the present invention;
[0028] Figure 2 This is a schematic diagram of the basic structure of the method of the present invention. Detailed Implementation
[0029] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0030] like Figure 1 As shown, the code error localization method based on a large language model of the present invention specifically includes the following steps:
[0031] 1) In the training process, a code dataset for code error localization learning is first constructed. Specifically, a Python code dataset is used. Each code data in the dataset contains a programming task q, its error code c, unit test cases t, and correct code s. For each code data in the dataset, the programming task q corresponds to several different error codes c, and each error code c corresponds to several unit test cases t and one correct code s. The unit test cases t for each error code c are run, and one unit test case t that fails to run is added to the code dataset.
[0032] 2) For each piece of code data in the code dataset, obtain tracing information based on the error code c and its unit test examples t, and then construct the code tracing file R for the error code c. t The code iterates through the error code c and its unit test case t for each piece of code in the code dataset. For each error code c and its unit test case t, the unit test case t is input into the error code c and run using a code debugging tool. The resulting error code c with comments and trace information is used as the code trace file R. t In practice, the code debugging tool used is the `traceback` module from the Python standard library. Specifically, for each line of code in the erroneous code `c`, after execution, it obtains all variables whose values changed in the current line of code and the execution order of the current line of code. If all variables in the current line of code did not change, a comment is added at the end of the current line to indicate that they did not change and the execution order. In practice, this can be commented as `NO CHANGE`. If there are three or fewer variables whose values changed in the current line of code, a comment is added at the end of the current line to indicate the final values of all variables whose values changed and the execution order. If there are more than three variables whose values changed in the current line of code, a comment is added at the end of the current line to indicate the final values of the first two and last variables whose values changed and the execution order. The final values of the variables whose values changed in the middle are replaced with ellipses. Finally, the erroneous code `c` with the commented tracing information is used as the code trace file `R`. t .
[0033] Code tracing file R t The specific text for each line is as follows:
[0034]
[0035] Where, r i This indicates the code tracing file R. t The i-th line of text in c; i This represents the i-th line of code for error code c; △v iThis represents the set of state changes of variables after the i-th line of code in error code c is executed, v i Let n represent the set of all variables that change in the i-th line of code for error code c, and let Δv represent the number of elements in the set. ik This represents the specific numerical value of the variable whose value changes after the execution of the i-th line of code in error code c. ik Represents set v i The k-th variable in the equation.
[0036] 3) Construct a diff-like error location file d based on the differences between each error code c and correct code s. Iterate through the error code c and its correct code s for each piece of code data in the code dataset. For each error code c and its correct code s, use a code version control tool to execute the diff command and process the differences to obtain the diff-like error location file d. Specifically, use Git to compare each line of code in error code c with each line of code in correct code s. If the current line of code in error code c is completely different from every line of code in correct code s, or if it does not exist in correct code s, add an error symbol marker to the beginning of the current line of code in error code c and concatenate the current line with a string. If the current line of code in error code c is the same as one line of code in correct code s, add a whitespace character marker to the beginning of the current line of code in error code c and concatenate the current line with a string. Finally, use the marked error code c as the error location file d.
[0037] The text of each line in the diff-like file d is as follows:
[0038]
[0039] Where, d i This represents the text on the i-th line of file d where the error is located; <space>_ represents a whitespace character; · represents a string concatenation symbol; '-' represents an error symbol marker; L represents the set of lines in error location file d marked with the '-' symbol that are identified as having a code error.
[0040] 4) According to the code tracing file R t After training a large language model (LLM) with the error location file d, an error locator is obtained that identifies and locates code errors. Each error code c, correct code s, and code trace file R are then used to train the LLM. t The error localization file d is used to construct the error localization learning data {c,s,R}. t The dataset D is constructed by combining the data from the error localization training datasets {c, s, R}. Simultaneously, each error localization training data point {c, s, R} is constructed. t The error location text T, denoted as d, includes explanatory text, programming task q, error code c, and code trace file R. t And the textual representation of the error location file d.
[0041] In practice, CodeLlama is used as the base Large Language Model (LLM) for training the error locator. For each data point {c,s,R} in the error localization training dataset D... t Given a programming task and a corresponding error code and its code trace file, construct an error location text T in the following format: "Instructions: Given a programming task and a corresponding error code and its code trace file, please provide the error location file containing the error code. In the error location file, use '-' to indicate lines that may need to be deleted or modified. Programming task: q, Error code: c, Code trace file: R" t Error location file: d.
[0042] The error localization training dataset D and its error localization texts T are input into the Large Language Model (LLM) for training. The LLM's tokenizer transforms the error localization texts T into text embeddings, and the log-likelihood function (negative log-likelihood function) is selected as the final loss function. Then, backpropagation is used to adjust the learnable parameters of the LLM. During training, the set of model parameters corresponding to the minimum final loss function is used as the final model parameters of the LLM, thus obtaining an error localizer for identifying and locating code errors.
[0043] 5) After performing the same operation as in step 2), input the code data to be located into the error locator for processing. After processing, the error locator generates a file d′ with the error code location of the current code data and displays it on the monitor, thus realizing code error location.
[0044] In the inference process, the code data to be located includes the programming task q′ to be located, its error code c′, and unit test cases t′. After performing the same operation as in step 2) on the code data to be located, the code trace file R to be located is obtained. t The program task q′ to be located, its error code c′, and the code trace file R are included. t The input error locator processes the data, which includes the programming task q' to be located, its error code c', and the code trace file R. t The code is converted into a text embedding Q' to be located. The text embedding Q' is then input into the error locator for processing, generating a file d' with error code locations for the current code data. The file d' with error code locations uses error symbols to mark erroneous lines of code, clearly indicating the potential error locations and parts that need correction. Error symbols are used to indicate lines of code that need modification or deletion, and these are displayed on the screen. For example, the programming task is: "Problem: Alice wants to buy N items. She has M coupons and K dollars. The price of the i-th item is A..." i Each coupon offers a 50% discount on the item (rounded). Write "yes" if she can afford it, otherwise write "no". (The question is related to...) Figure 2 Code tracing file R t The input is fed into the trained error locator, and the final result is an error location file in a diff-like format with error locations.
[0045] like Figure 2 As shown in the diff-like error location file, the diff-like error location file adds relevant program tracing information as comments after each line of code, marking the variable execution order number and the corresponding variable value for each line of code. For example, "#(0)n=3,m=2" means that the input values of n and m in this line are 3 and 2, and the execution order number of the current code is 0. "#(4)NO CHANGE" means that the value of no variable in this line has changed, and the execution order number of the current code is 4. In the error code, a '-' symbol is added before the deleted or modified line. For example, "for iin range(n-1):" is incorrect, and the correct code should be modified to "for iin range(n):".
[0046] The effectiveness of the method of this invention on the cleaned code error localization learning dataset is shown in Table 1 below:
[0047] Table 1
[0048]
[0049] Among them, few-shot learning (Few-Shoting) and fine-tuning (Fine-Tuning) are commonly used large model learning algorithms; the accuracy (ACC) ∈ [0,1], the larger the value of the accuracy (ACC), the higher the proportion of completely correct error localization; the evaluation metric (EL) ∈ [0,1], the smaller the value of the evaluation metric (EL), the lower the proportion of error localization results in the code; the evaluation metric (NL) ∈ [0,1], the smaller the value of the evaluation metric (NL), the lower the proportion of errors not being identified.
[0050] As can be seen, compared with the Few-Shoting model, the indicators ACC, EL, and NL of this invention on the same dataset are improved by 41.34%, 15.33%, and 6.0%, respectively; compared with the Fine-Tuning model, the indicators ACC, EL, and NL of this invention on the same dataset are improved by 6.0%, 5.0%, and 4.0%, respectively.
[0051] This invention also designs a code error localization system, including a preprocessing module, an error localization learning module, and an error localization generation module. The preprocessing module is used to obtain runtime tracing information and difference information of the code through a code dataset, thereby constructing a code tracing file R for the erroneous code c. t And a diff-like format error location file d; the error location learning module is used to learn from the code tracing file R. t After training the Large Language Model (LLM) with the error location file d, an error locator is obtained to identify and locate code errors. The error location generation module is used to process the code data to be located and its code tracking file with the error locator to generate a file d′ with the error code location of the current code data and display it on the screen to realize code error location.
[0052] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented using various computer languages. This application is described with flowcharts of methods, systems, and computer program products according to embodiments of this application.< / space>
Claims
1. A code error localization method based on a large language model, characterized in that, include: 1) Construct a code dataset for code error localization learning. Each piece of code in the dataset contains a programming task q and its error code c, a unit test example t, and a correct code s. 2) For each piece of code data in the code dataset, obtain tracing information based on the error code c and its unit test examples t, and then construct the code tracing file R for the error code c. t ; 3) Construct an error location file d in diff format based on the differences between each error code c and correct code s; 4) According to the code tracing file R t After training the large language model LLM with the error localization file d, an error locator is obtained to identify and locate code errors; 5) After performing the same operation as in step 2), input the code data to be located into the error locator for processing. After processing, the error locator generates a file d′ with the error code location of the current code data and displays it on the monitor, thus realizing code error location.
2. The code error localization method based on a large language model according to claim 1, characterized in that: In step 1), for each piece of code data in the code dataset, there is a programming task q, each programming task q corresponds to several different error codes c, each error code c corresponds to several unit test cases t and one correct code s, each unit test case t of each error code c is run, and one of the unit test cases t that fails to run is added to the code dataset.
3. The code error localization method based on a large language model according to claim 2, characterized in that: In step 2), the error code c and its unit test case t for each piece of code in the code dataset are traversed. For each error code c and its unit test case t, the unit test case t is input into the error code c and run using a code debugging tool. The resulting error code c with commented trace information is used as the code trace file R. t Specifically, for each line of code in error code c, after execution, all variables whose values changed in the current line of code and their execution order are obtained. If all variables in the current line of code did not change, an unchanged marker and the execution order are connected as comments at the end of the current line of code. If there are three or fewer variables whose values changed in the current line of code, the final values of all variables whose values changed and their execution order are connected as comments at the end of the current line of code. If there are more than three variables whose values changed in the current line of code, the final values of the first two and last variables whose values changed and their execution order are connected as comments at the end of the current line of code, with the final values of variables whose values changed in the middle replaced by ellipses. Finally, error code c with commented tracing information is used as the code tracing file R. t .
4. The code error localization method based on a large language model according to claim 1, characterized in that: In step 3), the error code c and its correct code s for each piece of code data in the code dataset are traversed. For each error code c and its correct code s, the difference information is obtained by executing the diff command through the code version control tool, resulting in an error location file d in a diff-like format. Specifically, each line of code in error code c is compared with each line of code in correct code s. If the current line of code in error code c is different from each line of code in correct code s or does not exist in correct code s, an error symbol is added to the beginning of the current line of code in error code c and the current line of code is concatenated with a string. If the current line of code in error code c is the same as one of the lines of code in correct code s, a whitespace character is added to the beginning of the current line of code in error code c and the current line of code is concatenated with a string. Finally, the marked error code c is used as the error location file d.
5. The code error localization method based on a large language model according to claim 1, characterized in that: In step 4), each error code c, correct code s, and code tracing file R are... t The error localization file d is used to construct the error localization learning data {c,s,R}. t The dataset D is constructed by combining the data from the error localization training datasets {c, s, R}. Simultaneously, each error localization training data point {c, s, R} is constructed. t The error location text T, denoted as d, includes explanatory text, programming task q, error code c, and code trace file R. t And the textual representation of the error location file d.
6. The code error localization method based on a large language model according to claim 5, characterized in that: In step 4), the error localization learning dataset D and its various error localization texts T are input into the Large Language Model (LLM) for training. The LLM's word segmenter transforms the error localization texts T into text embeddings, and the log-likelihood function is selected as the final loss function, i.e., the negative log-likelihood function. Then, backpropagation of errors is used to adjust the learnable parameters of the LLM. During the training process, the set of model parameters corresponding to the minimum final loss function is taken as the final model parameters of the LLM, and finally, an error locator for identifying and locating code errors is obtained.
7. The code error localization method based on a large language model according to claim 1, characterized in that: In step 5), the code data to be located includes the programming task q′ to be located, its error code c′, and unit test cases t′. After performing the same operation as in step 2), the code trace file R to be located is obtained. t The program task q′ to be located, its error code c′, and the code trace file R are included. t The input error locator processes the data, which includes the programming task q' to be located, its error code c', and the code trace file R. t The text embedding Q' is converted into the text to be located. The text embedding Q' is then input into the error locator for processing, generating a file d' with the current code data and the error code location. The error code line is marked with an error symbol in the file d' with the error code location and displayed on the monitor.
8. A code error localization system applicable to the method described in any one of claims 1-7, characterized in that, include: The preprocessing module is used to obtain runtime tracing and difference information from the code dataset, thereby constructing the code tracing file R for error code c. t And the error location file d in diff format; Error localization learning module, used to trace files in R based on code. t After training the large language model LLM with the error localization file d, an error locator is obtained to identify and locate code errors; The error location generation module is used to generate a file d′ with the error code location of the current code data after processing it with the code data to be located and its code tracking file using an error locator, and then display it on the screen to achieve code error location.
9. An electronic device, characterized in that, include: A memory and a processor are coupled to each other, wherein the memory stores program data, and the processor invokes the program data to perform the method as described in any one of claims 1-8.
10. A computer-readable storage medium storing program data thereon, characterized in that, When the program data is executed by the processor, it implements the method as described in any one of claims 1-8.