Method for correcting code defects and computing equipment

Abstract

The invention discloses a method for correcting code defects and computing equipment, and relates to a code correction system, the code correction system comprises a visual analysis model and a correction agent, the correction agent comprises a basic model and a plurality of callable functions, the basic model is a plain text large language model, a defect correction request for a target item is obtained, and the defect correction request is sent to the computing equipment; the defect correction request comprises a defect description text and a defect description image; generating a corresponding image description code according to the defect description image by utilizing the visual analysis model; and carrying out code correction on the target item by utilizing the correction agent according to the image description code and the defect description text, so that image information and text information in a defect correction request can be fully utilized, and the success rate of code correction is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification belong to the technical field of data processing, and particularly relate to a method for correcting code defects and a computing device. BACKGROUND

[0002] In the current software development process, large language models (LMMs) are being used more and more widely. Developers only need to input their software development requirements in the form of natural language into the large language model, and the large language model can automatically generate corresponding code. This process separates the tedious code writing work from the development process, and developers only need to act as "designers" to complete software development, greatly improving the efficiency of software development.

[0003] Further, developers are also trying to further apply large language models to code correction. Unlike code generation, the quality of the code generated by the large language model is higher in the code correction scenario - the corrected code must be adapted to the complete project code. On the one hand, developers need to locate the corresponding problem fragment in the project code according to the relevant defect report, in order to accurately replace the problem fragment with the corrected code in the subsequent process; on the other hand, even if the problem fragment is accurately located, the developer also needs to correct the problem fragment in a form that conforms to the project specification, so that the corrected code can run normally in the project.

[0004] When using a large language model for this process, in order to assist the large language model in understanding the software project or the defect report, the developer will input the relevant materials (usually in the form of images) together with the defect report into a multimodal large language model (MLLM) to generate corresponding corrected code. However, the performance of the multimodal large language model is not satisfactory, and the corrected code generated is difficult to directly apply to the software project.

[0005] Therefore, the present specification provides a method for correcting code defects and a computing device to at least partially solve the above problems. SUMMARY

[0006] The purpose of the embodiments of the present specification is to provide a method for correcting code defects and a computing device, comprising:

[0007] The first aspect of the present specification provides a method for correcting code defects, the method relates to a code correction system, the code correction system comprises a visual analysis model and a correction agent, the correction agent comprises a base model and a plurality of callable functions, the base model is a pure text large language model, and the method comprises:

[0008] obtaining a defect correction request for a target project, the defect correction request including defect description text and a defect description image;

[0009] generating corresponding image description code according to the defect description image by using the visual analysis model;

[0010] correcting the target project according to the image description code and the defect description text by using the correction agent.

[0011] The second aspect of the present specification provides a computing device, including a memory and a processor, the memory storing executable code, and the processor executes the executable code to implement the method of the first aspect.

[0012] The technical solution for correcting code defects provided in the present embodiment can make full use of image information and text information in the defect correction request, and improve the success rate of code correction. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments described in the present specification, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0014] Figure 1 is the architecture schematic diagram of the multi-modal large language model in the present specification;

[0015] Figure 2 is the structural schematic diagram of a code correction system in an embodiment of the present specification;

[0016] Figure 3 is the flow schematic diagram of a method for fine-tuning a visual analysis model in an embodiment of the present specification;

[0017] Figure 4 is the flow schematic diagram of a method for correcting code defects in an embodiment of the present specification;

[0018] Figure 5 is the flow schematic diagram of code correction in an embodiment of the present specification;

[0019] Figure 6 is the flow schematic diagram of a method for iteratively updating the corrected code in an embodiment of the present specification. DETAILED DESCRIPTION

[0020] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0021] The following is a brief introduction to the technical terms and concepts used in this manual:

[0022] Intelligent Agent: An intelligent agent is a software system capable of operating autonomously with minimal human intervention. It possesses the following core capabilities: making action decisions based on the environment, autonomously executing tasks based on these decisions, and iteratively making decisions and executing tasks based on feedback from the execution results until a preset goal is achieved. Currently, the main implementation of intelligent agents involves using a large language model as the core for receiving user tasks and making action decisions. An external intelligent agent toolkit is used to execute the action decisions made by the large language model. Specifically, the large language model in the intelligent agent can be pre-trained and combined with prompting engineering and retrieval enhancement generation schemes. This large language model can autonomously arrange task processing flows (e.g., the task processing flow can be input into the large language model in the form of prompt words) and output call statements for the intelligent agent toolkit (e.g., the functional introduction, interaction methods, and other relevant information of the intelligent agent toolkit can be externally attached to the large language model in the form of a database). This intelligent agent toolkit may include several function tools required for processing related tasks (e.g., if an intelligent agent is designed for data processing tasks, then the corresponding function tools may include data editing functions, data reading functions, and calculation functions, etc.). The large language model can transmit call statements to the corresponding function tools through a pre-defined interface, and receive the execution results from the function tools through the corresponding interface. The agents involved in this embodiment include a correction agent and a test agent.

[0023] Multimodal Large Language Model (MLLM): A multimodal large language model is a deep learning system that extends the traditional plain text large language model (which only accepts text information as input and outputs only text information). It can simultaneously process information from multiple modalities, including text, images, audio, and video. Specifically, the multimodal application capability of a multimodal large language model is reflected in two aspects: firstly, it can accept information from multiple different modalities as input; secondly, the output can contain information from multiple different modalities. The multimodal large language model involved in this embodiment includes a visual parsing model and a core large language model for the testing agent.

[0024] Before introducing the technical solution in this embodiment, it is important to clarify that the core of using a large language model for code correction lies in replacing manual tasks such as understanding project code and locating problematic segments. In other words, in practical applications, it should be assumed that the user submitting the code correction request in this embodiment does not have a deep understanding of the project code to be corrected or the problem itself. The relevant information that the user can obtain and submit to the large language model should be limited to existing, directly accessible information. For example, when a defect correction request targets a front-end page, the front-end page image displaying the defect can be submitted to the large language model as relevant information (i.e., the defect description image in this embodiment). The front-end page image, as the result of the front-end code's execution, records the structured information contained in the front-end code in a high-density, visual format. If the large language model can fully understand the front-end page image, it can obtain corresponding guidance information regarding the defect correction request.

[0025] To utilize front-end page diagrams, a common technical solution is to use a multimodal large language model that can simultaneously accept images and text as input to process defect correction requests. However, this solution often falls short of the requirements of real-world applications. Observation of the multimodal large language model's output for defect correction requests reveals that it fails to understand the code logic shown in the front-end page diagrams, and also fails to output code that conforms to the project's required format.

[0026] The problem may stem from the inherent characteristics of multimodal large language models. Typically, the architecture of a multimodal large language model is as follows: Figure 1As shown in the diagram. Referring to this architecture, the processing flow of the multimodal large language model is as follows: First, the original signal (image, video, audio, text, etc.) of each modality is encoded into corresponding feature representations through the encoding layer corresponding to each modality. Then, using the cross-modal alignment module corresponding to each modality, the feature representations of each modality are transformed into a unified feature space. Finally, the transformed feature representations are processed using a unified large language model (usually a plain text large language model) to obtain the generated result. During the pre-tuning of the multimodal large language model, the parameters of both the cross-modal alignment module and the plain text large language model are adjusted.

[0027] It's easy to understand that, compared to a pure text-based large language model that only processes text, a multimodal large language model learns the generation logic of multiple modalities using the same parameter scale. During the fine-tuning of a multimodal large language model, the text processing capability of a pure text-based large language model is inevitably diluted by its ability to process other modalities. Therefore, on the one hand, the code generation capability of a multimodal large language model cannot compare with that of a pure text-based large language model; on the other hand, the ability of a multimodal large language model to understand images is also affected by its text processing capabilities.

[0028] Based on this Figure 2 A schematic diagram of a code correction system according to one embodiment of this specification is shown. Figure 2 As shown in this embodiment, the code correction system includes a visual parsing model and a correction agent. The visual parsing model is a multimodal large language model capable of processing images, while the base model in the correction agent is a plain text large language model. Specifically, the visual parsing model can be obtained by fine-tuning a pre-trained multimodal large language model, enhancing its ability to understand images and generate corresponding code based on them; the base model does not need to have multimodal capabilities, but only needs to be fine-tuned according to the requirements of calling the agent tool library and generating code.

[0029] Therefore, when processing defect correction requests, the visual analysis model is first used to process the defect description image corresponding to the defect correction request to obtain the corresponding image description code. This image description code may contain structured information about the corresponding defect or item shown in the defect description image. Then, the image description code in text form and the defect description text are input into the correction agent. Thus, the correction agent only needs to process the text to complete the code correction task. In this embodiment, through the separate design of the visual analysis model and the correction agent, the task of understanding the defect description image is separated from the task of correcting the code, which can fully leverage the processing capabilities of the visual analysis model and the correction agent for their respective modalities.

[0030] To ensure that the visual analysis model can accurately generate image description codes in the corresponding format based on the defect description image, the visual analysis model needs to be fine-tuned to a certain extent.

[0031] Figure 3 The diagram illustrates a method for fine-tuning a visual resolution model according to an embodiment of this specification. This method can be executed by any computing device or server with computing capabilities; this specification does not impose any limitations on this method. Specifically, the method includes:

[0032] Step S301: Obtain the training sample set.

[0033] This training sample set consists of several training samples, each composed of a sample description image and its corresponding code label. Following the previous introduction to image description codes, to ensure that the image description codes generated by the visual analysis model meet the requirements in both format and content, for any sample description image, the code label corresponding to that sample description image, after compilation and execution, will result in the sample description image itself. For example, when the sample description image is a system design diagram, the code label can be in Mermaid format; when the sample description image is a front-end page diagram, the code label can be in HTML format.

[0034] Since there are currently no open-source datasets for system design diagrams and workflow diagrams that meet the requirements for the training sample set in this embodiment, this embodiment will provide a method for constructing a training sample set. First, an open-source code generation training set (e.g., the SWE-bench training set) is obtained. This training set may include code function descriptions and corresponding code snippets. Then, a sample generation model (which can also be a large language model) is used to generate a corresponding system design diagram or workflow diagram representing the structure or logic of the code snippets based on the code function descriptions and code snippets in the training set. It should be noted that the sample generation model can be constrained by prompts to generate the system design diagram and workflow diagram in a way that "code that produces the corresponding image after execution," meaning the output of the sample generation model is code rather than an image. Therefore, the code output by the sample generation model is used as a code label, and the image obtained after executing the code label is used as the corresponding sample description image, thus obtaining a training sample. The format of the code output by the sample generation model can be preset to the format of the aforementioned image description codes.

[0035] In some implementations, before executing the code input to the sample generation model to obtain the corresponding image, the code can be cleaned through methods such as syntax verification (syntax checking of the code, testing it one by one against the official rendering engine of the corresponding format, filtering out code with syntax errors, and ensuring that the code can be parsed correctly), rendering testing (actually rendering the code that has passed syntax verification to generate the corresponding image, and filtering out code that cannot be rendered successfully), quality checking (quality evaluation of successfully rendered images, and excluding samples with unclear or incomplete rendering results), data alignment verification (ensuring that each valid code and its corresponding rendered image form a correct one-to-one correspondence, avoiding incorrect pairing), and duplicate filtering (detecting and filtering duplicate function definitions and similar UML structures) to improve the data quality of the training sample set obtained in this embodiment.

[0036] Step S303: For any training sample in the training sample set, input the sample description image corresponding to the training sample into the visual analysis model to determine the prediction code output by the visual analysis model.

[0037] Specifically, taking a training sample as an example, in step S303, the sample description image corresponding to the training sample and the code parsing prompts can be input into the visual parsing model so that the visual parsing model outputs the corresponding predicted code. The code parsing prompts at least include a task description of the code parsing task. In some implementations, the code parsing prompts may also include auxiliary information such as image parsing examples to help the visual parsing model understand the code parsing task.

[0038] Step S305: Adjust the parameters of the visual analysis model based on the difference between the code label corresponding to the training sample and the predicted code.

[0039] Once the prediction code is determined, the parameters of the visual analysis model can be adjusted based on the difference between the prediction code and the corresponding code label, with the goal of making the prediction code as close as possible to the code label.

[0040] Through such Figure 3 The method shown in this embodiment can train a visual analysis model that can understand various types of charts and generate corresponding codes.

[0041] Figure 4A flowchart illustrating a method for correcting code defects according to an embodiment of this specification is shown. This method is executed using a code correction system, which can be deployed on any computing device or server with computing capabilities. The computing device or server may have a user-facing interactive interface, through which it receives defect correction requests from users and provides feedback on the corrected code. In some implementations, the code correction system includes a visual analysis model and a correction agent, which includes a basic model and an agent toolkit consisting of several callable functions. The following discussion uses this code correction system as the execution entity to illustrate this method. Figure 4 The method shown is described below, and the method includes:

[0042] Step S401: Obtain a defect correction request for the target project, wherein the defect correction request includes defect description text and defect description image.

[0043] First, the system can obtain user defect correction requests through an interactive interface. These requests display the target item to be corrected and descriptive information about the defect—a defect description text and a defect description image. The defect description text includes at least a direct description of the defect. This description can be determined based on the defect reporting scenario. For front-end defects, the direct description might be a description of discrepancies between the front-end page and the design, such as "the button corresponding to the submit function is not working." For back-end defects, the direct description might be a runtime error report or a description of discrepancies between the runtime results and the design. In some implementations, the defect description text may also include the user's expected outcome for the defect fix, such as "after clicking the button corresponding to the submit function, the corresponding file is submitted, and the user is redirected to the feedback interface." The content of the defect description image can be found in the previous section.

[0044] In some implementations, the defect description image may include, in addition to the front-end page diagram, a system design diagram or workflow diagram for the function corresponding to the defect. That is, the defect description image is not limited to describing the defect itself, but may also describe the function or system in which the defect occurs.

[0045] Specifically, the target project can be a development project for a software product or a part of a software product. The defect correction request shows the target project, that is, it shows the code library containing the code corresponding to the target project. Therefore, in the subsequent step S405, the code library corresponding to the target project can be corrected.

[0046] Step S403: Using the visual analysis model, generate corresponding image description code based on the defect description image.

[0047] After receiving a user's defect correction request, the first step is to input the defect description image from the request into a visual analysis model. This model then generates corresponding image description code based on the image. This image description code, once executed, produces the corresponding defect description image. For example, for a front-end page diagram, the image description code can be in HTML format; for a system design diagram or workflow diagram, the image description code can be in formats such as Mermaid, PlantUML, or Graphviz DOT.

[0048] It should be noted that when performing step S503, the execution process of step S303 can be referred to, and the defect description image and the corresponding code parsing prompt words can be input into the visual parsing model to determine the corresponding image description code. Step S405: Using the correction agent, the target item is corrected according to the image description code and the defect description text.

[0049] The concept of intelligent agents has been briefly introduced above. The following will describe in detail the specific process of code correction by the intelligent agent in conjunction with the application scenario of this embodiment.

[0050] As mentioned above, the corrective agent includes a base model and an agent toolkit for that base model. This toolkit includes several callable functions required for code correction scenarios. Furthermore, the corrective agent can be linked to the target project's codebase. In some implementations, the callable functions in this toolkit may include: read, write, edit, glob, grep, and bash functions. The base model can collaboratively perform code correction using these callable functions through corresponding call statements. The base model can also be any open-source plain text language model, such as Qwen3 or DeepSeek-Coder-v2; this specification does not impose any restrictions on this.

[0051] Figure 5 The diagram shows a flowchart of code correction in one embodiment of this specification. The code correction process is the specific process of step S405, which can be executed by the correction agent, including step S501 (defect location), step S503 (code generation) and step S505 (code replacement).

[0052] In step S501, the base model generates a first call statement for the plurality of callable functions based on the image description code and the defect description text.

[0053] After generating the first invocation statement, the basic model can transmit the first invocation statement to the agent tool library of the corrective agent through a preset interface. The first invocation statement can indicate the callable functions required for defect localization and the invocation instructions for each callable function.

[0054] In some implementations, after the intelligent agent toolkit receives the first invocation statement, it first calls the bash function to start the command-line interpreter. This bash function, through environment configuration, grants the command-line interpreter permission to operate on the target project's codebase. Then, in the command-line interpreter, it executes a call to the glob function to locate several candidate files corresponding to the defect description text within the codebase. Further, in the command-line interpreter, it executes a call to the greb function to locate several candidate fragments corresponding to the defect description text within the located candidate files. Subsequently, each candidate fragment is fed back to the base model through a predefined interface. Finally, the base model determines the target fragment to be repaired from the candidate fragments based on the defect description text.

[0055] In step S503, the base model generates the correction code corresponding to the target segment based on the target segment, the image description code, and the defect description text.

[0056] In step S505, the base model generates a second call statement for the plurality of callable functions based on the correction code. The second call statement shows the correction code and is used to correct the target fragment in the code library.

[0057] After generating the second invocation statement, the basic model can transmit the second invocation statement to the agent tool library of the corrective agent through a preset interface. The second invocation statement can show the callable functions required for defect localization and the invocation instructions for each callable function.

[0058] In some implementations, after the agent tool library receives the second invocation statement, it similarly calls the bash function to start the command-line interpreter; subsequently, in the command-line interpreter, the write function is called to write to the position corresponding to the target fragment in the code library, or the edit function is called to edit the target fragment in the code library; after editing or writing is completed, the agent tool library can also feed back the corresponding execution result to the base model.

[0059] It should be noted that, although Figure 5 The defect location step, code generation step, and code replacement step are described independently. However, in actual use cases, when performing step S405, it is only necessary to combine the steps according to... Figure 5The defined workflow prompts, image description codes, and defect description text are input into the base model, enabling the base model to operate automatically without additional user intervention, as follows: Figure 5 The process shown automatically calls each callable function to correct the code.

[0060] Correspondingly, in executing such Figure 4 Prior to the method shown, the basic model can be fine-tuned using common fine-tuning methods for generative agents to enable it to generate syntactically correct calling statements. The specific fine-tuning methods will not be elaborated here.

[0061] Furthermore, under normal conditions, the corrected code generated in step S503 may still have defects, which may cause the target project to still not function properly after the target segment is corrected in step S505. To solve this problem, in this embodiment, step S504 (code testing) can also be performed between steps S503 and S505. In step S504, the functionality of the corrected code generated in step S503 can be tested. Then, in step S505, only after the corrected code passes the test will the base model generate a second calling statement based on the corrected code that has passed the test. This ensures the accuracy of code correction and avoids unnecessary operations on the codebase.

[0062] In some implementations, step S504 can be performed by a correction agent. Specifically, the correction agent can be connected to a test case library, which may include test cases corresponding to the target project. In addition, the computing device or server deployed with the code correction system may also have the necessary software environment to run the test cases to support the test steps.

[0063] Therefore, in step S504, the base model first generates a third invocation statement based on the target fragment. The base model then transmits the third invocation statement to the corresponding agent tool library via a preset interface. This third invocation statement may indicate the callable functions required for code testing and the invocation instructions for each callable function.

[0064] In some implementations, after the agent toolkit receives the first call statement, it first calls the bash function to start the command-line interpreter. The bash function configures the command-line interpreter to have permission to operate on the test case library through environment configuration. Then, it executes the call instruction for the glob function in the command-line interpreter and determines several test cases corresponding to the corrected code in the test case library. Subsequently, each determined test case is fed back to the base model through a preset interface.

[0065] Next, the base model generates a fourth invocation statement based on several determined test cases and the corrected code. The base model can then transmit this fourth invocation statement to the corresponding agent tool library via a pre-defined interface. This fourth invocation statement is used to call various callable functions (usually bash functions) to test the functionality of the corrected code based on the determined test cases.

[0066] In some implementations, after the agent toolkit receives the first call statement, it first calls the bash function to start the command-line interpreter. This bash function configures the command-line interpreter to meet the environmental requirements for running each test case. Then, it executes each test case one by one in the command-line interpreter, determining the test result for each test case. Finally, the agent toolkit feeds back the test results to the base model through a preset interface. The test results include a first part and a second part. The first part indicates whether the corrected code can run normally, and the second part indicates the results of the corrected code running under several test cases. Therefore, if the first part of the test results indicates that the corrected code can run normally, and the second part indicates that the results of the corrected code running under each test case meet the expected results, then the corrected code has passed the test. In some implementations, the expected results corresponding to the corrected code can be determined based on the defect correction request submitted by the user; in other implementations, each test case may also have a corresponding expected result, thus the expected results corresponding to the corrected code can be referenced from the expected results corresponding to the test cases.

[0067] Of course, it's also necessary to consider that if the code modifications target the front-end page, the results will be displayed as images. Since the base model is a plain text large language model (as mentioned earlier, plain text large language models have stronger code generation capabilities), the base model alone cannot compare the actual results with the expected results. Under these conditions, this... Figure 2 The code correction system shown may also include a test agent, which consists of a test model and a corresponding agent tool library. The test model may be a multimodal large language model, and the agent tool library of the test model may refer to the correction agent.

[0068] Therefore, in step S504, the testing agent is used to test the functionality of the corrected code according to several test cases corresponding to the corrected code, and the test results corresponding to the corrected code are determined. The test results include a first content and a second content. The first content indicates whether the corrected code can be executed normally, and the second content indicates the running results of the corrected code under several test cases. Based on the test results, it is determined whether the corrected code has passed the test. If the test results indicate that the corrected code can be executed normally and can pass the several test cases, the test result is considered passed. Otherwise, the test results are all considered failed.

[0069] Of course, it must also be considered that, in reality, even manual code corrections often cannot be completed in one go, and the corrected code generated by executing step S503 only once is often difficult to pass direct testing. Therefore... Figure 6 This specification illustrates a flowchart of a method for iteratively updating correction code according to an embodiment of the present invention. When the correction code fails testing, this method can be executed. Figure 6 The method shown will continue until the updated, corrected code passes the test. Specifically, this is as follows: Figure 6 The methods shown can still be executed by the code correction system, including:

[0070] Step S601: Determine the supplementary description text based on the test results of the previous round.

[0071] It should be noted that, due to this... Figure 6 The method shown is executed after the code test fails, i.e., as... Figure 6 The method shown is executed after step S504. For example, Figure 6 For the first round of the method shown, the test result determined in step S504 can be used as the test result of the previous round.

[0072] Following the previous description of step S504, if the test result only contains text, the corrective agent can be used to determine the supplementary descriptive text corresponding to the test result. This supplementary descriptive text further describes the new defect introduced by the corrective code. In some implementations, the test result can be directly used as the supplementary descriptive text, or the corrective agent or test agent can be used to summarize the test result to obtain the supplementary descriptive text. This specification does not impose any restrictions on this.

[0073] If the test results contain images, then on the one hand, the corrective agent can be used to determine the supplementary description text corresponding to the first content in the test results (usually, the first content is represented in text form); on the other hand, the second content in the test results (the running results of the corrective code under several test cases may be represented in image form) can be used as the supplementary description image, and the supplementary description code corresponding to the supplementary description image can be determined using the visual parsing model.

[0074] Step S603: Using the correction agent, update the correction code corresponding to the previous round according to the supplementary description text, and determine the correction code corresponding to the current round.

[0075] Specifically, the execution method of step S603 can be found in the descriptions of steps S501-S503. The supplementary description text, along with workflow prompts, image description codes, and defect description text, are input into the basic model. This allows the basic model to automatically call the available functions to update and obtain the correction code corresponding to the current round. If the test results of the previous round included images, the supplementary description code determined in step S601, along with the supplementary description text, workflow prompts, image description codes, and defect description text, must also be input into the basic model.

[0076] Furthermore, following the introduction to step S601, regarding... Figure 6 For the first round of the method shown, the corrected code determined in step S503 can be used as the test result of the previous round.

[0077] Step S605: Using the test agent, test the functionality of the correction code corresponding to the current round, and determine the test result corresponding to the current round; if the current round of testing passes, end the iteration.

[0078] Specifically, the execution method of step S605 can be found in the description of step S505, which will not be repeated here.

[0079] The above, combined with, Figure 6 The method shown in this embodiment proposes a standardized code correction scheme that ensures correctness and significantly improves the ability of the code correction system to generate effective and accurate corrected code.

[0080] In summary, such as Figure 4 The method shown can make full use of image and text information in the defect correction request to improve the success rate of code correction.

[0081] In this specification, the Large Language Model (LLM) may also be referred to simply as the Large Model. A Large Language Model is a natural language processing model based on deep learning techniques, typically with billions to hundreds of billions or even more parameters, possessing powerful language understanding and generation capabilities. Large Language Models can employ the Transformer architecture or its variants (such as GPT, BERT, etc.), which utilizes an attention mechanism to globally model sequential data, efficiently handling long-distance dependencies and thus performing exceptionally well in natural language tasks. Large Language Models learn the statistical features and semantic relationships of language through pre-training on large-scale corpora, giving them outstanding generalization capabilities. The core capabilities of Large Language Models include, but are not limited to: understanding contextual semantics, generating coherent and grammatically correct text, performing logical reasoning, and handling multi-task scenarios. Its usage typically includes two modes: direct inference and fine-tuning. In direct inference mode, the user guides the Large Language Model to generate specific outputs by designing prompts. Cue words can be task descriptions or instructions in text form, used to stimulate the semantic understanding and generation capabilities of large language models. In fine-tuning mode, large language models are further trained on small-scale datasets in specific domains to optimize their performance on specific tasks. The powerful generalization ability and flexibility of large language models make them an important tool in the field of artificial intelligence, providing efficient and accurate solutions for automated text generation and understanding.

[0082] In some embodiments, large language models can also understand and generate data from other modalities (such as visual and audio data). In this case, large language models can also be called multimodal large language models (MLLMs). MLLMs provide a richer and more natural interactive experience by integrating multiple types of input and output, such as text, images, and sound. The core advantage of MLLMs lies in their ability to process and understand information from different modalities and fuse this information to complete complex tasks. For example, MLLMs can analyze an image and generate descriptive text, or generate a corresponding image based on a text description. This cross-modal understanding and generation capability makes MLLMs widely applicable across multiple fields.

[0083] It should be noted that the key technologies of large language models can be found in the detailed description in the paper "A Survey of Large Language Models" (paper number: arXiv:2303.18223v16, published on March 11, 2025, public link: https: / / doi.org / 10.48550 / arXiv.2303.18223), and will not be repeated here.

[0084] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0085] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0086] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0087] While one or more embodiments of this specification provide the operational steps of the methods described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes the elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0088] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0089] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0090] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0091] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0092] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0093] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0094] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0095] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0096] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0097] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0098] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of these embodiments. Various modifications and variations can be made to these embodiments by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims.

Claims

1. A method for correcting code defects, the method relating to a code correction system, the code correction system including a visual parsing model and a correction agent, the correction agent including a base model and several callable functions, the base model being a plain text large language model, the method comprising: Obtain a defect correction request for the target project, the defect correction request including defect description text and defect description image; Using the visual analysis model, a corresponding image description code is generated based on the defect description image; Using the aforementioned corrective agent, code correction is performed on the target project based on the image description code and the defect description text.

2. The method as described in claim 1, wherein, The defect description image specifically includes at least one of the following: a system design diagram, a workflow diagram, or a front-end page diagram displaying the defect for the function corresponding to the defect.

3. The method as described in claim 2, wherein, The corrective agent is connected to the codebase of the target project; Using the aforementioned corrective agent, code correction is performed on the target item based on the image description code and the defect description text, specifically including: The base model generates a first call statement for the plurality of callable functions based on the image description code and the defect description text. The first call statement is used to locate and extract the target fragment corresponding to the defect correction request in the code library. The base model generates correction code corresponding to the target segment based on the target segment, the image description code, and the defect description text; The base model generates a second call statement for the plurality of callable functions based on the corrected code. The second call statement is used to correct the target fragment in the code library.

4. The method of claim 3, wherein, The code correction system also includes a test agent; After the base model generates the correction code corresponding to the target segment based on the target segment, the image description code, and the defect description text, the method further includes: The functionality of the corrected code is tested using the aforementioned test agent; The base model generates a second call statement for the plurality of callable functions based on the modified code, specifically including: The base model generates second call statements for the several callable functions based on the corrected code that has passed the test.

5. The method as described in claim 4, wherein the functionality of the corrected code is tested using the test agent, specifically including: Using the aforementioned testing agent, the functionality of the corrected code is tested according to several test cases corresponding to the corrected code, and the test results corresponding to the corrected code are determined. The test results include a first content and a second content. The first content indicates whether the corrected code can be executed normally, and the second content indicates the running results of the corrected code under several test cases. Based on the test results, it is determined whether the corrected code has passed the test. If the test results indicate that the corrected code can be executed normally and can pass the several test cases, the test result is considered to have passed the test.

6. The method of claim 5, wherein, If the corrected code fails the test, the method further includes: The corrected code is iteratively updated in several rounds, wherein the first round of update steps specifically includes: Based on the test results, determine the supplementary description text corresponding to the current round; Using the correction agent, the correction code is updated according to the supplementary description text corresponding to the current round, and the correction code corresponding to the current round is determined; Using the correction agent, the correction code is updated based on the supplementary description text corresponding to the current round to determine the correction code corresponding to the current round; The test agent is used to test the functionality of the corrected code corresponding to the current round and determine the test result for the current round. If the test for the current round passes, the iteration ends; otherwise, the iteration continues. The update steps for any subsequent round after the first round specifically include: determining the supplementary description text corresponding to the current round based on the test results of the previous round; Using the aforementioned correction agent, the correction code corresponding to the previous round is updated based on the supplementary description text corresponding to the current round, thereby determining the correction code corresponding to the current round; The test agent is used to test the functionality of the corrected code corresponding to the current round and determine the test result for the current round. If the test for the current round passes, the iteration ends; otherwise, the iteration continues.

7. The method of claim 6, wherein, If the defect description image is a front-end page image displaying the defect, the second content specifically includes: several test page images of the correction code under several test cases; Determining whether the corrected code passes the test based on the test results specifically includes: Based on the aforementioned test page images, determine whether the corrected code has passed the test.

8. The method of claim 7, wherein, When the defect description image is a front-end page image displaying the defect, supplementary description text is determined based on the results of the previous round of testing, specifically including: The supplementary descriptive text is determined based on the first content corresponding to the previous round, and the supplementary descriptive image is determined based on the second content corresponding to the previous round; Using the visual analysis model, the supplementary description code corresponding to the supplementary description image is determined; Using the aforementioned corrective agent, the corrective code corresponding to the previous round is updated based on the supplementary description text, specifically including: Using the aforementioned correction agent, the correction code corresponding to the previous round is updated based on the supplementary description text and the supplementary description code.

9. The method of claim 3, wherein, The visual analysis model is fine-tuned according to the following steps: Obtain a training sample set, which consists of several training samples. For any training sample, the training sample includes a sample description image and a code label corresponding to the sample description image. The sample description image includes at least one of a system design diagram, a workflow diagram, and a front-end page diagram. The code label can be executed to obtain the corresponding sample description image. For any training sample in the training sample set, the sample description image corresponding to the training sample is input into the visual analysis model to determine the prediction code output by the visual analysis model; The parameters of the visual analysis model are adjusted based on the difference between the code label corresponding to the training sample and the predicted code.

10. A computing device comprising a memory and a processor, the memory storing executable code, wherein the processor, when executing the executable code, implements the method as claimed in any one of claims 1-9.

Images