A defect report completion and optimization method and system based on multi-agent cooperation

Abstract

The application provides a defect report completion and optimization method and system based on multi-agent cooperation, and relates to the technical field of artificial intelligence, comprising: obtaining an original defect report; decoupling the completion and optimization task of the original defect report into three sequentially executed sub-tasks of defect analysis, report completion and quality evaluation; analyzing the input original defect report through an analysis agent, identifying the missing key information fields therein, and generating a completion suggestion based on the retrieved external knowledge; receiving the completion suggestion output by the defect analysis through a completion agent, generating and completing the missing content in the original defect report based on the completion suggestion, and obtaining a completed report; and evaluating the quality of the completed report through an evaluation agent. By task decoupling, the defect report processing flow is divided into three specialized stages of analysis, completion and evaluation, and a multi-agent cooperation and dynamic iterative optimization mechanism is adopted, so that the defect report is finally completed and optimized in a high-quality and automated manner.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a defect report completion and optimization method and system based on multi-agent collaboration. BACKGROUND

[0002] The statements in this section merely provide background technology related to the present disclosure and do not necessarily constitute prior art.

[0003] Software defect reports are important information carriers in software maintenance, and their quality directly determines the efficiency of developers in locating and repairing defects. A high-quality defect report usually contains three core elements: (1) observation behavior, i.e., abnormal or error phenomena exhibited by the software; (2) expected behavior, i.e., the correct behavior that the software should have under normal circumstances; (3) reproduction steps, i.e., a series of operation instructions that can stably reproduce the defect. These elements provide developers with the key context needed to diagnose and fix problems.

[0004] Currently, automated completion and optimization of defect reports have become a research hotspot in the field of software engineering. Existing technical solutions mainly fall into the following categories:

[0005] The first category is based on traditional machine learning methods, which can identify missing fields and complete them, but the generated content often has problems such as logical repetition, lengthy expression, lack of key details, and insufficient readability and practicality.

[0006] The second category is based on a single large language model (LLM), which generates content that has improved conciseness and logic, but is still limited by the cognitive load of a single model, prone to "hallucinations", and ignores key information such as environment configuration and boundary conditions, resulting in incomplete and inaccurate completion content.

[0007] The third category is a multi-agent collaboration framework (such as MetaGPT, AutoGen), which attempts to solve complex tasks through the division of labor and cooperation of multiple AI agents. However, these frameworks have obvious limitations: either they lack verification and iterative optimization mechanisms for generated results, making it difficult to ensure the final quality; or they use loose conversation-driven collaboration, which is prone to inconsistent context understanding, semantic drift, and low efficiency, and it is difficult to design optimal roles and interactions for specific tasks (such as defect report completion).

[0008] In summary, existing technologies in the field of automated defect report completion generally have low completion quality, lack reliable iterative optimization, and have low efficiency in multi-agent collaboration. Therefore, there is an urgent need for a new technical solution that can systematically address these issues and efficiently generate high-quality, complete defect reports. SUMMARY

[0009] To solve the above problems, the application provides a defect report completion and optimization method and system based on multi-agent cooperation, which divides the defect report processing flow into three specialized stages of analysis, completion and evaluation by task decoupling, and adopts multi-agent cooperation and dynamic iterative optimization mechanism, so that the three specialized large language model agents work together and cross-verify, and finally realize high-quality and automated completion and optimization of defect reports.

[0010] The first aspect of the application provides a defect report completion and optimization method based on multi-agent cooperation, comprising:

[0011] Obtaining an original defect report;

[0012] Decoupling the completion and optimization tasks of the original defect report into three sequentially executed sub-tasks of defect analysis, report completion and quality evaluation;

[0013] The defect analysis sub-task analyzes the input original defect report through the analysis agent, identifies the missing key information fields therein, and generates a completion suggestion based on the retrieved external knowledge;

[0014] The report completion sub-task receives the completion suggestion output by the defect analysis through the completion agent, generates and completes the missing content in the original defect report based on the completion suggestion, and obtains the completed report;

[0015] The quality evaluation sub-task evaluates the quality of the completed report through the evaluation agent; if the quality evaluation passes, the report after completion and optimization is output; if the quality evaluation fails, feedback information is generated according to the evaluation result, and the feedback information is sent to the defect analysis sub-task and / or the report completion sub-task, triggering the execution of a new round of defect analysis and / or report completion until the evaluation passes or the maximum number of iterations is reached.

[0016] Further, the analysis agent, the completion agent and the evaluation agent are all implemented by a large language model, and each agent is configured with a dedicated structured prompt instruction, which includes received information, team description and responsibility explanation.

[0017] Further, the generation of the completion suggestion based on the retrieved external knowledge comprises:

[0018] Retrieving external knowledge by using a hierarchical retrieval matching method, which comprises:

[0019] Calculating the similarity between the target defect report and a plurality of information centroids pre-stored in the knowledge base, the information centroids being generated based on clustering of different types of key information in historical defect reports;

[0020] Filtering Top-K candidate knowledge groups from the knowledge base based on the centroid similarity;

[0021] The similarity between the target defect report and the specific sentence set in the candidate knowledge group is calculated to determine the most relevant target knowledge from the candidate knowledge group.

[0022] Further, the information centroid includes an observation behavior centroid, an expected behavior centroid, and a reproduction step centroid.

[0023] Further, the analysis agent receives retrieval request feedback from the evaluation agent and adjusts the knowledge retrieval strategy based on the retrieval request feedback to regenerate the completion suggestion.

[0024] Further, the completion agent receives modification suggestion feedback from the evaluation agent and optimizes the content of the completed report based on the modification suggestion feedback, the optimized content including format standardization and / or content readability.

[0025] Further, the quality of the completed report is evaluated by the evaluation agent, specifically including:

[0026] The completed report is evaluated from three dimensions of format correctness, content integrity and readability, and information richness;

[0027] If the evaluation fails, the problem type is determined:

[0028] If it is a content semantic or format problem, a modification suggestion is sent to the completion agent;

[0029] If it is an information missing or insufficient information problem, a retrieval request is sent to the analysis agent.

[0030] The second aspect of the present application provides a defect report completion and optimization system based on multi-agent cooperation, comprising:

[0031] A report acquisition module is configured to acquire an original defect report;

[0032] A task decoupling module is configured to decouple the completion and optimization tasks of the original defect report into three sequentially executed sub-tasks of defect analysis, report completion, and quality evaluation;

[0033] A defect analysis module is configured to analyze the input original defect report by an analysis agent, identify the missing key information fields therein, and generate a completion suggestion based on the retrieved external knowledge;

[0034] A report completion module is configured to receive the completion suggestion output by the defect analysis through a completion agent, generate and complete the missing content in the original defect report based on the completion suggestion, and obtain a completed report;

[0035] A quality evaluation module is configured to evaluate the quality of the completed report by an evaluation agent;

[0036] a control module configured to control the defect analysis module and / or the report completion module to perform an iteration operation according to feedback information of the quality assessment module.

[0037] A third aspect of the present application provides a defect report completion and optimization device based on multi-agent cooperation, the device comprising a memory and a processor; the memory is configured to store a computer program; the processor is configured to implement the defect report completion and optimization method based on multi-agent cooperation as described above when executing the computer program.

[0038] A fourth aspect of the present application provides a computer readable storage medium, the storage medium stores a computer program, when the computer program is executed by a processor, the defect report completion and optimization method based on multi-agent cooperation as described above is implemented.

[0039] Compared with the prior art, the defect report completion and optimization method and system based on multi-agent cooperation provided by the present application has the following beneficial effects:

[0040] (1) In view of the technical problems that the cognitive load is high and hallucinations are easy to occur when a single model processes complex tasks, the present application splits the complex defect report processing flow into three independent stages of analysis, completion and evaluation, and assigns a dedicated agent to each stage, effectively reducing the cognitive load of each agent and clarifying the responsibility boundary, thereby achieving the technical effects of improving the reliability of task processing and the accuracy of generated content.

[0041] (2) In view of the technical problem that the traditional retrieval method has poor effect due to the missing of key information in the defect report, the present application first calculates the similarity with the information centroid for coarse screening, and then compares the specific sentence set for fine screening, achieving the technical effect of efficient and accurate retrieval of related knowledge under the condition of information missing, and providing reliable information support for high-quality completion.

[0042] (3) In view of the technical problem that there is a lack of effective quality control and iterative optimization mechanism in multi-agent cooperation, the present application provides a dynamic feedback and iterative optimization mechanism as well as an evaluation agent and a feedback path triggering mechanism, establishes a closed loop process of "analysis, completion and evaluation", and can perform multi-round cross-validation and collaborative optimization on substandard content, thereby achieving the technical effect of gradually improving the quality of defect reports.

[0043] (4) In view of the technical problem that the role division and interaction mode of the general multi-agent framework are not clear in specific tasks, the present application provides a structured prompt instruction, which defines clear input, role, responsibility and output requirements for each agent, so that it can accurately understand its own task, thereby achieving the technical effect of greatly improving the efficiency of multi-agent cooperation and the overall output quality. BRIEF DESCRIPTION OF DRAWINGS

[0044] The accompanying drawings, which are incorporated in and form a part of the specification, serve to further understand the present disclosure, and, together with the description, illustrate one or more exemplary embodiments of the present disclosure and explain the principles of the present disclosure, but are not intended to limit the present disclosure.

[0045] Figure 1 is a flowchart of a defect report completion and optimization method based on multi-agent cooperation provided by an embodiment of the present application;

[0046] Figure 2 is a method overall framework diagram provided by an embodiment of the present application;

[0047] Figure 3 is an analysis agent instruction setting example diagram provided by an embodiment of the present application;

[0048] Figure 4 is a hierarchical retrieval matching method flowchart provided by an embodiment of the present application;

[0049] Figure 5 is a completion agent instruction setting example diagram provided by an embodiment of the present application;

[0050] Figure 6 is an evaluation agent instruction setting example diagram provided by an embodiment of the present application;

[0051] Figure 7 is a line graph of the influence of the number of iterations on the performance of the model BLEU provided by an embodiment of the present application;

[0052] Figure 8 is a line graph of the influence of the number of iterations on the performance of the model SentenceBERT provided by an embodiment of the present application;

[0053] Figure 9 is a line graph of the influence of the number of iterations on the performance of the model ROUGE-L provided by an embodiment of the present application;

[0054] Figure 10 is a line graph of the influence of the number of iterations on the performance of the model METEOR provided by an embodiment of the present application;

[0055] Figure 11 is a bar chart of the performance of the method in processing Github and Eclipse data sets to complete each type of content reflecting BLEU performance provided by an embodiment of the present application;

[0056] Figure 12 is a bar chart of the performance of the method in processing Github and Eclipse data sets to complete each type of content reflecting SentenceBERT performance provided by an embodiment of the present application;

[0057] Figure 13is a bar chart reflecting the performance of ROUGE-L when the method provided by Embodiment One of the present application is used to process Github and Eclipse data sets to complete each type of content;

[0058] Figure 14 is a bar chart reflecting the performance of METEOR when the method provided by Embodiment One of the present application is used to process Github and Eclipse data sets to complete each type of content;

[0059] Figure 15 is a bar chart reflecting the performance of BLEU when the method provided by Embodiment One of the present application is used to process Mozilla and Tomcat data sets to complete each type of content;

[0060] Figure 16 is a bar chart reflecting the performance of SentenceBERT when the method provided by Embodiment One of the present application is used to process Mozilla and Tomcat data sets to complete each type of content;

[0061] Figure 17 is a bar chart reflecting the performance of ROUGE-L when the method provided by Embodiment One of the present application is used to process Mozilla and Tomcat data sets to complete each type of content;

[0062] Figure 18 is a bar chart reflecting the performance of METEOR when the method provided by Embodiment One of the present application is used to process Mozilla and Tomcat data sets to complete each type of content;

[0063] Figure 19 is a schematic diagram of a defect report completion and optimization system based on multi-agent collaboration provided by Embodiment Two of the present application. DETAILED DESCRIPTION

[0064] It should be noted that the following detailed description is merely exemplary in nature and is intended to provide further description of the present application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0065] It is to be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. It is further noted that the terms "comprise" (and variations of the term, such as "comprising," "includes," "including" and the like) as used herein are intended to allow for the presence of other elements or components. It is further noted that the claims can be drafted to exclude any elements or components except where such elements are specifically recited in a claim or where exclusion is readily apparent from the context of this disclosure to aliens of ordinary skill in the art.

[0066] In the absence of conflicts, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0067] All data acquisition of the present embodiment is based on compliance with laws and regulations and user consent, and legal application of data.

[0068] Embodiment one

[0069] As Figure 1 The present embodiment provides a defect report completion and optimization method based on multi-agent cooperation, comprising:

[0070] S1: obtaining an original defect report;

[0071] S2: decoupling the completion and optimization tasks of the original defect report into three sequentially executed sub-tasks of defect analysis, report completion and quality evaluation;

[0072] S3: the defect analysis sub-task analyzes the input original defect report through an analysis agent, identifies the missing key information fields therein, and generates a completion suggestion based on the retrieved external knowledge;

[0073] S4: the report completion sub-task receives the completion suggestion output by the defect analysis through a completion agent, generates and completes the missing content in the original defect report based on the completion suggestion, and obtains a completed report;

[0074] S5: the quality evaluation sub-task evaluates the quality of the completed report through an evaluation agent; if the quality evaluation passes, the report after completion and optimization is output; if the quality evaluation does not pass, feedback information is generated according to the evaluation result, and the feedback information is sent to the defect analysis sub-task and / or the report completion sub-task, triggering the execution of a new round of defect analysis and / or report completion, until the evaluation passes or the maximum number of iterations is reached.

[0075] Traditional single model methods, such as traditional machine learning or single large language model, need to undertake multiple responsibilities such as identifying missing, generating content and judging quality when processing full-process completion tasks, with high cognitive load, which easily leads to poor results, for example, machine learning methods may produce semantic discontinuity, and large language models may have hallucination phenomenon. The present application decouples the tasks of steps S2 to S4 and adopts a multi-agent division mechanism, so that each agent only focuses on a single sub-task, significantly reducing the cognitive load and structurally avoiding the inherent defects of full-process single models.

[0076] The existing multi-agent framework, such as MetaGPT or AutoGen, usually lacks an iterative verification mechanism for the generated results, and often generates once and ends, which has a quality risk. The iterative feedback mechanism (step S5) proposed in the present application controls semantic drift effectively through multiple rounds of cross-validation, and ensures that the final report is logically coherent and consistent with the facts.

[0077] Specifically, the analysis agent, the completion agent and the evaluation agent are all implemented by a large language model, and each agent is configured with a dedicated structured prompt instruction, which includes received information, team description and responsibility description.

[0078] The existing multi-agent framework mostly uses session-driven collaboration, which is prone to problems such as inconsistent context understanding and unclear responsibility boundaries. The present application provides clear input, role and output guidance for each agent through structured prompt instructions, so that the large language model can accurately locate its own responsibilities. For example, the evaluation agent only needs to focus on quality judgment and does not need to participate in content generation, thereby significantly improving collaboration efficiency. The responsibility description further clarifies the output specification, for example, the content generated by the completion agent needs to be consistent with the original context logic, and the input range is limited by combining the received information, for example, the analysis agent only generates suggestions based on relevant external knowledge, avoiding the generation of illusory content inconsistent with the facts due to ambiguous roles or unconstrained inputs, and improving the reliability of generated content. The team description and responsibility description in the structured prompt can also be tailored to the domain characteristics of the defect report, for example, clearly defining observation behavior, expected behavior and reproduction steps as core elements, avoiding the problem of insufficient professionalization of generated content due to insufficient domain knowledge of general large language models.

[0079] Specifically, generating a completion suggestion based on the retrieved external knowledge includes:

[0080] A hierarchical retrieval matching method is used to retrieve external knowledge, which includes:

[0081] The similarity between the target defect report and the plurality of information centroids pre-stored in the knowledge base is calculated, and the information centroids are generated based on clustering of different types of key information in historical defect reports;

[0082] Based on the centroid similarity, the Top-K candidate knowledge groups are selected from the knowledge base;

[0083] The similarity between the target defect report and the specific sentence set in the candidate knowledge group is calculated to determine the most relevant target knowledge from the candidate knowledge group.

[0084] Traditional retrieval methods, such as text similarity matching or embedding matching, are highly dependent on the complete information of the target report, and when the original report is missing key fields, the matching quality is low and the retrieval results are irrelevant. The hierarchical retrieval method adopted by the present application first performs coarse screening through the centroid, even if some type of information is missing, the matching can still be realized through the centroid of other types of information (the three types of centroids are respectively regarded as the "prototype representation" of the corresponding sentence type in the semantic space; for example, each centroid of the observation behavior sentence represents a "symptom pattern" of a similar defect. ) to achieve matching, and then fine screening through the sentence set to lock accurate knowledge, so that efficient retrieval can still be realized in the information missing scenario. The target knowledge retrieved is derived from the clustering summary of historical similar defect reports, rather than random external knowledge, ensuring that the completion suggestions generated by the analysis agent conform to the domain logic of the defect report, such as the reproduction law of software defects and the industry standard of expected behavior, providing high-quality knowledge basis for subsequent completion agents, and avoiding the generation of irrelevant or incorrect completion content. Centroid coarse screening can quickly narrow the retrieval range, solving the problem of low efficiency of full retrieval, and sentence set fine screening ensures the accuracy of the results, solving the problem of coarse screening results not being accurate enough, and the combination of the two meets the efficiency demand of a large number of reports needing to be processed quickly in the open source scene, and also takes into account the accuracy of knowledge retrieval.

[0085] Specifically, the information centroid includes an observation behavior centroid, an expected behavior centroid, and a reproduction step centroid.

[0086] Observation behavior, expected behavior, and reproduction step are the three core elements of a defect report, and the absence of any element will significantly prolong the repair time, for example, the absence of reproduction steps will cause the developer to be unable to reproduce the defect. The present application limits the type of information centroid to ensure that the retrieval process always revolves around these three core elements, avoiding the problem of missing core information in the completion suggestion due to the absence of retrieval dimensions. No matter which type of core information is missing in the original report, the system can match through the centroid of other types of information, for example, when the expected behavior is missing, similar reports can be located through the observation behavior centroid and the reproduction step centroid, thereby breaking the limitation that traditional retrieval cannot match when one type of information is missing, and improving the adaptability of the method to different incomplete reports. Defining the type of information centroid also provides a unified basis for the clustering process of historical defect reports, that is, clustering according to the three core elements, avoiding the problem of meaningless centroids due to chaotic clustering dimensions, and providing protection for the consistency and reliability of subsequent retrieval.

[0087] Specifically, the analysis agent receives the retrieval request feedback from the evaluation agent, and adjusts the knowledge retrieval strategy based on the retrieval request feedback to regenerate the completion suggestion.

[0088] The existing multi-agent method often has the problem of information fragmentation between agents, for example, the evaluation agent discovers the problem but cannot effectively feedback to the analysis agent. The present application introduces a feedback mechanism, so that the quality judgment of the evaluation agent can directly guide the retrieval behavior of the analysis agent, solving the problem that the quality of the generated suggestion is insufficient but cannot be optimized, thereby improving the accuracy and richness of the completed suggestion. The analysis agent adjusts the retrieval strategy according to the specific feedback of the evaluation agent, for example, when the feedback indicates that the environmental configuration information is missing, the similar report containing the environmental configuration can be retrieved specifically to avoid blind re-retrieval, reduce unnecessary consumption of computing resources, and improve iteration efficiency. When the evaluation agent judges that the completed content does not conform to the fact, it may be caused by the lack or irrelevance of the knowledge retrieved by the analysis agent, the analysis agent can supplement more relevant knowledge by adjusting the strategy to provide a more reliable basis for the completion agent, and reduce the risk of generating illusory content based on false knowledge from the source.

[0089] Specifically, the completion agent receives the modification suggestion feedback from the evaluation agent, and optimizes the content of the completed report based on the modification suggestion feedback. The optimized content includes format specification and / or content readability.

[0090] The report completed by the traditional machine learning method often has the problems of logical repetition and lengthy expression, for example, the repeated steps based on retrieval splicing may repeat the description; the single large language model method may ignore the format specification, for example, outputting unstructured text. The present application optimizes the format and readability to ensure that the final report has a clear structure, uses JSON format for easy access by developers, and has concise and non-redundant content, thereby improving the practicality of the report. The format specified report, such as the fixed fields containing observation behavior, expected behavior and reproduction step, can quickly locate the key information for the developer; the content optimized for readability, such as concise reproduction steps, can reduce the understanding time of the developer, thereby indirectly improving the defect repair efficiency. By specifying the optimization direction, i.e. format and readability, it is ensured that different original reports after completion meet the unified standard, avoiding the additional cost of developers adapting to different styles of reports due to the non-uniformity of report format or expression, especially in the application scenarios of multiple contributors and multiple projects in the open source community.

[0091] Specifically, the quality of the completed report is evaluated by the evaluation agent, which includes:

[0092] The completed report is evaluated from three dimensions of format correctness, content integrity and readability, and information richness;

[0093] If the evaluation fails, the problem type is determined:

[0094] If it is a content semantic or format problem, a modification suggestion is sent to the completion agent;

[0095] If it is an information missing or insufficient problem, a retrieval request is sent to the analysis agent.

[0096] The existing method often adopts the mode of non-discriminatory feedback to all agents when the evaluation is not passed, resulting in confusion of responsibilities between agents and low optimization efficiency. The present application avoids waste of resources and significantly improves iteration efficiency by problem type judgment and directional feedback mechanism, so that the completion agent only processes format or semantic problems, and the analysis agent only processes information insufficient problems. Three evaluation dimensions respectively cover formal specification, content-based quality and content value, forming a full-dimensional check on the quality of defect reports to avoid missing problems caused by missing evaluation dimensions, such as reports with correct format but insufficient information being misjudged as qualified. Through problem type subdivision, it is ensured that each problem can be processed by the corresponding responsible agent, avoiding no one to process the problem or multiple agents to repeatedly process the problem, further ensuring that the quality of the final report meets the needs of the developer, such as information richness meeting the technical needs of defect reproduction and repair.

[0097] In one specific embodiment, the method provided by the present application consists of four parts: task decoupling, analysis stage, completion stage and evaluation stage, and the overall framework is as shown in Figure 2 Firstly, in the task decoupling part, the completion and optimization tasks of the defect report are divided into three independent sub-stages: defect analysis, report completion and quality evaluation, which are respectively executed by different large language model agents, thereby effectively reducing the cognitive load of each agent. In order to clarify the responsibility boundary of each agent and improve the efficiency of its sub-task completion, a structured prompt template is designed, which includes receiving information, team description and responsibility explanation, which helps each agent to accurately understand the task requirements and ensure the output quality. In order to effectively realize "human expert cooperation", a dynamic feedback and iterative optimization mechanism is introduced, so that the three agents carry out multiple rounds of cross-validation and collaborative optimization around the same defect report.

[0098] In the specific execution process, the method of the present application completes the mapping from input to output through multi-agent collaboration under the condition of a given task T, that is, T: X→Y, where T represents the completion and optimization task of the defect report, X is the input original defect report content, and Y is the completed and optimized defect report. By introducing the mechanism of "multi-agent collaboration in stages", the method of the present application ensures the independence of each stage task while realizing efficient collaboration between different agents, thereby improving the quality of the defect report.

[0099] Task decoupling

[0100] In the real world, complex tasks often involve multiple links, roles and information flows. In order to complete such tasks efficiently and reliably, human experts usually adopt the mechanism of "staged processing and multi-role collaboration", which divides the task into several sub-tasks with clear boundaries and assigns them to different experts for independent completion, in order to reduce complexity and improve collaboration efficiency. Inspired by this, the originally highly coupled defect report completion and optimization task is refined into three consecutive stages: defect analysis → report completion → quality evaluation, and a dedicated large language model agent is configured for each stage. Among them, the analysis agent is responsible for identifying the missing key elements in the defect report; the completion agent supplements the necessary technical details and context information based on the analysis results; the evaluation agent evaluates the format, content integrity and readability, information richness, etc. of the completion content and provides feedback. This "role solidification" mechanism not only ensures clear task boundaries and clear division of labor, but also effectively reduces the cognitive load and illusion risk of large language models when processing complex tasks, improving the reliability and accuracy of generated content. In addition, in order to help each agent better understand and perform its role and responsibilities, a structured prompt template is designed. The template consists of three parts: (1) Receive information: define the input content and external knowledge that the agent may receive; (2) Team description: provide the collaboration environment, the functional roles of other agents and the background of the complex task; (3) Duty explanation: clearly define the responsibility boundaries and task list of the current agent, including output content and quality requirements. The introduction of structured prompts enables each agent to have a clear understanding of the context and role positioning when executing sub-tasks, thereby significantly enhancing overall collaboration efficiency and generation quality. In summary, by introducing the concept of "staged, multi-agent collaboration" into defect report completion and optimization tasks, this invention provides a scalable and controllable paradigm for the reliable and robust application of large language models in complex software engineering tasks. This not only improves the quality of defect reports, but also provides a reference for the implementation of multi-agent collaboration in engineering practice.

[0101] Analysis stage

[0102] In the method of the invention, based on the structured prompt instructions, the LLM is guided as an analysis agent to perform context analysis on the defect report content, focusing on identifying the missing field types in the report (such as observation behavior, expected behavior, reproduction steps, etc.). First, the analysis agent will focus on identifying and analyzing the key elements of the report content to determine the specific categories of missing information; then, the analysis agent will retrieve external knowledge related to the current report topic to find materials that can provide reference for the missing content, and further integrate these information to generate completion suggestions. In addition, the analysis agent may also receive feedback and retrieval requests from the evaluation agent, which requires the analysis agent to perform deeper external knowledge retrieval to form more comprehensive and detailed completion suggestions to support the execution of subsequent completion tasks. The process of this stage can be represented as:

[0103]

[0104] Whether a retrieval request is received from the evaluation agent during this process will determine the specific behavior of the analysis agent. If no retrieval request is received, the analysis agent will only respond to structured prompts. Under the guidance of the assessment agent, the current defect report X is analyzed, external knowledge is retrieved, and completion suggestions are generated. Otherwise, the analysis agent will combine the retrieval request, defect report X, and structured prompts received from the assessment agent. Under guidance, additional knowledge was retrieved and completion suggestions were updated. The structured instruction settings of the analysis agent were configured as follows: Figure 3 As shown.

[0105] Specifically, the analysis of the agent's work consists of the following four key steps.

[0106] Analyzing and identifying missing information. Existing research indicates that key fields such as observed behavior, expected behavior, and reproduction steps can help developers understand the cause and location of defects in a timely manner, thereby enabling efficient remediation. Observed behavior provides the actual manifestation of the defect, helping developers understand the current software problem. Expected behavior provides the ideal performance of the software and serves as a standard for problem remediation. Reproduction steps help developers reproduce the problem through specific operations, thus more efficiently locating and fixing defects. Therefore, the analysis agent first performs semantic analysis on the existing content in the report, identifying missing fields from the three dimensions of observed behavior, expected behavior, and reproduction steps.

[0107] Related knowledge retrieval. In the task of defect report completion and optimization, retrieving external knowledge is a crucial step. It helps the analyst to retrieve knowledge from the historical database that is consistent with the current report topic and similar in content, and further integrates to generate suggestions to complete the missing fields. Specifically, the analysis agent automatically calls tools to help it retrieve relevant web links, discussion content, and user solutions from the Internet, enabling the analysis agent to quickly obtain relevant contextual information from a large number of external resources. In addition, the discussions and problems of open source projects are usually concentrated on platforms such as Bugzilla, which provides a powerful RESTful API interface to support searching by keywords, defect numbers, products, versions, and other ways. The analysis agent can directly use the Bugzilla API to retrieve related knowledge. For example, using a Bugzilla query similar to the following: GET / bug?keywords=NaN&keywords=tf.select&product=TensorFlow&status=NEW to retrieve problems related to the tf.select function. This query will retrieve all new defect reports containing "NaN" and "tf.select", which may be related to the current defect. In this way, the analysis agent can efficiently obtain solutions, discussions, or repair suggestions related to the current defect. In this process, retrieving similar defect reports is crucial. However, due to the lack of key information in the target defect report, traditional text or embedding similarity matching methods result in low-quality matching due to information loss. Therefore, a hierarchical retrieval matching method framework is proposed, which first calculates the similarity of the centroid, and then compares the similarity of the specific sentence set, achieving multi-level optimized matching. Specifically, the three types of centroid learning process in the training stage summarizes the current knowledge topic by aggregating different types of key information (observation behavior, expected behavior, reproduction steps). This method can cluster and generate its own centroid for each type of sentence. In this way, no matter which type of information is missing in the target defect report (for example, missing expected behavior or reproduction steps), the system can still effectively match through other types of information, enhancing the adaptability and flexibility of different defect reports. In the matching stage, the system filters through the topk centroid candidates, narrowing down the matching range. Then, the target sentence set is compared in detail with the sentence set corresponding to multiple centroids. Such a matching method can more accurately capture the similarity between the target report and the candidate report, improving the relevance of the retrieval results. The above process is shown in FIG. 1. Figure 4

[0108] ​Generating completion suggestions. In addition to receiving the similar defect report group, the agent may have duplicates or irrelevant cases due to the retrieved knowledge. The analysis agent needs to further measure the similarity of the relevant knowledge fragments to the target task to ensure that the generated suggestions can effectively help the completion agent perform the completion task. For example, when retrieving external knowledge to help complete the "reproduction steps", the agent may retrieve knowledge fragments that are irrelevant or redundant to the task, such as "using tf.select to handle NaN values" or "adding a step to check for NaN before the tf.select operation". These knowledge pieces are not very valuable for the completion of "reproduction steps". Therefore, the structured instructions explicitly require the analysis agent to integrate relevant knowledge in a targeted manner to ensure that the generated suggestions can accurately and effectively support the completion process.

[0109] Receiving feedback. When the analysis agent receives a request from the evaluation agent to retrieve more knowledge, it uses these specific feedback as prompt information and automatically triggers a dynamic optimization process, i.e., the collaboration between the analysis agent and the completion agent enters a new iteration cycle. At this time, the analysis agent re-executes steps (1)-(3) and adjusts the knowledge retrieval strategy according to the requirements of the evaluation agent to further generate more detailed suggestions to help the completion agent perform a new round of missing content generation process.

[0110] Completion phase

[0111] In the defect report completion and optimization process, the work of the completion agent is carried out under the collaboration and supervision of the analysis agent and the evaluation agent. This process can be represented as:

[0112]

[0113] where X represents a shared variable. The completion agent, under the guidance of the input defect report text X, prompt instructions P, and completion suggestions suggest analyse , completes the report output of the key field content. The content in X is updated accordingly. In this phase, the completion agent is mainly responsible for two core responsibilities.

[0114] (1) Generating missing content based on the analysis agent's suggestions. The completion agent receives the completion suggestions including missing fields (e.g., "reproduction steps", "expected behavior", etc.) and details to be considered during the completion process, and uses LLM to perform text generation tasks. The generated content usually follows a specific format and structure as required by the instructions to ensure the completeness, clarity, and accuracy of the content. Note that the completion agent is only responsible for generating missing content and does not change the content of other fields.

[0115] (2) Content optimization based on the suggestions of the evaluation agent. The completion agent will adjust the generated content according to the feedback of the evaluation agent. It mainly includes two aspects: format and readability. In terms of format, the defect report is required to be strictly output in JSON format, and contains five parts: number, title, observed behavior, expected behavior, and reproduction steps. This makes the structure of the file clear and hierarchical, helping developers quickly understand the content when reviewing the report. Therefore, when the evaluation agent finds that the content format of the report is incorrect or inconsistent with the preset, it will return the suggestion "There is a problem with the report format, please reorganize the report content in JSON format". In terms of readability, when the evaluation agent provides suggestions such as "The content of the reproduction steps contains redundant and repetitive expressions, please reorganize this part of the content to ensure the simplicity of the report", the completion agent will optimize the corresponding part according to the requirements.

[0116] The behavior of the completion agent is guided by the structure prompt instructions to ensure that it performs the above responsibilities with high quality. The instruction settings are as shown in Figure 5 .

[0117] Evaluation phase

[0118] In this phase, the evaluation agent is responsible for evaluating the quality of the defect report and determining whether to approve the current version or return feedback suggestions based on the evaluation results. Its core responsibilities focus on evaluating report quality and dynamically selecting appropriate feedback paths. This process can be represented as:

[0119]

[0120] Where the output of the evaluation agent LLM evaluate contains two results: pass or suggestion (request). When the report quality does not pass, the evaluation agent returns the defect report content X to the completion agent for modification suggestions or to the analysis agent for retrieval requests under the guidance of the structured prompt instructions P. The specific structured instruction settings are shown in Figure 6 . The evaluation agent relies on the following three key technology modules.

[0121] Quality evaluation mechanism. The evaluation agent uses pre-defined prompts to guide the LLM to evaluate the quality of the completed content from three aspects: report format correctness, content completeness and readability, and information richness, to determine whether it meets the expected standards. Specifically, the report is required to be output in JSON format and contain the contents of the five fields: number, title, observed behavior, expected behavior, and reproduction steps. Completeness helps to check whether the missing content has been completed. Readability and information richness further ensure that the content of the report can effectively help developers to perform defect repair behavior.

[0122] Feedback path triggering mechanism. According to the type of problem identified in the evaluation process, the evaluation agent automatically selects the feedback path to support subsequent optimization. For example, if the agent determines that there are problems such as semantic ambiguity or errors in the report, it directly sends feedback to the completion agent and provides modification suggestions. If the evaluation result shows that the report lacks key information or the amount of completion content is insufficient, the feedback request is sent to the analysis agent, guiding it to retrieve relevant background knowledge and generate new completion suggestions, thereby performing a new round of completion process.

[0123] Multi-round review and re-evaluation mechanism. The evaluation agent supports a multi-round iterative review process, and for the revised version submitted by the completion agent, the system restarts the quality evaluation process until the current version of the defect report is passed. A maximum number of iterations N is defined, and when the maximum number of iterations is reached or the evaluation agent confirms that the report quality is qualified, the workflow terminates and outputs the final defect report content.

[0124] Experimental analysis

[0125] Research questions

[0126] In order to evaluate the performance of the method of the present application, the following five research questions are designed.

[0127] Research question one: how does the performance of the present application compare to existing defect report completion methods? Specifically, compare the performance of the present application method with the optimal method of defect report completion and optimization in the traditional machine learning paradigm and LLM paradigm in terms of BLEU, Sentence-BERT, ROUGE-L, and METEOR indicators.

[0128] Research question two: how does the performance of the present application method change based on different LLMs as agent experts? Select LLaMA, Qwen, and DeepSeek large language models with different parameter quantities as agent experts to evaluate the impact of different LLMs on the performance of the text method.

[0129] Research question three: what is the impact of the phased and multi-agent collaboration in the present application? That is, what is the contribution of each agent expert in the process of completing complex completion and optimization tasks?

[0130] Research question four: how does the number of iterations of feedback and the missing of different fields affect the performance of the present application method? Set the number of iterations of feedback as a hyperparameter and observe its impact on the performance of the text method. In addition, statistics on the performance of the present application method on different fields of completion content in the four indicators.

[0131] Research Question Five: Can the hierarchical retrieval matching method improve the accuracy of knowledge retrieval? Specifically, we will observe the impact of using the hierarchical retrieval matching method for similar defect report retrieval on performance. In addition, we will set the number of retrieved defect reports as a hyperparameter m, and observe its value on the performance of the method in the four indicators.

[0132] Experimental Setup

[0133] Experimental Data

[0134] Four widely used open source projects were selected to build the experimental data set, including Github, Eclipse, Mozilla and Tomcat, as shown in Table 1. The four data sets jointly cover the client, system, server and general open source full link ecology, and cover Java, C, C++ and other mainstream programming languages. For example, Github covers tool libraries, front-end frameworks and DevOps and other application scenarios, involving Java, Python and C++ programming languages. Eclipse involves editors, compilers, UI and tool chain integration, text specifications, and detailed reproduction information. Mozilla involves multi-process, graphics rendering and network protocol system-level issues with strict templates. Tomcat is mainly based on Java, emphasizing concurrency, resource management and configuration and other server-side features. To verify the performance of the method, 600 information-complete defect reports were randomly selected from each project, totaling 2400 defect reports. The defect reports in each project were processed by randomly deleting observation behavior, expected behavior and reproduction step information. This deletion strategy includes deleting one or two pieces of information in the observation behavior, expected behavior and reproduction step, so that each complete defect report corresponds to two incomplete defect report variants. Considering the computational cost of large language models, a stratified random sampling method was used to extract 50% (i.e. 300) from the incomplete samples of each project, resulting in 1200 experimental samples. The current experimental setup ensures data diversity while controlling the experimental scale. Since the experimental results need to be compared with the real complete defect reports, the element deletion simulates the incomplete report situation in the real scene. All data processing processes are completed through automated scripts to ensure experimental repeatability. In addition, 1200 complete defect reports were additionally selected to perform the hierarchical retrieval matching method training process.

[0135] Table 1. Detailed information of each data set

[0136]

[0137] Experimental Indicators

[0138] To comprehensively assess the quality of defect report completion, a multi-dimensional evaluation index system is adopted, including surface matching degree, semantic similarity, and text fluency. Four indicators are used: BLEU, SentenceBERT, ROUGE-L, and METEOR.

[0139] BLEU (Bilingual Evaluation Understudy): BLEU is widely used in natural language processing (NLP) text generation tasks. It is mainly used to quantify the similarity between automatically generated text and reference text. In the defect report completion task, BLEU evaluates the similarity by analyzing the overlap of n-grams between the completed defect report text and the actual defect report text.

[0140] SentenceBERT: SentenceBERT focuses on evaluating semantic similarity. It maps the completed defect report text and the actual defect report text to a unified vector space and calculates the cosine similarity between them to measure the semantic closeness. This method allows a deeper understanding of the similarity of the report content, rather than just relying on surface lexical matching.

[0141] ROUGE-L (Recall-Oriented Understudy for Gisting Evaluation): ROUGE-L captures sentence-level structural similarity by considering the order of words in generated and actual texts. This evaluation method not only focuses on word matching, but also values the arrangement of words in sentences, providing a more detailed understanding of text coherence and relevance. In the defect report completion task, the similarity is evaluated by comparing the LCS between the generated content and the actual content.

[0142] METEOR (Metric for Evaluation of Translation with Explicit ORdering): METEOR is a comprehensive evaluation index. Compared with traditional evaluation methods such as BLEU, it combines language features such as stems and synonyms, aligns words between generated and reference summaries, allows partial matching, and better handles different meaning expressions. At the same time, METEOR penalizes sequence differences, enhancing the evaluation of output fluency and grammar. Overall, METEOR is a comprehensive index that considers multiple factors to more accurately measure the quality of generated summaries. In this invention, it evaluates the quality by comparing the word-level matching between the generated text and the actual text.

[0143] Baseline

[0144] To fully demonstrate the advantages of the proposed method and more objectively evaluate the performance of the model, the following three baseline models are selected.

[0145] BEE: Automatically check the integrity of defect reports using machine learning techniques, and request the reporter to provide missing information by actively contacting them.

[0146] ChatBR: Identify the integrity of defect reports by using a fine-tuned BERT model, and implement automatic completion of defect reports with the help of LLM through prompt learning techniques.

[0147] AdbGPT: Help LLM acquire human knowledge and logical reasoning ability by using few-shot learning and chain-of-thought reasoning, so as to generate reproduction steps in a similar way to developers.

[0148] Tab: First train a BERT model to segment defect reports into four components, then perform title generation tasks on templated reports based on a single LLM.

[0149] BRMDS: By integrating PEFT strategies and carefully designed instruction datasets, it can generate comprehensive, accurate and easy-to-understand multi-dimensional summaries.

[0150] Large language models

[0151] To verify the universality of the proposed method for different LLMs, experiments were conducted on a variety of LLMs of different sizes. The following is a detailed introduction to these LLMs.

[0152] LLaMA: LLaMA is a series of open-source large language models developed by Meta, focusing on natural language processing tasks. The model performs outstandingly in the field of software engineering, and can assist in generating summaries, defect detection and software design, etc. In this invention, two versions of Llama-3-3B and Llama-3-8B are selected.

[0153] Qwen: Qwen is a Transformer-based language model developed by Alibaba Cloud, which is pre-trained through the task of predicting the next word, and is suitable for text generation, question answering, dialogue, translation and text classification, etc. The model provides multiple sizes from 0.5B to 72B parameters, and this invention selects two versions of Qwen-2.5-14B and Qwen-2.5-32B.

[0154] DeepSeek: DeepSeek, proposed by DeepSeek, Inc., has main functions including text generation, dialogue capability, code writing, mathematical calculation and reasoning tasks, etc. It can be integrated into various downstream systems or applications to provide intelligent dialogue and content generation services for users. The invention selects two versions of DeepSeek-R1 and DeepSeek-V3.

[0155] GPT: GPT, a Transformer-based autoregressive language model developed by OpenAI, is pre-trained on a large scale using next-token prediction. The model is suitable for text generation, question answering, dialogue, translation, summarization, information extraction and classification. The invention selects two versions of GPT-4o and GPT-o3.

[0156] Claude: Claude, a language model developed by Anthropic, is suitable for tasks such as text generation, question answering and dialogue, and performs well in long text understanding and document and image analysis. The invention selects two versions of Claude-3-Sonnet and Claude-3-Opus.

[0157] Parameter settings

[0158] For the invention and other methods, the API of different LLMs is used for access, which does not receive updates. In all experiments, the maximum token is set to 512. For the invention, the number of iterative feedbacks between roles is limited to 4 times. For the parameter settings during the running of the baseline methods, refer to the results they reported in the original paper.

[0159] Research question one: how does the invention perform compared with existing bug report completion methods?

[0160] Table 2. Performance comparison of the invention method and other baselines

[0161]

[0162] Table 2 shows the performance of the method of the present application (based on the running results of GPT-o3) compared with the existing baseline in terms of BLEU, SentenceBERT, ROUGE-L and METEOR indicators. As can be seen from the table, compared with ChatBR which is suboptimal, the performance of the method of the present application in the four indicators is increased by 10.41%, 7.52%, 13.55% and 16.64% respectively on the four public data sets. This shows that the method of the present application can more accurately identify the missing information of the defect report, generate more true and high-quality text content, and be more clear in language expression, in line with human language habits. The inventors believe that this advantage may be due to the task decoupling and role solidification helping the agent experts in different stages to complete the subtasks with high quality. In addition, compared with the one-time filling machine learning method and the method based on a single LLM, the performance of the method of the present application is significantly superior. This advantage comes from the iterative feedback mechanism that can promote the collaboration and supervision among multiple agents, fully exert the advantages of collective intelligence, realize multiple cross-validation and collaborative optimization of the same defect report, and thus significantly improve the report quality. In addition, the performance of the method of the present application is also superior to the AdbGPT method which uses few-shot learning and chain-of-thought reasoning to complete the defect report. It is worth mentioning that chain-of-thought reasoning is a successful prompting technique that solves reasoning problems by generating intermediate reasoning steps. However, unlike the present application, AdbGPT lacks further review and revision of the generated results when generating defect reports, and cannot perform timely quality control and adjustment at each stage, thereby limiting its overall performance. In contrast, the present application can ensure the quality of the final report through the dynamic feedback and iterative optimization mechanism.

[0163] Research question two: what is the impact of different LLMs as agent experts on the performance of the method of the present application?

[0164] Table 3. Impact of different LLMs on the performance of the method of the present application

[0165]

[0166] To investigate the performance of the model provided by the present application on different types and scales of LLMs, the defect report completion task was implemented based on Llama, Qwen, DeepSeek, GPT and Claude, respectively, and the experimental results are shown in Table 3. As can be seen from the table, the model performance improves with the increase of the parameter scale, which is reasonable. Because a larger scale model can better capture the details in the existing information of the defect report, thereby improving the quality of text generation, especially in complex natural language processing tasks, a larger model can understand and process more information, and then generate more accurate and coherent text. Secondly, among the LLMs used, it was found that GPT-o3 performed best in the defect report completion generation task, providing higher quality generation results than other models. For example, in terms of BLEU and SentenceBERT indicators, the performance of GPT-o3 was improved by 6.93% and 11.07% respectively compared to Llama-3-3B on four data sets. This advantage reflects the outstanding ability of GPT-o3 in generating text quality. Compared with GPT-4o, the advantage of GPT-o3 mainly lies in its processing capability for complex reasoning and logical analysis tasks. GPT-o3 focuses on the cooperation and feedback between multiple agents in the defect report completion task, and constantly optimizes the report content. In this task that requires high coordination and multi-round feedback, GPT-o3 can generate more complete content that meets the requirements through more accurate reasoning and logical analysis. Therefore, GPT-o3 can fully exert its advantages in handling such tasks and perform well. In addition, Claude focuses on stability and security, resulting in overly conservative and indecisive generated content, which may not have the depth of support and flexibility of GPT models in text generation tasks that require specific details to be supplemented.

[0167] Research question three: What is the impact of phased and multi-agent collaboration in the present application?

[0168] Since the completion agent is responsible for performing the text generation task of the missing field, in order to ensure the smooth completion of the task, the completion agent is not subjected to ablation experiments. However, in order to evaluate the contribution of the analysis agent and the evaluation agent in the completion process of the missing field, ablation experiments of different agents are conducted.

[0169] w / o analyze: indicates that the analysis agent is removed, and the defect report completion and optimization task is performed only by the cooperation of the completion agent and the evaluation agent. In this case, the role of the analysis agent is removed, and the completion agent relies on the feedback of the evaluation agent to supplement the missing content without any analysis and external knowledge guidance about the existing content of the report.

[0170] w / o evaluate: means removing the evaluation agent, relying only on the collaboration of the analysis agent and the completion agent to perform the defect report completion and optimization tasks. In this case, the role of the missing evaluation agent, although the analysis agent still performs contextual analysis on the report, retrieves external knowledge, and provides completion suggestions, lacks the evaluation and feedback of the completion content quality.

[0171] Table 4. Impact of missing different agent experts on the performance of defect report completion

[0172]

[0173] The experimental results are shown in Table 4. From the table, it can be seen that the absence of any role will have a significant impact on the performance of defect report completion and optimization. Specifically, when the analysis agent is missing, the performance of the generated defect report content on the four indicators decreases by 3.80%, 2.31%, 5.27%, and 5.45%, respectively. This result shows that the analysis agent plays a crucial role in the completion process, as they are responsible for identifying missing key information from the context and providing detailed completion suggestions. The absence of the analysis agent will result in a lack of proper guidance for the completion task, affecting the completeness and accuracy of the report. When the evaluation agent is missing, the performance of the generated defect report content on the four indicators decreases by 6.85%, 5.92%, 9.62%, and 10.77%, respectively. This shows that the evaluation agent plays a crucial role in the quality of the completion content, especially during the iterative feedback process. The evaluation agent checks and optimizes the completion content to ensure that the generated report meets high-quality standards, and the absence of this role disrupts the phased multi-agent collaboration mechanism in the completion process, resulting in a significant decrease in the quality of the completion content. In summary, the analysis agent and the evaluation agent play important roles in the team. The absence of any role disrupts the collaboration and feedback process between multiple agents, resulting in a decrease in the quality of the generated information. Therefore, in the method of the present invention, the analysis agent and the evaluation agent are indispensable, and only when they work together can the complex task be completed with high quality.

[0174] Research question four: How does the number of iterative feedbacks and the absence of different fields affect the performance of the method of the present invention?

[0175] To study the impact of the number of iterative feedbacks on the performance of the model of the present invention, the number of iterations is set as a hyperparameter t, and the impact of t = 2, t = 4, and t = 6 on the performance of the model of the present invention on the four evaluation indicators is observed, respectively. The experimental results are shown in Table 5. Figures 7 to 10

[0176] ​As can be seen from the figure, with the increase of the number of iterations t, the model performance presents a trend of first rising and then falling, and reaches the best performance at t=4. This shows that reasonable setting of the number of iteration feedbacks can effectively improve the model performance, but excessive multi-agent iteration may lead to performance decline. This phenomenon may be closely related to the hallucination phenomenon of LLM. Although the present application decouples the defect report completion and optimization into three independent stages through task decoupling, effectively reducing the complexity of the task and the cognitive load of the agent expert. But the multi-agent collaboration process may exacerbate the spread of false information. Specifically, with the increase of the number of iterations, the model may begin to "overfit" some error patterns, and even minor false information may be accumulated and spread in each iteration, eventually leading to the decline of the quality of the generated content. Therefore, it is particularly important to select a reasonable number of iteration feedbacks during the operation of the present application, which helps to better control the quality of the model generated content.

[0177] In addition, the inventors studied the performance of the method of the present application in completing different missing fields (observed behavior OB, expected behavior EB and reproduction step S2R) on four indicators. The experimental results are shown in Figures 11 to 18

[0178] From Figures 11 to 18 ​It can be seen that the present application performs best in completing the expected behavior field content, while it is more difficult to complete the reproduction step field. This phenomenon is reasonable. Specifically, the expected behavior describes the expected system behavior in the defect report, which is usually closely related to user needs, system design and functional expectations. Completing the expected behavior usually involves understanding of system functions and speculation of common usage patterns. Since the expected behavior is essentially a description of an ideal situation, it can usually be inferred from the existing system function design. For example, if the report describes a failure of a functional module, the completion agent can infer the behavior that the system should exhibit without defects according to the design goals of the functional module. Since the expected behavior usually has a relatively standardized expected behavior, the completion process is relatively simple, and the present application performs best in this regard. In contrast, completing the reproduction step not only involves detailed operation steps, but also needs to consider different environments, configurations and input data and other factors. Especially in the absence of direct records and detailed information, the completion of the reproduction step is very challenging. For example, the reproduction step may need to accurately describe the version of the operating system, the hardware configuration, the type and order of specific input data, etc. These information are often dynamic and often lack explicit records. If the reproduction step description in the defect report is not clear, the completion task becomes more complex and may require multiple iterations and more refined reasoning. However, the present application successfully takes advantage of team collaboration through the mechanism of phased multi-agent collaboration and to some extent makes up for the deficiencies in the completion process of the reproduction step. Even in complex and insufficient information situations, the quality of the generated content can be improved through multiple rounds of iterative feedback.

[0179] Research Question Five: How do hierarchical retrieval matching methods and different hyperparameters m affect the performance of the present application method?

[0180] To study the influence of hierarchical retrieval matching methods and hyperparameters k on the performance of the present application model, the inventors conducted relevant experiments to observe the performance of the generated content in four performance indicators. The experimental results are shown in Table 5. Among them, w / o match-2s represents the experimental results of removing the hierarchical retrieval matching method. For the hyperparameter m, the influence of m = 2, m = 4 and m = 6 on the performance of the present application model in the four evaluation indicators was observed.

[0181] Table 5. Influence of hierarchical retrieval matching methods and hyperparameters k on defect report completion performance

[0182]

[0183] As can be seen from the table, the hierarchical retrieval matching method proposed in the application significantly improves the quality of the completed content. Specifically, when the hierarchical retrieval matching method is not used for similar defect report group retrieval, the performance of the generated defect report content on the four indicators decreases by 3.07%, 3.22%, 3.22% and 3.48% respectively. This result shows that the knowledge provided by the hierarchical matching retrieval method is crucial to the analysis agent in generating the completion suggestion process. Similar defect reports can help the analysis agent quickly focus among numerous defect reports, understand the causes and forms of the current defect, and thus form the optimal completion suggestion. This not only avoids generating irrelevant or inaccurate content, but also effectively alleviates the impact of the illusion phenomenon. In addition, the influence of the number of similar defect report groups on the performance of the model is further analyzed. As can be seen from Table 5, when m=4, the performance of the completed text on the four indicators is optimal. The main reason for this phenomenon may be that when m=2, the provided knowledge is not enough to help the analysis agent form an accurate completion suggestion; and when m=6, since the prompt information is input to the analysis agent in the form of text, limited by the size of the input window, truncation phenomenon will occur, and all the retrieved defect reports cannot be completely input into the LLM. Therefore, selecting the appropriate number of defect reports has a significant impact on the performance of the method of the application. Nevertheless, the experimental results fully demonstrate the effectiveness of the hierarchical retrieval matching method.

[0184] Human evaluation

[0185] In order to evaluate the completion performance of the method of the application, the inventors recruited 8 experts with software development experience to conduct experiments, including 3 graduate students and 5 developers with more than three years of research and development experience. During the experiment, the participants completed the experiment independently and did not communicate with each other. The experiment focused on evaluating the completeness, clarity and reproducibility of the defect reports completed by different methods. Each participant was required to randomly select 20 defect reports from the four public data sets for completion. At the end of the experiment, the defect reports completed by different methods were displayed, and the participants were asked to provide feedback. The participants scored the defect reports through a 5-Likert scale. The experimental results were statistically tested at the set significance level (p<0.05).

[0186] Table 6. Results of human evaluation

[0187]

[0188] After comparing the contents of the reports completed by different methods, all participants expressed strong support for using the method of the present application in actual work, with average preference scores of 4.24, 4.08 and 4.46 for integrity, clarity and reproducibility. In order to further understand the effectiveness of the method of the present application, the inventors collected feedback from the participants and summarized two practical challenges faced when manually completing the defect report. First, it is very time-consuming to understand the context of the existing defect report, because it is necessary to master the development purpose of different projects, adapt to defect report content written by users with cultural differences and different skill levels, and analyze potential operations and components, etc. Second, it is not easy to obtain feasible suggestions to perform the completion process, and it is also difficult to determine the correctness of the completion content, resulting in that the quality of the final defect report is often closely related to the historical experience and expertise of the participants. This problem is particularly prominent for junior completion personnel who are not familiar with the application.

[0189] Discussion and threats to validity

[0190] The method of the present application only uses the widely recognized LLM (Llama, Qwen, DeepSeek, GPT and Claude) to study the influence of multi-agent collaboration on the performance of the final model. With the rapid development of LLM, although it is impossible to evaluate the precise influence of various LLMs on the performance of the present application, the model of the present application can always exhibit excellent defect report completion performance regardless of the agent expert constructed based on which LLM. This is because the method of the present application can introduce the idea of "human expert collaboration" into the defect report completion and optimization task, and ensure the high-quality completion of each sub-task based on the "multi-agent collaboration in stages" mechanism, dynamic feedback and iterative optimization mechanism, and the design of special structured instructions, rather than relying only on the advantages of LLM in natural language understanding ability. In addition, the inventors only focus on the text content of the defect report, ignoring the use of running screenshots submitted by users to help complete the report. However, although running screenshots can visually display the running results of the system when the defect occurs, further analysis of the experimental data shows that the observed behavior in the defect report is essentially a text description of the screenshot information by the user. Therefore, it is unnecessary to invest additional time and resources to process the screenshot information.

[0191] In addition, the inventors collected 2400 bug reports from four open-source projects, Github, Eclipse, Mozilla, and Tomcat. The reports are mostly in unstructured text form, and the length and quality of the text vary greatly. Since the specific bug report datasets used by other researchers differ, the inventors cannot assert that the performance of the models executed on the current dataset will be exactly the same on different datasets. However, the inventors provide detailed descriptions of the experimental settings, which will help to reproduce the inventors' experiments using more datasets in practical development.

[0192] Related work

[0193] Bug report completion

[0194] Existing research on improving the quality of bug reports mainly focuses on improving or completing key information such as observed behavior, expected behavior, and reproduction steps. For example, Feng et al. proposed the AdbGPT model, which uses few-shot learning and thought chains to help LLM generate reproduction steps; ReCDroid+ proposed by Zhao et al. uses lexical knowledge in bug reports to automatically reproduce crashes by improving program analysis and natural language processing tools Yakusu; Song et al. trained a classification model to determine missing key information and used an interactive chatbot system to request necessary information from the submitter. In addition, research on improving bug report titles has also attracted widespread attention. For example, Chen et al. proposed the iTAPE model based on the sequence-to-sequence framework to generate report titles; Zhang et al. fine-tuned the BART model to generate bug report titles; Rastkar et al. used natural language processing techniques to summarize bug reports to generate concise title information. Unlike the above methods, the method proposed by the invention divides the complex task of bug report completion and optimization into stages by introducing the idea of "human expert collaboration" and guides different agent experts to complete sub-tasks with high quality through structured instructions. This strategy aims to effectively utilize the powerful capabilities of LLM to understand and generate natural language text that meets task requirements.

[0195] Bug report evaluation

[0196] In the open source community, the quality of bug reports is crucial for developers to locate and fix bugs in a timely and accurate manner. Therefore, the research on bug report quality assessment has gradually gained attention. Most existing researches adopt machine learning (ML) or heuristic rules to classify the report quality by extracting various metrics related to bug reports. For example, Fan et al. extracted features from five dimensions, including reporter experience, collaboration network, completeness, readability, and text, and used a random forest classifier to identify effective bug reports; Zimmermann et al. designed CUEZILLA, which selected the information that bug fixers expected users to provide as features and used supervised machine learning algorithms to train a prediction model to assess the quality of bug reports. Unlike CUEZILLA, Vasumathi et al. used four different text classification techniques to build and evaluate classification models to predict key features based on historical bug fixing knowledge. In addition, Chaparro et al. developed DeMIBuD, which combines regular expressions, heuristic rules, natural language processing (NLP), and machine learning (ML) techniques to automatically detect missing observed behavior, expected behavior, and reproduction step elements. Unlike the above methods, the present invention realizes multi-agent collaboration based on a dynamic feedback and iterative optimization mechanism to complete and optimize bug report completion and optimization tasks.

[0197] Multi-agent collaboration

[0198] With the latest advancements in large language models, multi-agent collaboration in the real world has become very prominent, enabling them to perform complex tasks such as creative writing, reasoning, and decision-making. For example, the MetaGPT framework proposed by Hong et al. introduces meta-programming techniques, including structured workflows and processes in agent interactions, to decompose complex problems. CAMEL provides a role-playing framework where a task-specific agent and two cooperating LLM agents (user and assistant) complete tasks through role-based dialogues. AutoGen enables developers to define flexible agent behaviors and communication patterns, allowing LLM agents to collaborate through dialogues and handle complex tasks by breaking them down into manageable subtasks. Additionally, the self-coordination proposed by Dong et al. instantiates different domain experts through prompt learning to collaboratively handle the needs of each stage of a code generation task. In situations where goals conflict or resources are limited, agents prioritize their individual goals, leading to conflicts or opposition with others, causing competition problems among agents. Despite this, this competition can still develop towards collective goals, such as in debate, strategic games, and other scenarios. For example, LLMARENAb is an agent collaboration framework that primarily uses competition as the main type of collaboration, benchmarking in 7 dynamic game environments, enabling skills such as spatial reasoning, strategic planning, numerical reasoning, risk assessment, communication, opponent modeling, and team collaboration. CompeteAI also simulates a game environment where 2 agents play the roles of two restaurant managers competing for 50 customers, illustrating how structured competition drives agents to improve their strategies, consistent with several classic sociological and economic theories. In LEGO, a multi-agent collaboration framework is introduced to generate causal explanation, with pre-defined competitive collaboration links. In MetaQA, collaboration between LLM agents occurs during the early stages of training, where multiple specialist agents are trained together through a target combination that encourages agents to compete for the best candidate answer and identify the agent trained in the domain of the input question. Competitive methods offer some advantages, such as promoting robustness, strategic adaptability, and complex problem-solving capabilities within multi-agents. However, competition also presents challenges, including potential conflicts that require mechanisms to ensure that competition remains constructive and beneficial to overall system goals.

[0199] SUMMARY

[0200] In the present application, the inventors propose a method for defect report completion and optimization based on multi-agent collaboration. To achieve this goal, the inventors decouple the complex task into three stages: defect analysis, report completion, and quality evaluation, and design independent large language model agents for each stage based on structured instructions. To achieve efficient collaboration between multiple agents, a dynamic feedback and iterative optimization mechanism is introduced, allowing the three agents to cross-verify and collaboratively optimize around the same defect report. The semantic drift problem is effectively controlled, ensuring logical coherence and consistent completion content with facts. Compared with existing methods, the present application can effectively complete the completion task and gradually improve the quality of the completion content. Experimental results show that the performance of the present application on four public datasets is better than that of the most advanced baseline method.

[0201] Embodiment Two

[0202] As shown in Figure 19 , the present embodiment provides a defect report completion and optimization system based on multi-agent collaboration, comprising:

[0203] A report acquisition module for acquiring an original defect report;

[0204] A task decoupling module for decoupling the completion and optimization task of the original defect report into three sequentially executed sub-tasks of defect analysis, report completion, and quality evaluation;

[0205] A defect analysis module for analyzing the input original defect report through an analysis agent, identifying the missing key information fields therein, and generating completion suggestions based on the retrieved external knowledge;

[0206] A report completion module for receiving the completion suggestions output by the defect analysis through a completion agent, generating and completing the missing content in the original defect report based on the completion suggestions, and obtaining a completed report;

[0207] A quality evaluation module for evaluating the quality of the completed report through an evaluation agent; if the quality evaluation fails, feedback information is generated according to the evaluation results and sent to the defect analysis sub-task and / or the report completion sub-task, triggering a new round of defect analysis and / or report completion until the evaluation passes or the maximum number of iterations is reached.

[0208] Embodiment Three

[0209] The present embodiment provides a defect report completion and optimization device based on multi-agent collaboration, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to implement the above-mentioned defect report completion and optimization method based on multi-agent collaboration when executing the computer program.

[0210] The processor is connected with the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor executes the one or more computer programs stored in the memory, so that the electronic device executes the method in Embodiment One.

[0211] It should be understood that, in the embodiments, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0212] The memory can include read-only memory and random access memory, and provide instructions and data for the processor. Part of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0213] In the implementation process, each step of the above method can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software.

[0214] The method in Embodiment One can be directly embodied as hardware processor execution completion, or executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads information in the memory and combines hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0215] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software mode depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0216] Embodiment Four

[0217] Another embodiment of the present application provides a computer readable storage medium storing a computer program. When the computer program is executed by a processor, a defect report completion and optimization method based on multi-agent cooperation is realized.

[0218] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), or the like. In this application, the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present application. In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0219] Although the present application is disclosed as above, the protection scope of the present application is not limited to this. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, and these changes and modifications will fall within the protection scope of the present application.

Claims

1. A defect report completion and optimization method based on multi-agent cooperation, characterized in that, The method comprises the following steps: obtaining an original defect report; decoupling the completion and optimization task of the original defect report into three sequentially executed sub-tasks of defect analysis, report completion and quality evaluation; the defect analysis sub-task analyzes the input original defect report through an analysis agent, identifies the missing key information fields therein, and generates a completion suggestion based on the retrieved external knowledge; the report completion sub-task receives the completion suggestion output by the defect analysis through a completion agent, generates and completes the missing content in the original defect report based on the completion suggestion, and obtains a completed report; the quality evaluation sub-task evaluates the quality of the completed report through an evaluation agent; if the quality evaluation passes, the completed and optimized report is output; if the quality evaluation fails, feedback information is generated according to the evaluation result, and the feedback information is sent to the defect analysis sub-task and / or the report completion sub-task to trigger a new round of defect analysis and / or report completion until the evaluation passes or the maximum number of iterations is reached.

2. The method of claim 1, wherein, The analysis agent, completion agent and evaluation agent are all implemented by a large language model, and each agent is configured with a dedicated structured prompt instruction, which includes receiving information, team description and responsibility explanation.

3. The method of claim 1, wherein, The generation of the completion suggestion based on the retrieved external knowledge comprises: adopting a hierarchical retrieval matching method to retrieve external knowledge, which comprises: calculating the similarity between the target defect report and a plurality of information centroids pre-stored in the knowledge base, the information centroids being generated based on clustering of different types of key information in historical defect reports; selecting Top-K candidate knowledge groups from the knowledge base based on the centroid similarity; calculating the similarity between the target defect report and a specific sentence set in the candidate knowledge group to determine the most relevant target knowledge from the candidate knowledge group.

4. The method of claim 3, wherein, The information centroids include observation behavior centroids, expected behavior centroids and reproduction step centroids.

5. The method of claim 1, wherein, The analysis agent receives retrieval request feedback from the evaluation agent and adjusts the knowledge retrieval strategy based on the retrieval request feedback to regenerate the completion suggestion.

6. The method of claim 1, wherein, The completion agent receives modification suggestion feedback from the evaluation agent and optimizes the content of the completed report based on the modification suggestion feedback, the optimization content including format specification and / or content readability.

7. The method of claim 1, wherein, The quality evaluation of the completed report by the evaluation agent specifically comprises: evaluating the completed report from three dimensions of format correctness, content integrity and readability, and information richness; if the evaluation fails, determining the problem type: if it is a content semantic or format problem, sending a modification suggestion to the completion agent; if it is an information missing or insufficient information problem, sending a retrieval request to the analysis agent.

8. A defect report completion and optimization system based on multi-agent collaboration, characterized in that, The method comprises the following steps: a report acquisition module for acquiring an original defect report; a task decoupling module for decoupling the completion and optimization task of the original defect report into three sequentially executed sub-tasks of defect analysis, report completion and quality evaluation; The defect analysis module is configured to analyze an input original defect report by an analysis agent, identify missing key information fields therein, and generate a completion suggestion based on retrieved external knowledge; The report completion module is configured to receive the completion suggestion output by the defect analysis, generate and complete the missing content in the original defect report based on the completion suggestion, and obtain a completed report by a completion agent; The quality evaluation module is configured to evaluate the quality of the completed report by an evaluation agent; If the quality evaluation fails, feedback information is generated according to the evaluation result, and the feedback information is sent to the defect analysis subtask and / or the report completion subtask, triggering a new round of defect analysis and / or report completion until the evaluation passes or the maximum number of iterations is reached.

9. A defect report completion and optimization device based on multi-agent collaboration, characterized by, The device comprises a memory and a processor; the memory is used to store a computer program; the processor is used to realize the defect report completion and optimization method based on multi-agent collaboration according to any one of claims 1 to 7 when the computer program is executed.

10. A computer-readable storage medium, characterized in that, The storage medium has a computer program stored thereon, and when the computer program is executed by the processor, the defect report completion and optimization method based on multi-agent collaboration according to any one of claims 1 to 7 is realized.

Images