Large model-based multi-agent exploration-exploitation report generation method and system

Abstract

This invention discloses a multi-agent exploration-exploitation report generation method and system based on a large model, belonging to the field of artificial intelligence technology. Targeting scenarios such as research reports and industry analysis reports, this invention provides a framework for multi-agent coordinated planning, information retrieval, and tool invocation. Within this collaborative framework, it introduces a human-in-the-loop interaction mechanism, an exploration-exploitation dual-mode planning mechanism, and a dedicated citation quality control mechanism, thereby improving the report's structural rationality, content breadth and depth, and citation reliability. This invention can be used to generate structured research reports, industry analysis reports, and other long text documents requiring systematic information integration.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a multi-agent exploration-utilization report generation method and system based on a large model. Background Technology

[0002] With the improvement of large language model (LLM) capabilities, multi-agent automated research systems based on LLM have gradually emerged. Recent developments such as Storm, webThinker, several closed-source DeepResearch systems, and multi-agent research assistants built on frameworks like LangGraph, AutoGen, and n8n all attempt to generate structured research reports and industry analysis reports through agent division of labor, tool invocation, and iterative reasoning.

[0003] Although existing multi-agent systems have improved in task coverage and information integration compared to earlier RAG-based single-agent methods, several key technical shortcomings still exist in large-scale research report generation scenarios:

[0004] First, multi-agent collaboration still primarily relies on fixed processes or weak collaborative structures, and generally lacks effective human-in-the-loop mechanisms. Most systems employ a repetitive pipeline of "planning-retrieval-generation," lacking adaptive global coordination among agents. Furthermore, key decision-making stages (e.g., research brief confirmation, report outline determination, adoption of important conclusions and viewpoints) typically utilize fully automated generation processes, lacking interactive confirmation and feedback loops with users. When deviations occur in early planning or topic understanding, the system struggles to correct these deviations through multi-agent collaboration and lacks the ability to adjust its direction midway through user confirmation, causing errors to propagate across multiple iterations and ultimately be reflected in the final report.

[0005] Secondly, existing frameworks generally focus on convergent optimization, lacking systematic exploration capabilities. The report generation process typically involves supplementing content or refining chapters based on the initial outline, lacking a proactive mechanism for exploring "unknown thematic dimensions" and making it difficult to identify potential but important aspects within the research topic. In tasks requiring broad coverage and in-depth insights, such as research reports and industry analysis reports, this leads to incomplete information structure and insufficient perspectives.

[0006] Finally, inadequate citation quality control remains a common technical problem. While existing systems can automatically insert citations, they lack systematic citation verification mechanisms, such as checking the validity of citation links, the correspondence between citation content and the main text, the standardization of citation format, and the presence of low-quality citations. This results in significant deficiencies in the credibility, auditability, and traceability of automatically generated research / industry reports.

[0007] In summary, existing multi-agent deeppresearch report generation architectures still have shortcomings in terms of exploration capabilities, error propagation suppression, and citation quality control. Therefore, there is a need for an automated research and industry report generation method and system that can achieve exploration—utilizing balanced scheduling, structured planning evolution (dynamic outline), and citation quality assurance within a multi-agent collaborative framework. Summary of the Invention

[0008] The purpose of this invention is to solve the above-mentioned problems in the prior art and to provide a multi-agent exploration-utilization report generation method and system based on a large model.

[0009] The specific technical solution adopted in this invention is as follows:

[0010] In a first aspect, the present invention provides a multi-agent exploration-utilization report generation method based on a large model, comprising:

[0011] S1. Receive the report input by the user through the front-end interface to generate task instructions, construct a structured task description, and then the research supervision coordinator calls the tool agent cluster to generate an initial research brief and present it to the user through the front-end interface. Based on the human-in-the-loop confirmation mechanism, receive the user's adjustment and confirmation instructions and save the final research brief after user confirmation.

[0012] S2. Under the constraints of the final research briefing, the research supervision coordinator calls the dual-mode planning agent to generate the initial outline of the report, and continuously updates and optimizes the outline in an iterative manner. In each iteration, the optimization mode of the outline in this round must be determined based on the marginal benefit of the previous round, whether it is the exploration mode or the exploitation mode, and the marginal benefit of this round is calculated after the optimization is completed.

[0013] S3. After the iteration is completed, the outline is presented to the user through the front-end interface. Based on the human-in-the-loop confirmation mechanism, the user's adjustment and confirmation instructions are received. Then, the content generation agent generates chapter content by calling the large language model at the chapter level according to the outline confirmed by the user, and forms the report text. Finally, the citation optimization agent performs quality control and optimization on the citations in the text and generates a citation quality report. The final report text and citation quality report after citation optimization are output.

[0014] As a preferred embodiment of the first aspect above, in step S1, after receiving the report input by the user through the front-end interface and generating the task instruction, it is necessary to analyze whether there are any issues in the task instruction that require further clarification from the user. If so, a clarification request is initiated to the user through the front-end interface, and the original input task instruction and the clarification content input by the user are organized into a structured task description.

[0015] As a preferred embodiment of the first aspect above, in S1, after obtaining the structured task description, the research supervision coordinator first calls the tool agent cluster to extract the search query composed of keywords from the structured task description, and performs the first search in the data source containing industry information. The content generation agent summarizes the search results, extracts several candidate core analysis dimensions, and generates an initial research brief.

[0016] As a preferred embodiment of the first aspect above, in S2, after obtaining the final research brief, the research supervision coordinator calls the dual-mode planning agent to generate outline chapters that conform to the research brief in the utilization mode. Each chapter corresponds to a topic unit, and then it is presented to the user through the front-end interface. Based on the human-in-the-loop confirmation mechanism, the user's adjustment and confirmation instructions are received, and finally the outline confirmed by the user is saved as the starting point for iterative update and optimization.

[0017] As a preferred embodiment of the first aspect above, in S2, before continuously updating and optimizing the outline in an iterative manner, it is necessary to extract the topic units corresponding to each chapter from the initial outline and add them to the knowledge graph as initial topic nodes. At the same time, high-value insights are extracted from the existing search results and added to the knowledge graph as candidate concept nodes. The high-value insights are topic units whose value scores obtained by the dual-mode planning agent evaluation exceed a threshold.

[0018] As a preferred embodiment of the first aspect mentioned above, in S2, when the outline is continuously updated and optimized in an iterative manner, the process executed in each iteration is as follows:

[0019] First, determine whether the marginal revenue of the previous round is less than the preset revenue threshold. If it is less, the exploration mode is used in this round; otherwise, the exploitation mode is used.

[0020] In exploration mode, the dual-mode planning agent generates divergent retrieval instructions according to the multidimensional query framework and sends them to the tool agent cluster to perform the retrieval, obtain high-value insights that may be overlooked in the current outline, update them as candidate concept nodes to the knowledge graph, and reconstruct the current outline as new chapters.

[0021] In the utilization mode, the dual-mode planning agent identifies the chapters that need optimization for the current outline, generates targeted retrieval and optimization instructions for the knowledge gaps within the chapters, and uses the supplementary retrieval results to perform local incremental updates on the chapters that need optimization.

[0022] Then, for the outlines before and after this round of updates and optimizations, the information gain, semantic novelty and structural change between the two outline chapters are calculated, and the weighted sum of the three indicators is used as the marginal benefit of this round.

[0023] As a preferred embodiment of the first aspect mentioned above, in step S3, when the citation optimization agent performs quality control and optimization on the citations in the main text, it is necessary to structure all reference citation information in the report, verify the validity of citation links, repair invalid or incomplete citation links, enhance the citations of key paragraphs in the report, and automatically recommend supplementary authoritative industry research reports or official planning documents as support; finally, a citation quality report is generated for users to refer to and review.

[0024] Secondly, the present invention provides a report generation system, the functional modules of which include a user interaction module, a research supervision coordinator, a multi-agent planning layer including a dual-mode planning agent and a content generation agent, a tool agent cluster and a reference optimization agent, and the functional modules communicate with each other through a preset message protocol to form a multi-agent workflow that can realize the multi-agent exploration based on a large model and the report generation method as described in any of the first aspects above.

[0025] Thirdly, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, enables the implementation of the multi-agent exploration-utilization report generation method based on a large model as described in any of the first aspects above.

[0026] Fourthly, the present invention provides a computer electronic device, which includes a memory and a processor;

[0027] The memory is used to store computer programs;

[0028] The processor is configured to, when executing the computer program, implement the large-model-based multi-agent exploration-utilization report generation method as described in any of the first aspects above.

[0029] Compared with the prior art, the present invention has the following advantages:

[0030] This invention addresses scenarios such as research reports and industry analysis reports, providing a multi-agent coordinated planning, information retrieval, and tool invocation framework. It introduces human-in-the-loop interaction mechanisms, an exploration-utilization dual-mode planning mechanism, and a dedicated citation quality control mechanism within this multi-agent collaborative framework, thereby improving the report's structural rationality, content breadth and depth, and citation reliability. The multi-agent report generation method and system based on a large model provided by this invention can be used to generate structured research reports, industry analysis reports, and other long text documents requiring systematic information integration. Attached Figure Description

[0031] Figure 1 A schematic diagram illustrating the steps of multi-agent exploration based on a large model using report generation methods;

[0032] Figure 2 A schematic diagram of the module composition of the report generation system;

[0033] Figure 3 A schematic diagram of a computer electronic device;

[0034] Figure 4 This is a flowchart of the multi-agent report generation method based on a large model in the embodiment;

[0035] Figure 5 for Figure 4 Internal process diagram of the dual-mode planning sub-process;

[0036] Figure 6 for Figure 4 The diagram illustrates the internal process of the optimized subprocess. Detailed Implementation

[0037] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in various embodiments of the present invention can be combined accordingly without mutual conflict.

[0038] This invention provides a multi-agent exploration-utilization report generation method based on a large model. This method, grounded in a framework of multi-agent coordinated planning, information retrieval, and tool invocation, can generate structured research reports, industry analysis reports, and other long text documents requiring systematic information integration. For ease of description, this invention will collectively refer to these industry-related long text documents as "reports."

[0039] like Figure 1 As shown, in a preferred embodiment of the present invention, the above-described multi-agent exploration-exploitation report generation method based on a large model is implemented by steps S1 to S3. Figure 2 As shown, the entire method requires functional modules such as a user interaction module, a research supervision coordinator, a multi-agent planning layer, a tool agent cluster, and a citation optimization agent. These modules communicate via a pre-defined message protocol, forming an orchestratable and traceable multi-agent workflow. The user interaction module uses a front-end GUI interface, while the research supervision coordinator, multi-agent planning layer, tool agent cluster, and citation optimization agent are a series of agents based on a large model. The specific form of the large language model is not limited; it can be a general-purpose large model with generalization capabilities, or a specific large model can be trained separately for each required function. The purpose of each functional module is described below:

[0040] The user interaction module is used to receive user input of research topics, task requirements and feedback, and to display intermediate results and final reports during the generation process.

[0041] The Research Supervisor is used to maintain the global task state, schedule the execution order of other functional modules, and trigger human-in-the-loop interactions at preset key nodes.

[0042] The multi-agent planning layer includes a dual-mode planning agent and a content generation agent. The dual-mode planning agent is used to adaptively switch between "utilization mode" and "exploration mode" according to the current outline and report status, and generate the next planning instruction. The content generation agent is used to generate the corresponding chapter drafts according to the current outline nodes and related materials.

[0043] Tool intelligence agent clusters are used to call external retrieval and analysis tools or large model-based intelligence agents according to planning instructions to obtain and organize industry-related data;

[0044] The Citation Agent is used to extract, verify, repair, and enhance citations in the report, and output a citation quality report.

[0045] The knowledge graph and memory module is used to store the topic units extracted during the iteration process in the form of a knowledge graph (including key topic concept nodes and relationships between topics), and to store the execution logs during system operation.

[0046] Based on the above functional modules, the specific implementation of each of steps S1 to S3 of the multi-agent exploration-report generation method will be described in detail below.

[0047] S1. Receive the report generation task instructions input by the user through the front-end interface, construct a structured task description, and then the research supervision coordinator calls the tool agent cluster to generate an initial research brief, which is then presented to the user through the front-end interface. Based on the human-in-the-loop confirmation mechanism, it receives the user's adjustment and confirmation instructions and saves the final research brief after user confirmation.

[0048] It should be noted that the aforementioned report generation task instructions are the topic and requirements of the report input by the user. The topic can be the current development status of the target industry, while the requirements are the key directions or analytical dimensions that need to be focused on in the report. For example, the report generation task instruction could be "Help me research the current development status of industry A, focusing on industry layout, key enterprises, policy environment, and development bottlenecks." The report generation task instructions input by the user are often in natural language text format, which is not conducive to structured storage and extraction during subsequent analysis. Therefore, it is necessary to extract fields from the user input text and store them in a structured manner to form a structured task description.

[0049] Furthermore, the initial report generation task instruction input by the user may lack necessary information, such as unclear time and spatial scope of analysis, or ambiguous industry names. Therefore, in this embodiment of the invention, after receiving the report generation task instruction input by the user through the front-end interface in S1, it is necessary to analyze whether there are any issues in the task instruction that require further clarification from the user. If so, a clarification request is initiated to the user through the front-end interface, and the original input task instruction and the user's clarification content are organized into a structured task description.

[0050] It should also be noted that the aforementioned research brief is a concise text describing the report's analytical topic and core analytical dimensions, including at least the research scope, core issues, expected structure, and data source type. In embodiments of this invention, the research brief can incorporate a structured task description as a constraint, generated in conjunction with external retrieval tools and a large language model. Specifically, after obtaining the aforementioned structured task description, the research supervision coordinator first invokes a cluster of tool agents to extract a search query composed of keywords from the structured task description, and performs an initial search in a data source containing industry information. The content generation agent then summarizes the search results, extracts several candidate core analytical dimensions, and generates an initial research brief.

[0051] After generating the initial research brief, it needs to be presented to the user through a front-end interface, and the user's adjustment and confirmation instructions are received based on a human-in-the-loop confirmation mechanism. If the user is not satisfied with the initial research brief, they can input adjustment instructions through the front-end interface, specifying the specific content and direction of the adjustment. After the user inputs the adjustment instructions, the research supervision coordinator needs to call the tool agent cluster and content generation agent again to generate a new research brief based on the original structured task description and the user's input adjustment instructions. Then, it continues to be presented to the user through the front-end interface until the user confirms that the research brief meets the requirements, and then it can be saved as the final research brief.

[0052] The human-in-the-loop confirmation mechanism used in this invention aims to allow users to confirm, correct, and provide feedback on phased outputs at key nodes during the operation of a multi-agent framework, thereby reducing the accumulation and propagation of early deviations in multiple iterations.

[0053] S2. Under the constraints of the final research briefing, the research supervision coordinator calls the dual-mode planning agent to generate the initial outline of the report, and continuously updates and optimizes the outline in an iterative manner. In each iteration, the optimization mode of the outline in this round must be determined based on the marginal benefit of the previous round, whether it is the exploration mode or the exploitation mode, and the marginal benefit of this round is calculated after the optimization is completed.

[0054] It should be noted that the dual-mode planning agent is an agent based on a large model. Its core function is to plan the mode for generating or adjusting the report outline, which can be divided into two modes: exploration mode and exploitation mode. The detailed process of generating the initial outline and performing optimization is described below.

[0055] In an embodiment of the present invention, after obtaining the final research brief in S1, the research supervision coordinator calls the dual-mode planning agent to generate outline chapters that conform to the research brief in the utilization mode. Each chapter corresponds to a thematic unit, and each thematic unit is a dimension of industry analysis. For example, a common outline can be composed of thematic units such as general overview, industrial chain structure, industrial layout, representative enterprises, policy support, bottlenecks and challenges, suggestions and countermeasures, etc.

[0056] It should be noted that the outline in this invention consists of a series of chapters. Each chapter can be regarded as a first-level heading of the outline, and each chapter can be further subdivided into different subsections. These subsections can be structured in the order of second-level headings and third-level headings.

[0057] Once the outline chapters are obtained, the research supervision coordinator can present them to the user through the front-end interface. Based on the human-in-the-loop confirmation mechanism, it receives the user's adjustment and confirmation instructions, and finally saves the outline after user confirmation as the starting point for iterative updates and optimizations.

[0058] In an embodiment of the present invention, before iteratively updating and optimizing the outline, it is necessary to extract the topic units corresponding to each chapter from the initial outline and add them to the knowledge graph as initial topic nodes. At the same time, high-value insights are extracted from the existing search results and added to the knowledge graph as candidate concept nodes. The high-value insights are topic units whose value scores obtained by the dual-mode planning agent evaluation exceed a threshold. These topic units can be dimensions or perspectives of industry analysis or industry-related entities (such as companies, individuals, institutions, etc.).

[0059] In an embodiment of the present invention, when the outline is continuously updated and optimized in an iterative manner, the process executed in each iteration is as follows:

[0060] First, determine whether the marginal revenue of the previous round is less than the preset revenue threshold. If it is less, the exploration mode is used in this round; otherwise, the exploitation mode is used.

[0061] In exploration mode, the dual-mode planning agent generates divergent retrieval instructions according to the multidimensional query framework and sends them to the tool agent cluster to perform the retrieval, obtain high-value insights that may be overlooked in the current outline, update them as candidate concept nodes to the knowledge graph, and reconstruct the current outline as new chapters.

[0062] In the utilization mode, the dual-mode planning agent identifies the chapters that need optimization for the current outline, generates targeted retrieval and optimization instructions for the knowledge gaps within the chapters, and uses the supplementary retrieval results to perform local incremental updates on the chapters that need optimization.

[0063] Then, for the outlines before and after this round of updates and optimizations, the information gain, semantic novelty and structural change between the two outline chapters are calculated, and the weighted sum of the three indicators is used as the marginal benefit of this round.

[0064] It should be noted that in the exploration mode, the multi-dimensional query framework described above can preset the dimensions for retrieval, such as one or more of the following: historical, cross-domain, controversial, cutting-edge, practical, and theoretical dimensions. Retrieving industry-related information from different dimensions can collect as many high-value insights as possible. Divergent retrieval commands can be generated by a bimodal planning agent based on a large language model, generating industry information retrieval commands for each dimension, and then executing them by calling a cluster of tool agents, thereby obtaining retrieval results from different dimensions as broadly as possible. From these retrieval results, topic units (i.e., keywords) are extracted. The bimodal planning agent scores these topic units, assigning a value score to each. Topic units with value scores exceeding a threshold are considered high-value insights and added to the knowledge graph as candidate concept nodes. Furthermore, each high-value insight attached to the knowledge graph needs to have its value score recorded. The value score of each high-value insight can be obtained by the bimodal planning agent using the generalization ability of the large language model, comprehensively scoring novelty, relevance, and influence dimensions. When the knowledge graph is updated, it can be compared to see if there are any candidate concept nodes whose value scores exceed the score threshold but have not yet been included in the outline chapters. If so, the outline can be restructured, and the corresponding chapters can be automatically added or adjusted so that the new outline chapters contain the topic units corresponding to the candidate concept nodes.

[0065] In addition, in the utilization mode, the chapters of the outline need to remain unchanged, but targeted optimization of some weaker chapters is required based on the existing outline. Specific chapters requiring optimization can be identified from the current outline by the dual-mode planning agent based on the generalization ability of the large language model, prioritizing chapters with higher marginal benefits, lower quality, or those marked as "priority optimization" by the user through the front-end interface. For each chapter requiring optimization, knowledge gaps within the chapter can be identified (i.e., whether there are sections that need strengthening or whether new sections need to be added). For these knowledge gaps, the large language model can generate targeted retrieval and optimization instructions. The supplementary retrieval results are used to perform local incremental updates on the chapters requiring optimization. It is important to note that local incremental updates do not change the chapter's thematic units (i.e., the first-level headings remain unchanged); only the content within the chapter is adjusted.

[0066] Furthermore, when calculating the marginal benefit of the outlines before and after this round of updates and optimizations, three indicators need to be introduced: information gain, semantic novelty, and structural change. The weighted sum of these three indicators is used as the marginal benefit of this round. Specifically, the difference in information entropy calculated based on the topic distribution of the outlines before and after the iteration is used as the information gain of the outline chapters; semantic embedding is performed on the outlines before and after the iteration, and the difference between 1 and the cosine similarity is used as the semantic novelty; additionally, the rate of change in the number of outline chapters before and after the iteration is used as the structural change. Finally, the weighted sum of these three indicators yields the marginal benefit of this round.

[0067] During the iteration process, each iteration (i.e., each iteration step t) requires updating the outline of the optimization report and calculating the marginal revenue MG(t) for this round. When the next iteration (i.e., iteration step t+1) begins, it is necessary to first determine whether to adopt the exploration mode or the utilization mode for the t+1 iteration based on the marginal revenue MG(t). The judgment principle is that if the marginal revenue of the previous round is less than the preset revenue threshold, then the t+1 round adopts the exploration mode; otherwise, the t+1 round adopts the utilization mode.

[0068] However, in embodiments of the present invention, in order to avoid the difficulty in converging the outline due to excessively frequent exploration, and also to avoid long periods without exploration, the system can set periodic triggering rules. For example, after each use of exploration mode, the next round must be forced to use exploitation mode, and after exploitation mode has been executed continuously for a predetermined number of times, the next round must be forced to use one round of exploration mode.

[0069] S3. After the iteration is completed, the outline is presented to the user through the front-end interface. Based on the human-in-the-loop confirmation mechanism, the user's adjustment and confirmation instructions are received. Then, the content generation agent, according to the outline confirmed by the user, calls the large language model to generate chapter content at the chapter level to form the report text. Finally, the citation optimization agent performs quality control and optimization on the citations in the text, generating a citation quality report. Finally, the citation-optimized final report text and citation quality report are presented to the user through the front-end interface, or output in a predetermined format.

[0070] In embodiments of the present invention, when the content generation agent generates chapter content, it needs to generate the corresponding main text content for each chapter at the chapter level. When generating content for each chapter, the content generation agent analyzes and summarizes the relevant search results based on a large language model, and then outputs the corresponding main text content for that chapter according to the order of its subsections. Furthermore, when outputting the main text content, reference citation information needs to be marked. References include, but are not limited to, papers, books, statistical bulletins, development plans, industry research reports, annual reports of key enterprises, and authoritative media reports, etc.

[0071] In addition, in the embodiments of the present invention, the citation optimization agent is also implemented based on a large language model. When the citation optimization agent performs quality control and optimization on the citations in the main text, it needs to structure all reference citation information in the report, verify the validity of citation links, repair invalid or incomplete citation links, enhance the citations of key paragraphs in the report, and automatically recommend supplementary authoritative industry research reports or official planning documents as support; finally, a citation quality report is generated for users to refer to and review.

[0072] The format of the final report can be adjusted according to actual requirements. Optimal content may include a structured table of contents, chapters of the main text, and a list of references.

[0073] Therefore, in the report generation methods described in S1 to S3 above, the human-in-the-loop confirmation is moved forward from the traditional "post-event review of results" to key positions such as "problem definition (task description) - research brief generation - outline planning", thereby realizing the phased constraint and correction of the global behavior of the multi-agent system, and thus reducing the risk of the cumulative propagation of early deviations in the iteration.

[0074] Similarly, in another embodiment of the present invention, a report generation system is also provided. The functional modules of the report generation system include a user interaction module, a research supervision coordinator, a multi-agent planning layer including a dual-mode planning agent and a content generation agent, a tool agent cluster and a reference optimization agent. The functional modules communicate with each other through a preset message protocol to form a multi-agent workflow. The functional modules in the multi-agent workflow cooperate with each other to realize the above-mentioned multi-agent exploration-utilization report generation method based on a large model.

[0075] It should be noted that the method steps shown in S1 to S3 above, as well as the various functional modules in the report generation system, can essentially be implemented in the form of computer programs or software functional modules.

[0076] Therefore, based on the same inventive concept, such as Figure 3 As shown, the present invention also provides a computer electronic device corresponding to the multi-agent exploration-utilization report generation method based on a large model provided in the above embodiments, which includes a memory and a processor;

[0077] The memory is used to store computer programs;

[0078] The processor is configured to, when executing the computer program, implement the large model-based multi-agent exploration-utilization report generation method as described above.

[0079] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0080] Therefore, based on the same inventive concept, the present invention provides a computer-readable storage medium corresponding to the multi-agent exploration-utilization report generation method based on a large model. The storage medium stores a computer program that, when executed by a processor, can realize the multi-agent exploration-utilization report generation method based on a large model as described above.

[0081] Therefore, based on the same inventive concept, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, enables the implementation of the large-model-based multi-agent exploration-utilization report generation method as described above.

[0082] Specifically, in the computer-readable storage medium of the above three embodiments, the stored computer program is executed by a processor, which can perform the aforementioned steps S1 to S3.

[0083] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0084] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0085] It should also be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the embodiments provided in this application, the division of steps or modules in the system and method is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or steps may be combined or integrated together, and a module or step may also be split.

[0086] The present invention will further demonstrate the detailed implementation process and technical effects of the above-described S1~S3 steps of the multi-agent exploration based on a large model using the report generation method on a specific dataset through a specific embodiment, so as to facilitate understanding of the essence of the present invention.

[0087] The present invention will further demonstrate the detailed implementation process and technical effects of the above-described S1~S3 steps of the multi-agent exploration based on a large model using the report generation method in a specific example, so as to facilitate understanding of the essence of the present invention.

[0088] Example

[0089] The basic method flow of this embodiment is the same as the multi-agent exploration-utilization report generation method based on a large model shown in steps S1 to S3 above. It will not be completely repeated here. The main focus is on the specific case, step details and intermediate results of this embodiment.

[0090] This embodiment uses "Current Status of Guangxi's New Materials Industry Development" as its report topic to illustrate a specific application of the multi-agent report generation method based on a large model of the present invention in a regional industry research scenario. The specific process of this embodiment is as follows: Figure 4 , Figure 5 and Figure 6 As shown, the detailed steps are as follows:

[0091] Step 1: Receive the task and clarify the intent.

[0092] Users can input task instructions through the front-end interface, such as: "Help me research the current development status of the new materials industry in Guangxi, focusing on industrial layout, representative enterprises, policy environment, and development bottlenecks."

[0093] The user's input task instructions are transmitted to the requirements clarification module. Based on a large language model analysis, the requirements clarification module identifies whether further questions from the user are needed. If so, it automatically initiates clarification questions for the user, including but not limited to:

[0094] A1) Is there a time frame limit (e.g., the past three years)?

[0095] A2) The specific scope of new materials (whether it includes new energy vehicle materials, electronic information materials, new metal materials, biomedical materials, etc.);

[0096] A3) Target audience for the report (government decision-makers, industry investors, research institutions, etc.);

[0097] A4) Is it necessary to benchmark against other regions (such as the Guangdong-Hong Kong-Macao Greater Bay Area and the Yangtze River Delta)?

[0098] After the user clarifies and supplements the question through the interface, it will be organized into a structured task description and then confirmed by the user through the front-end interface. After the user confirms, the system will organize the above information into a structured task description and write it into the task global status as input for subsequent research briefings and outline planning.

[0099] Step 2: Perform the initial search and generate a research brief.

[0100] Under the scheduling of the research supervisory coordinator, the tool agent cluster performs initial web searches and literature searches based on structured task descriptions for keywords such as "Guangxi new materials industry", "Guangxi new materials industry development plan", and "key enterprises of Guangxi new materials".

[0101] The content-understanding agent summarizes the search results and extracts several candidate core analysis dimensions, such as:

[0102] B1) Overall industry size and growth trend;

[0103] B2) Industrial chain structure and sub-sectors (upstream raw materials, midstream processing, downstream applications);

[0104] B3) Regional layout and key industrial parks (Nanning, Liuzhou, Beibu Gulf Economic Zone, etc.);

[0105] B4) Representative companies and typical projects;

[0106] B5) Policy environment and support system;

[0107] B6) Innovation capability and technological foundation;

[0108] B7) Development Bottlenecks and Opportunities.

[0109] The dual-mode planning agent defaults to utilization mode at this stage, leveraging the generative capabilities of the large language model to further generate an initial research brief based on the extracted candidate core analysis dimensions. The research brief includes information on the research scope, core questions, expected structure, and data source types.

[0110] Step 3: Confirmation of human presence in the environment in the research brief.

[0111] The research supervision coordinator presents the research briefing generated in step 2 to the user through a front-end interface, allowing the user to adjust the research scope and analysis focus, for example:

[0112] C1) emphasizes the sub-sector of "new materials related to new energy vehicles";

[0113] C2) requires the addition of the dimension of "benchmarking against the new materials industry in the Guangdong-Hong Kong-Macao Greater Bay Area";

[0114] C3) The time range is limited to publicly available data from the past three years.

[0115] Upon receiving the adjustment instruction from the user, the revised research brief needs to be generated again based on the large language model's generation capabilities. This revised brief is then presented to the user for confirmation via the front-end interface. The confirmed research brief is written back to the global state by the `confirm_research_brief` node.

[0116] Step 4: Generate an initial outline and verify human presence in the loop.

[0117] Under the constraints of the research briefing, the research supervisory coordinator invokes the initial outline of the bimodal planning agent to generate a report. For example, including:

[0118] 1. Introduction and Research Background

[0119] 2. Overview of Guangxi's New Materials Industry

[0120] 3. Industrial chain structure and sub-sectors

[0121] 4. Regional Layout and Key Industrial Parks

[0122] 5. Representative companies and typical projects

[0123] 6. Policy Environment and Support System

[0124] 7. Innovation Capability and Technological Foundation

[0125] 8. Development Bottlenecks and Major Challenges

[0126] 9. Development Opportunities and Policy Recommendations

[0127] While generating the outline, the dual-mode planning agent predicts the importance score `importance_score (0~1)` and the expected length `expected_tokens` for each chapter. The large language model calculates the estimated weight wi for each chapter based on these two pieces of information and normalizes them to obtain the initial topic distribution.

[0128] The research supervisory coordinator will develop the initial outline. The importance of each chapter is presented to the user through the interface. Users can adjust the chapter order, merge or split chapters, and add chapters such as "Benchmarking Analysis with the Guangdong-Hong Kong-Macao Greater Bay Area" and "Export Opportunities for RCEP." The outline confirmed by the user is recorded as follows. And write it to the global state.

[0129] Step 5: Build the knowledge graph and initialize the outline state.

[0130] Knowledge graph and memory modules from the confirmed outline The initial topic nodes (i.e., the first-level headings of chapters) are extracted from the initial search results and marked as `covered = True`. Simultaneously, high-value insights (including perspectives, dimensions, and related entities related to industrial development) related to "industrial chain structure," "innovation capability," "policy environment," and "regional collaboration" are extracted from the initial search results and added to the knowledge graph as candidate concept nodes, marked as `covered = False`. Their importance is represented by the `value_score` field. In the knowledge graph, a node with a `covered` field of `True` indicates that the corresponding topic unit is already covered by the current outline, while a `covered` field of `False` indicates that the corresponding topic unit is not yet covered by the current outline.

[0131] The outline in the global state and the constructed knowledge graph serve as inputs for subsequent bimodal planning iterations.

[0132] Step 6: Iterative dual-mode planning based on marginal revenue.

[0133] During the outline refinement phase, under the scheduling of the research supervisory coordinator, the system repeatedly executes the following sub-steps 61 to 64 according to a preset cycle:

[0134] Sub-step 61: State assessment and mode decision.

[0135] The research supervisory coordinator reads the current outline for this round (iteration step t). (For the first iteration, i.e., when iteration step t=0, Right now At the same time, based on the marginal revenue MG(t-1) of the previous iteration, it is decided whether to adopt the exploration mode or the utilization mode in this round. If the marginal revenue MG(t-1) of the previous iteration is less than the threshold, the exploration mode is adopted in this round, and sub-step 62 is executed. If the marginal revenue MG(t-1) of the previous iteration is not less than the threshold, the utilization mode is adopted in this round, and sub-step 63 is executed.

[0136] Furthermore, during the iteration process, this embodiment sets the following mandatory rules: if the previous round adopted the exploitation mode and MG(t) is lower than the preset threshold, then the current round will be forced to execute an exploration mode, and after the exploration mode is completed, the next round will be forced to return to the exploitation mode; if the marginal benefit in the exploitation mode is less than the threshold within the preset number of rounds, then the exploration mode will be forced to execute a round.

[0137] Sub-step 62, Exploration Mode.

[0138] When the system determines that it needs to expand the field of view or that the exploration cycle requirement has been met, it enters exploration mode:

[0139] The dual-mode planning agent generates multi-dimensional divergent queries, such as constructing queries around themes like "comparison between Guangxi's new materials industry and eastern coastal provinces", "Guangxi's new materials export structure and RCEP opportunities", and "construction of major platforms and demonstration zones in the autonomous region".

[0140] The tool's intelligent agent performs parallel divergent retrieval of `web_search` to obtain potentially high-value information from regional statistical bulletins, development plans, industry research reports, key enterprise annual reports, and authoritative media reports.

[0141] The planning agent evaluates the value of the exploration results, taking into account novelty, relevance and impact, and gives a value_score, and writes high-value insights into the knowledge graph;

[0142] If a topic with a high value_score that was not covered in the previous outline is found (such as "benchmarking analysis with the Guangdong-Hong Kong-Macao Greater Bay Area", "new material project cluster for undertaking industrial transfer from the eastern region", "key material cluster related to new energy vehicles", etc.), the outline will be restructured under the condition that the preset threshold condition is met (in this embodiment, the threshold condition is set to value_score greater than 0.6), the corresponding chapters will be added or adjusted automatically, and the new chapters will be assigned an initial weight.

[0143] Sub-step 63: Utilize the pattern.

[0144] When the system is in utilization mode, targeted optimizations are performed based on the existing framework. The specific process is as follows:

[0145] Identify knowledge gaps, such as the lack of specific project cases in the "Representative Enterprises and Typical Projects" chapter, and the lack of data on the number of patents or scientific research platforms in the "Innovation Capability and Technological Foundation" chapter.

[0146] Generate queries such as "2023 Statistical Bulletin on the Output Value of New Materials in Guangxi", "List of Key New Materials Enterprises in Guangxi", and "Liuzhou New Materials Industrial Park Project".

[0147] The tool's intelligent agent performs web search to obtain more accurate statistics, policy provisions, and company information;

[0148] The planning intelligence agent optimizes the outline based on newly acquired information. For example, it adds subsections such as "Location and Transportation Cost Pressures" and "Insufficient Industrial Support" under the "Development Bottlenecks and Challenges" chapter, and supplements the "Window of Opportunity for Undertaking Industrial Transfer from the Eastern Region" section with relevant content.

[0149] Sub-step 64: Marginal revenue calculation and mode switching.

[0150] After each iteration, the dual-mode planning agent calculates the entropy difference based on the chapter weight distribution before and after the outline iteration to obtain the structural information gain. Simultaneously, it performs semantic embedding on the outline representations before and after iteration, calculates cosine similarity to obtain semantic novelty, and measures the degree of structural change through changes in the number of chapters and hierarchical levels. The system then weights and combines these indicators to obtain the marginal revenue MG(t).

[0151] In an embodiment of the present invention, the calculation process of the marginal revenue MG(t) can be implemented as follows:

[0152] 641) Topic Unit Extraction and Weight Calculation

[0153] Extract outline The chapters in the text are presented as a series of thematic units. n is the current number of topic units;

[0154] Calculate the weight for each thematic unit Weight It can be determined based on at least one or more of the following indicators:

[0155] i) The number of words and tokens in the existing text content under this topic;

[0156] ii) The number of search results, citations, or importance score associated with the topic;

[0157] iii) The topic's hierarchical position in the outline (e.g., top-level chapters may be given higher weight);

[0158] Weights of all thematic units After normalization, the topic distribution of each topic unit is obtained:

[0159]

[0160] The sum of the thematic distributions of all thematic units This is used to depict the proportion of attention given to each topic in the current outline.

[0161] 642) Measurement of information breadth and richness

[0162] Based on the above topic distribution, the structural information content of the current outline is calculated in the form of information entropy:

[0163]

[0164] The aforementioned structural information can characterize the distribution of the outline across different themes.

[0165] At the same time, the outline obtained after iteration is updated Repeat the above steps to obtain the new outline. Thematic distribution and structural information content The difference between the two structural information values ​​can be approximated as the structural information gain brought about by this iteration:

[0166]

[0167] 643) The system can also use text embedding models (such as BERT) to semantically encode the outlines before and after iteration, obtaining semantic vector representations of the two outlines respectively. and calculate The cosine similarity between the two is used as the semantic novelty, and the difference between 1 and the cosine similarity is used as the semantic novelty.

[0168]

[0169] Semantic novelty describes the degree of change in the semantic space relative to the original outline in this iteration.

[0170] 644) Structural change measurement

[0171] In this embodiment, the system measures the degree of structural change by comparing the rate of change in the number of chapters in the outline before and after the iteration:

[0172]

[0173] in and These represent the number of chapters (i.e., thematic units) in the outline after the iteration and before the iteration, respectively;

[0174] 645) Marginal Revenue Calculation and Mode Switching Strategy

[0175] The system is based on information gain. Semantic novelty and structural gain are used to construct the marginal benefit of this iteration. The calculation formula is:

[0176]

[0177] Where α, β, and γ are weighting coefficients used to balance the impact of changes in structural information content, semantic novelty, and structural variability on the overall return.

[0178] The termination condition for the above iteration is either outline convergence or reaching the maximum number of iterations. For the former, when MG(t) remains below the threshold for a specified number of iterations and the chapter structure tends to stabilize over several iterations, the system determines that the outline has converged at the convergence judgment node. In this embodiment, as a simplification, reaching the preset maximum number of iterations can be directly used as the iteration termination condition, and the outline status after iteration termination can be sent to the user for confirmation.

[0179] Step 7: Outline convergence confirmation and chapter text generation.

[0180] After the outline is converged, the research monitoring coordinator presents the current outline and key points of each chapter to the user for final confirmation, allowing the user to make minor adjustments to a few chapters (such as adjusting the position of the "Benchmarking Analysis" chapter, splitting "Opportunities and Countermeasures" into two chapters, etc.).

[0181] After the outline is confirmed, the content generation agent calls the large language model at the chapter level. Based on the various search results retrieved and summarized by the tool agent cluster for the chapter, combined with the chapter's key points, attributed insight summaries and relevant data tables, the agent generates the first draft of the main text for chapters such as "Overall Industry Overview", "Industry Chain Structure and Sub-sectors", "Regional Layout and Key Parks", "Representative Enterprises and Typical Projects", and "Development Bottlenecks and Opportunities".

[0182] Step 8: Citation quality optimization (corresponding to the CitationAgent process).

[0183] After the main body of the report is stabilized, the citation optimization agent is activated to uniformly process the statistical data sources, policy documents, industry research reports, and enterprise information involved in the report, including:

[0184] 1) Citation extraction: Structure all citation tags in the text;

[0185] 2) Validity verification: Check the reachability of the link and the authority of the publishing organization;

[0186] 3) Citation repair: For invalid or incomplete citations, supplement the issuing unit, document number, and publication date, or replace them with an accessible equivalent source;

[0187] 4) Enhanced citation: For key paragraphs such as "development bottlenecks" and "countermeasures and suggestions", the system will automatically recommend supplementary authoritative industry research reports or official planning documents as support.

[0188] The citation optimization agent also needs to generate a citation quality report, including the percentage of valid citations and a list of potentially problematic citations, for users to review.

[0189] Step 9: Output the report.

[0190] The main body of the report and the citation quality report are displayed to the user through the front end, and the user can confirm them. After the user confirms the main body of the report and the citation quality report, the system outputs the final research report, "Current Status of Guangxi's New Materials Industry Development," which includes a structured table of contents, chapters of the main body, and a list of references.

[0191] As can be seen from this embodiment, in the regional industry research task of "Current Status of Guangxi New Materials Industry Development", this invention can generate an industry research report with a reasonable structure, comprehensive perspective and credible citations by clarifying needs, confirming human-in-the-loop at the research briefing and outline stage, exploring and utilizing dual-mode planning based on marginal benefits, and optimizing citation quality at the final draft stage.

[0192] The embodiments described above are merely some preferred implementations of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A multi-agent exploration method based on a large model—utilizing report generation, characterized in that, include: S1. Receive the report input by the user through the front-end interface to generate task instructions, construct a structured task description, and then the research supervision coordinator calls the tool agent cluster to generate an initial research brief and present it to the user through the front-end interface. Based on the human-in-the-loop confirmation mechanism, receive the user's adjustment and confirmation instructions and save the final research brief after user confirmation. S2. Under the constraints of the final research briefing, the research supervision coordinator calls the dual-mode planning agent to generate the initial outline of the report, and continuously updates and optimizes the outline in an iterative manner. In each iteration, the optimization mode of the outline in this round must be determined based on the marginal benefit of the previous round, whether it is the exploration mode or the exploitation mode, and the marginal benefit of this round is calculated after the optimization is completed. In S2, when the outline is continuously updated and optimized iteratively, the process executed in each iteration is as follows: First, determine whether the marginal revenue of the previous round is less than the preset revenue threshold. If it is less, the exploration mode is used in this round; otherwise, the exploitation mode is used. In exploration mode, the dual-mode planning agent generates divergent retrieval instructions according to the multidimensional query framework and sends them to the tool agent cluster to perform the retrieval, obtain high-value insights that may be overlooked in the current outline, update them as candidate concept nodes to the knowledge graph, and reconstruct the current outline as new chapters. In the utilization mode, the dual-mode planning agent identifies the chapters that need optimization for the current outline, generates targeted retrieval and optimization instructions for the knowledge gaps within the chapters, and uses the supplementary retrieval results to perform local incremental updates on the chapters that need optimization. Then, for the outlines before and after this round of updates and optimizations, calculate the information gain, semantic novelty and structural change between the chapters of the two outlines, and use the weighted sum of the three indicators as the marginal benefit of this round. S3. After the iteration is completed, the outline is presented to the user through the front-end interface. Based on the human-in-the-loop confirmation mechanism, the user's adjustment and confirmation instructions are received. Then, the content generation agent generates chapter content by calling the large language model at the chapter level according to the outline confirmed by the user, and forms the report text. Finally, the citation optimization agent performs quality control and optimization on the citations in the text and generates a citation quality report. The final report text and citation quality report after citation optimization are output.

2. The multi-agent exploration-utilization report generation method based on a large model as described in claim 1, characterized in that, In step S1, after receiving the report input by the user through the front-end interface and generating the task instruction, it is necessary to analyze whether there are any issues in the task instruction that require further clarification from the user. If so, a clarification request is sent to the user through the front-end interface, and the original input task instruction and the clarification content input by the user are organized into a structured task description.

3. The multi-agent exploration-utilization report generation method based on a large model as described in claim 1, characterized in that, In step S1, after obtaining the structured task description, the research supervision coordinator first calls the tool agent cluster to extract the search query composed of keywords from the structured task description, and performs the first search in the data source containing industry information. The content generation agent summarizes the search results, extracts several candidate core analysis dimensions, and generates an initial research brief.

4. The multi-agent exploration-utilization report generation method based on a large model as described in claim 1, characterized in that, In step S2, after obtaining the final research brief, the research supervision coordinator calls the dual-mode planning agent to generate outline chapters that conform to the research brief in the utilization mode. Each chapter corresponds to a topic unit and is then presented to the user through the front-end interface. Based on the human-in-the-loop confirmation mechanism, the system receives the user's adjustment and confirmation instructions and finally saves the outline after user confirmation as the starting point for iterative update and optimization.

5. The multi-agent exploration-utilization report generation method based on a large model as described in claim 1, characterized in that, In step S2, before iteratively updating and optimizing the outline, it is necessary to extract the topic units corresponding to each chapter from the initial outline and add them to the knowledge graph as initial topic nodes. At the same time, high-value insights are extracted from the existing search results and added to the knowledge graph as candidate concept nodes. The high-value insights are topic units whose value scores obtained by the dual-mode planning agent evaluation exceed a threshold.

6. The multi-agent exploration-utilization report generation method based on a large model as described in claim 1, characterized in that, In S3, when the citation optimization agent performs quality control and optimization on the citations in the main text, it needs to structure all reference citation information in the report, verify the validity of citation links, repair invalid or incomplete citation links, enhance the citations of key paragraphs in the report, and automatically recommend supplementary authoritative industry research reports or official planning documents as support; finally, a citation quality report is generated for users to refer to and review.

7. A report generation system, characterized in that, The functional modules of the report generation system include a user interaction module, a research supervision coordinator, a multi-agent planning layer including a dual-mode planning agent and a content generation agent, a tool agent cluster and a citation optimization agent. The functional modules communicate with each other through a preset message protocol to form a multi-agent workflow that can realize the multi-agent exploration-report generation method based on a large model as described in any one of claims 1 to 6.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they can implement the large model-based multi-agent exploration-utilization report generation method as described in any one of claims 1 to 6.

9. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the multi-agent exploration-utilization report generation method based on a large model as described in any one of claims 1 to 6.

Images