Autonomous construction method of report generation agent based on knowledge enhanced large model

Abstract

The present application relates to the technical field of artificial intelligence large model, specifically to a report generation agent autonomous construction method based on knowledge enhanced large model, which comprises: knowledge extraction on business domain structured documents and unstructured texts and construction of domain knowledge graph, adoption of an improved progressive knowledge injection algorithm to integrate knowledge into general large language model parameters to form a knowledge enhanced base model. Extraction of task planning sequences and operation trajectories generated by historical reports to construct a demonstration sample library, and fine tuning of a primary report generation agent through a behavior cloning strategy. A reinforcement learning training field containing multiple rounds of complex tasks and diversified environmental disturbances is built, interactive data is collected to form a behavior experience pool, decision parameters are iteratively updated based on proximal policy optimization, and autonomous construction of the agent is completed. This method can improve the effect of domain knowledge fusion and enhance the task planning and autonomous decision-making ability of the agent in complex scenarios.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence large model technology, and in particular to a method for autonomously constructing a report generation agent based on a knowledge-enhanced large model. Background Technology

[0002] Existing report generation agent construction technologies are mostly based on general large language models, combining domain text for routine knowledge organization and model fine-tuning. Domain knowledge is often integrated into the model through direct concatenation or single injection, and the agent relies solely on basic supervised fine-tuning to optimize parameters. Some related technologies introduce reinforcement learning to optimize the agent's decision-making capabilities, but the training scenarios are limited to a single report generation task, without setting training conditions for complex tasks or environmental disturbances. Knowledge integration and agent training both adopt conventional technical paradigms.

[0003] When using conventional knowledge injection methods, general-purpose large language models suffer from insufficient integration of domain knowledge with model parameters, making them prone to knowledge forgetting and unable to accurately match the knowledge requirements for report generation in the business domain. Agents trained with single-supervised fine-tuning struggle to grasp complex task planning and tool invocation logic, resulting in poor scenario adaptability for behavioral decisions.

[0004] The training scenarios for reinforcement learning are too simplistic and cannot replicate the complex environments in which actual reports are generated. The agent's strategy exploration lacks experience in diverse environments, and the lack of interactive data means that the iteration of decision-making strategies lacks precise data support. This makes it impossible to achieve refined optimization of the agent's autonomous decision-making capabilities, and there are shortcomings in both the effectiveness of knowledge fusion and the completeness of the agent's autonomous construction. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method for autonomously constructing report-generating agents based on a knowledge-enhanced large model.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for autonomously constructing a report generation agent based on a knowledge-enhanced large model, comprising: Obtain structured documents and unstructured text from the original business domain and perform knowledge extraction to construct a domain knowledge graph; Based on the general large language model, an improved progressive knowledge injection algorithm is used to integrate the knowledge in the domain knowledge graph into the parameters of the general large language model, forming a knowledge-enhanced base model. Extract task planning sequences and operation trajectories from the historical generation records of target reports to construct a sample library of task planning and tool calls; Based on the knowledge-enhanced base model and the demonstration sample library, a behavior cloning strategy is used to perform supervised fine-tuning of the initial agent model to generate a primary report-generating agent. Set up a reinforcement learning training field that includes multiple rounds of complex report generation tasks and diverse environmental perturbations, so that the primary report generation agent can explore strategies in the reinforcement learning training field; During the strategy exploration process, the state, action and reward data generated by the interaction between the primary report generating agent and the reinforcement learning training field are collected to form an agent behavior experience pool. The decision strategy parameters of the primary report generation agent are iteratively updated using the agent's behavioral experience pool through a proximal policy optimization method. When the evaluation index reaches a preset threshold, the training terminates and the final report generation agent, which has been autonomously constructed, is output.

[0007] As a further aspect of the present invention, the step of acquiring structured documents and unstructured text from the original business domain to perform knowledge extraction and construct a domain knowledge graph includes: The structured document is subjected to pattern recognition and field parsing to extract entity descriptions, attribute definitions and numerical constraints, and generate structured initial entity attribute pairs. Semantic segmentation and named entity recognition are performed on the unstructured text to locate the core concepts, terms and key descriptions in the text fragments, forming a set of unstructured semantic fragments; The initial entity attribute pairs and the set of unstructured semantic fragments are input into a preset relation extraction model. The relation extraction model identifies and establishes semantic associations between different entities and concepts based on predefined business relationship categories. The extracted entities, attributes, and semantic relationships are instantiated into nodes and edges according to the graph construction specification. Knowledge fusion technology is used to eliminate ambiguity in entity reference and conflict in attributes, and finally integrated to form an interconnected domain knowledge graph.

[0008] As a further aspect of the present invention, based on the general large language model, an improved progressive knowledge injection algorithm is used to integrate knowledge from the domain knowledge graph into the parameters of the general large language model to form a knowledge-enhanced base model, including: Knowledge subgraphs are sampled from the domain knowledge graph and converted into text sequences that the model can understand, forming a knowledge injection corpus; The knowledge-injected corpus is mixed with the general training corpus, and a progressive knowledge-injected training set is generated according to a preset mixing ratio. During model training, the ratio of general training corpus to knowledge-injected corpus in the progressive knowledge-injected training set is dynamically adjusted so that the model training focuses on understanding general semantics in the early stage and gradually increases the proportion of knowledge-injected corpus in the middle and later stages of training to enhance the absorption and memorization of domain knowledge. During the backpropagation phase of model parameter updates, an additional knowledge-preserving gradient is calculated for the training loss derived from the knowledge-injected corpus. This guides the model parameters to adapt to domain knowledge while minimizing the overriding forgetting of the original general capabilities. After training is completed, the knowledge-enhanced base model is obtained.

[0009] As a further aspect of the present invention, the working principle of the improved progressive knowledge injection algorithm includes: Define a knowledge injection scheduler. At the beginning of each iteration of model training, the knowledge injection scheduler calculates the proportion of the knowledge injection corpus in the total training batch in the current cycle according to the current training round and the preset scheduling strategy. When constructing a training batch, based on the proportion calculated by the knowledge injection scheduler, a corresponding number of samples are randomly sampled from the general training corpus and the knowledge injection corpus respectively, and combined into a training batch. When calculating the total loss of a training batch, the loss component contributed by the samples from the knowledge-injected corpus in the training batch is multiplied by a dynamic knowledge retention coefficient, which increases with the training rounds, thereby producing a stronger memory reinforcement effect on domain knowledge in the later stages of training. The improved progressive knowledge injection algorithm, through the collaboration of the knowledge injection scheduler and the dynamic knowledge retention coefficient, controls the integration of domain knowledge into the general large language model in a smooth and stable manner.

[0010] As a further aspect of the present invention, based on the knowledge-enhanced base model and the demonstration sample library, a behavior cloning strategy is used to perform supervised fine-tuning of the initial agent model to generate a primary report-generating agent, including: The primary report generating agent possesses basic planning and execution capabilities; The architecture of the initial intelligent agent model is set with the knowledge-enhanced base model as the core, and a task planning header and a tool invocation header are added. Task planning sequences and operation trajectories are extracted from the demonstration sample library. The historical states and objectives in the task planning sequences are used as model inputs, and the corresponding subsequent planning actions and tool calls are used as labels for the model's expected outputs to construct supervised fine-tuning training samples. With the goal of maximizing the consistency between the output actions and labeled actions of the primary report generating agent on supervised fine-tuning training samples, all trainable parameters in the initial agent model, except for the core parameters of the knowledge-enhanced base model, are updated using the gradient descent algorithm. When the accuracy of the planned actions of the primary report generation agent on the validation set reaches a preset standard, supervised fine-tuning is stopped, and the primary report generation agent is obtained by imitating the demonstration behavior to perform preliminary task decomposition and tool selection.

[0011] As a further aspect of the present invention, the setup includes a reinforcement learning training field comprising multiple rounds of complex report generation tasks and diverse environmental perturbations, enabling the primary report generation agent to explore policies within the reinforcement learning training field, including: Design a task generator, which generates task descriptions by randomly combining reports with different themes, complexity requirements, and data source conditions based on preset templates and rules. An environment simulator is constructed, which receives tool call instructions output by the primary report generating agent, simulates the execution of data query, information analysis, and chart drawing tools, and returns simulation results containing noise, errors, or missing information. The task generator and the environment simulator are encapsulated to form the reinforcement learning training field, which provides an interactive and dynamically changing training environment for the primary report generation agent. The primary report generation agent learns better decision-making strategies through trial and error and environmental feedback in the training environment.

[0012] As a further aspect of the present invention, during the policy exploration process, the state, action, and reward data generated by the interaction between the primary report generating agent and the reinforcement learning training field are collected to form an agent behavior experience pool, including: In each interaction step of the reinforcement learning training field, the environmental state representation observed by the primary report generating agent, the planned action or tool call action selected by the primary report generating agent, the immediate reward value returned by the environment simulator, and the next state to which the environment transitions after the action is performed are recorded. The recorded environmental state representation, planned actions or tool call actions, immediate reward value, and next state are combined into a complete transition sample; The transfer samples generated from each interaction are stored in a randomly accessible experience replay buffer in chronological order, and continuously accumulated to form the agent behavior experience pool. The agent behavior experience pool stores the exploration experience of the primary report generating agent under different tasks and environmental states.

[0013] As a further aspect of the present invention, the step of using a proximal policy optimization method to iteratively update the decision policy parameters of the primary report-generating agent using the agent's behavioral experience pool, and terminating training when the evaluation index reaches a preset threshold, and outputting the autonomously constructed final report-generating agent, includes: A small batch of transfer samples is randomly sampled from the agent's behavioral experience pool; Based on the sampled transfer samples, calculate the probability ratio between the current primary report generating agent policy and the old policy, and calculate the policy substitution loss accordingly. Calculate the value function loss, which is used to evaluate the accuracy of the current primary report generating agent's prediction of the state value; The policy substitution loss and the value function loss are combined with preset weights, and a policy entropy regularization term is added to form the overall objective function for near-end policy optimization. The overall objective function is optimized using the gradient ascent algorithm, and the parameters of the decision policy network and value function network in the primary report generating agent are updated to complete one round of iterative update. After each iteration, the updated primary report generation agent is run on the evaluation task set to calculate the average task completion rate and report quality score. When the average task completion rate and report quality score exceed the preset evaluation index threshold multiple times consecutively, the model is determined to have converged, and the agent that has been trained at this time is output as the final report generation agent.

[0014] As a further aspect of the present invention, the calculation of the probability ratio between the current primary report-generating agent's policy and the old policy based on the sampled transfer samples, and the calculation of the policy substitution loss accordingly, includes: The environmental state representation in the sampled transfer sample is input into the decision policy network of the current primary report generating agent to obtain the action probability of selecting the recorded action in the sampled transfer sample under the current policy. Obtain the historical action probability of selecting the recorded action under the old strategy from the historical records associated with the sample transfer samples; Calculate the ratio of the probability of an action under the current policy to the probability of a historical action under the old policy to obtain the policy probability ratio; Using the policy probability ratio and the advantage function estimate in the sampled transfer samples, a policy substitution loss term is constructed. This policy substitution loss term encourages actions that bring higher advantage, while the pruning operation of the probability ratio prevents the current policy from updating too much relative to the old policy.

[0015] As a further aspect of the present invention, after each iteration update, the updated primary report generation agent is run on the evaluation task set to calculate the average task completion rate and report quality score, including: A fixed number of assessment tasks are selected from a pre-built set of assessment tasks, wherein the assessment tasks have standard answers or expert review criteria; In a controlled evaluation environment, each evaluation task is executed sequentially, driving the updated primary report generating agent to plan and execute from scratch until the final report is generated or the maximum number of interaction steps is reached and terminated. Based on the updated primary report, the agent generates reports that are complete in content, standardized in structure, and consistent with the standard answer, or, based on expert review criteria, quantitatively calculates the task completion score and report quality score for each evaluation task. The average task completion score and report quality score for all evaluation tasks are calculated to obtain the average task completion score and report quality score, which are used to evaluate the overall performance of the agent.

[0016] Compared with the prior art, the advantages and positive effects of the present invention are as follows: The improved progressive knowledge injection algorithm can gradually integrate structured knowledge from the domain knowledge graph into the parameters of a general large language model. This progressive knowledge injection method reduces information loss and conflicts during knowledge fusion, ensuring domain knowledge is stably retained within the model architecture. The knowledge-enhanced base model can align with the business domain's knowledge system and text generation logic. A demonstration sample library is constructed by extracting task planning sequences and operation trajectories from historical report generation records. Supervised fine-tuning of the initial agent model is performed using a behavior cloning strategy. The primary report generation agent can learn standardized task planning and tool invocation paths, quickly adapting to the procedural operation logic of report generation.

[0017] A reinforcement learning training environment is set up, incorporating multiple rounds of complex report generation tasks and diverse environmental perturbations. The novice agent can autonomously explore policies within these diverse training scenarios. The resulting state, action, and reward data comprehensively record the agent's behavioral characteristics, and the resulting behavioral experience pool directly provides data support for policy optimization. The decision-making policy parameters of the novice agent are iteratively updated using a proximal policy optimization method. Parameter adjustments can align with the agent's actual interactive behavior, continuously optimizing the agent's decision-making logic. The agent can adapt to complex report generation scenarios and various environmental changes, and the rationality and flexibility of its autonomous decision-making can be continuously improved. Attached Figure Description

[0018] Figure 1 This is a flowchart of the report generation agent autonomous construction method based on a knowledge-enhanced large model as described in this invention; Figure 2 A flowchart for constructing a domain knowledge graph; Figure 3 A flowchart for an improved incremental knowledge injection algorithm. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0021] See Figure 1 This invention provides a method for autonomously constructing a report generation agent based on a knowledge-enhanced large model. The specific implementation is as follows: Knowledge extraction processing is performed on structured documents and unstructured texts in the original business domain to construct a domain knowledge graph. Based on a general large language model, an improved progressive knowledge injection algorithm is used to integrate knowledge from the domain knowledge graph into the parameters of the general large language model, thus forming a knowledge-enhanced base model. Simultaneously, task planning sequences and operation trajectories are extracted from the historical generation records of the target report to construct a demonstration sample library for task planning and tool calls. Based on the aforementioned knowledge-enhanced base model and demonstration sample library, a behavior cloning strategy is used to perform supervised fine-tuning of an initial agent model, generating a basic report generation agent with basic capabilities. A reinforcement learning training field containing multiple rounds of complex report generation tasks and diverse environmental perturbations is set up to drive the basic report generation agent to autonomously explore strategies within this training field. During the exploration process, state, action, and reward data generated by the agent's interaction with the environment are collected to form an agent behavior experience pool. By using a proximal policy optimization method, the decision policy parameters of the primary report generation agent are iteratively updated using this experience pool. When its performance on the independent evaluation task set reaches a preset threshold, the training terminates and the final report generation agent that has been autonomously constructed is output.

[0022] In one embodiment of the present invention, when constructing a domain knowledge graph, refer to... Figure 2The process involves performing pattern recognition and field parsing on structured documents to extract entity descriptions, attribute definitions, and numerical constraints, generating a set of structured initial entity attribute pairs. For unstructured text, semantic segmentation and named entity recognition are performed to locate core concepts, terms, and key descriptions within text fragments, forming a set of unstructured semantic fragments. These initial entity attribute pairs and the set of unstructured semantic fragments are then input into a predefined relation extraction model. This model, based on predefined business relationship categories, identifies and establishes semantic associations between different entities and concepts. The extracted entities, attributes, and semantic associations are then instantiated into nodes and edges according to a graph construction specification. Knowledge fusion technology is used to eliminate ambiguities in entity referencing and attribute conflicts, ultimately integrating them to form an interconnected domain knowledge graph.

[0023] In practical implementation, the processing of raw data for a specific business area is exemplified by the financial analysis field. The structured document example is a spreadsheet of "Company Annual Financial Report," while the unstructured text example is a plain text file of "Industry Analysis Report." Pattern recognition is performed on the structured document "Company Annual Financial Report" to identify the column headings and data areas of the table. Entity descriptions are extracted through field parsing, such as the entity "Company A" and its attribute definition "Operating Revenue" and the corresponding numerical constraint "10 billion yuan." Semantic segmentation is performed on the unstructured text "Industry Analysis Report," dividing the long text into independent segments based on thematic paragraphs. Named entity recognition is then performed on these segments to locate the core concept "New Energy Vehicles," the term "Energy Density," and the key description "Market Size CAGR Exceeds 20%," forming a set of unstructured semantic segments.

[0024] In some embodiments, the initial entity attribute pairs generated in the preceding steps, along with a set of unstructured semantic fragments, are input into a preset relation extraction model. The relation extraction model identifies relationships based on predefined business relationship categories. These predefined categories include "belongs to," "competitive," and "upstream in the supply chain." The relation extraction model processes the text "Company A is a leading enterprise in the lithium battery industry" from one set of unstructured semantic fragments, identifying a semantic relationship of "belonging to" between the entity "Company A" and the concept "lithium battery industry." The relation extraction model then processes the description "Company B is a major competitor of Company A" from another set of unstructured semantic fragments, identifying a semantic relationship of "competitive" between the entities "Company A" and "Company B."

[0025] In practical implementation, the "Company A" and its attribute "Registered Capital" parsed from structured documents are aligned with the "Company A" and its key description "Registered in 2020" identified from unstructured text using knowledge fusion technology. Knowledge fusion technology detects that "Registered in 2020" is a specific value of the "Registered Capital" time attribute, thus merging the descriptive information under the "Registered Capital" attribute node of the "Company A" entity. Knowledge fusion technology also handles entity referential ambiguity; when the abbreviation "Company A" appears in unstructured text, contextual association confirms that it refers to the entity "Company A," thereby associating relevant information to the same node. Optionally, the relation extraction model calculates the deterministic score of relation extraction when establishing semantic relationships. The deterministic score of relation extraction is calculated using the following formula: in: The deterministic score represents the number of relations extracted. Represents a header entity based on semantic embedding. ,relation With tail entity The probability of association, Indicates based on text fragments The matching confidence of the predefined rule pattern. It is a hyperparameter that balances the weights of the two parts. This score is used for ranking and filtering edge confidence during subsequent knowledge fusion.

[0026] In practice, all nodes and edges ultimately integrated are instantiated according to the graph construction specifications. Nodes are instantiated as data objects with unique identifiers, types, and attributes, while edges are instantiated as connection objects with relation types, weights, and source information. Knowledge fusion technology eliminates attribute conflicts. When the "Number of Employees" attribute value of the same entity "Company A" parsed from different sources is inconsistent, the knowledge fusion technology selects a value as the standard value based on the authority and timestamp of the data source, or records multiple values ​​and their sources in the node attributes. Finally, all entity nodes, attribute nodes, and edges representing semantic relationships that have undergone fusion and disambiguation are stored in the graph database, forming an interconnected domain knowledge graph.

[0027] In one embodiment of the present invention, a knowledge-enhanced base model is formed by first sampling knowledge subgraphs from a domain knowledge graph and converting these subgraphs into text sequences that the model can understand, forming a knowledge-injected corpus. The knowledge-injected corpus is then mixed with general training corpus at a preset mixing ratio to generate a progressive knowledge-injected training set. During model training, the ratio of general training corpus to knowledge-injected corpus in the training set is dynamically adjusted, ensuring that the model focuses on understanding general semantics in the early stages of training and gradually increases the proportion of knowledge-injected corpus in the later stages to strengthen the absorption and memorization of domain knowledge. During the backpropagation phase of model parameter updates, an additional knowledge retention gradient is calculated for the training loss derived from the knowledge-injected corpus, guiding the model parameters to adapt to domain knowledge while minimizing the covering forgetting of original general capabilities. After training is completed, a knowledge-enhanced base model is obtained. The working principle of the improved progressive knowledge injection algorithm is as follows (see reference). Figure 3 A knowledge injection scheduler is defined. At the beginning of each training iteration, this scheduler calculates the proportion of knowledge-injected corpora in the total training batches within the current training round, based on the current training round and a preset scheduling strategy. When constructing training batches, a corresponding number of samples are randomly sampled from the general training corpus and the knowledge injection corpus, respectively, according to the proportion calculated by the scheduler, and combined into a training batch. When calculating the total loss of a training batch, the loss component contributed by the samples from the knowledge injection corpus in the batch is multiplied by a dynamic knowledge retention coefficient. This coefficient increases with each training round, thereby generating a stronger memory reinforcement effect on domain knowledge in the later stages of training. This algorithm, through the synergy of the knowledge injection scheduler and the dynamic knowledge retention coefficient, controls the integration of domain knowledge into the general large language model in a smooth and stable manner.

[0028] In specific implementation, the method involves the formation process from a domain knowledge graph to a knowledge-enhanced foundation model. The example scenario uses a knowledge graph from the financial analysis domain. Knowledge subgraphs are sampled from the domain knowledge graph. These subgraphs contain nodes and edges such as "Company A," "Lithium Battery Industry," "belongs to," "Company B," and "competition." The knowledge subgraphs are converted into a text sequence format that the model can understand. The conversion method uses descriptive sentences in the format of "head entity - [relationship] - tail entity" and "entity (attribute: value)," generating text sequences such as "Company A belongs to the lithium battery industry. Company B is a competitor of Company A. Company A (revenue: 10 billion yuan). Company A (registered capital date: 2020)," forming the knowledge-injected corpus. The general training corpus comes from publicly available internet text and book corpora. The knowledge-injected corpus and the general training corpus are mixed according to a preset initial mixing ratio: 5% for the knowledge-injected corpus and 95% for the general training corpus, generating an initial progressive knowledge-injected training set.

[0029] In some embodiments, the mixing ratio is dynamically adjusted during model training. Defined as N total training epochs, in the early stages when n is less than 0.3N, the proportion of knowledge-injected corpus in the training set is maintained at a low range of 5% to 10%, ensuring the model primarily focuses on understanding general semantics in the early stages of training. In the middle stages when n is greater than or equal to 0.3N and less than 0.8N, the knowledge injection scheduler linearly increases the proportion of knowledge-injected corpus, gradually increasing it from 10% to 40%. In the later stages when n is greater than or equal to 0.8N, the proportion of knowledge-injected corpus is increased and stabilized at a higher range of 50% to 60%, strengthening the absorption and memorization of domain knowledge. During the backpropagation stage of model parameter updates, when calculating the total loss, the cross-entropy loss component contributed by each sample from the knowledge-injected corpus in the training batch is multiplied by an additional knowledge retention gradient coefficient. This knowledge retention gradient coefficient is set to a constant greater than 1, guiding the model parameters to adapt to domain knowledge while minimizing the covering forgetting of original general capabilities. After training, a knowledge-enhanced base model is obtained.

[0030] An optional, improved progressive knowledge injection algorithm works as follows: A knowledge injection scheduler is defined. At the beginning of each iteration of model training, the scheduler calculates the proportion of knowledge-injected corpus in the total training batch within the current training epoch, based on the current training epoch number and a preset linear scheduling strategy. The preset linear scheduling strategy stipulates that the proportion of knowledge-injected corpus increases linearly from 5% in the first epoch to 50% in the last epoch. When constructing a training batch, based on the proportion calculated by the knowledge injection scheduler (assuming the current proportion is 20%), 80 samples are randomly sampled from the general training corpus, and 20 samples are randomly sampled from the knowledge-injected corpus, combining them into a training batch containing 100 samples. When calculating the total loss of the training batch, the loss component contributed by the samples from the knowledge-injected corpus in the training batch is multiplied by a dynamic knowledge retention coefficient, which increases with each training epoch. This dynamic knowledge retention coefficient... The calculation method is as follows: in: This represents the dynamic knowledge retention coefficient. This represents the hyperparameter indicating the strength of knowledge retention. Indicates the current training round. This represents the total number of training epochs. The dynamic knowledge retention coefficient increases linearly from slightly greater than 1 in the early stages of training to its maximum value in the later stages, thus generating a stronger memory reinforcement effect on domain knowledge in the later stages of training. The improved progressive knowledge injection algorithm, through the synergy of the knowledge injection scheduler and the dynamic knowledge retention coefficient, controls the integration of domain knowledge into the general large language model in a smooth and stable manner.

[0031] In one embodiment of the invention, a behavioral cloning strategy is used to generate a primary report generation agent, which possesses basic planning and execution capabilities. The initial agent model is structured with a knowledge-enhanced base model at its core, supplemented with task planning and tool invocation headers. Task planning sequences and operation trajectories are extracted from a demonstration sample library. The historical states and goals in the task planning sequences are used as model inputs, and the corresponding subsequent planning actions and tool invocations are used as labels for the model's expected output, constructing supervised fine-tuning training samples. With the goal of maximizing the consistency between the primary report generation agent's output actions and labeled actions on the training samples, all trainable parameters in the initial agent model, except for the core parameters of the knowledge-enhanced base model, are updated using a gradient descent algorithm. When the accuracy of the primary report generation agent's planning actions on the validation set reaches a preset standard, supervised fine-tuning is stopped, resulting in a primary report generation agent that mimics demonstration behavior to perform preliminary task decomposition and tool selection. The reinforcement learning training ground includes a task generator and an environment simulator. The task generator randomly combines report generation task descriptions with different themes, complexity requirements, and data source conditions based on preset templates and rules. The environment simulator receives tool call instructions from the primary report generation agent, simulates the execution of tools such as data querying, information analysis, and chart drawing, and returns simulation results containing noise, errors, or missing information. By encapsulating the task generator and the environment simulator, a reinforcement learning training ground is formed, providing the primary report generation agent with an interactive and dynamically changing training environment. In this environment, the agent learns better decision-making strategies through trial and error and environmental feedback.

[0032] In the specific implementation, the supervised fine-tuning and reinforcement learning training ground for the primary report generation agent are constructed, with the example scenario continuing to be based on the financial analysis field. The primary report generation agent possesses basic planning and execution capabilities, including understanding the task description, decomposing task steps, and selecting appropriate tools. The architecture of the initial agent model is set up with a knowledge-enhanced base model as the core processing module, with an additional task planning header and a tool invocation header. The task planning header is used to output the next sub-goal or planned action, and the tool invocation header is used to output the name and parameters of the specific tool to be invoked. A historical task planning sequence and operation trajectory are extracted from the sample library of task planning and tool invocation. The task planning sequence contains the historical state "Task: Analyze the competitiveness of company A. Completed steps: Collect company A's annual report." and the goal "Complete the competitiveness analysis report." The operation trajectory records the next planned action "Step: Query company A's industry position." and the tool invocation "Tool: Industry database query tool, parameter: {Company name: 'Company A'}". By concatenating the historical state and the target as the model input text, and using the corresponding subsequent planning actions and tool call instructions as the label text for the model's expected output, a supervised fine-tuning training sample is constructed.

[0033] In some embodiments, with the goal of maximizing the consistency between the output actions and labeled actions of the primary report generation agent on supervised fine-tuning training samples, a behavior cloning loss function is defined. Gradient descent is used to update all trainable parameters in the initial agent model, excluding the core parameters of the knowledge-enhanced base model. These trainable parameters include the network parameters of the task planning head and the tool invocation head. When the accuracy of the primary report generation agent's planned actions on the independent validation set reaches a preset standard, such as a stable accuracy of over 85% for three consecutive evaluation periods, supervised fine-tuning is stopped, resulting in a primary report generation agent that imitates the demonstrated behavior for initial task decomposition and tool selection. The reinforcement learning training ground setup includes a task generator and an environment simulator. The task generator combines tasks based on preset templates and rules. The preset template includes "Generate a report on {company} with {analysis dimensions}, requiring the inclusion of {factor 1}, {factor 2}, and {factor 3}, with the data source limited to {data source conditions}." The rule base defines a list of possible values ​​for {company}, {analysis dimensions}, {factors}, and {data source conditions}. The task generator randomly selects values ​​from various lists to fill in templates, generating a report generation task description such as "Generate a financial risk report on Company B, requiring it to include solvency, cash flow analysis, and peer comparison, with data sources limited to publicly available annual reports from the past three years."

[0034] Optional, the rules followed by the task generator are shown in Table 1: Table 1: Task Generator Rules Table The environment simulator receives tool invocation commands from the primary report generation agent, such as "Invoke financial data extraction tool, parameters: {Company: 'Company B', Year: [2021, 2022, 2023], Indicator: 'Debt-to-Asset Ratio'}". The environment simulator internally simulates the tool's operation logic and generates simulation results. These results may contain noise, such as a random fluctuation of ±0.5% in the returned debt-to-asset ratio value. They may also contain errors, such as returning a "Indicator does not exist" message for non-existent indicators, and missing information, such as returning null values ​​for some years. Encapsulating the task generator and environment simulator creates a reinforcement learning training ground, providing the primary report generation agent with an interactive and dynamically changing training environment. The agent learns better decision-making strategies through trial and error and environmental feedback within this training environment. This can be understood as the behavioral cloning loss function. The calculation method is as follows: in: Indicates behavioral cloning loss. This indicates the number of samples in a training batch. This indicates that the primary report-generating agent is in a given historical state. With the goal Under the given conditions, output actions consistent with the demonstration sample. The conditional probability. The gradient descent algorithm minimizes the behavioral cloning loss function. To update network parameters.

[0035] In one embodiment of the present invention, an agent behavior experience pool is formed during the policy exploration process. In each interaction step of the reinforcement learning training field, the environmental state representation observed by the primary report generator agent, the planned action or tool invocation action selected by the agent, the immediate reward value returned by the environment simulator, and the next state the environment transitions to after executing the action are recorded. The recorded state, action, reward, and next state are combined into a complete transition sample. The transition samples generated from each interaction are stored in chronological order in a randomly accessible experience replay buffer, continuously accumulating to form the agent behavior experience pool. This experience pool stores the exploration experience of the primary report generator agent under different tasks and environmental states. The decision policy parameters are iteratively updated using a proximal policy optimization method, and a small batch of transition samples is randomly sampled from the agent behavior experience pool. Based on the sampled transition samples, the probability ratio between the current primary report generator agent policy and the old policy is calculated, and the policy substitution loss is calculated accordingly. The value function loss is calculated, which is used to evaluate the accuracy of the current agent's prediction of the state value. The policy substitution loss and the value function loss are combined with preset weights, and a policy entropy regularization term is added to form the overall objective function of the proximal policy optimization. The gradient ascent algorithm is used to optimize the overall objective function, updating the parameters of the decision policy network and value function network in the primary report generation agent, completing one round of iterative updates. After each round of iterative updates, the updated primary report generation agent is run on the evaluation task set, and the average task completion rate and report quality score are calculated. When the average task completion rate and report quality score exceed the preset evaluation index thresholds multiple times consecutively, the model is considered to have converged, and the agent that has completed training at this point is taken as the output of the final report generation agent.

[0036] In practical implementation, the formation of the agent's behavioral experience pool and the iterative update process of the proximal policy optimization method are illustrated in the example scenario, which continues from the interaction of the primary report generation agent in the reinforcement learning training field. In each interaction step of the reinforcement learning training field, the environmental state representation observed by the primary report generation agent is recorded. This environmental state representation is a structured text description, including the current task objective, the sequence of executed steps, and the latest result summary returned by each tool. The planned action or tool invocation action selected by the primary report generation agent is recorded, such as the planned action "Next step: Perform financial ratio calculation," or the tool invocation action "Invoke calculation tool, parameters: {Formula: 'Current Ratio', Data: Current Assets, Current Liabilities}." The immediate reward value returned by the environment simulator based on the tool invocation result and task completion progress is recorded, such as +0.1 reward for successfully acquiring valid data, +0.5 reward for generating a report section, and -0.1 reward for executing an invalid operation. After executing an action, the environment transitions to the next state, which updates the sequence of executed steps and the tool result summary. The recorded environmental state representation, planned action or tool call action, immediate reward value, and next state are combined into a complete transition sample, as shown in Table 2.

[0037] Table 2: Transfer Sample Structure Table In some embodiments, the transition samples generated from each interaction are stored chronologically in a randomly accessible experience replay buffer. This buffer has a fixed capacity; when the number of stored transition samples exceeds the capacity, the oldest transition sample is overwritten by a newer one. The accumulated transition samples form an agent behavior experience pool, storing the exploration experience of the primary report generator agent under different tasks and environmental states. The decision policy parameters are iteratively updated using a proximal policy optimization method. In each iteration, a small batch of transition samples is randomly sampled from the agent behavior experience pool, with a fixed batch size of 64. Based on the sampled transition samples, the probability ratio between the current primary report generator agent policy and the old policy (defined by the policy network parameters before the previous iteration) is calculated. The value function loss is calculated to evaluate the accuracy of the current primary report generator agent's prediction of the state value. It is obtained by comparing the mean squared error between the state value predicted by the current value network and the reward target value calculated based on subsequent rewards from the sampled transition samples.

[0038] Optionally, the policy substitution loss and value function loss are combined with preset weights, and a policy entropy regularization term is added to form the overall objective function for proximal policy optimization. The policy entropy regularization term is used to encourage the policy to maintain a certain degree of randomness, which is beneficial for exploration. The overall objective function is optimized using the gradient ascent algorithm, updating the parameters of the decision policy network and value function network in the primary report generation agent, completing one round of iterative updates. After each round of iterative updates, the updated primary report generation agent is run on the evaluation task set to calculate the average task completion rate and report quality score. When the average task completion rate and report quality score exceed the preset evaluation index thresholds multiple times consecutively, for example, in five consecutive iterations, the average task completion rate is higher than 90% and the report quality score is higher than 85, the model is considered to have converged, and the agent that has been trained at this point is used as the output of the final report generation agent. It can be understood that the overall objective function for proximal policy optimization... The calculation method is as follows: in: Describe the overall objective function. The operator representing the expectation of empirical samples. This represents the probability ratio of the strategy. This represents the estimated value of the advantage function. Indicates the cutting factor. This represents the clipping function. This represents the value loss weighting coefficient. Represents the loss of the value function. Represents the entropy weighting coefficient. The entropy represents the policy. The gradient ascent algorithm maximizes the overall objective function. To update network parameters.

[0039] In one embodiment of the invention, a policy substitution loss is calculated by inputting the environmental state representation from the sampled transfer samples into the decision policy network of the current primary report-generating agent to obtain the action probability of selecting the recorded action in the sample under the current policy. Historical action probabilities of selecting the recorded action under the old policy are obtained from the historical records associated with the sampled transfer samples. The ratio of the action probability under the current policy to the historical action probability under the old policy is calculated to obtain the policy probability ratio. A policy substitution loss term is constructed using the policy probability ratio and the advantage function estimate from the sampled transfer samples. This loss term encourages actions that bring higher advantage, while the pruning operation of the probability ratio prevents the update step of the current policy from being too large relative to the old policy. The average task completion rate and report quality score are calculated by selecting a fixed number of evaluation tasks from a pre-constructed evaluation task set. These tasks have standard answers or expert review criteria. In a controlled evaluation environment, each evaluation task is executed sequentially, driving the updated primary report-generating agent to plan and execute from scratch until a final report is generated or the maximum number of interaction steps is reached. Based on the completeness of the agent's generated report, its structural standardization, and its consistency with the standard answer, or according to expert review criteria, the task completion score and report quality score for each evaluation task are quantitatively calculated. The average of the task completion score and report quality score for all evaluation tasks is then calculated to obtain the average task completion score and report quality score used to evaluate the overall performance of the agent.

[0040] In practical implementation, the calculation of policy substitution loss and the final evaluation of agent performance follow the example scenario, which continues from the iterative update phase of near-end policy optimization. The policy substitution loss is calculated based on sampled transition samples. These transition samples, randomly sampled from the agent's behavioral experience pool, contain an environmental state representation, a recorded action, an immediate reward value, and the next state. The environmental state representation from the sampled transition samples is input into the decision-making policy network of the current primary report-generating agent. The decision-making policy network outputs the probability of selecting the recorded action from the sampled transition sample under the current policy; this probability is a value between 0 and 1. Historical action probabilities for selecting the recorded action under the old policy are obtained from the historical records associated with the sampled transition samples. These historical action probabilities were calculated and stored by the policy network at the time the transition sample was generated. The ratio of the action probability under the current policy to the historical action probability under the old policy is calculated to obtain the policy probability ratio. This ratio reflects the change in the degree of preference for the same action between the old and new policies.

[0041] In some embodiments, a policy substitution loss term is constructed using the policy probability ratio and the advantage function estimate in the sampled transition samples. The advantage function estimate represents the expected advantage of taking an action relative to the average level in a given state, and is calculated using a generalized advantage estimation algorithm based on the immediate reward value and subsequent state value in the transition samples. The policy substitution loss term encourages actions that bring higher advantages by increasing the probability ratio of actions that bring a positive advantage function. Simultaneously, a pruning operation on the probability ratio prevents the current policy from updating too rapidly relative to the old policy; the pruning operation restricts the probability ratio to the interval [1-ε, 1+ε], where ε is a small hyperparameter, such as 0.2. Average task completion and report quality scores are calculated by selecting a fixed number of evaluation tasks from a pre-constructed evaluation task set, for example, 50 evaluation tasks. The evaluation task set is independent of the training task generator, and each evaluation task has a standard answer or expert review criteria.

[0042] Optionally, each evaluation task is executed sequentially in a controlled evaluation environment. The controlled evaluation environment uses a version of the simulator with logic consistent with the training environment but free of random noise, driving the updated primary report generation agent to plan and execute from scratch until a final report is generated or the maximum number of interaction steps is reached. Based on the updated primary report generation agent's report, the task completion score and report quality score for each evaluation task are quantitatively calculated based on the report's completeness, structural conformity, and consistency with the standard answer, or according to expert review criteria. The completeness score checks whether the report covers all elements required by the task; the structural conformity score checks the report's chapter division, logical coherence, and format; and the consistency score compares the agent's output with the pre-stored standard answer in terms of key conclusions and data. Expert review criteria can be a detailed scoring scale predefined by domain experts. It can be understood that the policy substitutes for the loss term. The calculation method is as follows: in: This indicates that the strategy replaces the loss term. This indicates the expectation for the current sampling batch. This represents the strategy probability ratio, which is the ratio of the probability of the current strategy action to the probability of historical actions of the old strategy. This represents the estimated value of the dominance function at time step t in the sampled transfer sample. It's a clipping range hyperparameter. and The functions work together to encourage dominant actions and limit the update magnitude. The arithmetic mean of the task completion score and report quality score for all evaluation tasks is calculated to obtain the average task completion score and report quality score used to evaluate the overall performance of the agent.

[0043] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for autonomously constructing a report-generating agent based on a knowledge-enhanced large model, characterized in that: The method includes: Obtain structured documents and unstructured text from the original business domain and perform knowledge extraction to construct a domain knowledge graph; Based on the general large language model, an improved progressive knowledge injection algorithm is used to integrate the knowledge in the domain knowledge graph into the parameters of the general large language model, forming a knowledge-enhanced base model. Extract task planning sequences and operation trajectories from the historical generation records of target reports to construct a sample library of task planning and tool calls; Based on the knowledge-enhanced base model and the demonstration sample library, a behavior cloning strategy is used to perform supervised fine-tuning of the initial agent model to generate a primary report-generating agent. Set up a reinforcement learning training field that includes multiple rounds of complex report generation tasks and diverse environmental perturbations, so that the primary report generation agent can explore strategies in the reinforcement learning training field; During the strategy exploration process, the state, action and reward data generated by the interaction between the primary report generating agent and the reinforcement learning training field are collected to form an agent behavior experience pool. The decision-making strategy parameters of the primary report-generating agent are iteratively updated using the agent's behavioral experience pool through a proximal policy optimization method. When the evaluation index reaches a preset threshold, the training terminates and the final report-generating agent, which has been autonomously constructed, is output.

2. The method for autonomously constructing a report-generating agent based on a knowledge-enhanced large model according to claim 1, characterized in that, The process of acquiring structured documents and unstructured text from the original business domain, performing knowledge extraction, and constructing a domain knowledge graph includes: The structured document is subjected to pattern recognition and field parsing to extract entity descriptions, attribute definitions and numerical constraints, and generate structured initial entity attribute pairs. Semantic segmentation and named entity recognition are performed on the unstructured text to locate the core concepts, terms and key descriptions in the text fragments, forming a set of unstructured semantic fragments; The initial entity attribute pairs and the set of unstructured semantic fragments are input into a preset relation extraction model. The relation extraction model identifies and establishes semantic associations between different entities and concepts based on predefined business relationship categories. The extracted entities, attributes, and semantic relationships are instantiated into nodes and edges according to the graph construction specification. Knowledge fusion technology is used to eliminate ambiguity in entity reference and conflict in attributes, and finally integrated to form an interconnected domain knowledge graph.

3. The method for autonomously constructing a report-generating agent based on a knowledge-enhanced large model according to claim 1, characterized in that, Based on the general large language model, an improved progressive knowledge injection algorithm is used to integrate knowledge from the domain knowledge graph into the parameters of the general large language model, forming a knowledge-enhanced base model, including: Knowledge subgraphs are sampled from the domain knowledge graph and converted into text sequences that the model can understand, forming a knowledge injection corpus; The knowledge-injected corpus is mixed with the general training corpus, and a progressive knowledge-injected training set is generated according to a preset mixing ratio. During model training, the ratio of general training corpus to knowledge-injected corpus in the progressive knowledge-injected training set is dynamically adjusted so that the model training focuses on understanding general semantics in the early stage and gradually increases the proportion of knowledge-injected corpus in the middle and later stages of training to enhance the absorption and memorization of domain knowledge. During the backpropagation phase of model parameter updates, an additional knowledge-preserving gradient is calculated for the training loss derived from the knowledge-injected corpus. This guides the model parameters to adapt to domain knowledge while minimizing the overriding forgetting of the original general capabilities. After training is completed, the knowledge-enhanced base model is obtained.

4. The method for autonomously constructing a report-generating agent based on a knowledge-enhanced large model according to claim 3, characterized in that, The working principle of the improved progressive knowledge injection algorithm includes: Define a knowledge injection scheduler. At the beginning of each iteration of model training, the knowledge injection scheduler calculates the proportion of the knowledge injection corpus in the total training batch in the current cycle according to the current training round and the preset scheduling strategy. When constructing a training batch, based on the proportion calculated by the knowledge injection scheduler, a corresponding number of samples are randomly sampled from the general training corpus and the knowledge injection corpus respectively, and combined into a training batch. When calculating the total loss of a training batch, the loss component contributed by the samples from the knowledge-injected corpus in the training batch is multiplied by a dynamic knowledge retention coefficient, which increases with the training rounds, thereby producing a stronger memory reinforcement effect on domain knowledge in the later stages of training. The improved progressive knowledge injection algorithm, through the collaboration of the knowledge injection scheduler and the dynamic knowledge retention coefficient, controls the integration of domain knowledge into the general large language model in a smooth and stable manner.

5. The method for autonomously constructing a report-generating agent based on a knowledge-enhanced large model according to claim 1, characterized in that, Based on the knowledge-enhanced base model and the demonstration sample library, a behavior cloning strategy is used to perform supervised fine-tuning of the initial agent model, generating a primary report-generating agent, including: The primary report generating agent possesses basic planning and execution capabilities; The architecture of the initial intelligent agent model is set with the knowledge-enhanced base model as the core, and a task planning header and a tool invocation header are added. Task planning sequences and operation trajectories are extracted from the demonstration sample library. The historical states and objectives in the task planning sequences are used as model inputs, and the corresponding subsequent planning actions and tool calls are used as labels for the model's expected outputs to construct supervised fine-tuning training samples. With the goal of maximizing the consistency between the output actions and labeled actions of the primary report generating agent on supervised fine-tuning training samples, all trainable parameters in the initial agent model, except for the core parameters of the knowledge-enhanced base model, are updated using the gradient descent algorithm. When the accuracy of the planned actions of the primary report generation agent on the validation set reaches a preset standard, supervised fine-tuning is stopped, and the primary report generation agent is obtained by imitating the demonstration behavior to perform preliminary task decomposition and tool selection.

6. The method for autonomously constructing a report-generating agent based on a knowledge-enhanced large model according to claim 1, characterized in that, The setup includes a reinforcement learning training ground with multiple rounds of complex report generation tasks and diverse environmental perturbations, enabling the primary report generation agent to explore policies within the training ground, including: Design a task generator, which generates task descriptions by randomly combining reports with different themes, complexity requirements, and data source conditions based on preset templates and rules. An environment simulator is constructed, which receives tool call instructions output by the primary report generating agent, simulates the execution of data query, information analysis, and chart drawing tools, and returns simulation results containing noise, errors, or missing information. The task generator and the environment simulator are encapsulated to form the reinforcement learning training field, which provides an interactive and dynamically changing training environment for the primary report generation agent. The primary report generation agent learns better decision-making strategies through trial and error and environmental feedback in the training environment.

7. The method for autonomously constructing a report-generating agent based on a knowledge-enhanced large model according to claim 1, characterized in that, During the policy exploration process, the state, action, and reward data generated by the interaction between the primary report-generating agent and the reinforcement learning training ground are collected to form an agent behavior experience pool, including: In each interaction step of the reinforcement learning training field, the environmental state representation observed by the primary report generating agent, the planned action or tool call action selected by the primary report generating agent, the immediate reward value returned by the environment simulator, and the next state to which the environment transitions after the action is performed are recorded. The recorded environmental state representation, planned actions or tool call actions, immediate reward value, and next state are combined into a complete transition sample; The transfer samples generated from each interaction are stored in a randomly accessible experience replay buffer in chronological order, and continuously accumulated to form the agent behavior experience pool. The agent behavior experience pool stores the exploration experience of the primary report generating agent under different tasks and environmental states.

8. The method for autonomously constructing a report-generating agent based on a knowledge-enhanced large model according to claim 1, characterized in that, The process involves using a proximal policy optimization method to iteratively update the decision policy parameters of the initial report-generating agent using the agent's behavioral experience pool. When the evaluation metric reaches a preset threshold, training terminates, and the final, autonomously constructed report-generating agent is output, including: A small batch of transfer samples is randomly sampled from the agent's behavioral experience pool; Based on the sampled transfer samples, calculate the probability ratio between the current primary report generating agent policy and the old policy, and calculate the policy substitution loss accordingly. Calculate the value function loss, which is used to evaluate the accuracy of the current primary report generating agent's prediction of the state value; The policy substitution loss and the value function loss are combined with preset weights, and a policy entropy regularization term is added to form the overall objective function for near-end policy optimization. The overall objective function is optimized using the gradient ascent algorithm, and the parameters of the decision policy network and value function network in the primary report generating agent are updated to complete one round of iterative update. After each iteration, the updated primary report generation agent is run on the evaluation task set to calculate the average task completion rate and report quality score. When the average task completion rate and report quality score exceed the preset evaluation index threshold multiple times consecutively, the model is determined to have converged, and the agent that has been trained at this time is output as the final report generation agent.

9. The method for autonomously constructing a report-generating agent based on a knowledge-enhanced large model according to claim 8, characterized in that, The sampled transfer samples are used to calculate the probability ratio between the current primary report-generating agent's policy and the old policy, and the policy substitution loss is calculated accordingly, including: The environmental state representation in the sampled transfer sample is input into the decision policy network of the current primary report generating agent to obtain the action probability of selecting the recorded action in the sampled transfer sample under the current policy. Obtain the historical action probability of selecting the recorded action under the old strategy from the historical records associated with the sample transfer samples; Calculate the ratio of the probability of an action under the current policy to the probability of a historical action under the old policy to obtain the policy probability ratio; Using the policy probability ratio and the advantage function estimate in the sampled transfer samples, a policy substitution loss term is constructed. This policy substitution loss term encourages actions that bring higher advantage, while the pruning operation of the probability ratio prevents the current policy from updating too much relative to the old policy.

10. The method for autonomously constructing a report-generating agent based on a knowledge-enhanced large model according to claim 8, characterized in that, After each iteration, the updated primary report generation agent is run on the evaluation task set to calculate the average task completion rate and report quality score, including: A fixed number of assessment tasks are selected from a pre-built set of assessment tasks, wherein the assessment tasks have standard answers or expert review criteria; In a controlled evaluation environment, each evaluation task is executed sequentially, driving the updated primary report generating agent to plan and execute from scratch until the final report is generated or the maximum number of interaction steps is reached and terminated. Based on the updated primary report, the agent generates reports that are complete in content, standardized in structure, and consistent with the standard answer, or, based on expert review criteria, quantitatively calculates the task completion score and report quality score for each evaluation task. The average task completion score and report quality score for all evaluation tasks are calculated to obtain the average task completion score and report quality score, which are used to evaluate the overall performance of the agent.

Images