Optimization method and device of large language model agent, and computer device

Abstract

This application relates to the field of large-scale model technology, and discloses an optimization method, apparatus, and computer device for a large language model agent. The method includes: acquiring task data containing domain category identifiers; performing initial task processing through the agent to generate a first deliverable; invoking a preset evaluation rule matching the domain for evaluation to obtain a first score; when the first score meets a trigger condition, generating a candidate skill based on the evaluation result and execution trajectory; after loading the candidate skill, the agent performs secondary processing on the same task to generate a second deliverable and evaluate it to obtain a second score; and, if it is confirmed that the candidate skill is actually invoked and the second score is higher than the first score, writing it into a persistent skill library. This method locates defects through domain-specific evaluation and drives skill generation, using a two-stage verification closed loop to ensure that only effective improvement strategies are solidified, thus achieving autonomous evolution of the agent's capabilities and continuous improvement of delivery quality.

Description

Technical Field

[0001] This application relates to the field of large model technology, and in particular to an optimization method, apparatus, and computer equipment for a large language model intelligent agent. Background Technology

[0002] Large language models have significantly improved the capabilities of natural language understanding and generation. Intelligent agents built based on such models are gradually being used to perform various professional tasks and complete work delivery through tool invocation and planning mechanisms.

[0003] In existing implementations of large language model agents, their task execution capabilities largely rely on pre-defined rule sets or manually orchestrated workflows. When relying on statically pre-defined rules to execute specialized tasks, the agent's fixed capability configuration cannot adaptively adjust and improve when faced with scenarios exceeding preset boundaries or when specific defects are exposed during execution. Error information and low-quality deliverables generated during task execution are difficult to capture and extract into reusable improvement strategies. Due to the lack of an autonomous evolution mechanism to transform execution feedback into effective capabilities, the agent repeatedly produces results that do not meet professional delivery standards on similar tasks, making it difficult to continuously improve the quality of deliverables. Summary of the Invention

[0004] Based on this, it is necessary to address the problem that the quality of deliverables is poor due to the reliance on static pre-defined rules for large language model agents to process specialized tasks in existing technologies. Therefore, an optimization method for large language model agents is proposed.

[0005] Firstly, an optimization method for a large language model agent is provided, the method comprising: Obtain task data for the target task, wherein the task data includes a domain category identifier; The target task is processed initially by a large language model agent to generate the first deliverable. Based on the domain category identifier, a preset domain evaluation rule is invoked to evaluate the first deliverable and obtain a first evaluation result, which includes a first score. When the first evaluation result meets the preset triggering conditions, the large language model agent performs attribution analysis on the text feedback and execution trajectory in the first evaluation result to locate the defective link in the initial task processing that caused the score of the first evaluation result to be lower than expected; it calls a preset knowledge search tool to obtain external reference knowledge based on the feature information of the defective link; it inputs the external reference knowledge, the text feedback in the first evaluation result, and the execution trajectory into the large language model agent to guide it to generate a coping strategy for the defective link and encapsulate it into the candidate skill; The candidate skills are loaded by the large language model agent, and secondary task processing is performed on the target task to generate a second deliverable. The second deliverable is evaluated to obtain a second evaluation result, which includes a second score; If it is confirmed that the candidate skill is invoked in the secondary task processing and the second score is higher than the first score, the candidate skill is written into the persistent skill library.

[0006] Secondly, an optimization device for a large language model agent is provided, the device comprising: The acquisition module is used to acquire task data of the target task, wherein the task data includes a domain category identifier; The processing module is used to perform initial task processing on the target task through a large language model agent and generate the first deliverable. The evaluation module is used to evaluate the first deliverable by invoking preset domain evaluation rules according to the domain category identifier, and obtain a first evaluation result, wherein the first evaluation result includes a first score; The generation module is used to, when the first evaluation result meets the preset triggering conditions, perform attribution analysis on the text feedback and execution trajectory in the first evaluation result through the large language model agent to locate the defective link in the initial task processing that caused the score of the first evaluation result to be lower than expected; call the preset knowledge search tool to obtain external reference knowledge based on the feature information of the defective link; input the external reference knowledge, the text feedback in the first evaluation result, and the execution trajectory into the large language model agent to guide it to generate a coping strategy for the defective link and encapsulate it into the candidate skill; The processing module is further configured to load the candidate skills through the large language model agent, perform secondary task processing on the target task, and generate a second deliverable; The evaluation module is also used to evaluate the second deliverable and obtain a second evaluation result, which includes a second score. The writing module is used to write the candidate skill into the persistent skill library when it is confirmed that the candidate skill is invoked in the secondary task processing and the second score is higher than the first score.

[0007] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described optimization method for a large language model agent.

[0008] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described optimization method for a large language model agent.

[0009] The beneficial effects of this application are: This application addresses the problem in existing technologies where large language model agents rely on static pre-defined rules to process specialized tasks, resulting in poor deliverable quality, by constructing a domain-specific evaluation-driven closed-loop self-evolution mechanism.

[0010] First, this application introduces pre-defined domain evaluation rules that match the task domain category identifier, performing a structured evaluation of the first deliverable generated by the agent's initial task processing, producing an evaluation result including a first score. This feature allows the evaluation of deliverable quality to no longer rely on generalized indicators, but rather align with the professional delivery standards of specific industries. The resulting evaluation feedback can accurately pinpoint specific defects in the execution process, providing a reliable basis for subsequent targeted improvements.

[0011] Secondly, when the first score meets the preset triggering conditions, the method generates candidate skills based on the evaluation results and the execution trajectory of the first task processing. The agent then loads the candidate skills and performs secondary processing on the same target task. This feature enables the agent to autonomously extract improvement strategies from execution defects and apply them, breaking the limitation of existing technologies where capability rules rely entirely on manual pre-setting and are fixed, and realizing targeted capability supplementation driven by task feedback.

[0012] Furthermore, this application compares the second score obtained from the secondary processing with the first score, and only writes the candidate skill into the persistent skill library after confirming that the candidate skill has been actually invoked and the score has improved. This feature uses a secondary verification closed loop as a prerequisite for skill solidification, ensuring that only empirically effective improvement strategies can enter the persistent capability set, avoiding the disorderly accumulation of invalid or redundant rules, and making the agent's capability evolution based on a measurable and verifiable objective foundation.

[0013] In summary, the method of this application enables large language model agents to have a complete autonomous evolution capability in the execution of professional tasks, from defect feedback to skill refinement, and then to empirical verification and solidification, thereby achieving continuous improvement in the quality of deliverables. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] in: Figure 1 This is a schematic diagram of the system architecture of a computer device in one embodiment; Figure 2 This is a flowchart of an optimization method for a large language model agent in one embodiment; Figure 3 This is a schematic diagram of a computer storage medium in one embodiment; Figure 4 This is a structural block diagram of an optimization device for a large language model agent in one embodiment; Figure 5 This is a structural block diagram of a computer device in one embodiment. Detailed Implementation

[0016] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0017] The optimization method for large language model agents provided in this application can be applied to, for example, Figure 1 The system architecture may include: computer equipment 100 and server 101.

[0018] The computer device 100 and the server 101 can communicate via a network, which serves as the medium for providing communication links between the aforementioned units. The network can include various types of wired or wireless communication links, such as: wired communication links including optical fiber, twisted pair, or coaxial cable, and wireless communication links including Bluetooth, Wi-Fi, or microwave communication links.

[0019] Computer device 100 is the core control node in this application responsible for task scheduling, evaluation execution, candidate skill generation, and skill library management. Computer device 100 can be an independent physical server, virtual server, cloud server instance, or computer device with sufficient computing power.

[0020] After receiving the target task, computer device 100 acquires task data and parses it to extract fields such as domain category identifier, occupation category identifier, task description information, and reference document identifier. Computer device 100 then calculates the complexity of the target task using its internally deployed complexity assessment module and stores the calculation result in an extended field of the task data.

[0021] Computer device 100 is also responsible for invoking preset domain evaluation rules to evaluate the first deliverable returned by server 101, generating a first evaluation result that includes a first score, sub-scores for each evaluation dimension, and textual feedback. Based on the first evaluation result and the execution trajectory of the initial task processing, computer device 100 generates candidate skills and writes them into a temporary candidate skill library. During the execution phase of step S5, computer device 100 reads the candidate skill content from the temporary candidate skill library through a preset tool call interface and sends it to server 101 as part of the context.

[0022] Computer device 100 evaluates the second deliverable returned by server 101 and obtains a second evaluation result. In step S7, computer device 100 verifies whether the candidate skill is invoked in the secondary task processing and whether the second score is higher than the first score. After verification, the candidate skill is migrated from the temporary candidate skill library and written into the persistent skill library.

[0023] Server 101 is the computing node where the large language model agent is deployed and runs in this application. Server 101 can be a standalone physical server, a virtual server, a cloud server instance, or a server cluster consisting of multiple computing nodes. Server 101 is internally deployed with a large language model agent, which refers to a large language model running instance capable of receiving natural language instructions, understanding task context, calling external tools, and generating text deliverables.

[0024] Server 101 establishes a data connection with computer device 100 through a preset network communication protocol and receives task processing requests sent by computer device 100. The data carried in the task processing request includes task description information, reference file content, and optional candidate skill context information.

[0025] The large language model agent in server 101 performs the initial task processing based on the task description information and reference file content sent by computer device 100, generates the first deliverable, and returns the complete content of the first deliverable and the execution trajectory of the initial task processing to computer device 100. The execution trajectory records the tool call sequence of the large language model agent during the initial task processing, the input parameters and output results of each tool call, and the timestamp information of each processing stage.

[0026] The large language model agent in server 101 receives context information containing candidate skill content sent by computer device 100, identifies and loads candidate skills during secondary task processing, applies the improved operation logic described in the candidate skills to reprocess the target task, generates a second deliverable, and returns the complete content of the second deliverable and the execution record of the secondary task processing to computer device 100.

[0027] It should be understood that Figure 1 The number and structure of the computer equipment and servers shown are for illustrative purposes only. The appropriate number of components and structure can be implemented according to specific needs.

[0028] Please see Figure 2 As shown, Figure 2 A flowchart illustrating the optimization method for a large language model agent provided in this application embodiment includes the following steps: S1. Obtain the task data for the target task, which includes the domain category identifier.

[0029] The computer equipment receives tasks from external systems or users, which are input in the form of structured task data. The task data contains at least one domain category identifier, which represents the business domain to which the target task belongs. The domain category identifier can be an industry category identifier, a job category identifier, or both. After acquiring the task data, the computer equipment uses it as the baseline input for all subsequent processing flows. The task data can originate from a single record pushed to the message queue by the task management platform, or from batch-imported structured files, such as JSONL (JSON Lines, a common text file format for storing structured data) or Parquet files. The computer equipment reads and parses the data line by line or record by record. The integrity of the task data is guaranteed by the upstream system, and the computer equipment can directly proceed to subsequent steps after receiving the data.

[0030] In one example, consider the task of writing a risk control report in the financial industry. The task data received by the computer device includes an industry category identifier "Finance Industry" and an occupational category identifier "Risk Control Analyst". The task data also includes a task description: "Please write a client risk assessment report based on the client transaction data in the attachment," and a reference document identifier pointing to a data table file recording transaction details for the past 12 months. The computer device loads this task data into memory as the complete context for this task processing.

[0031] S2. The large language model agent performs the initial task processing on the target task and generates the first deliverable.

[0032] The computer device invokes a large language model agent, inputting the task description and relevant contextual information of the target task. During its initial task processing, the large language model agent does not rely on any pre-written domain-specific skills, but only on its own general reasoning and tool-calling capabilities to complete the task. The large language model agent can autonomously plan its execution steps, invoking one or more external tools to obtain auxiliary information, such as searching knowledge bases, reading reference documents, and performing code calculations. After multiple steps of reasoning and tool calls, it generates a complete deliverable that meets the requirements of the target task. This deliverable is marked as the first deliverable. Simultaneously, the computer device records the complete execution trajectory of the large language model agent during this task processing, including the reasoning content of each step, the name of the tool invoked and its input parameters, the result data returned by the tool, and the jump logic between each step. The data structure of the execution trajectory can be a step-by-step structured log, with each log record corresponding to a reasoning step or tool-calling step.

[0033] In one example, following the previous task of writing a risk control report, the large language model agent, upon first processing the task, reads the transaction log data from the reference file. It discovers records with missing fields, calls a data statistics tool to calculate the missing percentage, and then uses its own reasoning ability to directly write a customer risk assessment report. The computer saves this report as the first deliverable. Simultaneously, the computer records the entire execution trajectory of this process, including the steps of reading the file, calling the data statistics tool and the statistical results, and the reasoning steps for writing each paragraph of the report. The execution trajectory shows that the agent did not perform any filling or removal of missing data; instead, it directly wrote the report based on the incomplete data.

[0034] S3. Based on the domain category identifier, invoke the preset domain evaluation rules to evaluate the first deliverable and obtain the first evaluation result, which includes the first score.

[0035] The computer device reads the domain category identifier obtained from S1 and performs a matching search in a pre-built evaluation rule base. This evaluation rule base stores domain evaluation rules for various domains. Each domain evaluation rule consists of predefined structured evaluation dimensions and corresponding scoring criteria, which can be generated by experts or by a large language model trained on historical high-quality delivery samples. Each set of domain evaluation rules includes structured evaluation dimensions adapted to that domain, as well as scoring criteria and scoring scales for each evaluation dimension. The computer device inputs the matched domain evaluation rules and the first deliverable into an evaluation model. The evaluation model scores the first deliverable dimension by dimension according to the evaluation rules and generates textual feedback. The evaluation model can be an independently deployed instance of a large language model, or it can be the large language model agent itself switched to evaluation mode. After the evaluation is completed, the computer device stores the structured results output by the evaluation model as the first evaluation result. The first evaluation result includes at least a comprehensive first score, as well as sub-scores for each evaluation dimension and corresponding textual feedback.

[0036] The evaluation model adopts a hierarchical structure based on a large language model in its overall architecture. The model uses a pre-trained large language model as the base model, and three functional layers are built on top of the base model.

[0037] The first layer is the input processing layer. This layer receives two sets of input data: the full text of the deliverable to be evaluated and the domain evaluation rules matched with the domain category identifier. The input processing layer concatenates the full text of the deliverable and the domain evaluation rules into a structured prompt text. This prompt text defines the evaluation task's objective, a complete list of evaluation dimensions, a description of the scoring criteria for each dimension, and the output format requirements in a clear instruction format. The input processing layer preprocesses any formatting marks or non-textual content that may exist in the deliverable, converting them into textual descriptions that the evaluation model can understand.

[0038] The second layer is the evaluation and inference layer. This layer is the core computational component of the evaluation model. Its function is to analyze and evaluate the deliverables dimension by dimension based on the prompt text constructed by the input processing layer. During the inference process, the base model reads the scoring criteria for each dimension sequentially according to the dimensional order specified by the domain evaluation rules. It compares the actual performance of the deliverable in that dimension with the scoring criteria, determines which level the deliverable has reached in the criteria, and generates the corresponding score and textual feedback for that dimension. The evaluation and inference layer supports multi-dimensional parallel analysis. The evaluations of each dimension are semantically independent; the evaluation conclusion of one dimension does not affect the judgment of other dimensions.

[0039] The third layer is the output generation layer. This layer aggregates and integrates the scores and textual feedback generated by the evaluation and reasoning layer, producing a structured evaluation result according to a pre-defined output template. This structured evaluation result includes a comprehensive score, sub-scores for each dimension, and corresponding textual feedback for each dimension. The calculation method for the comprehensive score is determined by the aggregation logic specified in the domain evaluation rules, and can be a weighted sum of the scores, an arithmetic average, or taking the lowest score, etc. The output generation layer validates the format of the final output to ensure that all output scores are valid values ​​and that there are no logical contradictions between the textual feedback and the scores.

[0040] The training of the evaluation model was completed using supervised fine-tuning. The training process consisted of three stages: training data construction, model fine-tuning, and model validation.

[0041] In the training data construction phase, a large number of historical deliverable samples are first collected from multiple business domains, with each sample corresponding to a real or simulated task output. Annotators or experts with relevant domain backgrounds manually annotate each deliverable sample dimension-by-dimensional according to the evaluation rules corresponding to that domain. The manual annotation includes a score for each evaluation dimension and detailed textual feedback for that dimension. During the annotation process, the same deliverable is independently annotated by at least two annotators. Samples with score differences exceeding a preset threshold are reviewed or arbitrated to ensure the consistency and authority of the annotation results. After annotation, each deliverable sample, along with its corresponding domain evaluation rules, the manually annotated score, and the textual feedback, is combined into a training sample, forming the training dataset. The training dataset is divided into a training set and a validation set according to a certain ratio.

[0042] During the model fine-tuning phase, a pre-trained large language model is used as the initial model, and the model parameters are fine-tuned using samples from the training set. The input to each training sample is a combination of deliverable content and domain evaluation rules. The training objective is to minimize the difference between the model's output score and text feedback and the human-annotated results. The fine-tuning process employs a standard language model training objective function, an adaptive learning rate algorithm for the optimizer, and adjustments to the training hyperparameters based on the evaluation accuracy metrics on the validation set. Evaluation accuracy refers to the ratio of the model's scores for each dimension to the human-annotated scores within a preset tolerance range.

[0043] During the model validation phase, the fine-tuned model is evaluated using samples from the validation set. Evaluation metrics include the mean absolute error of the scores, the consistency between the written feedback and the scores, and the pass rate of domain expert review of the evaluation results for each dimension. Only when all the above metrics meet the preset passing standards is the trained evaluation model deployed online for the actual evaluation task in step S3.

[0044] The specific process of the evaluation model performing the evaluation task in step S3 is as follows.

[0045] The evaluation model first receives two inputs: the first deliverable generated in step S2, and the domain evaluation rules matched by the computer device from the evaluation rule base based on the domain category identifier. The input processing layer of the evaluation model reads the entire deliverable, preserving the original logical hierarchy of paragraph structure, table data, and list information. Simultaneously, the input processing layer parses the domain evaluation rules into a set of structured evaluation instructions. Each instruction corresponds to an evaluation dimension, including the dimension's name, maximum score, score level, and descriptions of the achievement conditions for each level.

[0046] Subsequently, the evaluation and inference layer initiates a dimension-by-dimensional evaluation process. For each evaluation dimension defined in the domain evaluation rules, the evaluation and inference layer reads the full text of the scoring criteria for that dimension, retrieves relevant paragraphs or data from the deliverables, and analyzes whether the deliverables' performance on that dimension meets the conditions for each level in the scoring criteria. When the deliverables' performance fully or partially matches the conditions for a certain level, the evaluation and inference layer determines the score for that dimension based on the degree of matching. If the deliverables do not contain the content required for that dimension, that dimension receives a score of zero, and a corresponding missing information is generated as text feedback. Simultaneously with determining the score, the evaluation and inference layer generates a text feedback for that dimension, specifically explaining the reasons for the score. If the score is less than full marks, the feedback also explains the reasons for the lost points and areas for improvement.

[0047] Once the scores and text feedback for all evaluation dimensions have been generated, the output generation layer calculates the overall score by weighted summation or arithmetic average of the scores for each dimension, according to the comprehensive score aggregation logic specified in the domain evaluation rules. The overall score, the scores for each dimension, and the text feedback are assembled into a structured evaluation result, which is returned to the computer device as the first evaluation result. The computer device stores this first evaluation result for use as input in subsequent trigger condition judgments and learning sessions.

[0048] In one example, following the previous one, the computer device matches a set of domain evaluation rules called "Risk Control Report Quality Evaluation Standard" in the evaluation rule base based on the domain categories "financial industry" and "risk control analyst". This rule includes four evaluation dimensions: data integrity processing quality, risk assessment logic rigor, report structure standardization, and language professionalism, with each dimension worth 25 points, for a total of 100 points. The evaluation model evaluates the first deliverable according to this rule. The evaluation model finds that the report does not provide any explanation or processing for missing data, awarding 5 points for the "data integrity processing quality" dimension, along with text feedback stating, "The report did not identify and process missing fields in the transaction log, resulting in incomplete evaluation conclusions." The scores for the other three dimensions are 18, 20, and 19 points respectively. The computer device uses the comprehensive score of 62 points as the first score and stores the complete evaluation result, including all dimension scores and text feedback, as the first evaluation result.

[0049] S4. When the first evaluation result meets the preset triggering conditions, the large language model agent performs attribution analysis on the text feedback and execution trajectory in the first evaluation result to locate the defective link in the initial task processing that caused the score of the first evaluation result to be lower than expected; calls the preset knowledge search tool, uses the feature information of the defective link as the query basis to obtain external reference knowledge; inputs the external reference knowledge, the text feedback in the first evaluation result and the execution trajectory into the large language model agent to guide it to generate a coping strategy for the defective link and encapsulate it into candidate skills.

[0050] The computer device evaluates the initial assessment result, checking if it meets preset triggering conditions. These conditions can be a scoring threshold (triggered when the initial score falls below this threshold), a complexity strategy (triggered directly when the task is marked as high-complexity), or a combination of both. When the assessment indicates the triggering conditions are met, the computer device initiates a learning session. In this session, the computer device analyzes both the textual feedback from the initial assessment result and the execution trajectory of the first task processing. Based on the specific issues identified in the textual feedback, the computer device traces back the execution trajectory to pinpoint at least one inadequate operational step that led to the problem. This inadequate operational step could be an incorrect tool call, a missed processing step, or an unreasonable reasoning direction. Building on this, the computer device can further combine external knowledge searches to obtain improvement methods or reference examples for the inadequate operational step, generating a candidate skill containing specific operational guidance or improvement strategies. The generated candidate skill is temporarily stored in a temporary candidate skill library, separate from existing skills in the persistent skill library. The temporary candidate skill library refers to a temporary storage area used to temporarily store generated candidate skills in a single task self-evolution loop. Its lifecycle is usually tied to the current loop verification process. After verification, the skills are migrated to the persistent skill library, and the skills that fail verification and the data in the temporary library will be cleared.

[0051] Following the previous example, the computer device determined the first score to be 62 points, which is lower than the preset trigger threshold of 75 points, thus meeting the trigger condition. The computer device initiated a learning session and analyzed the text feedback in the first evaluation result: "The report did not identify and process missing fields in the transaction log." The computer device traced the execution path back and located that the agent skipped the data quality check step directly after reading the data, failing to identify the existence of missing fields. The computer device invoked external search capabilities to find industry practices related to "data missing handling" and obtained a reference method of "first calculating the missing percentage, and if it exceeds 10%, then filling or marking the missing values." Based on this, the computer device generated a candidate skill named "Data Quality Pre-Check," which includes: after reading the structured data file, first counting the number and percentage of missing fields column by column; for fields with a missing percentage exceeding 10%, clearly marking them in the report and explaining the processing method. This candidate skill was added to the temporary candidate skill library.

[0052] S5. Load candidate skills through a large language model agent, perform secondary task processing on the target task, and generate a second deliverable.

[0053] In this process, the computer selects candidate skills generated in S4 from a temporary candidate skill library, and instructs the large language model agent to re-execute the task processing for the same target task. The key difference from the initial task processing is that, during the second task processing, the large language model agent needs to explicitly load the entire content of the candidate skill through a pre-defined tool invocation interface. After loading, the large language model agent must follow the operational steps, rules, or strategies specified in the candidate skill during inference and tool invocation. The second task processing also involves multiple steps of inference and tool invocation, ultimately generating a new deliverable, which the computer marks as the second deliverable. The computer also records the complete execution trajectory of the second task processing, which will be used to subsequently verify whether the candidate skill was actually invoked.

[0054] Following the previous example, the computer device instructed the large language model agent to perform a second processing of the same risk control report writing task. After starting, the large language model agent first retrieved the complete content of the "Data Quality Pre-Check" candidate skill from a temporary candidate skill library through a skill loading tool interface and loaded it into the current session context. After reading the transaction flow data, the agent followed the requirements of the candidate skill and statistically analyzed the proportion of missing fields column by column. It found that the "Customer Occupation" field had a missing rate of 23%, so it specifically noted in the report that "The missing rate of the Customer Occupation field is 23%, and this field was not considered in this evaluation." Based on this, the agent continued to complete the remaining analysis and generated a second risk control report as the second deliverable. The computer device also fully recorded the execution trajectory of this processing, which clearly included the record of calling the skill loading tool to retrieve the "Data Quality Pre-Check" and the subsequent steps of performing missing value statistics.

[0055] S6. Evaluate the second deliverable to obtain a second evaluation result, which includes a second score.

[0056] The computer device uses the exact same domain evaluation rules and models as in S3 to evaluate the second deliverable. The evaluation dimensions and scoring criteria are strictly consistent with those for the first deliverable to ensure comparability between the two evaluations. The evaluation model outputs a structured second evaluation result, which includes at least a comprehensive second score, sub-scores for each evaluation dimension, and corresponding textual feedback. The computer device associates and stores the second evaluation result with the first evaluation result for subsequent comparative analysis.

[0057] Following the previous example, the computer equipment used the same set of "Risk Control Report Quality Evaluation Standards" and the same evaluation model to evaluate the second risk control report. The evaluation results showed that, because this report proactively identified and explained the missing data, the "Data Integrity Processing Quality" dimension improved from 5 points to 20 points, while the scores for the other three dimensions were 18, 21, and 20 points respectively. The computer equipment received a comprehensive score of 79 points as the second evaluation score, and the complete evaluation result, including scores for all dimensions and textual feedback, was stored as the second evaluation result.

[0058] S7. If it is confirmed that the candidate skill is called in the secondary task processing and the second score is higher than the first score, write the candidate skill into the persistent skill library.

[0059] The computer device rigorously verifies the validity of candidate skills, which involves two conditions that must be met simultaneously. The first verification condition is the verification of the invocation behavior. The computer device checks the secondary task processing execution trajectory recorded in S5 to confirm whether it contains a record of invocation of the candidate skill, specifically whether the agent has obtained the content of the candidate skill through the skill loading tool interface. The second verification condition is the verification of effect improvement. The computer device compares the second score with the first score to confirm whether the second score is higher than the first score. In a preferred embodiment, not only is a higher second score required than a higher first score, but the difference between the two must also reach a preset improvement threshold. This preset improvement threshold is a value used to determine whether the quality improvement is significant. It can be a fixed score difference (e.g., 5 points) or a percentage improvement relative to the initial score (e.g., 10%), and can be configured to be relevant to the task domain to improve the reliability of the verification. When both verification conditions are met, the computer device recognizes the candidate skill as a valid skill and migrates it from the temporary candidate skill library to the persistent skill library, completing the formal skill deposition. If any verification condition fails, the candidate skill is discarded and not persisted, thus ensuring that all skills added to the database have been empirically verified.

[0060] Following the previous example, the computer device performed two verifications. In the first verification, the computer device checked the execution trajectory of S5 and confirmed that it included a record of the large language model agent calling the skill loading tool to obtain "data quality pre-check," and that the subsequent missing value statistics steps were consistent with the operation instructions for this skill, thus the call behavior verification passed. In the second verification, the computer device compared the first score of 62 points with the second score of 79 points. The second score was 17 points higher than the first score, not only exceeding the first score but also exceeding the preset 5-point improvement threshold, thus the effect improvement verification passed. After both verifications passed, the computer device migrated the "data quality pre-check" candidate skill from the temporary candidate skill library to the persistent skill library, and this skill officially became a reusable capability asset of the large language model agent.

[0061] Furthermore, based on the initial evaluation results and the execution trajectory of the first task processing, candidate skills are generated, including: S41. Using a large language model agent, perform attribution analysis on the text feedback and execution trajectory in the first evaluation result to locate the defective link in the initial task processing that caused the score of the first evaluation result to be lower than expected.

[0062] The computer device inputs the textual feedback from the first evaluation result output in step S3 and the initial task processing execution trajectory recorded in step S2 into an attribution analysis session for the large language model agent. This attribution analysis session is configured to require the large language model agent to establish a causal mapping relationship between the textual feedback and the execution trajectory. Specifically, the large language model agent reads the problem descriptions indicated in the textual feedback, and for each problem description, it traces backward in the execution trajectory to find the semantically corresponding processing steps. When the operation content, order, or absence of a certain step forms a direct causal chain with the problem indicated in the textual feedback, the large language model agent marks that step as a defective link. The granularity of locating a defective link can be a specific tool call step, a piece of reasoning logic, or a stage consisting of a series of consecutive processing steps. The large language model agent generates an attribution description for each located defective link, including the location of the defective link in the execution trajectory, the specific operation content of the defective link, and the logical deduction of why the defective link caused the problem indicated in the textual feedback. After the attribution analysis is completed, the computer equipment will compile all the identified defective links and their attribution descriptions into a defective link list for use in subsequent steps.

[0063] For example, following the previous example, the text feedback from the first evaluation result indicated that "the report failed to identify and process missing fields in the transaction log, resulting in incomplete data for the evaluation conclusion." The computer device input this text feedback along with the execution trajectory of the initial task processing into the attribution analysis session. The large language model agent, tracing back through the execution trajectory, discovered that after reading the transaction log data file, the agent immediately jumped to the report writing stage without performing any data quality checks. Based on this, the large language model agent identified this jump as a defect and generated an attribution explanation: "The processing flow directly proceeded to step 3, report writing, after reading data in step 2, missing a data integrity verification step. This resulted in missing fields not being discovered and addressed, leading to an evaluation conclusion based on incomplete data, triggering the deduction item in the evaluation feedback." The computer device recorded this defect in the defect list.

[0064] S42. Invoke the preset knowledge search tool, use the characteristic information of the defective link as the query basis, and obtain external reference knowledge.

[0065] After locating the defect, the computer equipment invokes a pre-defined knowledge search tool. This tool is an external search interface independent of the large language model agent, capable of connecting to internal enterprise knowledge bases, technical document libraries, industry standard libraries, or public search engines. The computer equipment extracts feature information from the defect list generated by S41 as the basis for its query. This feature information may include the operation type involved in the defect, the corresponding technical terminology, and the problem category described in the text feedback. The computer equipment assembles this feature information into a query statement and sends it to the knowledge search tool for retrieval. The tool returns a set of document fragments or reference entries related to the query, each containing a title, a text summary, and a source identifier. The computer equipment sorts the returned results by relevance and selects a preset number of results with the highest relevance for retention as external reference knowledge. If the number of results returned by the knowledge search tool is insufficient or the relevance is below a preset threshold, the computer equipment can adjust the combination of feature information in the query statement and perform a new search.

[0066] For example, following the previous example, the computer equipment extracts feature information from the defect list, including the operation type "data reading" and the problem category "data missing handling." The computer equipment combines these two features into a query, "Data missing inspection and handling process after structured data reading," and sends it to the enterprise's internal knowledge base search interface. The knowledge search tool returns three relevant documents. The first one comes from a data analysis specification document, which states, "After the data file is loaded, a 'describe' operation should be performed to count the number and proportion of missing fields. Fields with a missing rate exceeding 15% need to be marked or filled before analysis." The computer equipment retains this content as external reference knowledge for the generation of subsequent candidate skills.

[0067] S43. Input external reference knowledge, textual feedback from the first evaluation results, and execution trajectory into the large language model agent to guide it in generating coping strategies for defective links and encapsulating them into candidate skills.

[0068] The computer device inputs three data sources into a skill generation session for the large language model agent: external reference knowledge acquired in step S42, textual feedback from the first evaluation result output in step S3, and the initial task processing execution trajectory recorded in step S2. The computer device sets instructions for the skill generation session requiring the large language model agent to generate an actionable coping strategy based on the external reference knowledge, the problems indicated by the textual feedback, and the processing flow in the execution trajectory. This coping strategy must clearly describe at which node of the task processing flow, what checks or operations are performed, what tools or methods are used, and what expected result is achieved. The language expression of the coping strategy needs to be standardized and instructive, enabling the large language model agent to directly understand and execute the strategy after loading it in subsequent tasks. The large language model agent encapsulates the generated coping strategy into a candidate skill. The data structure of the candidate skill includes at least three fields: skill name, applicable scenario description, and skill content text. The skill name is automatically generated by the large language model agent based on the core function of the coping strategy, following a unified naming format. The computer device writes the encapsulated candidate skills into a temporary candidate skill library, completing all processing in step S4.

[0069] For example, following the previous example, the computer device inputs the external reference knowledge "After the data file is loaded, a describe operation should be performed to count the number and proportion of missing fields," the text feedback "The report did not identify and process the missing fields in the transaction log," and the complete process from data reading to report writing in the initial execution trajectory, into the skill generation conversation. After analysis, the large language model agent generates a response strategy of "After the data file reading step is completed and before the analysis step begins, insert a data quality check step to count the number and proportion of missing records for each field column by column, and mark fields with a missing proportion exceeding 15% in the subsequent report in tabular form." The large language model agent encapsulates this response strategy as a candidate skill, named "Structured Data Missing Field Check and Annotation," with the applicable scenario description "Suitable for risk control reports and data analysis tasks processing structured data files such as CSV and Excel." The skill content is the complete description of the above response strategy. The computer device stores this candidate skill in a temporary candidate skill library.

[0070] In one possible implementation, S7, if it is confirmed that a candidate skill is invoked in the secondary task processing and the second score is higher than the first score, the candidate skill is written into the persistent skill library, including: S71. Verify whether the execution record of the secondary task processing contains a record of calling the candidate skill.

[0071] After completing the secondary task processing in step S5, the computer device has obtained a complete execution record for that processing. This execution record is stored in a structured sequence of steps, with each step containing the operation type identifier, input parameters, and output result. The computer device scans the execution record line by line, retrieving step entries with the operation type identifier "skill loading" or an equivalent category. When such an entry is found, the computer device further reads the input parameters of that entry and extracts the loaded skill name or skill identifier. The computer device then matches this skill name or skill identifier with the candidate skills generated in step S4. The matching method can be a complete string match or a fuzzy match based on semantic similarity calculation. If a match is successful, the computer device confirms that the candidate skill was actually invoked in the secondary task processing and marks the verification result as passed. If no skill loading type step entry is found in the execution record, or if the skill name recorded in the retrieved skill loading entries does not match any candidate skills, the computer device marks the verification result as failed. If the verification result fails, the candidate skill will be discarded directly and will not proceed to subsequent steps S72 and S73.

[0072] For example, following the previous example, the computer device scans each execution record of the secondary task processing generated in step S5. Step 3 of the execution record shows the operation type as "Skill Loading," and the skill name recorded in the input parameters as "Data Quality Pre-Check." The computer device matches this skill name with candidate skills in the temporary candidate skill library, finding that the two names are completely identical, confirming that the candidate skill was actually invoked in the secondary task processing. The computer device marks the verification result of S71 as passed.

[0073] S72. Verify whether the difference between the second score and the first score reaches the preset improvement threshold.

[0074] In this process, the computer device extracts a first score from the first evaluation result stored in step S3 and a second score from the second evaluation result stored in step S6. The computer device calculates the difference between the second score and the first score. A preset improvement threshold is a pre-configured numerical parameter, which can be an absolute score (e.g., 5 points) or a relative percentage (e.g., an improvement of at least 8%). When the preset improvement threshold is an absolute score, the computer device directly compares the calculated difference with the threshold. When the preset improvement threshold is a relative percentage, the computer device first divides the difference by the first score to obtain a relative improvement ratio, and then compares this ratio with the threshold. When the difference is greater than or equal to the preset improvement threshold, the computer device marks the verification result as passed, indicating that the introduction of the candidate skill has brought a measurable and substantial improvement to the task delivery quality. When the difference is less than the preset improvement threshold, the computer device marks the verification result as failed. Even if the second score is higher than the first score but the difference does not reach the threshold, the candidate skill is still considered to have an insignificant improvement effect, and the verification result is still failed. This candidate skill will be discarded and will not proceed to step S73.

[0075] For example, continuing from the previous example, the computer device extracts a first score of 62 points from the first evaluation result and a second score of 79 points from the second evaluation result. The computer device calculates the difference as 79 minus 62, which equals 17 points. The system's preset boost threshold is configured to 5 points. The computer device compares the difference of 17 points with the threshold of 5 points; 17 is greater than 5, confirming that the difference has reached the preset boost threshold. The computer device marks the verification result of S72 as passed.

[0076] S73. When the call record exists and the difference reaches the preset improvement threshold, write the candidate skill into the persistent skill library.

[0077] The computer device performs a logical AND operation on the S71 and S72 verification results. The computer device considers the candidate skill to have passed all validity checks in the two-stage verification loop if and only if both S71 and S72 verification results are passed. At this point, the computer device performs a persistent write operation on the candidate skill. The specific path of the write operation is to retrieve the candidate skill from the temporary candidate skill library, serialize it according to the data format specified in the persistent skill library, and append it to the storage file of the persistent skill library. The storage file of the persistent skill library can be a JSONL file, with each line recording one skill entry already added to the library. Before writing, the computer device can also perform evolution processing on the candidate skill, including checking whether there is functional overlap between the candidate skill and existing skills in the persistent skill library, merging overlapping entries, removing completely duplicate entries, and standardizing the skill name according to a unified naming convention. If either S71 or S72 verification result is failed, the computer device determines that the candidate skill has failed the validity check. At this point, the computer device does not perform a persistent write operation, but instead removes the candidate skill from the temporary candidate skill library to ensure that the skill will not enter the persistent skill library.

[0078] For example, following the previous example, both S71 and S72 verifications passed, satisfying both conditions simultaneously. The computer retrieves the candidate skill "Data Quality Pre-Check" from the temporary candidate skill library and checks the existing skill list in the persistent skill library. It finds no skill entries related to data quality checking, indicating no functional overlap or duplication. Following a unified naming convention, the computer confirms the skill name as "Structured Data Missing Field Check and Annotation," serializes it into a single line of JSONL format, and appends it to the persistent skill library file. This skill officially becomes a reusable capability asset for the large language model agent.

[0079] In one possible implementation, before writing candidate skills into the persistent skill library, the following steps are also included: A1. Merge, deduplicatize, and standardize the naming of candidate skills and existing skills in the persistent skill library.

[0080] After a candidate skill passes the dual verification in S71 and S72, the computer device does not immediately write it into the persistent skill library. Instead, it first performs a semantic similarity analysis between the candidate skill and all existing skills in the persistent skill library. The computer device uses an evaluation model from the same source as in S3 or an independent semantic matching model to calculate the semantic similarity between the skill content text of the candidate skill and the skill content text of each existing skill in the persistent skill library. The semantic similarity calculation can be vectorized: first, the two skill content texts are converted into high-dimensional semantic vectors, and then the cosine similarity or Euclidean distance between the two vectors is calculated to obtain a similarity value between 0 and 100%. When the semantic similarity between a candidate skill and an existing skill exceeds a preset merging threshold, such as 70%, the computer device determines that the two skills have overlapping functions and need to be merged. The specific merging process is as follows: the computer device inputs the content text, applicable scenario description, and skill name of two skills into the large language model agent. The large language model agent then merges the two pieces of content, taking the union of the operation steps, retaining the more complete execution logic, unifying terminology, and removing redundant descriptions to generate a merged skill content text and applicable scenario description. The merged skill inherits the identifier of the skill with the higher skill level or more recent update time, while the other skill is marked as pending deletion. If the semantic similarity between the candidate skill and multiple existing skills exceeds the merging threshold, all skills that meet the merging conditions, along with the candidate skill, are subjected to a multi-in-one fusion process.

[0081] For example, following the previous example, the verified candidate skill "Structured Data Missing Field Inspection and Annotation" has the following text: "After completing the step of reading the structured data file, count the number of missing records and the missing percentage for each field column by column. Fields with a missing percentage exceeding 15% will be annotated in tabular form in subsequent reports." The computer device performs semantic similarity calculations on this skill and each existing skill in the persistent skill library. It finds that there is already a skill in the library called "Pre-verification after Data File Reading," whose text is: "After reading a CSV or Excel file, check the integrity of each column of data and count the number of null values." The semantic similarity between the two skills is 82%, exceeding the 70% merging threshold. The computer device inputs the two skills into the large language model agent for fusion processing, generating the merged skill text: "After completing the step of reading the structured data file, count the number of missing records and the missing percentage for each field column by column. Fields with a missing percentage exceeding 15% will be annotated in tabular form in subsequent reports; for fields with a missing percentage not exceeding 15%, the processing can be skipped or the missing status can be noted." The merged skill inherits the identifier of "pre-verification after reading data file", and the original entry of the candidate skill is marked as pending deletion.

[0082] The deduplication and merging processes are seamlessly integrated, aiming to clean up skill entries with highly overlapping or even identical semantics. After merging, the computer device performs actual deletion operations on skill entries marked as pending deletion in the persistent skill library, removing the corresponding storage records from the persistent skill library file. Furthermore, the computer device performs semantic similarity checks again between the newly generated skill content and the remaining existing skills in the persistent skill library. If the semantic similarity between the newly generated skill content and an existing skill exceeds a higher duplication threshold, such as 95%, the computer device determines that the two skills are substantially duplicated. In this case, the computer device retains the skill with more complete content and more detailed operation steps, and deletes the other directly. Deduplication ensures that there are no two or more skill entries with the same or nearly identical functions in the persistent skill library, avoiding the expansion and redundancy of the skill library.

[0083] For example, following the previous example, the skill content generated after the merge process was compared again with another skill in the persistent skill library named "Report Data Source Labeling Specifications". The semantic similarity between the two skills was 38%, far below the 95% duplication threshold, and therefore did not constitute substantial duplication. The computer equipment only removed the original entry "Structured Data Missing Field Check and Labeling" previously marked for deletion from the library; the remaining skill entries remained unchanged. The number of skill entries in the library did not become redundant due to duplication.

[0084] The goal of naming standardization is to ensure that all skills in the persistent skill library follow a unified naming convention, facilitating subsequent skill retrieval, access, and manual review. The computer equipment has a pre-set naming standard template that specifies the components and order of skill names. For example, the naming standard may require skill names to follow the format of "operation object + operation action + applicable scenario," with each element connected by a specific separator, and each element using standardized business terminology rather than colloquial language. For each skill retained after merging and deduplication, the computer equipment extracts keywords and operation information from its skill content text and regenerates a standardized skill name according to the naming standard template. For newly generated skills after merging, the computer equipment generates standardized names simultaneously during skill fusion. For existing skill entries in the persistent skill library, if their names do not conform to the naming standard, the computer equipment also renames them. The renaming operation does not change the skill content text or applicable scenario description; it only modifies the skill name field. After naming standardization, all skill entries in the persistent skill library have uniformly formatted and semantically clear skill names.

[0085] For example, continuing from the previous example, the original skill name corresponding to the retained skill content after merging is "Pre-verification after data file reading," which does not conform to the computer device's preset naming convention template of "operation object plus operation action." The computer device extracts key information from the skill content, identifying the operation object as "structured data missing fields" and the operation action as "checking and labeling," and generates a standardized skill name "structured data missing field check and labeling" according to the template. The computer device then writes this name into the skill name field of the corresponding entry in the persistent skill library, replacing the original name "pre-verification after data file reading."

[0086] In one possible implementation, domain evaluation rules are a set of structured data objects pre-installed in the computer device. Each domain evaluation rule is mapped to one or more domain category identifiers, and this mapping is stored in the evaluation rule base of the computer device. Domain category identifiers are used to indicate the business scope to which the task belongs, and can be an industry name, a job title, or a combination of both. After the computer device obtains the task data of the target task in step S1, it extracts the domain category identifier from it and uses this identifier as a query key to match it in the evaluation rule base. The matching result is the domain evaluation rule applicable to the current task.

[0087] The internal structure of the domain evaluation rules consists of a set of evaluation dimensions, each representing an independent perspective for assessing the quality of deliverables. The number of evaluation dimensions is determined based on the characteristics of the domain, typically ranging from 3 to 8. These dimensions are designed to be complementary, ensuring that each dimension covers the quality requirements of the deliverables from different angles, and that there is no significant overlap in evaluation between them.

[0088] Each evaluation dimension corresponds to a scoring criterion. The scoring criterion defines the maximum score for that dimension, the scoring levels, and the achievement conditions for each level. Scoring levels are typically divided into 3 to 5 tiers, each corresponding to a score range or a fixed score. Achievement conditions are described using natural language, specifically outlining the quality requirements that the deliverable must meet to reach a certain level in that dimension. The description of achievement conditions combines qualitative and quantitative descriptions. Qualitative descriptions specify the characteristics the deliverable should possess or the processing to be completed, while quantitative descriptions provide specific quantitative thresholds or proportion requirements where applicable.

[0089] The domain evaluation rules also include a defined aggregation logic for the overall score. This aggregation logic specifies how the scores from each dimension are summed into a comprehensive score. Common aggregation methods include the arithmetic mean of the scores, weighted summation, and taking the lowest score. The weighting scheme for the weighted summation is also explicitly defined in the domain evaluation rules.

[0090] The first evaluation result is structured feedback data generated by the computer equipment after executing step S3. In addition to a comprehensive score, the first evaluation result fully records the sub-scores for each evaluation dimension and the corresponding textual feedback. This design allows the first evaluation result to not only answer the question of whether the deliverable is "good or bad overall," but also to precisely pinpoint "in which aspects the deliverable is good, in which aspects it is bad, and why it is bad."

[0091] The sub-scores are numerical values ​​given by the evaluation model in step S3 after evaluating each evaluation dimension according to the domain evaluation rules. The numerical range of the sub-scores is strictly limited to the full score range defined by the scoring criteria for that dimension. The sub-scores for each dimension are stored in key-value pairs, where the key is the dimension name and the value is the score value.

[0092] Text feedback consists of natural language comments generated by the evaluation model for each evaluation dimension. Each piece of text feedback corresponds strictly to the sub-score for that dimension, forming a matching relationship between score and explanation. The text feedback comprises three parts: the first part is an objective description of the deliverable's performance in that dimension, indicating what processing was completed and what content was presented; the second part is an explanation of the scoring basis, clarifying why the sub-score for that dimension corresponds to a certain level in the scoring criteria; the third part appears when the sub-score is below the maximum score, indicating the deliverable's shortcomings in that dimension and suggesting directions for improvement. When the sub-score reaches the maximum score, the third part can be omitted or replaced with a positive statement.

[0093] All data from the first evaluation results are stored in a unified structured format. The sub-scores and textual feedback for each evaluation dimension are established with the dimension name as the primary key and correspond to the definition of each dimension in the domain evaluation rules, so that the evaluation results can be used to quickly locate the comments and scores of any dimension.

[0094] In one possible implementation, the task data is a structured data set acquired by the computer device in step S1, which describes all attributes of the target task. In addition to the core field of domain category identifier, this set also includes three fields: occupation category identifier, task description information, and reference document identifier.

[0095] Occupational category identifiers are used to refer to the professional role simulated or served by the target task. Unlike domain category identifiers, which define task attributes from an industry perspective, occupational category identifiers define task attributes from a job function perspective. The value of an occupational category identifier can be an entry from a pre-defined list of occupational names, such as data analyst, risk control analyst, legal counsel, product manager, software engineer, etc. There is a relationship between occupational category identifiers and domain category identifiers, but they are not one-to-one. The same occupational category can serve multiple industry domains, and the same industry domain can contain multiple occupational categories. When a computer device acquires task data for a target task, the domain category identifier and occupational category identifier provide reference information for subsequent evaluation rule matching and capability scheduling from the industry and function dimensions, respectively.

[0096] The task description information is a natural language text description of the target task. This text description details the specific content that the target task needs to be completed, the background context of the task, the expected form and format requirements of the task output, and the special constraints that need to be followed during the task completion process. The task description information is the main input for the large language model agent when performing the first task processing in step S2. The large language model agent plans the execution steps, calls the necessary tools, and generates the first deliverable by reading and understanding the task description information.

[0097] Reference file identifiers are used to indicate the storage location or unique identifier of one or more reference files associated with the target task. Reference files are source data files that need to be read or processed during task processing; their format can be structured, semi-structured, or unstructured. Reference file identifiers are stored in the task data as a list; when a task involves multiple reference files, this list contains the identifiers of all relevant files. In step S2, the computer device locates and loads the corresponding reference file from the file storage system based on the reference file identifier, and provides its contents to the large language model agent for analysis and processing.

[0098] The logic for determining the domain category identifier is executed by the computer device during the task data receiving phase. When an external system or user submits a target task to the computer device, the value of the domain category identifier field can be directly provided in the task data. When the task data does not directly provide the domain category identifier, but provides an industry category field or an occupation category field, the computer device derives the domain category identifier from the industry category or occupation category according to preset mapping rules.

[0099] When both industry and occupation categories are provided in the task data, the computer device's determination strategy falls into two scenarios. The first scenario is that a pre-built evaluation rule base contains domain evaluation rules directly corresponding to the industry category. In this case, the computer device prioritizes the industry category as the primary value for the domain category identifier, while using the occupation category as an auxiliary identifier for more refined evaluation rule filtering or scheduling strategy selection. The second scenario is that a pre-built evaluation rule base does not contain domain evaluation rules directly corresponding to the industry category, but does contain domain evaluation rules directly corresponding to the occupation category. In this case, the computer device selects the occupation category as the domain category identifier. If the task data only provides an industry category and not an occupation category, or only provides an occupation category and not an industry category, the computer device directly uses the provided category field as the domain category identifier.

[0100] When the task data does not provide either an industry category or an occupation category, the computer device can analyze the semantic content of the task description information to extract implicit domain information to help determine the domain category identifier. Specifically, the computer device inputs the task description information into a domain classification model, which analyzes the keywords and contextual semantics in the task description and outputs the most likely domain category identifier. If the confidence level of the domain classification model's output is lower than a preset threshold, the computer device will use a system-preset default domain category identifier.

[0101] In one possible implementation, the preset triggering conditions include: the first score is lower than a preset score threshold, or the complexity of the target task meets a preset complexity strategy.

[0102] The preset trigger conditions are a set of decision conditions used by the computer device after generating the first evaluation result in step S3 to determine whether to initiate the skill self-evolution closed loop consisting of subsequent steps S4 to S7. These trigger conditions are determined by the computer device before step S4 is initiated. If the determination result is satisfactory, the candidate skill generation, secondary verification, and persistent writing processes of steps S4 to S7 are activated; if the determination result is unsatisfactory, the self-evolution process is skipped, and the computer device directly outputs the first deliverable as the final result and terminates the processing of the target task.

[0103] The preset trigger condition contains two parallel sub-conditions. The computer device uses a logical OR operation to determine the condition, meaning that if either of the two sub-conditions is true, the overall trigger condition is considered to be satisfied.

[0104] The first sub-condition is that the first score is lower than a preset score threshold. The computer device extracts the first score from the first evaluation result generated in step S3 and compares it with the preset score threshold in the system. The preset score threshold is set according to the system's minimum acceptable level of deliverable quality, and is usually set as an integer or floating-point number between 60% and 70% of the full score. When the first score is lower than the preset score threshold, the computer device determines that the current large language model agent's initial task processing has failed to meet the quality baseline, and its processing ability needs to be improved through a skill self-evolution closed loop. When the first score is greater than or equal to the preset score threshold, this sub-condition is not met.

[0105] The second sub-condition is that the complexity of the target task meets the preset complexity strategy. The preset complexity strategy is a set of rules used to evaluate task complexity, which may include quantitative evaluation methods for factors such as task description length, number of sub-steps, number of domains involved, and input file complexity. After acquiring task data in step S1, the computer device can synchronously or asynchronously evaluate the complexity of the task description information. The complexity evaluation is performed by a complexity evaluation module within the computer device. This module quantitatively analyzes factors such as the text length of the task description information, the number of sub-steps included in the task, the breadth of the domains involved in the task, and the diversity of the required output forms, outputting a complexity level or complexity score. The preset complexity strategy defines the rules for determining whether a task is considered "complex," such as a complexity score exceeding a preset complexity threshold, a complexity level reaching a specified level, or the task description containing a specific combination of keywords. When the complexity of the target task meets the judgment rules of the preset complexity strategy, the computer device considers the task to have sufficient complexity and representativeness, and the candidate skills corresponding to the shortcomings exposed in the task have high reuse value, making it worthwhile to invest computing resources in skill self-evolution. This sub-condition is not met when the complexity does not meet the strategy requirements.

[0106] In one example, the complexity calculation is performed using a multi-dimensional weighted summation method. There are a total of 4 dimensions involved in the calculation. Each dimension is scored independently, multiplied by a preset weight, and then summed to obtain the final complexity score, with a maximum score of 100.

[0107] The first dimension is the step complexity of the task description, with a weight of 35 points. The computer device performs semantic parsing on the task description information to extract the number of task steps it contains. The extraction method involves identifying keywords representing action sequences in the task description information, such as sequence connectors like "first," "then," "finally," "next," "simultaneously," and "on this basis," as well as structured markers like parallel verbs and step numbers. Each identified independent step description node is counted as one step point. A score of 1 to 2 steps earns 10 points in this dimension, 3 to 4 earns 20 points, 5 to 6 earns 28 points, and 7 or more earns 35 points.

[0108] The second dimension is the cross-domain involvement of the task, with a weight of 25 points. The computer device performs domain entity recognition on the text content of the task description information, counting the number of domain-related terms and the number of cross-domain categories. Domain-related terms are derived from a pre-built domain terminology database in the computer device, which maintains term sets separately for industry and occupation categories. When a term in the task description information simultaneously covers two or more industry or occupation categories, 8 points are added to this dimension for each additional category covered, up to a maximum of 25 points. When a term covers only a single category, this dimension receives 5 points.

[0109] The third dimension is the processing complexity of the reference files, with a weight of 25 points. The computer device reads the number of files in the reference file identifier list from the task data and determines the format type of the reference files based on the file extension or file header information. This dimension scores 0 points when the number of reference files is 0. It scores 5 points when the number of files is 1 and the format is a single type. It scores 10 points when the number of files is 1 but the format is a nested complex format. It scores 18 points when the number of files is 2 to 3 and involves 2 or more format types. It scores 25 points when the number of files is 4 or more or involves 3 or more format types.

[0110] The fourth dimension is the constraint complexity of the output requirements, with a weight of 15 points. The computer equipment identifies constraints in the task description information, extracting explicit constraints such as format requirements, word count requirements, chapter structure requirements, data precision requirements, and time range requirements. Each independent constraint is counted as 1 constraint point. A score of 0 to 1 constraint point earns 0 points, 2 to 3 earns 5 points, 4 to 5 earns 10 points, and 6 or more earns 15 points.

[0111] The scores for each of the four dimensions are multiplied by their respective weights and then summed to obtain the final complexity score, which ranges from 0 to 100. If the complexity score is greater than or equal to a preset complexity threshold of 50, the target task is considered a complex task.

[0112] The two sub-conditions complement each other: the first sub-condition triggers self-evolution from the perspective of task completion quality, ensuring that the capability shortcomings exposed by low-quality deliverables are captured and repaired in a timely manner; the second sub-condition triggers self-evolution from the perspective of the characteristics of the task itself, ensuring that the processing experience accumulated in high-value complex tasks is actively preserved, and even if the initial deliverable of such a task is rated reasonably, the skills formed in the process may still have a significant benefit on subsequent similar tasks.

[0113] In one possible implementation, S2, loading candidate skills through a large language model agent includes: S21. Obtain the content of candidate skills from the temporary candidate skill library through the preset tool call interface.

[0114] The temporary candidate skill library is a temporary storage area created by the computer device after candidate skills are generated in step S4. It is used to temporarily store relevant data of candidate skills throughout the entire lifecycle of the secondary task processing. The temporary candidate skill library and the persistent skill library are physically independent of each other, and their data lifecycles are different. Data in the persistent skill library is retained after the system restarts, while data in the temporary candidate skill library is either migrated to the persistent skill library or cleaned up and discarded based on the verification results after the secondary task processing is completed.

[0115] In step S4, the computer device writes the generated candidate skills into a temporary candidate skill repository in the form of structured data objects. Each candidate skill data object includes a skill name field, a skill content text field, a skill applicable scenario description field, and a skill generation timestamp field. The temporary candidate skill repository uses a key-value pair storage structure, with the unique identifier of the candidate skill as the key and the complete data object of the candidate skill as the value.

[0116] The pre-defined tool invocation interface is a set of standardized application programming interfaces exposed by the computer device. This interface follows a unified request-response protocol and serves as the sole channel for data interaction between the skill scheduling module and the external storage module within the computer device. The tool invocation interface includes multiple operation types, with the skill reading operation related to step S21 being the skill reading operation. The skill reading operation receives a skill identifier as input. The computer device's skill scheduling module initiates a read request to the temporary candidate skill library through this interface. The temporary candidate skill library searches for the skill identifier using its internal index. Upon successful search, the temporary candidate skill library serializes the corresponding candidate skill data object into a pre-defined data transmission format and returns it to the skill scheduling module. If the search fails, the temporary candidate skill library returns a null value or an exception flag. The computer device logs the exception and terminates the candidate skill loading process for the current secondary task.

[0117] Step S21 is executed at the beginning of step S5, before the large language model agent begins secondary task processing. The computer device's skill scheduling module iterates through the candidate skill identifier list returned by step S4, sequentially reading the complete content of each candidate skill from the temporary candidate skill library via a pre-defined tool call interface. After reading, all candidate skill data objects are cached in the computer device's memory for use in step S22.

[0118] For example, in the risk control report writing task, step S4 addresses the defect of "unidentified missing data fields" exposed in the initial task processing. It generates a candidate skill named "Structured Data Missing Field Check and Annotation," with the skill content stating, "After reading the structured data file, count the number of missing records and the missing percentage for each field column by column. Fields with a missing percentage exceeding 15% will be annotated in tabular form in subsequent reports." The skill's applicable scenario is described as "Suitable for tasks requiring integrity checks of structured data and the generation of quantitative reports." The computer assigns a unique identifier "cand_skill_001" to this candidate skill and writes it into the temporary candidate skill library.

[0119] Before initiating the secondary task processing in step S5, the computer device's skill scheduling module calls the skill reading operation in the preset tool invocation interface, passing in the input parameter "cand_skill_001". The tool invocation interface sends a read request to the temporary candidate skill library. The temporary candidate skill library locates the identifier in its internal index and returns a complete data object containing the skill name, skill content text, and applicable scenario description to the skill scheduling module. The skill scheduling module caches this data object in memory, completing the execution of step S21.

[0120] S22. Load the acquired candidate skills into the context of the large language model agent so that the large language model agent can apply the candidate skills in the secondary task processing.

[0121] Context is the collection of all information that a large language model agent can refer to when performing task processing. Context typically includes multiple components such as system instructions, task description information, historical dialogue records, tool call records, and skill descriptions. Each time a computer device invokes a large language model agent, it needs to assemble all the context information required for this invocation according to a predetermined format to form a complete set of context input data.

[0122] The core operation of step S22 is to embed the candidate skill content obtained from the temporary candidate skill library in step S21 into the context input data of the large language model agent in a specific format. The embedding method is to add a new skill description block in the context. This block is distinguished from other context content by clear boundary markers. The boundary markers are used to inform the large language model agent that the skill described in this block is one that it can call upon in this task processing.

[0123] The internal format of the skill description block is defined by a pre-set skill context template on the computer device. This template specifies the presentation order and style of the three fields: skill name, skill content text, and skill applicable scenario. The computer device formats and renders the candidate skill data objects cached in step S21 according to the template, and appends the rendered text to the end of the context input data or to a specified position related to the skill.

[0124] After embedding the candidate skill content, the computer device submits the assembled contextual input data to the large language model agent, along with the original task description information and reference file content. Upon receiving the contextual input data, the large language model agent parses the skill name and content text in the skill description block to understand the operational logic of the applicable skills. During the execution of secondary task processing, when the task flow progresses to the stage where the candidate skill is applicable, it actively invokes that skill to complete the corresponding processing. The existence of the candidate skill in the context does not force the large language model agent to invoke it; rather, it provides the agent with an optional enhancement capability.

[0125] For example, following the previous example, the computer device's skill scheduling module holds a complete data object for the candidate skill "Structured Data Missing Field Check and Annotation" in the cache. The computer device formats and renders this data object according to a preset skill context template. The rendered skill description block text is: "Available Skill Description: Skill Name [Structured Data Missing Field Check and Annotation], Skill Content [After the step of reading the structured data file is completed, the number of missing records and the missing percentage for each field are counted column by column. Fields with a missing percentage exceeding 15% are annotated in a table format in subsequent reports], Applicable Scenarios [Suitable for tasks that require integrity checks on structured data and the generation of quantitative reports]."

[0126] The computer device appends the skill description block to the context input data of the large language model agent, assembling it together with the system instruction "You are a risk control analyst, please complete the risk control report writing based on the following task description and available skills," as well as the task description information and transaction log file content, to form a complete context. Upon receiving this context, during the report generation process, when it identifies the need to read a CSV file and perform data quality checks, it retrieves the available skill "Structured Data Missing Field Check and Annotation" from the context. It then performs column-by-column statistical analysis and annotation of the missing fields according to the operation steps described in the skill description, applying the skill to the secondary task processing.

[0127] Please see Figure 3 As shown, in one embodiment, an optimization apparatus for a large language model agent is provided, the apparatus comprising: The acquisition module 301 is used to acquire task data of the target task, wherein the task data includes a domain category identifier; Processing module 302 is used to perform initial task processing on the target task through a large language model agent and generate a first deliverable; The evaluation module 303 is used to evaluate the first deliverable by calling preset domain evaluation rules according to the domain category identifier, and obtain a first evaluation result, wherein the first evaluation result includes a first score; The generation module 304 is used to, when the first evaluation result meets the preset triggering conditions, perform attribution analysis on the text feedback and execution trajectory in the first evaluation result through the large language model agent to locate the defective link in the initial task processing that caused the score of the first evaluation result to be lower than expected; call the preset knowledge search tool to obtain external reference knowledge based on the feature information of the defective link; input the external reference knowledge, the text feedback in the first evaluation result and the execution trajectory into the large language model agent to guide it to generate a coping strategy for the defective link and encapsulate it into candidate skills; The processing module 302 is further configured to load the candidate skills through the large language model agent, perform secondary task processing on the target task, and generate a second deliverable; The evaluation module 303 is further configured to evaluate the second deliverable and obtain a second evaluation result, the second evaluation result including a second score; The writing module 305 is used to write the candidate skill into the persistent skill library when it is confirmed that the candidate skill is invoked in the secondary task processing and the second score is higher than the first score.

[0128] For further details regarding the implementation of the above technical solution by each module in the optimization device for the large language model agent, please refer to the description in the optimization method for the large language model agent provided in the above application embodiments, which will not be repeated here.

[0129] It should be noted that the device 3 provided in the above embodiments, when executing the optimization method of the large language model agent, is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the above functions. In addition, the optimization device for the large language model agent provided in the above embodiments and the optimization method embodiment for the large language model agent belong to the same concept, and its implementation process is detailed in the method embodiment, which will not be repeated here.

[0130] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0131] See Figure 4 The diagram shown is a schematic of a computer storage medium provided in an embodiment of this application. The computer storage medium can store multiple instructions (i.e., ... Figure 4 The computer program shown above), the instructions are adapted to be loaded and executed as described above. Figure 2 The method steps of the illustrated embodiment can be found in the following documentation for detailed execution. Figure 2 The specific details of the illustrated embodiments will not be elaborated here.

[0132] This application also provides a computer program product that stores at least one instruction, which is loaded and executed by the method for optimizing a large language model agent as described in the above embodiments.

[0133] Please see Figure 5 This document provides a schematic diagram of the structure of a computer device according to an embodiment of this application. Figure 5 As shown, the computer device 500 may include: at least one processor 501, at least one network interface 504, user interface 503, memory 505, and at least one communication bus 502.

[0134] The communication bus 502 is used to enable communication between these components.

[0135] The user interface 503 may include input units such as a mouse and keyboard.

[0136] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0137] The processor 501 may include one or more processing cores. The processor 501 connects to various parts within the computer device 500 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 505, and by calling data stored in memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 501 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 501 and may be implemented as a separate chip.

[0138] The memory 505 may include random access memory (RAM) or read-only memory. Optionally, the memory 505 may include a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 505 may also be at least one storage device located remotely from the aforementioned processor 501. Figure 5 As shown, the memory 505, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and application programs.

[0139] exist Figure 5In the computer device 500 shown, the user interface 503 is mainly used to provide an input interface for the user and to obtain the user's input data; while the processor 501 can be used to call the application program stored in the memory 505 and specifically execute, such as Figure 2 The method shown can be referred to for details. Figure 2 As shown, it will not be elaborated further here.

[0140] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a disk, optical disk, read-only memory, or random access memory, etc.

[0141] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. An optimization method for a large language model intelligent agent, characterized in that, include: Obtain task data for the target task, wherein the task data includes a domain category identifier; The target task is processed initially by a large language model agent to generate the first deliverable. Based on the domain category identifier, a preset domain evaluation rule is invoked to evaluate the first deliverable and obtain a first evaluation result, which includes a first score. When the first evaluation result meets the preset triggering conditions, the large language model agent performs attribution analysis on the text feedback and execution trajectory in the first evaluation result to locate the defective link in the first task processing that caused the score of the first evaluation result to be lower than expected. The system invokes a preset knowledge search tool, uses the feature information of the defective link as the query basis, and obtains external reference knowledge. The external reference knowledge, the text feedback in the first evaluation result, and the execution trajectory are input into the large language model agent to guide it in generating a coping strategy for the defective link and encapsulating it into candidate skills. The candidate skills are loaded by the large language model agent, and secondary task processing is performed on the target task to generate a second deliverable. The second deliverable is evaluated to obtain a second evaluation result, which includes a second score; If it is confirmed that the candidate skill is invoked in the secondary task processing and the second score is higher than the first score, the candidate skill is written into the persistent skill library.

2. The method according to claim 1, characterized in that, The step of writing the candidate skill into the persistent skill library when it is confirmed that the candidate skill is invoked in the secondary task processing and the second score is higher than the first score includes: Verify whether the execution record of the secondary task processing contains a call record for the candidate skill; Verify whether the difference between the second score and the first score reaches a preset improvement threshold; When the call record exists and the difference reaches the preset improvement threshold, the candidate skill is written into the persistent skill library.

3. The method according to claim 1, characterized in that, Before writing the candidate skills into the persistent skill library, the method further includes: The candidate skills are merged with existing skills in the persistent skill library, deduplicated, and their names are normalized.

4. The method according to claim 1, characterized in that, The domain evaluation rules include structured evaluation dimensions corresponding to the domain category identifier and scoring criteria for each evaluation dimension; The first evaluation result also includes sub-scores and textual feedback for each evaluation dimension.

5. The method according to claim 1, characterized in that, The task data also includes occupational category identifiers, task description information, and reference document identifiers; The domain category identifier is determined based on the industry category or occupation category in the task data.

6. The method according to claim 1, characterized in that, The preset triggering conditions include: the first score is lower than a preset score threshold, or the complexity of the target task meets a preset complexity strategy.

7. The method according to claim 1, characterized in that, The loading of the candidate skills through the large language model agent includes: The content of the candidate skills is obtained from a temporary candidate skill library through a preset tool call interface; The acquired candidate skills are loaded into the context of the large language model agent so that the large language model agent can apply the candidate skills in the secondary task processing.

8. An optimization device for a large language model intelligent agent, characterized in that, include: The acquisition module is used to acquire task data of the target task, wherein the task data includes a domain category identifier; The processing module is used to perform initial task processing on the target task through a large language model agent and generate the first deliverable. The evaluation module is used to evaluate the first deliverable by invoking preset domain evaluation rules according to the domain category identifier, and obtain a first evaluation result, wherein the first evaluation result includes a first score; The generation module is used to perform attribution analysis on the text feedback and execution trajectory in the first evaluation result through the large language model agent when the first evaluation result meets the preset triggering conditions, and to locate the defective link in the first task processing that causes the score of the first evaluation result to be lower than expected. The system invokes a preset knowledge search tool, uses the feature information of the defective link as the query basis, and obtains external reference knowledge. The external reference knowledge, the text feedback in the first evaluation result, and the execution trajectory are input into the large language model agent to guide it in generating a coping strategy for the defective link and encapsulating it into candidate skills. The processing module is further configured to load the candidate skills through the large language model agent, perform secondary task processing on the target task, and generate a second deliverable; The evaluation module is also used to evaluate the second deliverable and obtain a second evaluation result, which includes a second score. The writing module is used to write the candidate skill into the persistent skill library when it is confirmed that the candidate skill is invoked in the secondary task processing and the second score is higher than the first score.

9. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the optimization method for a large language model agent as described in any one of claims 1 to 7.

Images