Multi-dimensional evaluation-based agent closed-loop optimization method, device and program product
Patent Information
- Application Number
- CN202611004322.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-07
- Publication Date
- 2026-10-23
AI Technical Summary
[0004]在模型训练维度，基于反馈的强化学习方法（如直接偏好优化、群体相对策略优化等）虽能利用奖励模型或大模型评分优化策略，但训练所用的奖励口径与上游评估体系相割裂，改进仅停留在模型层，无法作用于编排层，也不具备问题自动归因能力
[0044]本公开的积极进步效果在于：本公开的基于多维度评估的智能体优化方法、装置及程序产品，针对一目标模型、一目标编排框架（harness）与一目标基准测试（benchmark），以及多个参考编排框架，均在该目标benchmark上运行，采集统一轨迹；构成目标测试结果和参考测试结果，以多维度评分细则对轨迹打分、定位偏弱维度；进行归因和改进参数的确定。能够使评估结果不止于报告，而在同一度量空间内直接驱动诊断、改进与验收；自动判定智能体弱点归属于编排层还是模型层，避免人工判定；以最低成本路径优先改进，优先修复编排层，仅在必要时训练模型，防止改进过程污染评估基准，使所测提升为真泛化；另外还使每轮改进可度量、可复现和可追溯。
Smart Images
Figure CN122965557A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of software engineering, and in particular to methods, apparatus and program products for optimizing intelligent agents. Background Technology
[0002] An intelligent agent typically includes a model and an orchestration framework (harness). A benchmark is a standardized test set and scoring system used to evaluate the capabilities of AI.
[0003] Existing optimization methods for intelligent agents mainly follow four relatively independent paths: evaluation, model training, orchestration layer optimization, and self-improvement closed loop, which are not yet interconnected. In the evaluation dimension, current tests of intelligent agents on a set of tasks use task pass rate or result score as the main output indicators, with some extensions supporting procedural dimensions (such as number of steps, number of tool calls, etc.). Judges often use rule-based procedures or large model scoring (LLM-as-judge), but evaluation is only used as an endpoint judgment. Multidimensional results do not participate in the construction of downstream improvement signals, nor do they distinguish the attribution of problems.
[0004] In terms of model training, although feedback-based reinforcement learning methods (such as direct preference optimization, group relative policy optimization, etc.) can optimize strategies by using reward models or large model scoring, the reward caliber used in training is disconnected from the upstream evaluation system. The improvement is limited to the model layer and cannot be applied to the orchestration layer, nor does it have the ability to automatically attribute problems.
[0005] In terms of orchestration layer optimization, methods such as automatic suggestion word search, tool configuration search, and pipeline structure modification often take a single indicator as the optimization target, lack a multi-dimensional process profile, and are not linked to the model training path.
[0006] In terms of self-improvement loop, while multi-role collaboration or self-synthesized tasks / self-generated rewards can drive model iteration, the loop is entirely limited to the model side, does not involve orchestration layer repair, and cannot achieve attribution across orchestration frameworks. The above four directions each solve local problems, but there is a lack of a unified data flow and feedback mechanism between evaluation, training, optimization, and attribution, and a systematic improvement solution that can connect the entire chain has not yet been formed. Summary of the Invention
[0007] The technical problem to be solved by this disclosure is to overcome the defects in the prior art and provide a method, device and program product for intelligent agent closed-loop optimization based on multi-dimensional evaluation that can realize self-diagnosis and closed-loop optimization.
[0008] This disclosure solves the above-mentioned technical problems through the following technical solution:
[0009] A closed-loop optimization method for intelligent agents, the method comprising:
[0010] The first pass rate of the target model placed in the target orchestration framework in the target benchmark test is obtained, and the second pass rate of the target model placed in the reference orchestration framework in the target benchmark test is obtained, wherein the reference orchestration framework includes multiple frameworks.
[0011] Test attribution is determined based on the dispersion of the first and second pass rates and the highest pass rate.
[0012] Optionally, determining the test attribution based on the dispersion of the first pass rate and the second pass rate, as well as the highest pass rate, specifically includes:
[0013] Based on the fact that the highest pass rate is lower than the first preset threshold, the test attribution is determined to be the model layer;
[0014] If the highest pass rate is higher than the first preset threshold and the dispersion is higher than the second preset threshold, then the test attribution is determined to be the orchestration layer.
[0015] If the highest pass rate is higher than the first preset threshold and the dispersion is lower than the second preset threshold, then it is determined that no attribution test is required.
[0016] Optionally, the method further includes:
[0017] Obtain the target test results of the target model placed in the target orchestration framework in the target benchmark test, and the reference test results of the target model placed in the reference orchestration framework in the target benchmark test;
[0018] Based on the test attribution, the orchestration layer generates improvement parameters to improve the target orchestration framework according to the target test results and the reference test results; or,
[0019] Based on the test attribution model layer, the model is retrained according to the target test result and the reference test result.
[0020] Optionally, the target test results of the target model placed in the target orchestration framework in the target benchmark test are obtained; and the reference test results of the target model placed in the reference orchestration framework in the target benchmark test are obtained; specifically including,
[0021] The target model is collected and placed into the target orchestration framework to generate the target trajectory in the target benchmark test, thus generating the target test results; and...
[0022] The reference test results are generated by collecting the target model, placing it into the reference orchestration framework, and using the reference trajectory in the target benchmark test.
[0023] Optionally, the target test result includes multiple first scores corresponding to multiple dimensions, and the reference test result includes multiple second scores corresponding to multiple dimensions;
[0024] The orchestration layer, based on the test attribution, generates improvement parameters to improve the target orchestration framework according to the target test results and the reference test results; specifically, it includes:
[0025] Weak dimensions are determined based on the first and second scores corresponding to multiple dimensions.
[0026] The orchestration configuration is determined based on the first and second scores corresponding to the weaker dimensions.
[0027] The target orchestration framework is improved based on the orchestration configuration.
[0028] Optionally, the target test result includes multiple first scores corresponding to multiple dimensions, and the reference test result includes multiple second scores corresponding to multiple dimensions;
[0029] The step of retraining the model based on the test attribution model layer, according to the target test result and the reference test result, specifically includes:
[0030] Weakness dimensions are determined based on the first and second scores corresponding to multiple dimensions;
[0031] Training data is selected based on the aforementioned weakness dimensions;
[0032] The target model is trained based on the selected training data.
[0033] Optionally, the method further includes:
[0034] Retesting is performed on the target base test based on the improved target model and target orchestration framework;
[0035] or,
[0036] Retesting is performed on the target base test based on the target model and the improved target orchestration framework.
[0037] Another aspect of this disclosure provides an intelligent agent closed-loop optimization device, comprising:
[0038] The pass rate acquisition module is used to acquire the first pass rate of the target model placed in the target orchestration framework in the target benchmark test, and the second pass rate of the target model placed in the reference orchestration framework in the target benchmark test, wherein the reference orchestration framework includes multiple frameworks.
[0039] The attribution module is used to determine test attribution based on the dispersion of the first pass rate and the second pass rate, as well as the highest pass rate.
[0040] In another aspect of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and for running on the processor, wherein the processor executes the computer program to implement the intelligent agent closed-loop optimization method described in any one of the preceding descriptions.
[0041] In another aspect, this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the agent closed-loop optimization method described above.
[0042] In another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the agent closed-loop optimization method as described above.
[0043] Based on common knowledge in the field, the above-mentioned preferred conditions can be combined arbitrarily to obtain various preferred embodiments of this disclosure.
[0044] The positive advancements of this disclosure are as follows: The agent optimization method, apparatus, and program products based on multi-dimensional evaluation disclosed herein operate on a target benchmark, collecting a unified trajectory for a target model, a target harness, a target benchmark, and multiple reference harnesses; constructing target test results and reference test results; scoring the trajectory and identifying weak dimensions using multi-dimensional scoring rules; and determining attribution and improvement parameters. This enables the evaluation results to go beyond mere reporting, directly driving diagnosis, improvement, and acceptance within the same metric space; automatically determining whether agent weaknesses belong to the harness layer or the model layer, avoiding manual judgment; prioritizing improvements along the lowest-cost path, prioritizing the repair of the harness layer, and training the model only when necessary, preventing the improvement process from contaminating the evaluation benchmark and ensuring that the measured improvement is true generalization; furthermore, it makes each round of improvement measurable, reproducible, and traceable. Attached Figure Description
[0045] Figure 1 A flowchart of an agent closed-loop optimization method based on multi-dimensional evaluation provided in Embodiment 1 of this disclosure;
[0046] Figure 2 A flowchart of an agent closed-loop optimization method based on multi-dimensional evaluation, which is another embodiment of Embodiment 1 of this disclosure;
[0047] Figure 3 A schematic diagram of the intelligent agent closed-loop optimization device based on multi-dimensional evaluation provided in Embodiment 2 of this disclosure;
[0048] Figure 4 This is a schematic diagram of an electronic device provided in Embodiment 3 of this disclosure. Detailed Implementation
[0049] The present disclosure is further illustrated below by way of embodiments, but the present disclosure is not limited to the scope of the embodiments described herein.
[0050] The prefixes such as "first" and "second" used in this disclosure are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. The use of ordinal numbers and other prefixes used to distinguish descriptive objects in this disclosure does not constitute a limitation on the described objects. The description of the described objects is given in the context of the embodiments, and the use of such prefixes should not constitute unnecessary restrictions. Furthermore, in the description of this embodiment, unless otherwise stated, "multiple" means two or more.
[0051] Existing intelligent agent performance evaluation and improvement are disconnected. Evaluation outputs remain at the report level, requiring human interpretation, diagnosis, and design of improvement actions. Furthermore, improvement signals and evaluation criteria are inconsistent, training reward / preference data are designed separately, and training objectives and evaluation metrics are difficult to corroborate. Moreover, there is no automatic attribution of problems; when performance is poor, it is difficult to distinguish between insufficient model capability and improper orchestration framework configuration. Additionally, improvement paths are singular and lack cost awareness, either focusing solely on orchestration adjustments or model training, lacking automatic judgment and switching between the two, and lacking a mechanism for prioritizing low-cost paths. This disclosure and prior art... The essential difference in evaluation methods lies in the following: the same set of multi-dimensional scoring rules serves as the evaluation criteria, diagnostic basis, training data screening basis, and acceptance criteria, while the reward for reinforcement learning adopts the objective signal of task pass rate consistent with the acceptance; the difference in performance of the same model under multiple orchestration frameworks is used to automatically determine the problem's affiliation, belonging to the orchestration layer or the model layer, and a cost-aware dual path is implemented, prioritizing the repair of untrained orchestrations and upgrading the model for reinforcement learning when necessary; at the same time, training tasks that do not intersect with the target benchmark are used to ensure that the improvement is generalizable, reproducible, and does not pollute the benchmark.
[0052] Example 1
[0053] Figure 1 This is a flowchart of an agent optimization method provided in Embodiment 1 of this disclosure. The optimization method includes:
[0054] S10, obtain the first pass rate of the target model placed in the target orchestration framework in the target benchmark test, and the second pass rate of the target model placed in the reference orchestration framework in the target benchmark test, wherein the reference orchestration framework includes multiple frameworks;
[0055] S20, determine test attribution based on the dispersion of the first pass rate and the second pass rate and the highest pass rate.
[0056] Specifically, determining the test attribution based on the dispersion of the first pass rate and the second pass rate and the highest pass rate includes:
[0057] S21, based on the fact that the highest pass rate is lower than the first preset threshold, determine the test attribution to the model layer;
[0058] S22, based on the fact that the highest pass rate is higher than the first preset threshold and the dispersion is higher than the second preset threshold, the test attribution is determined to be the orchestration layer;
[0059] S23, based on the fact that the highest pass rate is higher than the first preset threshold and the dispersion is lower than the second preset threshold, it is determined that no attribution test is required.
[0060] Dispersion is typically expressed as standard deviation or variance. If the highest pass rate is below a first preset threshold, it indicates a problem with the model itself, regardless of the dispersion level; no harness can achieve a high pass rate for this model. Conversely, if the highest pass rate is high (above the first preset threshold) and the dispersion is high (above the second preset threshold), it indicates that the model itself is not problematic; the high dispersion is due to issues with the orchestration layer, which is attributed to the target orchestration framework. Furthermore, if the highest pass rate is above the first preset threshold (high pass rate) and the dispersion is low, it indicates that both the model and the harness are functioning correctly, and attribution testing is unnecessary.
[0061] To facilitate calculation and judgment, the pass rate and dispersion mentioned above can be normalized. The first preset threshold and the second preset threshold can be determined as needed. For example, the first preset threshold is 50%, and the second preset threshold is preferably 3. These can be set as required.
[0062] For example, the target model is the model under test, which can be selected from various commonly used models, such as GLM-5.1, the target framework (harness) is goose, the reference frameworks are codeact, claude_code, opencode, mini-swe, and the target benchmark is SWE-bench Verified, which includes 500 real-world bug-fixing tasks.
[0063] GLM-5.1 is simultaneously set in both the target harness=goose and the reference harness={codeact, claude_code, opencode, mini-swe}. All five are run on the same batch of 500 problems on SWE-bench Verified to obtain the pass rate. This pass rate is typically determined by the objective test included with the target benchmark, without the need for judges. For example, the hidden official test included with SWE-bench Verified can be used to run the test once, recording 1 for all passes and 0 for the rest, without additional judging, ultimately yielding the overall pass rate. For instance, the target harness's first pass rate is 62.2%, while the second pass rates are: codeact 74.2%, claude_code 73.4%, opencode 74.2%, and mini-swe 75.8%.
[0064] The average of multiple pass rates is The dispersion is expressed by the standard deviation:
[0065] ;
[0066] in, This represents the first pass rate and the second pass rate, and n represents the quantity. In this embodiment, n=5.
[0067] Therefore, the standard deviation can be obtained as follows: ;
[0068] Furthermore, the highest pass rate is 74.1%, which is greater than 50%. Therefore, it can be determined that this embodiment should be attributed to the orchestration layer.
[0069] In this embodiment, the pass rate obtained by testing the target model and target orchestration framework, as well as the target model and reference orchestration framework, in the same benchmark automatically determines whether the problem is at the model layer or the orchestration layer. This avoids manual diagnosis and evaluation, and can also point the way for subsequent improvements. It can also avoid the problem that the model may cater to the judge when making judgments and rewards.
[0070] As another implementation method, such as Figure 2 As shown, the method further includes:
[0071] S30, obtain the target test results of the target model placed in the target orchestration framework in the target benchmark test, and the reference test results of the target model placed in the reference orchestration framework in the target benchmark test;
[0072] The target test result includes multiple first scores corresponding to multiple dimensions, and the reference test result includes multiple second scores corresponding to multiple dimensions.
[0073] Step S30, which involves obtaining the target test result of the target model placed in the target choreography framework in the target benchmark test and the reference test result of the target model placed in the reference choreography framework in the target benchmark test, specifically includes: collecting the target trajectory of the target model placed in the target choreography framework in the target benchmark test to generate the target test result; and collecting the reference trajectory of the target model placed in the reference choreography framework in the target benchmark test to generate the reference test result.
[0074] Specifically, the target model is placed into the target orchestration framework and tested in the target benchmark to obtain the target test trajectory. The target model is also placed into various reference orchestration frameworks and tested in the target benchmark to obtain the respective reference test trajectories. Specifically, a unified trajectory is intercepted and collected at the interface where the orchestration framework calls the model (model call layer). Each step of the trajectory includes input, model output, tool call and return, token consumption, error information, etc. For example, instead of parsing the individual logs of each harness, each request and return is recorded as a "request-return" pair at its unified "call model" interface, stored in raw_pairs.jsonl. Each step includes fields such as finish_reason, completion_tokens, content, and tool_calls, thus ensuring that the trajectories of different frameworks such as Goose and Mini-SWE are naturally consistent and directly comparable.
[0075] The trajectory and the corresponding scoring rules for each dimension are input into the evaluation model (LLM-as-judge), for example, using GPT-5.5 and keeping the wheels fixed to ensure consistency, in order to obtain the score for each dimension.
[0076] The dimensions include: format failure rate, read-before-modify rate, effective step rate, blind retries rate, post-repair verification, etc. For example, as shown in Table 1 below, the dimensions obtained include, but are not limited to, those listed in the table. The dimensions used in each evaluation do not necessarily have to include all dimensions; a subset of dimensions can be selected.
[0077] Table 1:
[0078]
[0079] For example, for the post-fix verification dimension, the input evaluation model GPT-5.5 is: trajectory file + prompt "After the last code modification, have you run any verification (test / reproduction script)? Run = 1, No run, submit directly = 0".
[0080] The actual trajectory was: ...Step 38: edit sympy / core / mul.py (last code modification) → Step 39: submit directly, without any run / test in between.
[0081] GPT-5.5 Output:
[0082] - points: 0
[0083] - Basis: "The last code modification was in step 38; the code was then committed in step 39. There was no test / reproduction script execution throughout the process → the changes were not verified, so the result was 0."
[0084] For example, regarding the blind retry rate, the input evaluation model GPT-5.5 is: trajectory file + prompt "After the tool reports an error, if the next step is to retry with almost the same parameters without reading the error or performing a diagnosis, count it as a blind retry; number of blind retry ÷ total number of retries after the error. The lower the better."
[0085] In the actual trajectory file: Step 8, run pytest tests / x.py → reported ModuleNotFoundError: no module 'foo'; Step 9, run pytest tests / x.py again as before (no error, foo not installed) → same error; Step 10, retry as before → same error; Step 14, after another run error, Step 15, first cat the error and then change the command (this time it was not blind).
[0086] The GPT-5.5 output is:
[0087] - points: 0.67;
[0088] - Basis: "After an error occurs, there are a total of 3 retry opportunities; steps 9 and 10 are retrying with the same parameters as before without diagnosis (blind), and step 15 reads the error message first and then adjusts (not blind). 2 / 3≈0.67."
[0089] Then, by combining the multi-dimensional outputs of each trajectory in all benchmark test questions, a first score and a second score corresponding to multiple dimensions are obtained.
[0090] For example, as shown in Table 2 below, the first three columns of the table list the target harness test results, including the first score in multiple dimensions.
[0091] Table 2:
[0092]
[0093] S40, based on the test attribution, the orchestration layer generates improvement parameters to improve the target orchestration framework according to the target test results and the reference test results;
[0094] S41, Weak dimensions are determined based on the first and second scores corresponding to multiple dimensions;
[0095] Specifically, for the first and second scores corresponding to each dimension, the average score of each dimension is calculated. The difference between the first score and the average score is then calculated. If the difference is higher than a third preset threshold, the dimension is determined to be a weak dimension. For example, as shown in the table above, the first score for each dimension is in the third column, and the difference between the first score and the average score of each dimension is in the fifth column. For example, if the third preset threshold is 5 or 3, then the weak dimension can be determined to be the format failure rate. The weak dimensions determined in this way may be one or more.
[0096] If all differences are less than the third preset threshold, then the dimension corresponding to the largest difference is determined to be the weaker dimension.
[0097] S42, determine the orchestration configuration based on the first and second scores corresponding to the weaker dimension;
[0098] Specifically, a repair agent, an orchestration agent with the ability to read and modify source code, such as ClaudeCode, is used to determine the orchestration configuration. For example, the aforementioned weak dimension is the format failure rate, and the specific improvement scheme can be determined jointly based on the evidence of the failure steps and the cross-framework configuration difference. Returning to the failure step evidence from the "format failure rate"—the final step has finish_reason=length, completion_tokens max_tokens=4096, content is empty, and no tool calls; then comparing the configuration with the optimal mini-swe for this dimension, it is found that goose has hardcoded 4096 and has not projected the existing model_limits.output=16384 in the model file. This configuration difference is the direction for repair. The repair agent, in this example, is ClaudeCode, which adjusts GOOSE_MAX_TOKENS from 4096 to 16384, and after optimal confirmation on the development set that does not intersect with the target benchmark, applies it to goose; the model parameters are not updated throughout the process. The second round works similarly, returning to the failed step from the "step budget" dimension (some trajectories hit the upper limit at step 50, and the last step stdout indicates "the maximum number of actions that do not require user intervention has been reached"), and the DEFAULT_AGENT_MAX_STEPS value is changed from 50 to 200 based on cross-frame differential.
[0099] S43, Improve the target orchestration framework based on the orchestration configuration.
[0100] The method also includes,
[0101] S50, based on the test attribution model layer, retrain the model according to the target test result and the reference test result; specifically including:
[0102] S51, determine the weakness dimensions based on the first and second scores corresponding to multiple dimensions; this determination step is the same as S41 above.
[0103] S52, Filter training data based on the aforementioned weakness dimension;
[0104] Specifically, a training set is first selected. The training set questions are taken from existing task sets that do not overlap with the target benchmark, such as real repository issue-repair tasks from SWE-Gym, OpenSWE, etc. The training set is deduplicated with SWE-bench Verified by instance ID (repository + PR / issue number) to ensure that the training set has zero overlap with the target benchmark. The target benchmark is only evaluated throughout the process and does not participate in training or selection.
[0105] First, assign capability labels to the candidate training set. This includes labels such as whether a question requires writing a self-reproduced script and running a test to determine correctness (corresponding to post-fix verification M57), whether it requires locating multiple files to correct (corresponding to the read-before-correction rate M07), and whether an error requires diagnosis before retrying (corresponding to the blind retry rate M10). These labels can be automatically determined by rules or evaluation agents with the same criteria as the evaluation, based on the question metadata (issue text, number of involved files, whether a test is included, etc.). For example, this can still be handled by the unified judge GPT-5.5, using the same model and criteria as the scoring. Then, based on the weaker dimensions, increase the proportion of questions with the corresponding labels in the training set—for example, if M57 and M07 (dimensional labels) are weak in this round, increase the proportion of questions requiring self-written test verification and those requiring cross-file location, while decreasing the proportion of other types. The ratio is adjustable, allowing the gradient to focus on the weaker capability families.
[0106] S53, Train the target model based on the selected training data.
[0107] A fixed training orchestration framework is used, with SWE-Agent as the training harness. For each training question, multiple trajectories are rolled out online to form a group. This group of rollouts is required for the relative advantage of the group and is a sampling of the reinforcement learning algorithm itself. The reward only uses the task pass rate. If the patch passes the official test for the question, it is recorded as 1, otherwise it is recorded as 0, thus saving the need for a separate reward model, and the reward is completely consistent with the acceptance criteria.
[0108] The truncated degradation trajectory, for example, if it hits "finish_reason==length and completion_tokens==max_tokens and content length==0 and tool call count==0", which is the same truncation criterion used in the first step of format recognition failure on goose, the patch will be empty and will inevitably fail the test, so it will naturally get zero reward and does not need to be manually removed.
[0109] Within the same question group, the relative advantage of the group is considered. The model was updated (group relative policy optimization of the valueless model), and the loss was only calculated for tokens generated by the model itself; the content returned by the tool was not included in the loss. After about 150 steps, the pass rate of Qwen3-4B (the target model) on SWE-agent and on the untrained SWE-bench Verified platform increased from about 12.6% to about 20.6%.
[0110] As another implementation, the method further includes:
[0111] S60, based on the improved target model and target orchestration framework, performs retesting in the target base test;
[0112] Alternatively, retesting can be performed in the target base test based on the target model and the improved target orchestration framework.
[0113] Put the modified model (model) and the target harness (improved target harness) back into the target benchmark (e.g., SWE-bench Verified) for retesting; require that the task pass rate (determined by official testing) does not decrease but increases, or that the format failure rate and other dimensions are not lower than the preset threshold, etc.
[0114] The test model version (GLM-5.1 / Qwen3-4B), harness version, and key configuration values, such as the versions of goose (GOOSE_MAX_TOKENS, DEFAULT_AGENT_MAX_STEPS, swe-agent, etc.), evaluation agent version (GPT-5.5, fixed across rounds to ensure consistency), benchmark and training set snapshots, random seeds, and various thresholds, were recorded round by round, which improved the pass rate from 62.2% to 72.0%. This optimization path can be replayed, compared round by round, and audited.
[0115] The agent closed-loop optimization method based on multi-dimensional evaluation in this embodiment uses the same set of multi-dimensional scoring rules to simultaneously serve as evaluation criteria, diagnostic basis, training data screening basis, and acceptance criteria. It automatically determines the problem's attribution (or model layer) based on the differences in the performance of the same model under multiple orchestration frameworks, and implements a cost-aware dual path of prioritizing untrained orchestration repair and upgrading the model for reinforcement learning when necessary. At the same time, the model training adopts training tasks that do not intersect with the target benchmark to ensure that the improvement is generalizable, reproducible, and does not pollute the benchmark.
[0116] This approach ensures that evaluation results go beyond mere reports, directly driving diagnosis, improvement, and acceptance within the same metric space; it automatically determines whether agent weaknesses belong to the orchestration layer or the model layer, avoiding manual judgment; it prioritizes improvements along the lowest-cost path, i.e., it prioritizes orchestration layer repairs and only upgrades the model when necessary; it prevents the improvement process from polluting the evaluation benchmark and from pandering to the judge, ensuring that the measured improvements are true generalizations; and it also makes each round of improvement measurable, reproducible, and traceable.
[0117] Example 2
[0118] Corresponding to the foregoing method embodiments, this disclosure also provides embodiments of an intelligent agent closed-loop optimization device based on multi-dimensional evaluation.
[0119] Figure 3 This is a schematic diagram of a closed-loop optimization device for intelligent agents based on multi-dimensional evaluation. The system includes:
[0120] The pass rate acquisition module 1 is used to acquire the first pass rate of the target model placed in the target orchestration framework in the target benchmark test, and the second pass rate of the target model placed in the reference orchestration framework in the target benchmark test, wherein the reference orchestration framework includes multiple frameworks.
[0121] Diagnostic module 2 is used to determine test attribution based on the dispersion of the first pass rate and the second pass rate, as well as the highest pass rate.
[0122] Specifically, the diagnostic module 2 determines the test attribution to the model layer based on the fact that the highest pass rate is lower than a first preset threshold;
[0123] If the highest pass rate is higher than the first preset threshold and the dispersion is higher than the second preset threshold, then the test attribution is determined to be the orchestration layer.
[0124] If the highest pass rate is higher than the first preset threshold and the dispersion is lower than the second preset threshold, then it is determined that no attribution test is required.
[0125] Dispersion is typically expressed as standard deviation or variance. If the highest pass rate is below a first preset threshold, it indicates a problem with the model itself, regardless of the dispersion level; no harness can achieve a high pass rate for this model. Conversely, if the highest pass rate is high (above the first preset threshold) and the dispersion is high (above the second preset threshold), it indicates that the model itself is not problematic; the high dispersion is due to issues with the orchestration layer, which should be attributed to the orchestration layer. Furthermore, if the highest pass rate is above the first preset threshold (high pass rate) and the dispersion is low, it indicates that both the model and the harness are functioning correctly, and attribution testing is unnecessary.
[0126] To facilitate calculation and judgment, the pass rate and dispersion mentioned above can be normalized. The first preset threshold and the second preset threshold can be determined as needed. For example, the first preset threshold is 50%, and the second preset threshold is preferably 3. These can be set as required.
[0127] In this embodiment, the pass rate obtained by testing the target model and target orchestration framework, as well as the target model and reference orchestration framework, in the same benchmark automatically determines whether the problem is at the model layer or the orchestration layer. This avoids manual diagnosis and evaluation, and can also point the way for subsequent improvements. It can also avoid the problem that the model may cater to the judge when making judgments and rewards.
[0128] In another embodiment, the device further includes:
[0129] Multi-dimensional evaluation module 3 obtains the target test results of the target model placed in the target orchestration framework in the target benchmark test, and the reference test results of the target model placed in the reference orchestration framework in the target benchmark test;
[0130] The target test result includes multiple first scores corresponding to multiple dimensions, and the reference test result includes multiple second scores corresponding to multiple dimensions.
[0131] The multi-dimensional evaluation module 3 further includes a trajectory acquisition module 31, which is used to collect the target trajectory of the target model placed in the target choreography framework in the target benchmark test to generate the target test result; and to collect the reference trajectory of the target model placed in the reference choreography framework in the target benchmark test to generate the reference test result.
[0132] Specifically, a unified trajectory is intercepted and collected at the interface where the orchestration framework calls the model (model call layer). Each step of the trajectory includes input, model output, tool call and return, token consumption, error information, etc. For example, instead of parsing the individual logs of each harness, each request and return is recorded as a "request-return" pair at its unified "model call" interface and stored in raw_pairs.jsonl. Each step includes fields such as finish_reason, completion_tokens, content, and tool_calls, so that the trajectories of different frameworks such as Goose and Mini-SWE are naturally consistent and can be directly compared.
[0133] The trajectory and the corresponding scoring rules for each dimension are input into the evaluation model (LLM-as-judge), for example, using GPT-5.5 and keeping the wheels fixed to ensure consistency, in order to obtain the score for each dimension.
[0134] The device further includes: an orchestration self-healing module 4, used to generate improvement parameters to improve the target orchestration framework based on the test attribution and the orchestration layer, according to the target test results and the reference test results;
[0135] Specifically, the orchestration self-healing module 4 includes:
[0136] The first diagnostic module 41 is used to determine the weaker dimension based on the first and second scores corresponding to multiple dimensions.
[0137] Specifically, for the first and second scores corresponding to each dimension, the average score of each dimension is calculated. The difference between the first score and the average score is then calculated. If the difference is higher than a third preset threshold, the dimension is determined to be a weak dimension. For example, as shown in Table 2, the first score for each dimension is in the third column, the difference between the first score and the average score of each dimension is in the fifth column, and the third preset threshold can be set as needed, such as 5 or 3. In this case, the weak dimension can be determined as the format failure rate. The weak dimensions determined in this way may be one or more.
[0138] If all differences are less than the third preset threshold, then the dimension corresponding to the largest difference is determined to be the weaker dimension.
[0139] Repair module 42 is used to determine the orchestration configuration based on the first score and the second score corresponding to the weak dimension; and to improve the target orchestration framework based on the orchestration configuration.
[0140] The device further includes:
[0141] Model repair module 5 is used to retrain the model based on the test attribution model layer, according to the target test result and the reference test result; specifically including:
[0142] The second diagnostic module 51 is the same as the first diagnostic module 41, or it can be used in conjunction with the first diagnostic module.
[0143] Filtering module 52 is used to filter training data based on the weakness dimension;
[0144] Specifically, a training set is first selected. The training set questions are taken from existing task sets that do not overlap with the target benchmark, such as real repository issue-repair tasks from SWE-Gym, OpenSWE, etc. The training set is deduplicated with SWE-bench Verified by instance ID (repository + PR / issue number) to ensure that the training set has zero overlap with the target benchmark. The target benchmark is only evaluated throughout the process and does not participate in training or selection.
[0145] First, assign capability labels to the candidate training set. This includes labels such as whether a question requires writing a self-reproduced script and running a test to determine correctness (corresponding to post-fix verification M57), whether it requires locating multiple files to correct (corresponding to the read-before-correction rate M07), and whether an error requires diagnosis before retrying (corresponding to the "blind retry rate M10"). These labels can be automatically determined by rules or evaluation agents with the same criteria as the evaluation, based on the question metadata (issue text, number of involved files, whether a test is included, etc.). For example, this can still be handled by the unified judge GPT-5.5, using the same model and criteria as the scoring. Then, based on the weaker dimensions, increase the proportion of questions with the corresponding labels in the training set—for example, if M57 and M07 (dimensional labels) are weak in this round, increase the proportion of questions requiring self-written test verification and those requiring cross-file location, while decreasing the proportion of other types. The ratio is adjustable, ensuring the gradient is concentrated on the weaker capability families.
[0146] Training module 53 trains the target model based on the selected training data.
[0147] A fixed training orchestration framework is used, with SWE-Agent as the training harness. For each training question, multiple trajectories are rolled out online to form a group. This group of rollouts is required for the relative advantage of the group and is a sampling of the reinforcement learning algorithm itself. The reward only uses the task pass rate. If the patch passes the official test for the question, it is recorded as 1, otherwise it is recorded as 0, thus saving the need for a separate reward model, and the reward is completely consistent with the acceptance criteria.
[0148] The device may also include a verification module 6 for retesting the target basic test based on the improved target model and target orchestration framework.
[0149] Alternatively, retesting can be performed in the target base test based on the target model and the improved target orchestration framework.
[0150] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs.
[0151] Example 3
[0152] Figure 4 This is a schematic diagram of the structure of an electronic device according to an example embodiment of the present disclosure. The electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the intelligent agent closed-loop optimization method based on multi-dimensional evaluation as described in any of the above embodiments. Figure 4 The electronic device 40 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.
[0153] like Figure 4 As shown, the electronic device 40 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 40 may include, but are not limited to: at least one processor 41, at least one memory 42, and a bus 43 connecting different system components (including memory 42 and processor 41).
[0154] Bus 43 includes a data bus, an address bus, and a control bus.
[0155] The memory 42 may include volatile memory, such as random access memory (RAM) 421 and / or cache memory 422, and may further include read-only memory (ROM) 423.
[0156] The memory 42 may also include a program tool 425 (or utility) having a set (at least one) program module 424, such program module 424 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.
[0157] The processor 41 executes various functional applications and data processing by running computer programs stored in the memory 42, such as the agent closed-loop optimization method based on multi-dimensional evaluation provided in any of the above embodiments.
[0158] Electronic device 40 can also communicate with one or more external devices 44 (e.g., keyboard, pointing device, etc.). This communication can be performed via input / output (I / O) interface 45. Furthermore, electronic device 40 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 46. As shown, network adapter 46 communicates with other modules of electronic device 40 via bus 43. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 40, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.
[0159] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.
[0160] Example 4
[0161] This disclosure also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the agent closed-loop optimization method based on multi-dimensional evaluation provided in any of the above embodiments.
[0162] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.
[0163] Example 5
[0164] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the agent closed-loop optimization method based on multi-dimensional evaluation as described above.
[0165] The program code for executing the computer program product of this disclosure can be written in any combination of one or more programming languages, and the program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.
[0166] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.
Claims
1. A closed-loop optimization method for intelligent agents, characterized in that, The method includes: The first pass rate of the target model placed in the target orchestration framework in the target benchmark test is obtained, and the second pass rate of the target model placed in the reference orchestration framework in the target benchmark test is obtained, wherein the reference orchestration framework includes multiple frameworks. Test attribution is determined based on the dispersion of the first and second pass rates and the highest pass rate.
2. The agent closed-loop optimization method as described in claim 1, characterized in that, The determination of test attribution based on the dispersion of the first pass rate and the second pass rate, as well as the highest pass rate, specifically includes: Based on the fact that the highest pass rate is lower than the first preset threshold, the test attribution is determined to be the model layer; If the highest pass rate is higher than the first preset threshold and the dispersion is higher than the second preset threshold, then the test attribution is determined to be the orchestration layer. If the highest pass rate is higher than the first preset threshold and the dispersion is lower than the second preset threshold, then it is determined that no attribution test is required.
3. The agent closed-loop optimization method as described in claim 1, characterized in that, The method further includes: Obtain the target test results of the target model placed in the target orchestration framework in the target benchmark test, and the reference test results of the target model placed in the reference orchestration framework in the target benchmark test; Based on the test attribution, the orchestration layer generates improvement parameters to improve the target orchestration framework according to the target test results and the reference test results; or, Based on the test attribution model layer, the model is retrained according to the target test result and the reference test result.
4. The agent closed-loop optimization method as described in claim 3, characterized in that, Obtain the target test results of the target model placed in the target orchestration framework in the target benchmark test; and the reference test results of the target model placed in the reference orchestration framework in the target benchmark test; specifically including, The target model is collected and placed into the target orchestration framework to generate the target trajectory in the target benchmark test, thus generating the target test results; and... The reference test results are generated by collecting the target model, placing it into the reference orchestration framework, and using the reference trajectory in the target benchmark test.
5. The agent closed-loop optimization method as described in claim 3, characterized in that, The target test result includes multiple first scores corresponding to multiple dimensions, and the reference test result includes multiple second scores corresponding to multiple dimensions; The orchestration layer, based on the test attribution, generates improvement parameters to improve the target orchestration framework according to the target test results and the reference test results; Specifically, it includes: Weak dimensions are determined based on the first and second scores corresponding to multiple dimensions. The orchestration configuration is determined based on the first and second scores corresponding to the weaker dimensions; The target orchestration framework is improved based on the orchestration configuration.
6. The agent closed-loop optimization method as described in claim 3, characterized in that, The target test result includes multiple first scores corresponding to multiple dimensions, and the reference test result includes multiple second scores corresponding to multiple dimensions; The step of retraining the model based on the test attribution model layer, according to the target test result and the reference test result, specifically includes: Weakness dimensions are determined based on the first and second scores corresponding to multiple dimensions; Training data is selected based on the aforementioned weakness dimensions; The target model is trained based on the selected training data.
7. The agent closed-loop optimization method as described in claim 3, characterized in that, The method further includes: Retesting is performed on the target base test based on the improved target model and target orchestration framework; or, Retesting is performed on the target base test based on the target model and the improved target orchestration framework.
8. A closed-loop optimization device for intelligent agents, characterized in that, include: The pass rate acquisition module is used to acquire the first pass rate of the target model placed in the target orchestration framework in the target benchmark test, and the second pass rate of the target model placed in the reference orchestration framework in the target benchmark test, wherein the reference orchestration framework includes multiple frameworks. The attribution module is used to determine test attribution based on the dispersion of the first pass rate and the second pass rate, as well as the highest pass rate.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and for running on the processor, characterized in that, When the processor executes the computer program, it implements the intelligent agent closed-loop optimization method according to any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the agent closed-loop optimization method as described in any one of claims 1-7.