An automated research agent system and method for large language model fine-tuning

Abstract

This invention relates to an automated research agent system and method for fine-tuning large language models (LLMs). The system includes a researcher module, driven by the large language model or through multi-agent collaboration, configured to receive user-input task objectives, complete literature reviews, formulate experimental plans, perform fine-grained analysis of experimental results, and manage memory; an executor module, a code agent with coding and execution capabilities, configured to execute the experimental plan, including data construction, model training and evaluation, and feed back the experimental results to the researcher for analysis; bidirectional communication between the researcher module and the executor module; and a tool library that introduces a fine-grained analysis mechanism to maximize the feedback value of each experiment and guide subsequent optimization. This invention achieves closed-loop automation of LLM fine-tuning.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and to an automated research agent system and method for fine-tuning large language models (LLMs), and more particularly to an agent system and method that can automatically complete the entire process of LLM fine-tuning (from scheme design, data construction to model training and evaluation). Background Technology

[0002] With the development of artificial intelligence technology, the capabilities of large-scale models are constantly improving, approaching or even surpassing the level of human experts in some fields. Utilizing the capabilities of large-scale models to assist and accelerate scientific research has become a research hotspot in recent years. Existing research can be broadly categorized into two types: 1) AI-Augmented Research: Utilizing LLM to assist in literature reviews, paper writing, and code generation. While this can improve research efficiency to some extent, it serves only as a supplement to manual research. These tools are typically used as auxiliary means for researchers and do not form a fully automated closed loop. 2) Autonomous Research Agents: These further explore end-to-end automation, resulting in two representative technologies: One type is End-to-End AI Researchers, such as AI Scientist and AI-Researcher systems, which can realize the complete process from idea generation to paper writing. However, this type of work mainly verifies the feasibility of end-to-end automation, with relatively broad optimization objectives and no in-depth exploration of specific tasks (such as LLM fine-tuning). The other type is Evolutionary Search Agents, such as AlphaEvolve, ML-Master, AIDE, and R&D Agent, which introduce evolutionary algorithms or tree search (such as Monte Carlo Tree Search MCTS) into the scientific discovery process.

[0003] Evolutionary search agents formalize machine learning engineering (MLE) or scientific discovery as a search and optimization problem within a vast solution space. They draw inspiration from evolutionary algorithms (such as evolutionary strategies) or tree search, utilizing LLM as the core "mutation" or "inference" engine. Through an iterative cycle of "propose-validate-select," they automatically explore, optimize, and discover high-performance solutions. R&D Agent decouples machine learning engineering tasks into two phases: research and development. It utilizes dynamic programming and inference pipelines to systematically explore idea generation and implementation verification. AIDE formalizes automated machine learning engineering as a tree search problem in the code space, leveraging large language models to draft, debug, and improve solutions, achieving efficient iteration within the code space. ML-Master integrates balanced multi-path exploration and guided inference modules, using Monte Carlo tree search to explore solution paths in parallel and relying on adaptive memory to achieve closed-loop optimization of exploration and inference. AlphaEvolve deeply combines large language models with evolutionary computation, achieving automated code generation and optimization through code evolution loops, achieving breakthroughs in algorithm discovery and industrial system optimization.

[0004] However, when the aforementioned evolutionary search agent technology is applied to large language model fine-tuning tasks, its limitations become apparent, making it difficult to adapt to the technical characteristics and engineering requirements of large language model fine-tuning. This is mainly reflected in three aspects: 1) High verification costs limit the search scale. The cost of a single experiment in LLM fine-tuning (such as using LoRA or full-parameter fine-tuning) is extremely high, including computational costs: each fine-tuning requires multiple rounds of gradient updates on a large number of model parameters, which usually requires several or even dozens of GPUs to run for several hours. A complete experimental verification (training + evaluation) may consume tens or even hundreds of dollars of computing power resources; time costs: the turnaround time for experiments is long, usually in the form of hours or even days. Traditional evolutionary search requires hundreds of iterations and evaluation of thousands of solutions, which is completely unacceptable on the time scale of LLM fine-tuning; evaluation noise: the performance evaluation of LLM itself has high variance (affected by random seeds). In order to obtain reliable feedback, it is often necessary to conduct multiple repeated experiments, which further amplifies the verification cost; 2) The mismatch between the characteristics of the search space and the evolutionary operators. Existing evolutionary agents are usually limited to mutation operations in the code or hyperparameter space. However, for fine-tuning large language models, performance improvement depends not only on the selection of hyperparameters, but more importantly on the quality and structural characteristics of the training data, such as the diversity of instruction data, task difficulty, and format standardization. Therefore, how to achieve effective mutation or crossover operations in the text semantic space to generate higher-quality training data has become a significant problem that current evolutionary learning frameworks have not fully solved, especially in vertical domain tasks—the performance of these tasks is highly dependent on the quality of the training data, and simple text editing methods (such as code-level modifications in AlphaEvolve) cannot guarantee the semantic consistency of the generated data or its effectiveness in teaching tasks; 3) The contradiction between broad exploration and deep optimization: evolutionary algorithms excel at maintaining diversity in a broad space and avoiding getting trapped in local optima. However, for LLM fine-tuning, due to experimental cost limitations, large-scale broad exploration is usually unsustainable. We need a more sample-efficient optimization method that can quickly learn from a few failed experiments and accurately adjust the direction of the next attempt, rather than relying on blind trial and error. Therefore, although existing autonomous research agents theoretically possess the potential for automated LLM fine-tuning, their "high-overhead, low-efficiency" exploration mechanism makes them difficult to implement in practice.

[0005] In summary, existing technologies suffer from the following drawbacks: Limited search space: The aforementioned approximate solutions (such as AutoML based on MCTS) primarily target limited, structured spaces such as hyperparameters or network structures. They cannot handle the more core and complex elements of LLM fine-tuning, especially the training data itself. Massive amounts of training data cannot be directly used as "parameters" in the LLM context for searching and optimization, posing a significant obstacle to achieving full-process automation. Low computational efficiency: Existing evolutionary or tree-search-based methods typically require parallel sampling and validation of numerous schemes. For tasks like LLM fine-tuning, which can take hours or even days and consume significant GPU resources, this "broad-based" exploration strategy is computationally unacceptable, resulting in extremely low iteration efficiency. Insufficient utilization of feedback information: Existing methods typically only utilize the final evaluation metric (such as accuracy) as feedback signals. However, a single LLM fine-tuning experiment contains rich information, such as changes in the loss curve, performance differences across different subsets, and even failed cases. This information is wasted in existing solutions and fails to guide the next round of more precise optimization. Summary of the Invention

[0006] The technical problem addressed by this invention is that current LLM fine-tuning training heavily relies on human expert experience, resulting in long development cycles, high trial-and-error costs, and difficulty in achieving end-to-end automation. Existing automated research intelligent agent technologies primarily target machine learning model optimization or algorithm design tasks, focusing on optimization within a single space such as hyperparameter search, innovation point generation, or code generation. These technologies cannot effectively handle the open-ended, multi-dimensional exploratory problems involved in LLM fine-tuning, such as processing massive training data, optimizing complex algorithm designs, and combining hyperparameters. Furthermore, the long experimental cycle and high computational cost of LLM fine-tuning render traditional evolutionary algorithms, which rely on large-scale sampling and verification, inefficient or even infeasible.

[0007] This invention aims to provide an automated research agent system and method for LLM fine-tuning. Specifically, it provides an agent system and method capable of automating the entire LLM fine-tuning process (from scheme design and data construction to model training and evaluation), continuously optimizing the performance of large language models within limited resources and time budgets, even reaching or surpassing the design level of human experts. The agent system is named TREX (TRee-based EXplorer). This system achieves closed-loop automation of LLM fine-tuning through the collaborative work of two core modules: a "Researcher" and an "Executor," along with a high-performance data processing library (AIDP, AI Data Processing library). The Researcher module is responsible for analyzing tasks, reviewing literature, and developing experimental plans; the Executor module is a code agent integrated with a GPU cluster, responsible for executing the experimental plan, including data construction, model training, and evaluation, and feeding back the experimental results to the Researcher for analysis. The AIDP data processing library introduces a fine-grained analysis mechanism to maximize the feedback value of each experiment and guide subsequent optimization directions. Meanwhile, the system employs a Monte Carlo Tree Search (MCTS)-based optimization strategy to generate new experimental schemes by continuously mining and optimizing the most promising paths from historical experiments, rather than blindly sampling.

[0008] The first aspect of this invention provides an automated research agent method for fine-tuning large language models, comprising the following steps: S1. Provide a basic training dataset and perform grid search on key hyperparameters, using experiments as the root node of the search strategy to establish a performance baseline; S2. Selecting Experimental Nodes: Selecting nodes from the existing experimental tree using a sequence decision optimization algorithm. Optimize it as a node to be optimized; S3. Expanding New Experimental Nodes: The researcher module generates differentiated new experimental schemes based on the historical experimental records of the nodes to be optimized and combined with literature knowledge. These new experimental schemes are not simple parameter perturbations, but can include more advanced changes, such as "adopting different data filtering strategies", "introducing a new data synthesis method" or "adjusting the loss function in the training process". S4. Experiment Execution and Verification: The executor module receives the new experimental scheme, parses the new experimental scheme into executable code, calls the AI ​​data processing library (AIDP tool) to build the dataset, submits the computing power cluster task for model training and evaluation, and synchronously feeds back the experimental results of the new experimental node. S5, Backpropagation Update: The experimental results of the new experimental node are used as rewards and propagated back along the path from the new node to the root node, updating the visit count and cumulative reward of all nodes along the way; and S6. Determine whether the current model performance has reached the preset target or whether the resource budget has been exhausted. If not, return to step S2 and continue node selection and iterative optimization. If the target has been reached, terminate the process, output the large language model fine-tuning implementation strategy and model, and execute the user-specified downstream task based on the large language model fine-tuning implementation strategy and model to significantly improve the model's performance on the corresponding task within a limited resource and time budget. By fine-tuning the general base model, its ability on a specific task (such as legal question answering) is enhanced, making it adaptable to the corresponding task requirement scenario (such as a legal consultation system). That is, compared with the base model before training, the model automatically fine-tuned by the system will have a significant performance improvement on the corresponding downstream task.

[0009] Furthermore, the sequence decision optimization algorithm can be any one of Monte Carlo Tree Search (MCTS), reinforcement learning-based strategy (RL), or Bayesian optimization.

[0010] Furthermore, in step S1, the optimal experiment is used as the root node of the search strategy to establish a performance baseline. The optimal experiment is based on achieving the best performance on the performance evaluation metric specified in the training task. The performance evaluation metric is part of the task definition and serves as the input to the system (e.g., the task is to train a "legal expert model," and the performance metric is the accuracy score on the "LawBench" evaluation set).

[0011] Further, in step S2, select a node. Includes the following steps: Based on the initialized experimental tree, calculate the maximum upper bound confidence interval (UCT) value of the tree for all nodes of the experimental tree; and The node with the largest UCT value is selected as the node to be optimized. The formula for calculating the UCT value is as follows: in, It is a node Number of visits, It is the number of visits to its parent node. It is a node Cumulative rewards, cumulative rewards Defined as the normalized value of the main evaluation indicators (such as accuracy and ROUGE score) of this experiment, with the constant C controlling the balance between exploration and exploitation; Furthermore, in step S2, when calculating the UCT values ​​of all nodes in the experimental tree, the context configured for the node to be optimized includes: Experimental trajectory It refers to all intermediate experiments from the initial experimental node to the current node to be optimized; Sibling nodes are nodes that are related to each other. They share a common parent node and are configured to avoid redundant exploration and experimentation schemes similar to existing nodes; and Key nodes in the experimental tree, including those that bring significant performance improvements or experimental failures, are configured to globally share important experimental conclusions.

[0012] Furthermore, in step S3, the differentiated new experimental scheme includes one or more of the following: adopting different data filtering strategies, introducing new data synthesis methods, and adjusting the loss function during the training process.

[0013] Furthermore, in step S4, in order to further improve efficiency, in a single iteration, the executor module trains a batch of models with different configurations in parallel based on the new experimental scheme (for example, fine-tuning a few hyperparameters based on the selected scheme), and uses the result of the best performing model as the cumulative reward of the new experimental node to update the UCT value.

[0014] Furthermore, after step S4, fine-grained analysis is also included: The researcher module performs index decomposition, cross-model comparison, and failure case analysis on the experimental results, stores the analysis results in a structured manner in the experimental history tracker, and updates the system's internal memory; and / or The fine-grained analysis also includes: Gradient information analysis identifies the most important data samples or network layers for a specific task by analyzing the gradients of model parameters, thereby enabling more refined optimization.

[0015] Furthermore, the metric is decomposed into scores for multiple sub-tasks, analyzing the model's performance across different dimensions. The cross-model comparison compares the current model with the baseline model and the historical best model on the same test set, sample by sample. The bad-case analysis identifies the worst-performing sample categories or domains and packages the failed samples and their prediction results. The executor packages these "failed case" samples and their prediction results and feeds them back to the researchers. By analyzing these cases, researchers can identify the model's weaknesses and propose targeted optimization schemes in the next plan (e.g., generating more synthetic training data for these failed cases).

[0016] A second aspect of the present invention provides an automated research agent system for fine-tuning large language models, comprising: The Researcher module, driven by a large language model or through multi-agent collaboration, is configured to receive user-inputted task objectives, complete literature reviews, develop experimental plans, perform fine-grained analysis of experimental results, and manage memory. The Executor module, a code-based intelligent agent capable of writing and executing code, is configured to execute experimental plans, including data construction, model training and evaluation, and to feed back the experimental results to researchers for analysis; the researcher module and the Executor module communicate bidirectionally; and The toolkit introduces a fine-grained analysis mechanism to maximize the feedback value of each experiment and guide subsequent optimization.

[0017] Furthermore, the tool library includes The AI ​​Data Processing Library (AIDP) is configured to support the execution module in constructing complex training data processing; the AI ​​Data Processing Library includes one or more atomic capabilities among data loading, scoring, generation, filtering, data augmentation strategy search, graph neural network sample modeling, and data crawling. The academic search tool is configured to provide academic paper and knowledge retrieval services to the researcher module, enabling it to complete literature reviews and learn cutting-edge methodologies within its target niche; and The experiment history tracker is configured to record all experiment configurations, code, results, and logs to form a structured database, providing support for the researcher module to retrieve and analyze historical experiment records.

[0018] Furthermore, the researcher module integrates a memory management strategy that selectively loads only the historical records most relevant to the current exploration path when generating a new plan.

[0019] This invention proposes an intelligent agent capable of efficiently exploring the LLM fine-tuning scheme space under extremely high experimental cost constraints. The method of this invention has the following capabilities: Metacognition and reasoning: The ability to filter and optimize ideas through in-depth reasoning before conducting experiments, rather than resorting to brute force. Experience accumulation and transfer: The ability to extract transferable insights from historical experiments (e.g., "increasing the difficulty of mathematical samples" is more effective than "simply increasing the learning rate"), thereby guiding subsequent explorations; Fine-grained feedback utilization: It is necessary not only to utilize the final evaluation score, but also to diagnose the defects of the solution and propose targeted improvement strategies from intermediate feedback such as training loss curves and error patterns on the validation set.

[0020] This invention has at least the following beneficial effects: 1) It achieves higher-dimensional automation: Existing technologies only search within a predefined hyperparameter space, while this invention incorporates the construction and optimization of the training data itself into an automated closed loop. Through the collaboration of the AIDP library and the researcher module, the system can autonomously design complex strategies such as data cleaning, filtering, and synthesis, solving the problem that massive amounts of data cannot directly participate in automatic optimization; 2) It significantly improves exploration efficiency and is more suitable for high-cost tasks: Existing technologies rely on large-scale parallel sampling, which is not suitable for LLM fine-tuning. This invention adopts a UCT-based optimal exploration strategy, which performs deep optimization from the historical best path in each iteration, rather than blindly trying. This "few but excellent" strategy greatly reduces invalid experiments, making it possible to continuously optimize LLM within a limited computational budget. Experiments have proven that... The present invention demonstrates that this strategy can steadily and continuously improve performance, outperforming the greedy strategy; 3) High utilization of feedback information, making optimization more directional: Existing technologies only use the final index as feedback, resulting in low information utilization; This invention introduces a fine-grained analysis mechanism, especially the mining and utilization of "failure cases," which enables the intelligent agent to understand why the model failed, thereby generating more targeted improvement schemes in the next iteration (such as supplementing data for specific failure types), achieving a leap from "black-box parameter tuning" to "white-box diagnostic optimization." Experiments show that after introducing failure case analysis, the system performance improves faster and the final score is higher; 4) More creative and interpretable scheme generation: Existing technologies usually generate new schemes by simple parameter perturbation or mutation. This invention is driven by the "researcher" LLM, which can combine literature knowledge and historical experimental results to generate complex training schemes containing new ideas and methods (e.g., "using the method of paper X to perform Y kinds of data enhancement"), making the entire exploration process closer to the scientific research paradigm of human experts, and making the generated schemes easier to understand and analyze; 5) The large language model fine-tuning implementation strategy and model derived from the technical solution of this invention can be used in the field of artificial intelligence to achieve the following technical effects: significantly improve the performance indicators of large language models on specific downstream tasks (such as legal question answering, financial analysis, etc.) within limited resources and time budgets; automate the traditional fine-tuning process that relies on expert experience, from literature review, experimental design, execution evaluation to iterative optimization, without human intervention throughout the entire process, greatly reducing the R&D threshold and human resource costs. Attached Figure Description

[0021] To further illustrate the above and other advantages and features of the various embodiments of the present invention, a more specific description of the embodiments of the invention will be presented with reference to the accompanying drawings. It is to be understood that these drawings depict only typical embodiments of the invention and are therefore not intended to limit its scope. In the drawings, identical or corresponding parts will be indicated by identical or similar reference numerals for clarity.

[0022] Figure 1The following diagram illustrates the system architecture of an automated research agent in some embodiments of the present invention; Figure 2 Experimental tree structures for Monte Carlo Tree Search (MCTS) in some embodiments of the present invention are shown; Figure 3 Experimental results of the automated research agent system in some embodiments of the present invention on the two tasks of Tombench and OpenFin are shown. Figure 4 This invention illustrates the impact of different search strategies on the performance of automated research agent systems in some embodiments; Figure 5 An ablation experiment was shown to investigate the impact of bad case analysis on the performance of automated research intelligent agent systems. Figure 6 An ablation experiment is shown: exploring the impact of AIDP tools on the performance of automated research intelligent agent systems. Detailed Implementation

[0023] It should be noted that the components in the accompanying drawings may be shown exaggerated for illustrative purposes and may not be to scale.

[0024] In this invention, the various embodiments are merely intended to illustrate the solutions of the invention and should not be construed as limiting.

[0025] In this invention, unless otherwise specified, the quantifiers “a” and “one” do not exclude scenarios involving multiple elements.

[0026] It should also be noted that, in the embodiments of the present invention, only a portion of the parts or components may be shown for clarity and simplicity. However, those skilled in the art will understand that, under the teachings of the present invention, the required parts or components can be added as needed for specific scenarios.

[0027] It should also be noted that within the scope of this invention, the terms "same", "equal", and "equal to" do not mean that the two values ​​are absolutely equal, but allow for a certain reasonable error. In other words, the terms also cover "substantially the same", "substantially equal", and "substantially equal to".

[0028] It should also be noted that in the description of this invention, the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not explicitly or implicitly suggest that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0029] In this invention, the modules of the system according to the invention can be implemented using software, hardware, firmware, or a combination thereof. When a module is implemented using software, its function can be implemented through computer program flow. For example, the module can be implemented using code segments (such as code segments in languages ​​like C and C++) stored in a storage device (such as a hard disk, memory, etc.), wherein the corresponding function of the module can be implemented when the code segment is executed by a processor. When a module is implemented using hardware, its function can be implemented by setting a corresponding hardware structure. For example, the module's function can be implemented by hardware programming a programmable device such as a field-programmable gate array (FPGA), or by designing an application-specific integrated circuit (ASIC) that includes multiple transistors, resistors, capacitors, and other electronic devices. When a module is implemented using firmware, the module's function can be written into a read-only memory such as an EPROM or EEPROM in the form of program code, and the corresponding function of the module can be implemented when the program code is executed by a processor. In addition, some functions of the module may need to be implemented by separate hardware or by working in cooperation with the hardware. For example, the detection function is implemented by a corresponding sensor (such as a proximity sensor, accelerometer, gyroscope, etc.), the signal transmission function is implemented by a corresponding communication device (such as a Bluetooth device, infrared communication device, baseband communication device, Wi-Fi communication device, etc.), the output function is implemented by a corresponding output device (such as a display, speaker, etc.), and so on.

[0030] Furthermore, the embodiments of the present invention describe the process steps in a specific order. However, this is only for the convenience of distinguishing each step, and is not a limitation on the order of each step. In different embodiments of the present invention, the order of each step can be adjusted according to the process.

[0031] This embodiment provides an automated research agent method for fine-tuning large language models. The core exploration method is an improved Monte Carlo Tree Search (MCTS), specifically designed to address the high cost of LLM fine-tuning. The process is as follows: Figure 2 As shown. Includes the following steps: S1. Initialization (Root Node): The system first conducts an initial experiment to establish a baseline. This includes providing a basic training dataset and performing a grid search on key hyperparameters. The optimal experiment is used as the root node of the search strategy to establish a performance baseline. S2. Selecting Experimental Nodes: Search for MCTS nodes using a Monte Carlo tree from the existing experimental tree. Optimize it as a node to be optimized; S3. Expanding New Experimental Nodes: The researcher module generates differentiated new experimental schemes based on the historical experimental records of the nodes to be optimized and combined with literature knowledge. These new experimental schemes are not simple parameter perturbations, but can include more advanced changes, such as "adopting different data filtering strategies", "introducing a new data synthesis method" or "adjusting the loss function in the training process". S4. Experiment Execution and Verification: The executor module receives the new experimental plan, parses the new experimental plan into executable code, calls the AI ​​data processing library (AIDP tool) to build the dataset, submits the computing power cluster task for model training and evaluation, synchronously feeds back the experimental results of the new experimental node, and returns detailed experimental results. S5. Backpropagation Update: The experimental result (Q value) of the new experimental node is used as a reward and propagated back along the path from the new node to the root node, updating the number of visits and cumulative rewards (N value and Q value) of all nodes along the way. S6. Determine whether the current model performance has reached the preset target or whether the resource budget has been exhausted. If not, return to step S2 and continue node selection and iterative optimization. If the target has been reached, terminate the process, output the optimal large language model fine-tuning implementation strategy and model, and execute the user-specified downstream tasks based on the large language model fine-tuning implementation strategy and model to significantly improve the model's performance on the corresponding tasks within the limited resources and time budget.

[0032] In step S2, select the node. Includes the following steps: Based on the initialized experimental tree, the maximum upper bound confidence interval (UCT) value of the tree for all nodes in the experimental tree is calculated; when generating a new scheme for node v, the context provided to the LLM consists of the following parts: experimental trajectory This refers to all intermediate experiments from the initial experimental node to the current node to be optimized; sibling nodes are those related to the node... They share a common parent node and are configured to avoid redundant exploration and experimental schemes similar to existing nodes; and key nodes in the experimental tree, including nodes that bring significant performance improvements or experimental failures, are configured to globally share important experimental conclusions; and The node with the largest UCT value is selected as the node to be optimized. The formula for calculating the UCT value is as follows: in, It is a node Number of visits, It is the number of visits to its parent node. It is a node Cumulative rewards, cumulative rewards Defined as the normalized value of the main evaluation indicators (such as accuracy and ROUGE score) of this experiment, with the constant C controlling the balance between exploration and exploitation; In step S4, to further improve efficiency, in a single iteration, the executor module trains a batch of models with different configurations in parallel based on the new experimental scheme (for example, fine-tuning a few hyperparameters based on the selected scheme), and uses the result of the best performing model as the cumulative reward of the new experimental node to update the UCT value.

[0033] To extract the maximum amount of information from each expensive experiment, the evaluation phase of this invention not only calculates the final index, but also includes fine-grained analysis after step S4: The researcher module performs metric decomposition, cross-model comparison, and bad-case analysis on the experimental results. The analysis results are then structured and stored in the experimental history tracker and the system's internal memory is updated. Metric decomposition involves breaking down the overall metric into scores for multiple sub-tasks for complex tasks, analyzing the model's performance across different dimensions. Cross-model comparison compares the current model with the baseline model and the historical best model on the same test set, sample by sample. Bad-case analysis identifies the worst-performing sample categories or domains and packages the failed samples and their predictions. The executor packages these "failed case" samples and their predictions and feeds them back to the researcher. By analyzing these cases, the researcher can identify the model's weaknesses and propose targeted optimization strategies for the next iteration (e.g., generating more synthetic training data for these failed cases).

[0034] This embodiment also provides an automated research agent system for fine-tuning large language models, including: The researcher module, driven by a large language model or through multi-agent collaboration, is configured to receive user-input task objectives (which may involve fine-tuning a downstream task to improve LLM capabilities, such as enhancing the model's ability to call tools, answer professional knowledge questions in finance or law, etc.); conduct literature reviews through an integrated academic search engine to understand state-of-the-art methods and common data construction and training strategies in the field; formulate experimental plans: based on the literature review and historical experimental results, formulate plans for the next round of experiments, which may include: data construction schemes (such as using AIDP tools for data cleaning, filtering, and synthesis), selection of training algorithms, hyperparameter configuration, etc.; perform fine-grained analysis and memory management of experimental results: receive detailed experimental results returned by the executor, analyze and summarize them, and update them in internal memory. To address the context overflow problem caused by long-sequence experiments, a memory management strategy is designed to selectively load only the historical records most relevant to the current exploration path when generating a new plan. The executor module, acting as the system's "hands and feet," is a code-intelligent agent capable of writing and executing code. It is configured to execute experimental plans, including data construction, model training and evaluation, and to feed back experimental results to researchers for analysis. The researcher module and the executor module communicate bidirectionally. Specifically, the executor module's functions include: plan parsing and code generation: parsing the experimental plans developed by researchers into executable code; in particular, it can call atomic tools in the AIDP tool library to combine them into complex data processing pipelines; executing experimental tasks: interacting with the computing cluster via API to complete computationally intensive tasks such as batch data processing, model training, and evaluation; experimental monitoring and feedback: monitoring the cluster task progress and summarizing experimental results, logs, model checkpoints, and other information, returning them to the researchers; and more. The toolkit introduces a fine-grained analysis mechanism to maximize the feedback value of each experiment and guide subsequent optimization. The toolkit includes: an AI Data Processing Library (AIDP), configured to support the executor module in constructing complex training data processing; the AI ​​Data Processing Library includes one or more atomic capabilities such as data loading, scoring, generation, filtering, data augmentation strategy search, graph neural network sample modeling, and data crawling; an academic search tool, configured to provide the researcher module with academic paper and knowledge retrieval services, enabling it to complete literature reviews and learn cutting-edge methods in its target fine-tuning field; and an experiment history tracker, configured to record the configuration, code, results, and logs of all experiments and form a structured database, providing the researcher module with support for historical experiment record retrieval and analysis.

[0035] AI Data Processing Library: Provides a suite of high-performance atomic data processing tools for LLM training scenarios, including data loading, scoring (e.g., calculating perplexity), generation (e.g., generating synthetic data using LLM, constructing preference pairs), and filtering (e.g., deduplication, score-based filtering). These atomic capabilities form the foundation for executors to build complex data pipelines, ensuring the reliability and efficiency of data processing.

[0036] Academic Search: Allows the Researcher module to access academic paper databases for literature, data retrieval, and knowledge acquisition.

[0037] Experiment History Tracker: Records the configuration, code, results, and logs of each experiment, forming a structured historical database.

[0038] The researcher module integrates a memory management strategy that selectively loads only the historical records most relevant to the current exploration path when generating a new plan.

[0039] The feasibility and effectiveness of this invention have been proven through numerous experiments.

[0040] Experimental platform: An FT-Bench benchmark set containing 10 tasks was built, covering a variety of scenarios such as general capability enhancement and vertical domain adaptation.

[0041] Comparison scheme: The automated research agent system (hereinafter referred to as TREX system) used Qwen3-1.7B as the baseline model, and successively carried out one round of baseline construction and twenty rounds of iterative optimization. The final optimized model was systematically compared with the most advanced Qwen3-235B-2507 model in the current open source field, as well as the model obtained by human experts for fine-tuning and optimizing the same task.

[0042] Experimental results: Table 1. Final experimental optimization results of the TREX system on ten FT-Bench tasks and comparison with Qwen3-235B-2507. Performance Improvements: As shown in Table 1, the TREX system delivers significant performance improvements across all 10 tasks, and in several tasks (such as TOMG-Bench, OpenFinData, Hoc, and ACI-Bench), it even surpasses the performance of human expert fine-tuning or the model performance of Qwen3-235B-2507 (see Table 1). Figure 3 ). Figure 3This section compares the TREX end-to-end experimental results with the optimization results from human experts. In Experiment A, TREX used Gemini3-pro-thinking as the researcher model, while in Experiment B, it used Qwen3-next-80B-thinking. (↑) represents the optimized growth curve. FEVO-R32B is the model result fine-tuned by human experts based on Qwen2.5-32B, and FEVO-R32B-0 is the model result fine-tuned by human experts based on Qwen2.5-32B-Instruct.

[0043] Perform ablation experiments: Figure 4 The results demonstrate that the MCTS-based exploration strategy outperforms both greedy and state-of-the-art strategies. Figure 4 The experimental score trajectories (dashed lines) and corresponding score growth curves (solid lines) are shown for three search strategies (MCTS search strategy, greedy strategy, and sequence strategy). Figure 5 This validated the crucial role of fine-grained analysis (especially failure case analysis) in improving the performance of the TREX system. Figure 5 A comparison of the experimental score trajectory (dashed line) and the corresponding score growth curve (solid line) based on observable and unobservable error cases; Figure 6 This verifies the indispensability of the AIDP toolkit in ensuring the smooth execution of experimental procedures and improving results. Figure 6 Comparison of experimental score trajectories (dashed lines) and corresponding score growth curves (solid lines) with and without AIDP enabled.

[0044] Strategy effectiveness analysis: The statistical data in Table 2 shows that "adjusting data" and "adjusting the training process" are the most effective strategies for improving performance, while "constructing synthetic data" also shows great potential.

[0045] Table 2 Comparison of execution count and efficiency of different TREX operators These experimental results fully demonstrate that the technical solution proposed in this invention is feasible, effective, and robust.

[0046] In some embodiments, in addition to using MCTS as the exploration strategy, other sequential decision optimization algorithms can be used as alternatives, such as reinforcement learning (RL) based strategies (e.g., the PPO algorithm), to model the entire iterative optimization process as a Markov decision process, using an RL agent to generate the next round of experimental schemes. Alternatively, Bayesian optimization can be used to probabilistically model the space of high-cost experimental schemes and select the most promising scheme for evaluation.

[0047] In some embodiments, the researcher module is driven by a single LLM. An alternative could be a multi-agent collaborative system, for example, assigning a "literature analyst" agent specifically for literature review, a "program designer" agent for planning, and a "results analyst" agent for post-review analysis. Through discussion and collaboration among multiple specialized agents, research tasks are completed collaboratively.

[0048] In some embodiments, the implementation of the AIDP library may not be limited to the atomic capabilities listed in the paper. Its functionality can be extended, for example, to integrate automatic data augmentation strategy search, or to leverage graph neural networks to model relationships between data samples, data crawler construction, etc., thereby achieving more intelligent data selection.

[0049] In some embodiments, fine-grained analysis may go beyond simply analyzing failure cases. Alternatives may include gradient information analysis, which identifies the most important data samples or network layers for a specific task by analyzing the gradients of model parameters, enabling more refined optimization.

[0050] This invention is not limited to fine-tuning large language models; its core idea—an intelligent agent system that automates complex, open-ended exploratory tasks through a closed loop of "plan-execution-verification-analysis"—can be widely applied to other high-cost scientific research and engineering fields. Training other types of AI models: training strategies that can be used to automatically explore and optimize other large-scale models (such as diffusion models, visual Transformers).

[0051] Construction of high-quality AI training data: The system can automatically conduct research on public datasets, iterate training data schemes, and generate and execute data processing code according to training tasks, thereby automatically constructing a complete data production pipeline and accumulating high-quality professional domain training data through continuous iteration to improve data quality.

[0052] While some embodiments of the present invention have been described in this application, those skilled in the art will understand that these embodiments are merely illustrative. Numerous variations, alternatives, and improvements will arise in those skilled in the art under the teachings of this invention without departing from its scope. The appended claims are intended to define the scope of the invention and thereby cover methods and structures within the scope of the claims themselves and their equivalents.

Claims

1. An automated research agent method for fine-tuning large language models, characterized in that, Includes the following steps: S1. Provide a basic training dataset and perform grid search on key hyperparameters, using experiments as the root node of the search strategy to establish a performance baseline; S2. Select nodes from the existing experimental tree using a sequence decision optimization algorithm. Optimize it as a node to be optimized; S3. Generate differentiated new experimental schemes based on the historical experimental records of the nodes to be optimized and combined with literature knowledge; S4. Parse the new experimental scheme into executable code, call the AI ​​data processing library to build a dataset, submit the computing power cluster task for model training and evaluation, and synchronously feed back the experimental results of the new experimental nodes; S5. The experimental results of the new experimental node are used as rewards and propagated backward along the path from the new node to the root node, updating the number of visits and cumulative rewards of all nodes along the way. S6. Determine whether the current model performance has reached the preset target or whether the resource budget has been exhausted. If not, return to step S2 and continue node selection and iterative optimization. If the target has been reached, terminate the process, output the large language model fine-tuning implementation strategy and model, and execute the user-specified downstream tasks based on the large language model fine-tuning implementation strategy and model.

2. The automated research agent method for fine-tuning large language models according to claim 1, characterized in that, The sequence decision optimization algorithm can be any one of Monte Carlo Tree Search (MCTS), reinforcement learning-based strategy (RL), or Bayesian optimization.

3. The automated research agent method for fine-tuning large language models according to claim 1, characterized in that, In step S2, select the node. Includes the following steps: Based on the initialized experimental tree, calculate the maximum upper bound confidence interval (UCT) value of the tree for all nodes of the experimental tree; and The node with the largest UCT value is selected as the node to be optimized. The formula for calculating the UCT value is as follows: in, It is a node Number of visits, It is the number of visits to its parent node. It is a node Cumulative rewards, cumulative rewards Defined as the normalized value of the main evaluation index of this experiment, the constant C controls the balance between exploration and utilization.

4. The automated research agent method for fine-tuning large language models according to claim 3, characterized in that, In step S2, when calculating the UCT value of all nodes in the experimental tree, the context configured for the node to be optimized includes: Experimental trajectory It refers to all intermediate experiments from the initial experimental node to the current node to be optimized; Sibling nodes are nodes that are related to each other. They share a common parent node and are configured to avoid redundant exploration and experimentation schemes similar to existing nodes; and Key nodes in the experimental tree, including those that bring significant performance improvements or experimental failures, are configured to globally share important experimental conclusions.

5. The automated research agent method for fine-tuning large language models according to claim 4, characterized in that, In step S3, the differentiated new experimental scheme includes one or more of the following: adopting different data filtering strategies, introducing new data synthesis methods, and adjusting the loss function during training.

6. The automated research agent method for fine-tuning large language models according to claim 4, characterized in that, In step S4, based on the new experimental scheme, a batch of models with different configurations are trained in parallel, and the result of the best performing model is used as the cumulative reward of the new experimental node to update the UCT value.

7. The automated research agent method for fine-tuning large language models according to claim 1, characterized in that, Following step S4, fine-grained analysis is also included: The experimental results are then subjected to index decomposition, cross-model comparison, and failure case analysis. The analysis results are then structured, stored, and the system's internal memory is updated. And / or The fine-grained analysis also includes: Gradient information analysis.

8. An automated research agent system for fine-tuning large language models, characterized in that, include: The researcher module, driven by a large language model or through multi-agent collaboration, is configured to receive user-inputted task objectives, complete literature reviews, formulate experimental plans, perform fine-grained analysis of experimental results, and manage memory. The executor module, which is a code-intelligent agent with the ability to write and execute code, is configured to execute the experimental plan, including data construction, model training and evaluation, and to feed back the experimental results to the researcher for analysis; the researcher module and the executor module communicate bidirectionally; as well as The toolkit introduces a fine-grained analysis mechanism to maximize the feedback value of each experiment and guide subsequent optimization.

9. The automated research agent system for fine-tuning large language models according to claim 8, characterized in that, The tool library includes The AI ​​data processing library is configured to support the execution module in constructing complex training data processing; the AI ​​data processing library includes one or more atomic capabilities among data loading, scoring, generation, filtering, data augmentation strategy search, graph neural network sample modeling, and data crawling. The academic search tool is configured to provide academic paper and knowledge retrieval services to the researcher module, enabling it to complete literature reviews and learn cutting-edge methods in the target fine-tuning field. as well as An experiment history tracker is configured to record the configuration, code, results, and logs of all experiments and form a structured database.

10. The automated research agent system for fine-tuning large language models according to claim 8, characterized in that, The researcher module integrates a memory management strategy that selectively loads only the historical records most relevant to the current exploration path when generating a new plan.

Images