Large language model control fine tuning method and system based on multi-task cooperative regulation and control

Abstract

The invention discloses a large language model control fine tuning method and system based on multi-task cooperative regulation and control, and belongs to the technical field of large language models. Generating a gating coefficient and an initial dynamic evaluation signal through the intelligent regulation and control network; a task difficulty index is obtained by combining multi-index fusion and historical moving average, and the sampling probability and the exclusive learning rate are dynamically adjusted; weighting the fusion gradient and carrying out back propagation to update parameters; and closed-loop feedback monitoring is carried out and parameters of the regulation and control network and the scheduling policy device are optimized. The system comprises a data coding module, a collaborative intelligent regulation and control module, a dynamic balance control module, a joint optimization module and a closed-loop feedback module. According to the method, gradient conflicts among tasks are relieved, the problems of convergence instability and performance imbalance are solved, the multi-task training efficiency, convergence stability and generalization ability of a large language model are improved, and the method can be widely applied to multi-class multi-task learning scenes.

Description

Technical Field

[0001] This invention belongs to the field of large language model technology, specifically relating to a method and system for fine-tuning large language model control based on multi-task collaborative regulation. Background Technology

[0002] With breakthroughs in the application of Large Language Models (LLM) in natural language understanding and generation, multi-task learning capability has become its core competitiveness. LLM obtains the foundation of language representation through large-scale pre-training, and after multi-task fine-tuning, general knowledge can be transferred to downstream tasks such as text understanding, question answering, and summary generation, achieving "one-time training, multi-task applicability", reducing deployment and maintenance costs, and showing significant potential in industrial scenarios and academic research.

[0003] However, existing multi-task fine-tuning methods suffer from several drawbacks due to outdated collaborative mechanisms. First, static collaborative strategies struggle to resolve gradient conflicts, relying on manually preset static weights or simple gradient operations, which cannot adapt to dynamic gradient relationships. For example, many related patents and studies involve fixed strategy combinations or post-hoc parameter adjustments, failing to resolve real-time conflicts during training and easily leading to parameter oscillations. Second, they lack "dynamic balancing" control capabilities. Fixed sampling or polling resource allocation strategies cannot adjust resources according to the real-time convergence state of the task, resulting in overfitting of the model to simple tasks and underfitting of complex tasks. Third, collaborative frameworks lack closed-loop self-evolution capabilities. Static weights or heuristic scheduling strategies have fixed logic during training and cannot self-correct according to the task state, leading to resource waste and optimization dilemmas. Summary of the Invention

[0004] In view of this, the purpose of this invention is to propose a large language model control fine-tuning method and system based on multi-task collaborative regulation, so as to solve or partially solve the problems mentioned in the background art.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a method for fine-tuning large language models based on multi-task collaborative regulation, comprising: The training dataset from the specified task is input into a shared large language model, a learnable task identifier vector is assigned to each task, and the input data is uniformly encoded and preprocessed to generate task-aware feature representations. Through a learnable intelligent control network, the task identifier vector and the task-aware feature representation are received, and a gating coefficient for dynamically adjusting the task gradient weights and an initial dynamic evaluation signal for evaluating the current convergence state of the task are generated. Based on the initial dynamic evaluation signal, the final dynamic evaluation signal is generated by combining multi-index fusion and historical difficulty index moving average processing. Then, the sampling probability and task-specific learning rate of the specified task in the training batch are dynamically adjusted by the scheduling strategy. Based on the gating coefficient and the scheduling result dynamically adjusted by the scheduling strategy, the gradients of the specified task are weighted and fused to form a joint optimization objective, and backpropagation is performed to synchronously update the parameters of the shared large language model and the parameters of the intelligent control network. During the training iteration, the performance indicators and convergence status of the specified task are continuously monitored, and the internal parameters of the intelligent control network and the scheduling strategy are adjusted according to the monitoring results to form an adaptive optimization mechanism.

[0006] As a preferred scheme for the large language model control fine-tuning method based on multi-task collaborative regulation, the unified encoding preprocessing includes word segmentation formatting, feature encoding and task identification fusion. The word segmentation formatting process segments the input text into words through a word segmenter and uses a padding strategy to unify the length of the input sequence to a preset length L, generating a TokenID sequence. The feature encoding inputs the TokenID sequence into a shared large language model encoder and extracts the encoder hidden layer representation as the task's basic features. The task identifier fusion extracts aggregated features from basic features and concatenates them with learnable task identifier vectors to generate a unified feature representation for task awareness.

[0007] As a preferred scheme for the large language model control fine-tuning method based on multi-task collaborative regulation, the structure of the learnable intelligent regulation network is a lightweight feedforward neural network or attention network, the output gating coefficient is normalized to the range of [0,1] by the Sigmoid function, and the initial dynamic evaluation signal is mapped to a non-negative value by the Softplus function. The learnable intelligent control network is integrated inside the Transformer layer of the large language model. It achieves task-aware feature modulation by element-wise multiplying or adding the gating coefficients with the attention weights of the attention network or the activation values ​​of the feedforward network.

[0008] As a preferred scheme for the large language model control fine-tuning method based on multi-task collaborative regulation, the objects of the multi-indicator fusion include loss change rate, gradient magnitude, prediction entropy and model output uncertainty index; the historical difficulty index moving average processing introduces the moving average of the historical difficulty index of the specified task to smooth instantaneous fluctuations.

[0009] As a preferred scheme for the large language model control fine-tuning method based on multi-task collaborative regulation, the scheduling strategy uses a Softmax function based on a temperature coefficient to map the task difficulty index to the sampling probability, and reduces the temperature coefficient according to a preset annealing strategy during training, so that the sampling distribution shifts from uniform exploration to focus on high-difficulty tasks; the task-specific learning rate is adjusted through a learning rate factor.

[0010] As a preferred scheme for the large language model control fine-tuning method based on multi-task collaborative regulation, when the difficulty index of a certain task exceeds the first threshold in N consecutive training cycles, the task reweighting mechanism is triggered to apply L1 or L2 norm regularization constraints to the gradient of the task, or to directly apply a decay factor to the gating coefficient of the task. During the training iteration, an adaptive optimizer is configured separately for the learnable intelligent control network. When the convergence speed of a certain task is detected to be lower than the second threshold, the learning rate of the control parameters of the intelligent control network related to the task is reduced to weaken the parameter update amplitude and smooth the convergence process.

[0011] As a preferred scheme for large language model control and fine-tuning methods based on multi-task collaborative regulation, the performance indicators of the specified task are continuously monitored, including loss curve, convergence trend, accuracy, precision, recall, F1 score, task difficulty index and task balance index. Based on the performance indicators of the specified task, convergence efficiency index, generalization ability index, gradient stability index and task interaction influence index are analyzed.

[0012] This invention also provides a large language model control fine-tuning system based on multi-task collaborative regulation, comprising: The multi-task data encoding module is used to input the training dataset from the specified task into a shared large language model, assign a learnable task identifier vector to each task, and perform unified encoding preprocessing on the input data to generate task-aware feature representations. The collaborative intelligent control module is used to receive the task identifier vector and the task-aware feature representation through a learnable intelligent control network, generate a gating coefficient for dynamically adjusting the task gradient weights, and an initial dynamic evaluation signal for evaluating the current convergence state of the task. The dynamic balance control module is used to generate the final dynamic evaluation signal based on the initial dynamic evaluation signal, combined with multi-index fusion and historical difficulty index moving average processing, and then dynamically adjust the sampling probability and task-specific learning rate of the specified task in the training batch through the scheduling strategy. The joint optimization module is used to perform weighted fusion of the gradients of a specified task based on the gating coefficients and the scheduling results dynamically adjusted by the scheduling strategy, to form a joint optimization objective, and to perform backpropagation to synchronously update the parameters of the shared large language model and the parameters of the intelligent control network. The adaptive optimization module is used to continuously monitor the performance indicators and convergence status of a specified task during training iterations, and adjust the internal parameters of the intelligent control network and the scheduling strategy based on the monitoring results, thus forming an adaptive optimization mechanism.

[0013] As a preferred scheme for a large language model control fine-tuning system based on multi-task collaborative regulation, the multi-task data encoding module includes: The unified encoding preprocessing includes word segmentation formatting, feature encoding, and task identifier fusion. The word segmentation formatting process segments the input text using a word segmenter and uses a padding strategy to unify the length of the input sequence to a preset length L, generating a TokenID sequence. The feature encoding process inputs the TokenID sequence into a shared large language model encoder and extracts the encoder's hidden layer representation as the task's basic features. The task identifier fusion process extracts aggregated features from the basic features and concatenates them with a learnable task identifier vector to generate a task-aware unified feature representation.

[0014] As a preferred solution for a large language model control fine-tuning system based on multi-task collaborative regulation, the collaborative intelligent regulation module includes: The structure of the learnable intelligent control network is a lightweight feedforward neural network or attention network. The output gating coefficients are normalized to the range of [0,1] by the Sigmoid function, and the initial dynamic evaluation signal is mapped to a non-negative value by the Softplus function. The learnable intelligent control network is integrated inside the Transformer layer of the large language model. It achieves task-aware feature modulation by element-wise multiplying or adding the gating coefficients with the attention weights of the attention network or the activation values ​​of the feedforward network.

[0015] As a preferred scheme for a large language model control fine-tuning system based on multi-task collaborative regulation, the dynamic balance control module includes: The objects of the multi-indicator fusion include loss change rate, gradient magnitude, prediction entropy, and model output uncertainty index; the historical difficulty index moving average processing introduces the moving average of the historical difficulty index of the specified task to smooth instantaneous fluctuations. The scheduling strategy uses a temperature coefficient-based Softmax function to map the task difficulty index to the sampling probability, and reduces the temperature coefficient according to a preset annealing strategy during training, so that the sampling distribution shifts from uniform exploration to focus on high-difficulty tasks; the task-specific learning rate is adjusted by a learning rate factor.

[0016] As a preferred scheme for a large language model control fine-tuning system based on multi-task collaborative regulation, the adaptive optimization module includes: When the difficulty index of a task exceeds the first threshold for N consecutive training cycles, the task reweighting mechanism is triggered, which applies L1 or L2 norm regularization constraints to the gradient of the task, or directly applies a decay factor to the gating coefficient of the task. During the training iteration, an adaptive optimizer is configured separately for the learnable intelligent control network. When the convergence speed of a certain task is detected to be lower than the second threshold, the learning rate of the control parameters of the intelligent control network related to the task is reduced to weaken the parameter update amplitude and smooth the convergence process. Continuously monitor the performance metrics of a specified task, including loss curve, convergence trend, accuracy, precision, recall, F1 score, task difficulty index, and task balance index. Based on the performance metrics of the specified task, analyze convergence efficiency index, generalization ability index, gradient stability index, and task interaction impact index.

[0017] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the large language model control fine-tuning method based on multi-task collaborative regulation, as described in the first aspect or any possible implementation thereof.

[0018] Fourthly, the present invention provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform steps in the large language model control fine-tuning method based on multi-task collaborative regulation, as described in the first aspect or any possible implementation thereof.

[0019] The beneficial effects of the technical solution provided by this invention are as follows: First, this invention introduces a learnable intelligent control network that dynamically generates gating coefficients based on task identifier vectors and task-aware feature representations, replacing the traditional static weight strategy and adjusting the weights of each task gradient in the shared model parameter update. This network can be integrated into the Transformer layer of a large language model, achieving task-aware feature modulation through element-wise operations with attention weights or feedforward network activation values. This effectively reconciles the contradictions between different task gradient directions, guiding parameters to update in a common direction that is better for all tasks, thus ensuring the overall optimization effect and generalization boundary of the model.

[0020] Second, this invention generates a difficulty index that reflects the long-term convergence status of a task by combining multiple indicators such as loss change rate and gradient magnitude with a historical difficulty index moving average. Based on this, the scheduling strategy dynamically adjusts the sampling probability through a Softmax function with a temperature coefficient and adapts the learning rate to a specific learning rate through a learning rate factor, so that training resources are intelligently tilted towards high-difficulty, low-convergence tasks. This solves the problem of overfitting for simple tasks and underfitting for complex tasks caused by traditional fixed sampling / polling strategies, and ensures multi-task collaborative convergence.

[0021] Third, this invention monitors multi-dimensional performance indicators such as task loss curve, accuracy, F1 score, and difficulty index in real time, analyzes convergence efficiency, generalization ability, gradient stability, and the impact of task interactions, and can periodically or trigger-based adjust the intelligent control network parameters and key parameters of the scheduling strategy. At the same time, it configures an adaptive optimizer for the intelligent control network, which automatically reduces the learning rate of relevant control parameters when the task convergence speed is too low, and triggers gradient regularization or gating coefficient decay when the task difficulty index continuously exceeds the threshold, so as to avoid training oscillation, overfitting, or divergence, and realize self-correction and continuous optimization in the training process.

[0022] Fourth, based on the intelligent control network signal decision, the scheduling results and gating coefficients work together to achieve joint optimization, forming a dual mechanism of data-level sampling control and optimization-level gradient adjustment. At the same time, the modular architecture supports the flexible integration of the intelligent control network into Transformer layers of different depths, and the scheduling strategy can be customized according to the scenario. It can adapt to various multi-task training scenarios such as natural language processing, question answering and summarizing generation, dialogue systems, and multimodal tasks, providing a unified and efficient solution for LLM fine-tuning in different fields.

[0023] Fifth, this invention can accelerate the overall convergence speed across multiple tasks and reduce ineffective training iterations caused by parameter oscillations and resource waste; at the same time, it ensures the performance balance of the model across tasks, avoiding the drag of single-task performance on overall generalization ability. Its ability to adapt to multiple tasks with a single training iteration, combined with an efficient fine-tuning mechanism, can reduce the cost of model deployment and maintenance in multi-task scenarios, providing high-quality and highly reliable LLM support for applications such as industrial-grade intelligent assistants, content generation, and multimodal interaction. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are merely embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the process of a large language model control fine-tuning method based on multi-task collaborative regulation provided in an embodiment of the present invention; Figure 2 This is a flowchart of the multi-task input data processing and encoding process in the large language model control fine-tuning method based on multi-task collaborative regulation provided in this embodiment of the invention. Figure 3 This is a flowchart of the collaborative intelligent control process in the large language model control fine-tuning method based on multi-task collaborative control provided in the embodiments of the present invention. Figure 4The flowchart of dynamic balance control in the large language model control fine-tuning method based on multi-task collaborative regulation provided in the embodiments of the present invention; Figure 5 The flowchart of joint optimization and backpropagation in the large language model control fine-tuning method based on multi-task collaborative regulation provided in the embodiments of the present invention; Figure 6 This is a flowchart of the feedback optimization process in the large language model control fine-tuning method based on multi-task collaborative regulation provided in the embodiments of the present invention. Figure 7 This is a schematic diagram of the architecture of a large language model control fine-tuning system based on multi-task collaborative regulation provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0027] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this invention should have the ordinary meaning understood by those skilled in the art to which this invention pertains. The terms "comprising" or "including," or similar words used in the embodiments of this invention, mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects.

[0028] With the widespread application and continuous breakthroughs of Large Language Models (LLMs) in natural language understanding and generation, their multi-task learning and generalization capabilities have become key indicators for measuring the core competitiveness of models. Through pre-training on large-scale corpora, LLMs acquire a powerful foundation of language representations, while multi-task fine-tuning further enables them to transfer learned general knowledge to multiple downstream tasks. Specifically, in various natural language processing scenarios such as text understanding, question answering, summary generation, dialogue systems, and sentiment analysis, LLMs can achieve knowledge interaction and reuse by sharing the underlying network structure and parameters, significantly improving the model's generalization performance and task adaptability. This "train once, apply to multiple tasks" characteristic not only reduces the cost of model deployment and maintenance but also demonstrates great potential in open-domain applications, intelligent assistants, content generation, and other industrial scenarios, as well as in academic research directions such as few-shot / zero-shot learning.

[0029] However, despite the significant theoretical advantages of multi-task learning, existing multi-task fine-tuning methods based on large language models still face a series of key technical bottlenecks in practical applications due to outdated collaborative mechanisms, which restrict their full performance potential. First, static collaborative strategies struggle to resolve gradient conflicts. Due to fundamental differences in learning objectives, data distributions, and decision boundaries across different tasks, the gradient directions generated during parameter updates may be inconsistent or even contradictory. Existing methods often rely on manually preset static loss weights or simple gradient operations; these fixed collaborative rules cannot adapt to the dynamically changing gradient relationships during training. For example, even in the field of model fusion, the adaptive fusion method mentioned in patent CN120597991A is essentially based on a weighted combination of multiple vertical-domain models using fixed strategies, without addressing the dynamic coordination of gradients during training. Similarly, the modal expansion method based on parameter fusion and decoupling proposed in patent CN120105352B, while involving model parameter adjustment, still focuses on fixed strategy combinations at the model level, rather than dynamic gradient coordination during training. Recent research, such as the training-free model merging scheme proposed by Pcb-Merging, adjusts parameter coefficients by balancing parameter competition, but its intervention occurs after training is complete, failing to resolve real-time gradient conflicts during training. Furthermore, while data-free knowledge fusion methods merge models in the parameter space, as static, post-hoc fusion strategies, they still cannot achieve dynamic gradient coordination during training. Sentiment classification and entity recognition tasks compete at the feature extraction level, and static strategies cannot perceive and reconcile this contradiction that changes with the training phase. This causes model parameters to oscillate during optimization, creating a "seesaw" effect that ultimately affects the overall optimization performance and generalization boundary of the model.

[0030] Secondly, traditional methods lack the ability to pursue and control dynamic equilibrium. Different tasks exhibit significant heterogeneity in sample size, learning difficulty, and convergence dynamics. For example, when training text classification and machine translation tasks simultaneously, the former typically involves a large amount of data and converges quickly, while the latter has a complex structure and slow training speed. Existing resource allocation strategies based on fixed sampling or polling are blind and cannot be adjusted according to the real-time convergence status of the task. This leads to the model being prone to overfitting to simple tasks or tasks with large sample sizes, while underfitting to complex tasks or tasks with small sample sizes. The result is not balanced convergence, but rather a static imbalance in performance, making it difficult to achieve the expected goal of collaborative optimization. This is indirectly reflected in studies such as "Multi-dimensional FeatureFusion-based Runtime Prediction Approach for Cloud Workflow Tasks," where multi-dimensional feature fusion primarily serves prediction accuracy rather than the dynamic balance of resources and attention during training. It is worth noting that the adaptive learning rate optimization method based on gradient variance and temporal decay proposed in patent CN120297350A, although involving dynamic parameter adjustment, mainly targets the optimization of a single model or task and fails to address the dynamic balance problem of resources and attention across multiple tasks. Similarly, the Pcb-Merging method achieves post-processing parameter balancing by calculating task vector similarity, rather than real-time dynamic resource allocation during training. From a cross-domain perspective, the study "Micro-Conflict Analysis in Collaborative Governance" points out that conflict can be seen as a dynamic factor rather than a static obstacle. This further corroborates the need for dynamic regulation to transform inter-task competition into a manageable factor in multi-task learning, rather than adopting static strategies.

[0031] Furthermore, the lack of closed-loop self-evolution capabilities in existing collaborative frameworks is another key factor restricting the efficiency of multi-task learning. The static weights or heuristic scheduling strategies used in traditional multi-task fine-tuning have fixed internal logic after training begins. For example, fixed-weight loss weighting methods cannot perceive changes in task difficulty at different training stages; the formula of uncertainty-based weighting rules cannot be optimized. Even the task-adaptive gating network mentioned in patent CN118194917B relies primarily on predefined mechanisms (such as z-loss penalty and expert competition) for adjustment, lacking the ability to iterate backward based on the global training state. In contrast, the data-driven robot closed-loop joint optimization method proposed in patent CN119328777A, while forming a closed-loop system through reinforcement learning, is mainly applied to optimizing robot task execution strategies, rather than the self-evolution of optimization strategies in large language model multi-task fine-tuning itself. Similarly, the Pcb-Merging method uses intelligent optimization algorithms to search for mixing coefficients, but this is external hyperparameter optimization during the model fusion stage, rather than the self-adjustment of internal network parameters during training. While recent paradigms for training-free model merging (such as the ZipIt! method) offer an alternative approach, their core principle is to merge pre-trained models, similarly avoiding dynamic optimization and evolution within the training loop. The MuDSC framework, a prime example of training-free merging, further confirms that such methods, by avoiding the training process, naturally cannot achieve dynamic optimization and evolution within the training loop. This one-time, unchanging collaborative model cannot intelligently respond and self-correct based on the actual state of each task during training. This not only wastes computational resources but also leads to the optimization process getting stuck in local optima, manifesting as severe oscillations in training loss, slow convergence, and even training divergence.

[0032] In summary, in complex and heterogeneous multi-task application scenarios, the fundamental limitation of existing methods lies in the static, superficial, and unadaptive nature of their collaborative mechanisms. How to break through the traditional, rigid collaborative paradigm and establish a dynamic equilibrium convergence control mechanism capable of real-time task status perception, intelligent decision-making, and precise resource scheduling, while possessing self-optimization capabilities, has become a core research direction for improving the performance and practicality of multi-task fine-tuning of large language models. Therefore, the main objective of this invention is to provide a dynamic equilibrium convergence control method and system for large language models based on multi-task collaborative intelligent regulation. By introducing a learnable intelligent regulation mechanism and a dynamic equilibrium scheduling strategy, a closed-loop optimization system integrating perception, decision-making, and execution is constructed to solve the problems of gradient conflict, convergence instability, and performance imbalance in existing multi-task fine-tuning methods. The following are the specific contents of the embodiments of this invention.

[0033] See Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6 This invention provides a method for fine-tuning a large language model based on multi-task collaborative regulation, comprising the following steps: S1. Multi-task input data processing and encoding: The system first receives training datasets from K different tasks. For the raw input data of the k-th task... The system executes a unified, standardized processing procedure, which specifically includes the following steps: First, perform word segmentation and formatting:

[0034] The input text is segmented using a tokenizer, and a padding strategy is employed to unify the length of the input sequence to L, generating corresponding TokenID sequences. This ensures that the data input dimensions are consistent across different tasks, facilitating subsequent batch calculations and feature alignment.

[0035] Secondly, perform feature encoding:

[0036] The standardized sequence Input to a shared large language model encoder In this process, the corresponding hidden layer representation is extracted to obtain the feature representation corresponding to the task. This process realizes a non-linear mapping from the input text to a high-dimensional feature space, providing a foundation for feature sharing between tasks.

[0037] Subsequently, the task identifiers were merged:

[0038] Assign a learnable embedding vector to each task. and from the feature representation Extract aggregated features (e.g., the hidden vector corresponding to the [CLS] label) The two are concatenated to generate a unified feature representation for task awareness. =Concat( , This representation integrates task features and semantic context information, preserving both shared features between tasks and the specificity of each task.

[0039] Through the above step S1, standardized preprocessing and unified feature encoding of multi-task input data are achieved, providing a standardized input interface for the task gating and difficulty adaptive scheduling module.

[0040] S2, Cooperative intelligent control signal generation: In step S2, the system performs collaborative intelligent control signal generation. This is based on the task-aware feature representation obtained in step S1. The system uses a learnable intelligent control network (denoted as...) ) Calculate the dynamic control signal corresponding to each task to achieve collaborative optimization and adaptive balance among tasks.

[0041] First, perform signal calculations:

[0042] Intelligent control network for input features The process is performed, and the gating signal and evaluation signal for this task are output, denoted as follows: and These two signals are used for dynamic adjustment of the task gradient and real-time evaluation of the training status, respectively.

[0043] Secondly, perform signal normalization:

[0044] For the gating signal section, the system will use the initial gating input. After normalization using the Sigmoid activation function, the coefficients are mapped to the (0,1) interval to obtain the final dynamic gating coefficients. This coefficient is used to finely control the gradient weights of tasks during the shared model parameter update process, thereby achieving dynamic balance and conflict mitigation among tasks. For the evaluation signal section, the system will initially evaluate the input. After positive mapping using the Softplus function, a non-negative dynamic evaluation signal is obtained. :

[0045] This signal reflects the convergence status and learning progress of the task, and is used for subsequent task difficulty estimation and resource scheduling strategy adjustment. Through the above signal calculation and normalization process, the system generates collaborative intelligent control signals that can be updated in real time. These signals provide the core basis for subsequent gradient weighted optimization and adaptive allocation of training resources, ensuring the stability and efficiency of the multi-task training process.

[0046] S3, Joint Optimization and Backpropagation: In step S3, the system performs joint optimization and backpropagation to achieve collaborative learning among multiple tasks and dynamic updates of shared parameters. This step mainly includes two parts: weighted total loss calculation and synchronous backpropagation and parameter update.

[0047] First, a weighted total loss calculation is performed, and the system calculates the original losses for each task. With the corresponding gating coefficient Multiply and then sum to form a joint optimization objective function:

[0048] This objective function introduces task-gated weights to achieve adaptive contributions of different tasks to the overall optimization, enabling the weight allocation among tasks to be dynamically adjusted according to the model training state.

[0049] Secondly, synchronous backpropagation and parameter updates are performed. The system calculates the weighted total loss. Then, backpropagation is performed using it as the optimization objective to calculate the gradient. The model parameters here. It is a composite parameter set that also includes the main model parameters. Parameters of the task gating weight control network The parameter update process can be represented as:

[0050] In the formula, Indicates the first The composite model parameters (including the main model and control network parameters) at the next iteration. This indicates the updated parameters. For learning rate, This is the gradient of the total loss with respect to the parameters. Through the above update mechanism, the system synchronously optimizes the main model and the task control network in each iteration. The main model learns how to complete multiple tasks, while the control network learns how to dynamically allocate task weights. The two evolve together to achieve a joint improvement in multi-task performance.

[0051] S4. Monitoring and Feedback: During training, the system continuously monitors the dynamics of each task to provide a basis for intelligent control and optimization decisions. This process includes two core components: performance monitoring and convergence state evaluation.

[0052] During the performance monitoring phase, the system periodically records the losses of each task. and performance metrics on the validation set Parameters such as accuracy and F1 score are used to reflect the immediate training performance of the task and the generalization ability of the model.

[0053] During the convergence state evaluation phase, the system integrates multiple real-time metrics to calculate the task difficulty index for each task. This is used to quantify the convergence difficulty and training status of the task. The calculation process is as follows:

[0054] In the formula, This represents a negative value indicating the rate at which the loss decreases; the slower the loss decreases, the smaller this value will be. This represents the magnitude of the gradient, used to measure the drasticness of the task parameter updates; Represents the predicted distribution The information entropy reflects the uncertainty of the model in the current task; To balance the weights, it is used to control the relative influence of different indicators in the comprehensive evaluation.

[0055] Through the above evaluation mechanism, the system can identify tasks that converge slowly or perform unstablely during training in real time, providing a quantitative basis for subsequent scheduling strategy adjustments and resource allocation, thereby achieving dynamic training optimization and self-balancing regulation among multiple tasks.

[0056] S5, Dynamic Balance Convergence Control: Convergence Dilemma Index Based on S4 Evaluation The system implements dynamic balance control through a scheduling strategy.

[0057] The resource scheduling decision-making process is as follows: S51. Dynamic Sampling Probability: Unlike the standard Softmax function, this embodiment uses a temperature coefficient. The Softmax function maps the dilemma index to the sampling probability of each task in the next training batch. This directs resources toward highly challenging tasks.

[0058] S52. Differentiated Learning Rate: Calculate a unique learning rate factor for each task based on the difficulty index. And adjust its effective learning rate accordingly. :

[0059] S53, Closed-loop feedback: Update the sampling probability and learning rate configuration Feedback is sent to the training data loader and optimizer. Simultaneously, monitoring results can also be used to trigger adjustments to the intelligent network. Fine-tuning of internal parameters.

[0060] The application scenarios of the method of this invention are as follows: In multi-task fine-tuning of natural language processing, this invention can simultaneously train sentiment analysis, news classification, and named entity recognition tasks. The sentiment analysis task uses the SST-2 dataset, and the model aims to predict the sentiment (positive or negative) of movie reviews. The system dynamically calculates task gradient weights through a task gating network. For tasks with slow convergence during training, its gradient influence is increased, and its sampling probability and learning rate are improved through a difficulty-adaptive strategy, thereby enhancing the model's overall performance across all tasks. The news classification task uses the AGNews dataset, classifying news texts into four categories. The system adjusts the gradient contribution through a gating network to minimize gradient interference between tasks, while maintaining training efficiency and model convergence stability. The named entity recognition task uses the CoNLL-2003 dataset to identify entities such as person names, place names, and organization names in text. For entity recognition tasks with slow convergence, the system appropriately increases the sampling frequency and learning rate to ensure sufficient training resources for this task, achieving training balance with other tasks.

[0061] In question-answering and summarization scenarios, this invention can simultaneously train extractive question answering, generative question answering, and text summarization tasks. The extractive question answering task uses the SQuAD dataset to extract answers from text based on questions. A task-gating network dynamically calculates the gradient contribution, ensuring the model's accuracy in the extraction task. The generative question answering task uses the NaturalQuestions dataset to generate natural language answers based on questions. The system uses a difficulty-adaptive strategy to identify tasks with higher training difficulty, assigning them higher learning rates and sampling weights to ensure fully optimized generation capabilities. The text summarization task uses the CNN / DailyMail dataset to generate concise summaries of articles. A gating network adjusts gradient weights to reduce interference with the question-answering task, thereby achieving collaborative optimization across multiple tasks.

[0062] In training the dialogue system, this invention can simultaneously train intent recognition, slot filling, and dialogue generation tasks. The intent recognition task uses the SNIPS dataset to map user input to intent categories. The system appropriately reduces the sampling probability of the intent category based on the task's convergence, freeing up training resources for other tasks. The slot filling task also uses the SNIPS dataset to identify slot information in user input. A task gating network controls its gradient contribution, maintaining a balance among tasks. The dialogue generation task uses the Persona-Chat dataset to generate natural responses based on context. The system increases its learning rate and sampling frequency through a difficulty-adaptive strategy, ensuring the generation task receives sufficient training resources and improving overall dialogue quality.

[0063] In multimodal task scenarios, this invention can simultaneously train text understanding and image description generation tasks. The text understanding task can be trained using a news classification dataset, and the system adjusts the gradient contribution of the text task through a gating network. The image description generation task uses the MSCOCO dataset to generate descriptive text from images. The system recognizes that this task is relatively difficult to train, so its learning rate and sampling frequency are appropriately increased, while the text task gradient is balanced, achieving efficient training and overall performance optimization for multimodal tasks.

[0064] It should be noted that the method of this embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this embodiment, and these multiple devices will interact with each other to complete the large language model control fine-tuning method based on multi-task collaborative regulation.

[0065] It should be noted that the above description describes some embodiments of the present invention. In some cases, the described actions or steps can be performed in a different order than that shown in the above embodiments and the desired result can still be achieved. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0066] See Figure 7 Based on the same inventive concept, and corresponding to the methods of any of the above embodiments, this invention also provides a large language model control fine-tuning system based on multi-task collaborative regulation, comprising: The multi-task data encoding module 100 is used to input the training dataset from the specified task into a shared large language model, assign a learnable task identifier vector to each task, and perform unified encoding preprocessing on the input data to generate task-aware feature representations. The collaborative intelligent control module 200 is used to receive the task identifier vector and the task-aware feature representation through a learnable intelligent control network, generate a gating coefficient for dynamically adjusting the task gradient weights, and an initial dynamic evaluation signal for evaluating the current convergence state of the task. The dynamic balance control module 300 is used to generate a final dynamic evaluation signal based on the initial dynamic evaluation signal, combined with multi-index fusion and historical difficulty index moving average processing, and then dynamically adjust the sampling probability and task-specific learning rate of the specified task in the training batch through the scheduling strategy. The joint optimization module 400 is used to perform weighted fusion of the gradient of the specified task based on the gating coefficient and the scheduling result after dynamic adjustment by the scheduling strategy to form a joint optimization objective, and to perform backpropagation to synchronously update the parameters of the shared large language model and the parameters of the intelligent control network. The adaptive optimization module 500 is used to continuously monitor the performance indicators and convergence status of a specified task during the training iteration process, and adjust the internal parameters of the intelligent control network and the scheduling strategy based on the monitoring results to form an adaptive optimization mechanism.

[0067] In this embodiment, the multi-task data encoding module 100 includes: The unified encoding preprocessing includes word segmentation formatting, feature encoding, and task identifier fusion. The word segmentation formatting process segments the input text using a word segmenter and uses a padding strategy to unify the length of the input sequence to a preset length L, generating a TokenID sequence. The feature encoding process inputs the TokenID sequence into a shared large language model encoder and extracts the encoder's hidden layer representation as the task's basic features. The task identifier fusion process extracts aggregated features from the basic features and concatenates them with a learnable task identifier vector to generate a task-aware unified feature representation.

[0068] In this embodiment, the collaborative intelligent control module 200 includes: The structure of the learnable intelligent control network is a lightweight feedforward neural network or attention network. The output gating coefficients are normalized to the range of [0,1] by the Sigmoid function, and the initial dynamic evaluation signal is mapped to a non-negative value by the Softplus function. The learnable intelligent control network is integrated inside the Transformer layer of the large language model. It achieves task-aware feature modulation by element-wise multiplying or adding the gating coefficients with the attention weights of the attention network or the activation values ​​of the feedforward network.

[0069] In this embodiment, the dynamic balance control module 300 includes: The objects of the multi-indicator fusion include loss change rate, gradient magnitude, prediction entropy, and model output uncertainty index; the historical difficulty index moving average processing introduces the moving average of the historical difficulty index of the specified task to smooth instantaneous fluctuations. The scheduling strategy uses a temperature coefficient-based Softmax function to map the task difficulty index to the sampling probability, and reduces the temperature coefficient according to a preset annealing strategy during training, so that the sampling distribution shifts from uniform exploration to focus on high-difficulty tasks; the task-specific learning rate is adjusted by a learning rate factor.

[0070] In this embodiment, the adaptive optimization module 500 includes: When the difficulty index of a task exceeds the first threshold for N consecutive training cycles, the task reweighting mechanism is triggered, which applies L1 or L2 norm regularization constraints to the gradient of the task, or directly applies a decay factor to the gating coefficient of the task. During the training iteration, an adaptive optimizer is configured separately for the learnable intelligent control network. When the convergence speed of a certain task is detected to be lower than the second threshold, the learning rate of the control parameters of the intelligent control network related to the task is reduced to weaken the parameter update amplitude and smooth the convergence process. Continuously monitor the performance metrics of a specified task, including loss curve, convergence trend, accuracy, precision, recall, F1 score, task difficulty index, and task balance index. Based on the performance metrics of the specified task, analyze convergence efficiency index, generalization ability index, gradient stability index, and task interaction impact index.

[0071] The apparatus in the above embodiments is used to implement a large language model control fine-tuning method based on multi-task collaborative regulation in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0072] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the large language model control fine-tuning method based on multi-task collaborative regulation described in any of the above embodiments.

[0073] Figure 8 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 610, a memory 620, an input / output interface 630, a communication interface 640, and a bus 650. The processor 610, memory 620, input / output interface 630, and communication interface 640 are interconnected internally via the bus 650.

[0074] The processor 610 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0075] The memory 620 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 620 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 620 and is called and executed by the processor 610.

[0076] The input / output interface 630 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0077] The communication interface 640 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0078] Bus 650 includes a pathway for transmitting information between various components of the device, such as processor 610, memory 620, input / output interface 630, and communication interface 640.

[0079] It should be noted that although the above-described device only shows the processor 610, memory 620, input / output interface 630, communication interface 640, and bus 650, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0080] The electronic device described in the above embodiments is used to implement a large language model control fine-tuning method based on multi-task collaborative regulation in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0081] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, the present invention also provides a non-transitory computer-readable storage medium storing computer instructions, which are used to cause the computer to execute a large language model control fine-tuning method based on multi-task collaborative regulation as described in any of the above embodiments.

[0082] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0083] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute a large language model control fine-tuning method based on multi-task collaborative regulation as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0084] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention is limited to these examples; within the framework of the invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity.

[0085] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of the invention, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of the invention, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of the invention will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of the invention, it will be apparent to those skilled in the art that the embodiments of the invention may be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0086] Although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAMDRAM) may be used with the embodiments discussed.

[0087] The embodiments of this invention are intended to cover all such substitutions, modifications, and variations falling within the scope of the claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this invention should be included within the scope of protection of this invention.

Claims

1. A large language model control fine-tuning method based on multi-task collaborative regulation, characterized in that, include: The training dataset from the specified task is input into a shared large language model, a learnable task identifier vector is assigned to each task, and the input data is uniformly encoded and preprocessed to generate task-aware feature representations. Through a learnable intelligent control network, the task identifier vector and the task-aware feature representation are received, and a gating coefficient for dynamically adjusting the task gradient weights and an initial dynamic evaluation signal for evaluating the current convergence state of the task are generated. Based on the initial dynamic evaluation signal, the final dynamic evaluation signal is generated by combining multi-index fusion and historical difficulty index moving average processing. Then, the sampling probability and task-specific learning rate of the specified task in the training batch are dynamically adjusted by the scheduling strategy. Based on the gating coefficient and the scheduling result after dynamic adjustment by the scheduling strategy, the gradient of the specified task is weighted and fused to form a joint optimization objective, and backpropagation is performed to synchronously update the parameters of the shared large language model and the parameters of the intelligent control network. During the training iteration, the performance indicators and convergence status of the specified task are continuously monitored, and the internal parameters of the intelligent control network and the scheduling strategy are adjusted according to the monitoring results to form an adaptive optimization mechanism.

2. The large language model control fine-tuning method based on multi-task collaborative regulation according to claim 1, characterized in that, The unified encoding preprocessing includes word segmentation formatting, feature encoding and task identification fusion. The word segmentation formatting process segments the input text into words using a word segmenter and uses a padding strategy to unify the length of the input sequence to a preset length L, generating a TokenID sequence. The feature encoding inputs the TokenID sequence into a shared large language model encoder and extracts the encoder hidden layer representation as the task's basic features. The task identifier fusion extracts aggregated features from basic features and concatenates them with learnable task identifier vectors to generate a unified feature representation for task awareness.

3. The method for fine-tuning large language models based on multi-task collaborative regulation according to claim 1, characterized in that, The structure of the learnable intelligent control network is a lightweight feedforward neural network or attention network. The output gating coefficients are normalized to the range of [0,1] by the Sigmoid function, and the initial dynamic evaluation signal is mapped to a non-negative value by the Softplus function. The learnable intelligent control network is integrated inside the Transformer layer of the large language model. It achieves task-aware feature modulation by element-wise multiplying or adding the gating coefficients with the attention weights of the attention network or the activation values ​​of the feedforward network.

4. The method for fine-tuning large language models based on multi-task collaborative regulation according to claim 1, characterized in that, The objects of the multi-indicator fusion include loss change rate, gradient magnitude, prediction entropy and model output uncertainty index; the historical difficulty index moving average processing introduces the moving average of the historical difficulty index of the specified task to smooth instantaneous fluctuations.

5. The method for fine-tuning large language models based on multi-task collaborative regulation according to claim 1, characterized in that, The scheduling strategy uses a temperature coefficient-based Softmax function to map the task difficulty index to the sampling probability, and reduces the temperature coefficient according to a preset annealing strategy during training, so that the sampling distribution shifts from uniform exploration to focus on high-difficulty tasks; the task-specific learning rate is adjusted by a learning rate factor.

6. The method for fine-tuning large language models based on multi-task collaborative regulation according to claim 1, characterized in that, When the difficulty index of a task exceeds the first threshold for N consecutive training cycles, the task reweighting mechanism is triggered, which applies L1 or L2 norm regularization constraints to the gradient of the task, or directly applies a decay factor to the gating coefficient of the task. During the training iteration, an adaptive optimizer is configured separately for the learnable intelligent control network. When the convergence speed of a certain task is detected to be lower than the second threshold, the learning rate of the control parameters of the intelligent control network related to the task is reduced to weaken the parameter update amplitude and smooth the convergence process.

7. The method for fine-tuning large language model control based on multi-task collaborative regulation according to claim 1, characterized in that, Continuously monitor the performance metrics of a specified task, including loss curve, convergence trend, accuracy, precision, recall, F1 score, task difficulty index, and task balance index. Based on the performance metrics of the specified task, analyze convergence efficiency index, generalization ability index, gradient stability index, and task interaction impact index.

8. A large language model control fine-tuning system based on multi-task collaborative regulation, characterized in that, include: The multi-task data encoding module is used to input the training dataset from the specified task into a shared large language model, assign a learnable task identifier vector to each task, and perform unified encoding preprocessing on the input data to generate task-aware feature representations. The collaborative intelligent control module is used to receive the task identifier vector and the task-aware feature representation through a learnable intelligent control network, generate a gating coefficient for dynamically adjusting the task gradient weights, and an initial dynamic evaluation signal for evaluating the current convergence state of the task. The dynamic balance control module is used to generate the final dynamic evaluation signal based on the initial dynamic evaluation signal, combined with multi-index fusion and historical difficulty index moving average processing, and then dynamically adjust the sampling probability and task-specific learning rate of the specified task in the training batch through the scheduling strategy. The joint optimization module is used to perform weighted fusion of the gradients of a specified task based on the gating coefficients and the scheduling results dynamically adjusted by the scheduling strategy, to form a joint optimization objective, and to perform backpropagation to synchronously update the parameters of the shared large language model and the parameters of the intelligent control network. The adaptive optimization module is used to continuously monitor the performance indicators and convergence status of a specified task during training iterations, and adjust the internal parameters of the intelligent control network and the scheduling strategy based on the monitoring results, thus forming an adaptive optimization mechanism.

9. The large language model control fine-tuning system based on multi-task collaborative regulation according to claim 8, characterized in that, In the multi-task data encoding module: The unified encoding preprocessing includes word segmentation formatting, feature encoding, and task identifier fusion. The word segmentation formatting process segments the input text using a word segmenter and uses a padding strategy to unify the length of the input sequence to a preset length L, generating a TokenID sequence. The feature encoding process inputs the TokenID sequence into a shared large language model encoder and extracts the encoder's hidden layer representation as the task's basic features. The task identifier fusion process extracts aggregated features from the basic features and concatenates them with a learnable task identifier vector to generate a task-aware unified feature representation. In the collaborative intelligent control module: The structure of the learnable intelligent control network is a lightweight feedforward neural network or attention network. The output gating coefficients are normalized to the range of [0,1] by the Sigmoid function, and the initial dynamic evaluation signal is mapped to a non-negative value by the Softplus function. The learnable intelligent control network is integrated inside the Transformer layer of the large language model. It achieves task-aware feature modulation by element-wise multiplying or adding the gating coefficients with the attention weights of the attention network or the activation values ​​of the feedforward network. In the dynamic balance control module: The objects of the multi-indicator fusion include loss change rate, gradient magnitude, prediction entropy, and model output uncertainty index; the historical difficulty index moving average processing introduces the moving average of the historical difficulty index of the specified task to smooth instantaneous fluctuations. The scheduling strategy uses a temperature coefficient-based Softmax function to map the task difficulty index to the sampling probability, and reduces the temperature coefficient according to a preset annealing strategy during training, so that the sampling distribution shifts from uniform exploration to focus on high-difficulty tasks; the task-specific learning rate is adjusted by a learning rate factor.

10. The large language model control fine-tuning system based on multi-task collaborative regulation according to claim 8, characterized in that, In the adaptive optimization module: When the difficulty index of a task exceeds the first threshold for N consecutive training cycles, the task reweighting mechanism is triggered, which applies L1 or L2 norm regularization constraints to the gradient of the task, or directly applies a decay factor to the gating coefficient of the task. During the training iteration, an adaptive optimizer is configured separately for the learnable intelligent control network. When the convergence speed of a certain task is detected to be lower than the second threshold, the learning rate of the control parameters of the intelligent control network related to the task is reduced to weaken the parameter update amplitude and smooth the convergence process. Continuously monitor the performance metrics of a specified task, including loss curve, convergence trend, accuracy, precision, recall, F1 score, task difficulty index, and task balance index. Based on the performance metrics of the specified task, analyze convergence efficiency index, generalization ability index, gradient stability index, and task interaction impact index.

Images