Multi-model collaboration method used in low-data environment

Abstract

The invention discloses a multi-model collaboration method used in a low data environment, which comprises the following steps of: constructing an expert model pool, and determining an initial position, an initial speed and a utility value of each expert model; dynamically updating the speed and position of each expert model in the expert model pool through collaborative search, and evaluating the global optimal position of the expert model pool and the personal optimal position of each expert model; in the collaborative search process, the collaborative search path of each expert model in the expert model pool is dynamically adjusted, and each expert model is optimized; judging whether the current collaborative search process is ended or not according to convergence states of all expert models, and if so, returning a global optimal model of the expert model pool as a final optimization result; and in the expert model pool, selecting the expert model of which the utility value is greater than the utility threshold as a teacher model, selecting the expert model of which the utility value is smaller than the utility threshold and which needs the same task type as a student model, and performing knowledge distillation guidance on the student model through the teacher model so as to optimize the student model.

Description

Technical Field

[0001] The present invention relates to the field of multi-model collaboration technology, and in particular to a multi-model collaboration method for use in a low-data environment. Background Art

[0002] Large language models (LLMs), such as GPT, BERT, and BLOOM, have demonstrated powerful capabilities in the field of natural language processing (NLP) and have made significant progress in many tasks. These models have acquired rich knowledge reserves through pre-training, so they perform well in unsupervised learning, few-shot learning, and zero-shot learning. Although LLMs have general language understanding capabilities, in practical applications, it is difficult for a single model to consistently maintain high performance across domains and tasks. Different tasks often have different language characteristics, semantic requirements, and knowledge domains (such as medicine, law, science, etc.), requiring LLMs to be adaptively adjusted to better meet the requirements of specific tasks. In multi-task domains, LLMs usually need to handle different types of tasks at the same time, and even need to share knowledge and parameters between different tasks. Multi-task learning can help models achieve synergistic effects between multiple tasks, but it also brings complexity in model training, tuning, and deployment. LLMs that adapt to multi-task domains sometimes also involve cross-domain collaborative optimization, and the model needs to be fine-tuned in a targeted manner to meet the needs of specific application domains. In order to adapt to new tasks and multi-task fields, some current research methods include task-adaptive fine-tuning, mixed expert models (MoE), model combination and weighted fusion. These methods usually require a large amount of data resources and computing resources to support the switching and optimization of models between multiple tasks through multiple iterative training or routing selection of expert models. Existing methods have the following technical problems: Current LLM adaptation methods often rely on a large amount of labeled data for effective task fine-tuning. However, in some specific domains or few-sample tasks, the lack of sufficient training data leads to poor adaptive fine-tuning results. In addition, the high cost of data preparation and annotation limits the adaptability of LLM in low-data environments.

[0003] LLMs that adapt to multi-task domains usually require multiple iterations of model training and updating. This not only consumes significant computing resources, but also increases the difficulty of deployment, especially when the model is large. For example, the training of mixed expert models requires a specific routing mechanism to assign different tasks to corresponding experts to improve performance, but the routing overhead is high and the flexibility is limited.

[0004] In multi-task learning, the model needs to share parameters and knowledge between tasks. However, the requirements of multiple tasks are usually different, which may cause the model to optimize one task while damaging the performance of other tasks, which is called task interference. This makes it difficult for LLM to maintain efficient versatility and accuracy in multi-task scenarios.

[0005] Most existing adaptation methods lack flexibility and are difficult to achieve dynamic modular combination. Taking "weighted model fusion" as an example, this type of method usually assumes a fixed combination of model weights and lacks the ability to adjust in real time according to task requirements. For multi-task fields, flexible combination and modular adjustment are the key to effectively adapt to task changes, but existing methods show obvious limitations in this regard.

[0006] In practical applications, the model needs to have strong generalization capabilities to adapt to new tasks or domains that have not been seen before. However, a single LLM still has significant deficiencies in cross-domain and cross-task adaptability, and it is difficult to maintain consistent high performance between different tasks. At the same time, task adaptation needs to be completed as efficiently as possible with limited resources, and existing methods perform poorly in low-data, low-computing environments.

[0007] The purpose of this invention is to design a multi-model collaboration method for low-data environments in response to the above-mentioned problems in the prior art. Summary of the invention

[0008] In view of this, the purpose of the present invention is to propose a multi-model collaboration method for low-data environment, which can solve the above-mentioned problems.

[0009] The present invention provides a multi-model collaboration method for low-data environment, comprising: Build an expert model pool and determine the initial position, initial speed and utility value of each expert model; Dynamically update the speed and position of each expert model in the expert model pool through collaborative search, and evaluate the global best position of the expert model pool and the personal best position of each expert model; During the collaborative search process, the collaborative search path of each expert model in the expert model pool is dynamically adjusted to optimize each expert model; Determine whether the current collaborative search process is finished based on the convergence status of all expert models. If so, return the global best model in the expert model pool as the final optimization result. In the expert model pool, the expert model with a utility value greater than the utility threshold is selected as the teacher model, and the expert model with a utility value less than the utility threshold that requires the same task type is selected as the student model. The teacher model is used to guide the student model through knowledge distillation to optimize the student model.

[0010] Furthermore, the expert model pool is constructed to determine the initial position, initial speed and utility value of each expert model, including: Select several large language expert models with different task expertise to build a starting expert model pool; The representation of the expert model weight in the parameter space is defined as the initial position of the expert model, and the expert model randomly points to another expert model as the initial speed of the expert model; The utility value of the expert model on a given task or multiple tasks is evaluated through the utility function. The utility function is expressed as follows: , in, Indicates that the model i The utility value on the subtasks, is the weight of the corresponding task, m The total number of tasks.

[0011] Further, dynamically updating the speed and position of each expert model in the expert model pool through collaborative search, and evaluating the global best position of the expert model pool and the personal best position of each expert model includes: The speed of updating the expert model is determined by the inertia of the expert model, the personal best position, and the global best position. The update rules are as follows: , in, is the velocity vector of expert model i, is the inertia weight, , , are the attraction or repulsion coefficients for the personal best, the global best, and the global worst, respectively, , , is a factor randomly drawn from a uniform distribution in [0,1], is the personal optimal position of expert model i, is the global best position, is the current position of expert model i; The position of the expert model is updated by the speed after the expert model is updated. The update formula is as follows: , in, is the step size coefficient, is the current position of expert model i; After each position update, the utility value of each expert model at the new position is calculated, and the personal and global best positions are updated according to the utility value.

[0012] Furthermore, the update formula of the personal best position of the expert model i is as follows: , in, represents the personal best position of expert model i at the tth iteration, represents the current position of expert model i at the tth iteration, Indicates that the expert model i is at position The utility value at is the utility value of the personal best position of expert model i at the t-1th iteration.

[0013] Furthermore, the update formula of the global optimal position is as follows: , in, N is the total number of models in the population, represents the current position of model i at the tth iteration, Indicates that the expert model i is at position The utility value at which argmax is found The largest model index j, Indicates the position of the current model index j.

[0014] Furthermore, after each position update, the utility value of each expert model at the new position is calculated, and the personal and global optimal positions are updated according to the utility value, including: If the utility value of the expert model at the current position is greater than the utility value of the historical personal best position, the current position is updated to the new personal best position; Among all expert models, find the expert model with the highest current utility value and record its position as the new global optimal position; Among all expert models, find the model with the lowest utility value and record its position as the global worst position.

[0015] Furthermore, in the collaborative search process, the collaborative search path of each expert model in the expert model pool is dynamically adjusted, and the optimization of each expert model includes: The expert model is optimized by dynamically adjusting the inertia coefficient of the expert model. The inertia coefficient adjustment formula is as follows: , in, is the initial inertia, k is the current number of iterations, and K is the maximum number of iterations; Evaluate the invalid expert model through the utility value after several iterations of the expert model, and reset the invalid expert model; The expert model is optimized by dynamically scaling the step size of the expert model. The step size adjustment formula is as follows: , in, is the initial step length, is the step size reduction factor, and k is the current iteration number.

[0016] Further, the invalid expert model is evaluated by the utility value after several iterations of the expert model, and resetting the invalid expert model includes: If the expert model fails to improve its utility value in several consecutive iterations, the expert model is considered an invalid model; If the expert model is positioned as an invalid expert model, its position is set to its personal best position and its velocity is reset to zero.

[0017] Further, judging whether the current collaborative search process is finished according to the convergence status of all expert models, and returning the global best model in the expert model pool as the final optimization result includes: If the utility value increase of all expert models is less than the change threshold, the collaborative search converges. The calculation formula is as follows: , in, is the utility value of expert model i at the tth iteration, m is the number of consecutive iterations, is the change threshold; Calculate the position distance of all expert models in the weight space. If the distance variance of all expert models is less than the distance threshold, all expert models converge. Compare the utility values ​​of each iteration of all expert models, and select the global best position of the iteration with the highest utility value as the final optimization result; Output the expert model parameters corresponding to the global optimal position, and use the utility value at this position as the final utility value. The calculation formula is as follows:

[0018]

[0019] in, is the optimal model position finally determined, is the final global optimal position, is the final utility value of the best model, is the final utility value.

[0020] Furthermore, the teacher model is used to guide the student model in knowledge distillation, and the optimization of the student model includes: Input the new task data into the teacher model to obtain soft label data, input the new task data into the student model to obtain predicted distribution data, and soften the soft label data by distillation temperature scaling; The distillation loss is calculated through the soft label data and the predicted distribution data, the supervision loss is calculated through the new task data, and the joint loss function is constructed by combining the distillation loss and the supervision loss. With the goal of minimizing the joint loss function, the student model is optimized and the trained student model is combined with other task data as a new teacher model for multiple rounds of distillation.

[0021] Beneficial effects of the present invention: First, multi-task model optimization requires determining the initial state (position and speed) and utility value, and establishing a clear optimization starting point. Through the initial parameters and utility values, we can clearly understand the capabilities of each expert model for different tasks. This lays the foundation for subsequent collaborative search and dynamic adjustment, and improves the overall optimization efficiency of the algorithm.

[0022] Second, by dynamically evaluating the global best value of the group and the personal best value of each model in each round of search, we can help adjust the optimization path. Through the global best position and the personal best position, we can guide all models to optimize in a better direction and accelerate group collaboration. Each model updates the path according to its own best utility value to avoid invalid search.

[0023] Third, by re-adjusting the path of the stagnant model (repositioning it around the current global optimal point), the optimization potential is activated. The search weight and step size parameters are dynamically adjusted to switch the model between exploration and local optimization. In multi-task scenarios, the search path of each expert model is dynamically adjusted according to different task requirements.

[0024] Fourth, dynamically determine whether the optimization conditions are met based on the convergence state to reduce unnecessary iterations. Find the global solution with the highest utility value and output the optimal model and its parameters.

[0025] Fifth, the knowledge transfer method of strong teaching and weak learning is introduced through the distillation mechanism to help inefficient models learn quickly and improve their utility value. Weak models learn the ability of other tasks in multi-task scenarios through distillation to make up for the lack of task adaptability. To make up for the shortcomings of low-data environments, avoid the waste of resources by eliminating weak models, and allow basic models to continue to participate in collaboration through efficient distillation. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 It is a flow chart of the method of this embodiment. DETAILED DESCRIPTION

[0028] To facilitate understanding by those skilled in the art, the structure of the present invention is now further described in detail with reference to the embodiments and the accompanying drawings. It should be understood that the steps mentioned in the present embodiment, unless otherwise specified in their order, can be adjusted in order according to actual needs, and can even be executed simultaneously or partially simultaneously.

[0029] like Figure 1 As shown, an embodiment of the present invention provides a multi-model collaboration method for a low-data environment, including: S1 builds an expert model pool and determines the initial position, initial speed, and utility value of each expert model; S101 selects several large language expert models with different task expertise to build a starting expert model pool; In this step, these expert models can be models that have been fine-tuned for specific tasks or trained on data from different fields (referring to single-task models that have achieved the best results in various fields, such as expert models specifically for translation, expert models specifically for solving mathematical problems, and expert models specifically for text writing. These models are obtained from open source networks or trained by themselves) to ensure diversity. The diversity of the expert model pool is crucial to the subsequent collaborative search process because diversity can improve the overall generalization ability. The number of models in the initial expert model pool can be adjusted according to the complexity of the task and computing resources. By default, at least 10 expert models should be selected to ensure effective exploration during the collaborative search process.

[0030] S102 defines the representation of the expert model weight in the parameter space as the initial position of the expert model, and defines the expert model randomly pointing to another expert model as the initial speed of the expert model; In this step, the position of each expert model is determined by the representation of its model weights in the parameter space. The diversity of positions helps to cover a wider range of task requirements. The speed of each expert model defines the direction and stride of the model's movement in the weight space. The speed setting determines the model's exploration ability. The initial speed can be randomly set to point to another randomly selected expert model to promote the diversity and randomness of the search. To avoid all expert models from gathering in one area too early, the initial speeds are designed to be dispersed to achieve more extensive exploration and optimization.

[0031] S103 evaluates the utility value of the expert model on a given task or multiple tasks through a utility function. The utility function is expressed as follows: , in, Indicates that the model i The utility value on each subtask is is the weight of the corresponding task, m The total number of tasks.

[0032] In this step, the utility function is used to evaluate the performance of the model on a given task or multiple tasks (utility value). The utility function can select appropriate evaluation indicators (such as accuracy, recall, etc.) according to the task requirements. is the weight of the corresponding task (used to balance the relative importance of multiple tasks), and then maps the performance indicator of the model to a scalar value , so as to serve as the target of model optimization in the collaborative search process.

[0033] S2 dynamically updates the speed and position of each expert model in the expert model pool through collaborative search, and evaluates the global best position of the expert model pool and the personal best position of each expert model; In this step, collaborative search is performed through a swarm optimization algorithm, which is a method of finding the optimal solution in a complex high-dimensional search space through the collaboration and interaction of multiple individuals (or "agents", "particles", "models", etc.). It simulates the collaborative behavior of natural groups (such as bird flocks, fish schools, ant colonies, etc.) to achieve efficient global optimization and problem solving. The collaborative search process based on the swarm optimization algorithm is the core of the collaborative optimization of multi-expert models. By dynamically updating the speed and position of each expert model, it moves towards the optimal solution, thereby achieving the effect of task adaptability optimization. The following are the specific steps of the process: S201 updates the speed of the expert model through the inertia of the expert model, the personal best position, and the global best position. The update rules are as follows: , in, is the velocity vector of expert model i, is the inertia weight, , , are the attraction or repulsion coefficients for the personal best, the global best, and the global worst, respectively, , , is a factor randomly drawn from a uniform distribution in [0,1], is the personal optimal position of expert model i, is the global best position, ; In this step, the speed of each expert model determines its direction and stride in the weight space. The speed update is based on three main factors: inertia, personal best position , the global optimal position g, is the inertia weight, which controls the degree to which the current speed is maintained. A higher value helps explore more unknown areas; , , are the attraction or repulsion coefficients for the personal best, global best, and global worst, respectively, controlling the degree of influence of the model at the personal best position, the global best position, and the global worst position; , , It is a factor randomly drawn from a uniform distribution in [0,1], used to introduce randomness and increase the diversity and robustness of the search.

[0034] is the personal best position of expert model i, which refers to the position of the highest utility value encountered by expert model i during the iteration process. Each model will update its utility value after each iteration and compare it with the current personal best value. If the current utility value is higher, the personal best position will be updated. For model i at the tth iteration, the update formula of its personal best position is as follows: , in, represents the personal best position of expert model i at the tth iteration, represents the current position of expert model i at the tth iteration, Indicates that the expert model i is at position The utility value at is the utility value of the personal best position of expert model i at the t-1th iteration.

[0035] is the global best position, is the global worst position. The global best position It refers to the optimal position of all models in the current group, that is, the position with the highest utility value. The determination of the global optimal position is based on the utility values ​​of all models. The model with the highest utility value is found and the current position of the model is set as the global optimal position. At the tth iteration, the update formula of the global optimal position is as follows: , in, N is the total number of models in the population, represents the current position of model i at the tth iteration, Indicates that the expert model i is at position The utility value at which argmax is found The largest model index j, Indicates the position of the current model index j.

[0036] S202 updates the position of the expert model according to the speed after the expert model is updated. The updating formula is as follows: ， in, is the step size coefficient, is the current position of expert model i; In this step, is the step size coefficient, which controls the magnitude of the expert model's movement in the new direction. Values ​​promote exploration, allowing the model to move quickly in weight space; smaller A larger value will help with finer adjustments and improve local convergence.

[0037] S203 After each position update, the utility value of each expert model at the new position is calculated, and the personal and global best positions are updated according to the utility value.

[0038] S2031 If the utility value of the expert model at the current position is greater than the utility value of the historical personal best position, then the current position is updated to a new personal best position; In this step, for each model, if the utility value of the current position is better than the utility value of its historical personal best position, the current position is updated to the new personal best position This updating approach ensures that each model records the best state it has encountered during the collaborative search process.

[0039] S2032 finds the expert model with the highest current utility value among all expert models, and records its position as the new global optimal position; In this step, among all models, the model with the highest current utility value is found and its position is recorded as the new global optimal position. This position represents the optimal solution direction of the entire model group at present, and other models can move closer to this position to improve their utility value.

[0040] S2033 finds the model with the lowest utility value among all expert models and records its position as the global worst position.

[0041] In this step, the model with the lowest utility value is found and its position is recorded as the global worst position. The setting of the global worst position is used to exclude the model from moving to the bad area and ensure the overall optimization direction of the model in the collaborative search process.

[0042] During the collaborative search process, S3 dynamically adjusts the collaborative search path of each expert model in the expert model pool to optimize each expert model; In this step, a dynamic adjustment mechanism is set up to enhance the effectiveness and flexibility of the collaborative search process. This mechanism ensures that the expert model can achieve a balance between exploration and convergence in the collaborative search process through a multi-level adjustment strategy, thereby achieving efficient optimization. The search path refers to the trajectory of the expert model's exploration in the parameter space, that is, the process of gradually adjusting its position (parameter configuration) during the collaborative search process. This path records the direction, stride, and position change of each model in the optimization space, and finally converges to a position close to the optimal solution.

[0043] S301 optimizes the expert model by dynamically adjusting the inertia coefficient of the expert model. The inertia coefficient adjustment formula is as follows: , in, is the initial inertia, k is the current number of iterations, and K is the maximum number of iterations; In this step, in each iteration of collaborative search, the present invention designs a mechanism to dynamically adjust the search path. This mechanism adjusts the inertia coefficient, step size, and attraction / repulsion coefficient of each model according to the progress of the expert model and the overall convergence of the group. The inertia coefficient 𝜔 determines the degree of dependence of each model on its current speed. In order to promote exploration in the early stages of search, a higher inertia coefficient can be set to encourage the model to search the weight space in a larger range; as the number of iterations increases, the inertia coefficient is gradually reduced so that the model gradually gathers near the known efficient solution. This formula ensures that in the early stage Larger, later Gradually decrease.

[0044] S302 evaluates the invalid expert model through the utility value after several iterations of the expert model, and resets the invalid expert model; In this step, during the collaborative search process, some models may fail to achieve significant improvement in multiple iterations, resulting in a "stagnation" phenomenon. To solve this problem, the present invention introduces a "repositioning" mechanism to reposition these invalid models to more promising search areas to stimulate new optimization directions. The specific steps are as follows: S3021 If the expert model fails to improve its utility value in several consecutive iterations, the expert model is deemed to be an invalid model; In this step, if a model is continuously If the model fails to improve its utility value in the iterations, it is considered an invalid model. This stagnation may indicate that the model has fallen into a local optimal solution or lacks optimization space in the current direction.

[0045] S3022 If the expert model is positioned as an invalid expert model, its position is set to its personal best position and its speed is reset to zero.

[0046] In this step, once a model is marked as invalid, its position is reset to its personal best position and its velocity is reset to zero. This is equivalent to "restarting" the model, allowing it to restart the search process from a better initial position. This process ensures that the invalid model no longer wastes computing resources and has the potential to find a better solution in a new direction.

[0047] S303 optimizes the expert model by dynamically scaling the step size of the expert model. The step size adjustment formula is as follows: , in, is the initial step length, is the step size reduction factor, and k is the current iteration number.

[0048] In this step, the step size The dynamic adjustment of is also crucial in the collaborative search process. A large step size is conducive to exploring new areas in the weight space, while a small step size is conducive to refining the optimization results. Therefore, the step size adjustment mechanism ensures that the exploration speed is accelerated in the early stage of the search, and gradually focuses on the refinement and optimization of the local optimal solution in the later stage. The present invention adopts a method of exponentially reducing the step size. Assuming that the initial step size is , then after each iteration, the step size is reduced to: ,in, is the step size reduction factor (usually between 0.9 and 0.99), and k is the current iteration number. This ensures that the model moves quickly in the early stages, and gradually reduces the step size in the later stages to enter the local optimization stage. In some cases, in order to avoid the model falling into the local optimal solution, the step size can be temporarily enlarged in several iterations (for example *2), so that the model jumps out of the local area and re-explores a wider weight space, and then returns to the normal step size reduction process.

[0049] S4 determines whether the current collaborative search process is finished according to the convergence status of all expert models, and if so, returns the global best model in the expert model pool as the final optimization result.

[0050] In this step, at the end of the collaborative search process of the present invention, it is necessary to set a reasonable termination condition to determine whether a satisfactory optimization effect has been achieved. The setting of the termination condition is crucial. It is necessary to ensure that the search is terminated after a better solution is found to avoid wasting resources, and to avoid premature termination to ensure the comprehensiveness of the search process. Detection of the convergence state is one of the main termination conditions in the collaborative search process. Specifically, if most of the expert models in the group do not have a significant increase in utility value in multiple consecutive iterations, it indicates that the search process of the group has stabilized and entered a convergence state. At this point, it can be considered that the search has approached the optimal solution and the search process is terminated. The steps to judge convergence are as follows: S401 If the utility value increase of all expert models is less than the change threshold, the collaborative search converges. The calculation formula is as follows: , in, is the utility value of expert model i at the tth iteration, m is the number of consecutive iterations, is the change threshold; In this step, a threshold value for the change in utility value is set , if in m consecutive iterations, the utility value of all models is improved by less than , then the search is judged to have converged. In this step, it is judged that the collaborative search has converged, and it is necessary to further judge the position distance of the expert model in the weight space.

[0051] S402 calculates the position distance of all expert models in the weight space. If the distance variance of all expert models is less than the distance threshold, all expert models converge. In this step, the position distance of all models in the weight space is calculated. If the positions of all members of the model group are concentrated in a small range (for example, the distance variance of all models is less than a certain threshold), the search process of all expert models is considered to have converged. This indicates that the model group has been concentrated in one area during the search process, indicating that a better solution has been found.

[0052] At the end of the collaborative search process, the global best position in the group of expert models represents the model with the highest utility value in the current group. The expert model corresponding to this position is the final optimal solution, and its parameter configuration has been optimized to the optimal state during the search process. The specific steps are as follows: S403 compares the utility values ​​of each iteration of all expert models, and selects the global best position of the iteration with the highest utility value as the final optimization result; In this step, during the entire collaborative search process, the global optimal utility value after each iteration is recorded. , and compare the utility values ​​of all records at the end of the search, and determine the global best position of the iteration with the highest utility value as the final solution.

[0053] S404 outputs the expert model parameters corresponding to the global optimal position, and uses the utility value at this position as the final utility value. The calculation formula is as follows:

[0054]

[0055] in, is the optimal model position finally determined, is the final global optimal position, is the final utility value of the best model, is the final utility value.

[0056] In this step, the final global optimal position The corresponding model parameter output and the utility value at that position As the final utility value , this global optimal model will serve as the optimal solution for the entire collaborative search process to meet the needs of specific tasks (such as classification, translation, image recognition, etc.).

[0057] S5 selects the expert model with a utility value greater than the utility threshold in the expert model pool as the teacher model, and selects the expert model with a utility value less than the utility threshold that requires the same task type as the student model. The teacher model is used to guide the student model in knowledge distillation and optimize the student model.

[0058] In this step, select the best performing model from the expert model pool As a teacher model, its task The utility value on is high, and the output distribution is suitable for the task There is a clear guiding significance. Choose new tasks that need to adapt to the same task type Expert Model As a student model, its utility value is in the task The specific knowledge distillation guidance steps are as follows: S501 inputs the new task data into the teacher model to obtain soft label data, inputs the new task data into the student model to obtain predicted distribution data, and softens the soft label data by distillation temperature scaling; In this step, the task New data is fed into the teacher model and student models , generating the corresponding output: teacher model output Represents soft label data, student model output Represents the predicted distribution data. The output distribution of the teacher model can be softened by scaling the temperature C: , in, is the output of the teacher model, is the output of the student model, is the temperature parameter. The temperature C adjusts the probability distribution of the teacher model to make the output probability distribution more "soft". When , the output distribution of the teacher model will be smooth, and this smooth probability distribution is more likely to convey the potential connections between categories.

[0059] S502 calculates the distillation loss through the soft label data and the predicted distribution data, calculates the supervision loss through the new task data, and constructs a joint loss function by combining the distillation loss and the supervision loss; In this step, the distillation loss is measured by the KL divergence, which calculates the KL divergence between the output distributions of the student model and the teacher model. The calculation formula is as follows: , in, is the KL divergence, For soft label data, Represents predictive distribution data; Combined with the true label of the new task data, calculate the supervision loss of the student model: , in, is the true label of the new task data, CE is a universal loss function, and the corresponding loss function can be selected according to the actual task requirements; Combining distillation loss and supervision loss to construct a joint loss function, the calculation formula is as follows: ; in, and is the loss weight.

[0060] S503 optimizes the student model with the goal of minimizing the joint loss function, and combines the trained student model as a new teacher model with other task data for multiple rounds of distillation.

[0061] In this step, the distillation temperature is gradually adjusted. and loss weight By minimizing the joint loss function, the student model gradually approaches the performance of the teacher model. The trained student model is used as the new teacher model, combined with other task data, and multiple rounds of distillation are performed to enhance the multi-task adaptability of the model. In this way, the student model not only adapts to the current task, but also gradually generalizes, helping to expand the task capabilities of the expert model pool. The soft labels of the teacher model make up for the lack of supervision caused by insufficient data and help the student model learn more efficiently. Knowledge distillation combined with group collaborative optimization can dynamically improve the multi-task performance of the entire model group and make the model pool more balanced.

[0062] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0063] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0064] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0065] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0066] It should be noted that in the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claim. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0067] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0068] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0069] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0070] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms should not be understood as necessarily referring to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

Claims

1. A multi-model collaboration method for low-data environments, characterized in that: include: Build an expert model pool and determine the initial position, initial speed and utility value of each expert model; Dynamically update the speed and position of each expert model in the expert model pool through collaborative search, and evaluate the global best position of the expert model pool and the personal best position of each expert model; During the collaborative search process, the collaborative search path of each expert model in the expert model pool is dynamically adjusted to optimize each expert model; Determine whether the current collaborative search process is finished based on the convergence status of all expert models. If so, return the global best model in the expert model pool as the final optimization result. In the expert model pool, the expert model with a utility value greater than the utility threshold is selected as the teacher model, and the expert model with a utility value less than the utility threshold that requires the same task type is selected as the student model. The teacher model is used to guide the student model through knowledge distillation to optimize the student model.

2. The multi-model collaboration method for low-data environment according to claim 1, characterized in that: The step of constructing an expert model pool and determining the initial position, initial speed, and utility value of each expert model includes: Select several large language expert models with different task expertise to build a starting expert model pool; The representation of the expert model weight in the parameter space is defined as the initial position of the expert model, and the expert model randomly points to another expert model as the initial speed of the expert model; The utility value of the expert model on a given task or multiple tasks is evaluated through the utility function. The utility function is expressed as follows: ， in, Indicates that the model i The utility value on the subtasks, is the weight of the corresponding task, m The total number of tasks.

3. The multi-model collaboration method for low-data environment according to claim 2, characterized in that: The method of dynamically updating the speed and position of each expert model in the expert model pool through collaborative search and evaluating the global best position of the expert model pool and the personal best position of each expert model includes: The speed of updating the expert model is determined by the inertia of the expert model, the personal best position, and the global best position. The update rules are as follows: ， in, is the velocity vector of expert model i, is the inertia weight, , , are the attraction or repulsion coefficients for the personal best, the global best, and the global worst, respectively, , , is a factor randomly drawn from a uniform distribution in [0,1], is the personal optimal position of expert model i, is the global best position, is the current position of expert model i; The position of the expert model is updated by the speed after the expert model is updated. The update formula is as follows: ， in, is the step size coefficient, is the current position of expert model i; After each position update, the utility value of each expert model at the new position is calculated, and the personal and global best positions are updated according to the utility value.

4. The multi-model collaboration method for low-data environment according to claim 3, characterized in that: The update formula of the personal best position of the expert model i is as follows: ， in, represents the personal best position of expert model i at the tth iteration, represents the current position of expert model i at the tth iteration, Indicates that the expert model i is at position The utility value at is the utility value of the personal best position of expert model i at the t-1th iteration.

5. The multi-model collaboration method for low-data environment according to claim 3, characterized in that: The update formula of the global optimal position is as follows: ， in, N is the total number of models in the population, represents the current position of model i at the tth iteration, Indicates that the expert model i is at position The utility value at which argmax is found The largest model index j, Indicates the position of the current model index j.

6. The multi-model collaboration method for low-data environment according to claim 3, characterized in that: After each position update, the utility value of each expert model at the new position is calculated, and the personal and global optimal positions are updated according to the utility value, including: If the utility value of the expert model at the current position is greater than the utility value of the historical personal best position, the current position is updated to the new personal best position; Among all expert models, find the expert model with the highest current utility value and record its position as the new global optimal position; Among all expert models, find the model with the lowest utility value and record its position as the global worst position.

7. The multi-model collaboration method for low-data environment according to claim 1, characterized in that: In the collaborative search process, dynamically adjusting the collaborative search path of each expert model in the expert model pool and optimizing each expert model includes: The expert model is optimized by dynamically adjusting the inertia coefficient of the expert model. The inertia coefficient adjustment formula is as follows: ， in, is the initial inertia, k is the current number of iterations, and K is the maximum number of iterations; Evaluate the invalid expert model through the utility value after several iterations of the expert model, and reset the invalid expert model; The expert model is optimized by dynamically scaling the step size of the expert model. The step size adjustment formula is as follows: ， in, is the initial step length, is the step size reduction factor, and k is the current number of iterations.

8. The multi-model collaboration method for low-data environment according to claim 7, characterized in that: The invalid expert model is evaluated by the utility value after several iterations of the expert model, and resetting the invalid expert model comprises: If the expert model fails to improve its utility value in several consecutive iterations, the expert model is considered an invalid model; If the expert model is positioned as an invalid expert model, its position is set to its personal best position and its velocity is reset to zero.

9. The multi-model collaboration method for low-data environment according to claim 1, characterized in that: The step of judging whether the current collaborative search process is finished according to the convergence status of all expert models, and returning the global best model in the expert model pool as the final optimization result includes: If the utility value increase of all expert models is less than the change threshold, the collaborative search converges. The calculation formula is as follows: ， in, is the utility value of expert model i at the tth iteration, m is the number of consecutive iterations, is the change threshold; Calculate the position distance of all expert models in the weight space. If the distance variance of all expert models is less than the distance threshold, all expert models converge. Compare the utility values ​​of each iteration of all expert models, and select the global best position of the iteration with the highest utility value as the final optimization result; Output the expert model parameters corresponding to the global optimal position, and use the utility value at this position as the final utility value. The calculation formula is as follows: in, is the optimal model position finally determined, is the final global optimal position, is the final utility value of the best model, is the final utility value.

10. The multi-model collaboration method for low-data environment according to claim 1, characterized in that: The teacher model is used to guide the student model through knowledge distillation, and the optimization of the student model includes: Input the new task data into the teacher model to obtain soft label data, input the new task data into the student model to obtain predicted distribution data, and soften the soft label data by distillation temperature scaling; The distillation loss is calculated through the soft label data and the predicted distribution data, the supervision loss is calculated through the new task data, and the joint loss function is constructed by combining the distillation loss and the supervision loss. With the goal of minimizing the joint loss function, the student model is optimized and the trained student model is combined with other task data as a new teacher model for multiple rounds of distillation.

Images