Sub-model weight optimization method for ensemble learning of large language model

Abstract

The invention provides a sub-model weight optimization method for ensemble learning of a large language model, and relates to the technical field of data processing, and the method comprises the steps: generating an initial sub-model weight group, combining and coding the initial sub-model weight group into a binary gene character string, and taking the binary gene character string as an initial population of evolutionary computation; constructing a proxy experiment environment, generating a proxy corpus by randomly sampling the original pre-training corpus, and selecting at least three representative natural language processing tasks as proxy tasks for evaluating the adaptive value of the gene character string; based on the evaluation result of the proxy task, selecting a high-fitness gene character string by adopting a roulette method, and generating a next-generation gene character string through crossover and mutation operations; and S2 and S3 are iteratively executed until convergence, an optimal sub-model weight combination is obtained, and task-independent meta-learner fine tuning training is carried out on the original full-amount corpus to obtain a final integrated model.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method for optimizing the weights of sub-models in ensemble learning of large language models. Background Technology

[0002] Large language models, as a core technology in the field of natural language processing, are widely used in scenarios such as text generation, semantic understanding, and multimodal interaction. With the continuous expansion of the parameter scale of open-source models such as ChatGLM and LLaMA, ensemble learning technology has built a better solution system through multi-model collaborative operation. Specifically, this technology covers the entire process from model selection and weight allocation to output fusion, including key aspects such as a stacked ensemble framework based on genetic algorithms and task relevance analysis. Among related technologies, CN111553117B uses a genetic algorithm to optimize the combination of primary learners and achieves intrinsic viscosity control in the polyester production process through iterative search. Its technical system has verified the feasibility of ensemble learning in industrial scenarios.

[0003] However, existing ensemble learning methods, when directly using the full corpus for sub-model weight search, suffer from computational complexity as high as O(N^K) (where K is the number of sub-models), lacking optimization mechanisms for the characteristics of large language models. This can lead to wasted computational resources or degraded model performance. Specifically, traditional genetic algorithms, in task-agnostic meta-learner training, require single-iteration optimization on petabyte-scale corpora, consuming hours to days of GPU time. Furthermore, existing agent task evaluation systems (such as the SST-2 / MNLI / SQUAD three-task combination) lack a mechanism to verify the correlation between weight selection strategies and full corpus performance. Moreover, in roulette wheel-based selection strategies, the use of fitness value normalization formulas does not achieve co-optimization with agent environment construction parameters (such as sampling rate and task combination). These technical limitations not only affect the efficiency of ensemble learning but may also lead to performance degradation of the final model on key NLP tasks. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, the first objective of this invention is to propose a method for optimizing sub-model weights in large language model ensemble learning.

[0006] The second objective of this invention is to propose a sub-model weight optimization device for integrated learning of large language models.

[0007] The third objective of this invention is to provide an electronic device.

[0008] The fourth objective of this invention is to provide a computer-readable storage medium.

[0009] The fifth objective of this invention is to provide a computer program product.

[0010] To achieve the above objectives, a first aspect of the present invention proposes a method for optimizing sub-model weights in large language model ensemble learning, comprising:

[0011] S1, generate the initial sub-model weight combination and encode it into a binary gene string, which serves as the initial population for evolutionary computation; S2. Construct a proxy experimental environment, generate proxy corpus by randomly sampling the original pre-training corpus, and select at least three representative natural language processing tasks as proxy tasks to evaluate the fitness value of the gene string. S3, based on the evaluation results of the agent task, uses the roulette wheel method to select gene strings with high fitness values, and generates the next generation gene strings through crossover and mutation operations; S4 iterates through S2 and S3 until convergence, obtains the optimal sub-model weight combination, and performs task-agnostic meta-learner fine-tuning on the original full corpus to obtain the final ensemble model.

[0012] In one embodiment of the present invention, the step of generating the initial sub-model weight combination and encoding it into a binary gene string includes: S11, using Bernoulli distribution Independent sampling generates "0" or "1" in a binary string; S12, encode the weight combination of each sub-model into a length of The binary string, where The number of sub-models, The weights are discretized into different levels.

[0013] In one embodiment of the present invention, the step of constructing the proxy experimental environment includes: S21, select SST-2, MNLI, and SQUAD v1.1 as the agent tasks; S22, using a weighted evaluation formula Calculate the fitness value, where and This is the preset task weight coefficient.

[0014] In one embodiment of the present invention, the step of evaluating the results based on the agent task includes: S31, Set the crossover probability To swap the sub-model weight encodings at corresponding positions in the two gene tuples; S32, Configure mutation probability Each gene bit in the binary string is randomly flipped with that probability.

[0015] In one embodiment of the present invention, the step of iteratively executing S2 and S3 until convergence includes: S41, the number of iterations is limited to no more than 50; S42, set the sampling rate of the proxy corpus to be no less than 10% and no more than 20%.

[0016] In one embodiment of the present invention, it further includes: S5 uses GENFUSER as the meta-learner, and inserts data into the sequence...<extra_id_i> Separator and append weight markers to the input The encoding is a fused input, where Indicates the first The weight parameters of each sub-model.

[0017] To achieve the above objectives, a second aspect of the present invention proposes a sub-model weight optimization device for large language model ensemble learning, comprising: The initial weight encoding module is used to generate the initial sub-model weight combination and encode it into a binary gene string, which serves as the initial population for evolutionary computation. The proxy environment construction module is used to build a proxy experimental environment. It generates proxy corpus by randomly sampling the original pre-training corpus and selects at least three representative natural language processing tasks as proxy tasks to evaluate the fitness value of the gene string. The evolutionary operation module is used to select gene strings with high fitness values ​​using a roulette wheel method based on the evaluation results of the agent task, and generate the next generation of gene strings through crossover and mutation operations; The iterative optimization and fine-tuning module is used to iteratively execute S2 and S3 until convergence, obtain the optimal sub-model weight combination, and perform task-independent meta-learner fine-tuning training on the original full corpus to obtain the final ensemble model.

[0018] To achieve the above objectives, a third aspect of the present invention provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of the first aspects.

[0019] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of the first aspects.

[0020] To achieve the above objectives, a fifth aspect of the present invention provides a computer program product that, when executed by a processor, implements the method described in any one of the first aspects.

[0021] The technical solutions provided by the embodiments of the present invention bring at least the following beneficial effects: This invention effectively reduces computational resource consumption and training time in large language model ensemble learning by introducing a sub-model weight search strategy based on evolutionary computation and a proxy experimental environment evaluation mechanism, thereby improving the performance and generalization ability of the ensemble model in multi-task scenarios.

[0022] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0023] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating a method for optimizing sub-model weights in ensemble learning of a large language model, as provided in an embodiment of the present invention. Figure 2 This is a flowchart illustrating a method for optimizing sub-model weights in ensemble learning of a large language model, as provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of sub-model weight search in a sub-model weight optimization method for large language model ensemble learning provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the sub-model weight optimization device for ensemble learning of a large language model provided in an embodiment of the present invention. Detailed Implementation

[0024] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0025] Figure 1 This is a flowchart illustrating a method for optimizing sub-model weights in ensemble learning of a large language model, as provided in an embodiment of the present invention. Figures 1-3 As shown, the method includes the following steps: S1 generates the initial sub-model weight combination and encodes it into a binary gene string, which serves as the initial population for evolutionary computation.

[0026] Specifically, in the technical solution of this invention, "generating an initial sub-model weight combination and encoding it into a binary gene string as the initial population for evolutionary computation" is the starting point of the evolutionary computation process in the entire ensemble learning system. Its technical implementation is based on random initialization and binary encoding strategies, and has a clear mathematical foundation and engineering implementation path.

[0027] At the technical implementation level, this step first generates a set of weight combinations for an ensemble learning sub-model through random sampling, serving as the first generation of the population for evolutionary computation. Specifically, each weight combination... For a corresponding candidate integration strategy, where Indicates the size of the initial population. Indicates the first in the population Each individual weight combination represents the contribution ratio of each sub-model in the ensemble process. However, to accommodate the binary operation mechanism of evolutionary computation, these weights are discretized and encoded as binary strings. Each bit in the binary string represents the weight state of a sub-model, and its value... or From Bernoulli distribution The samples were obtained independently to ensure that the initial population had sufficient diversity.

[0028] At the parameter level, the key parameters involved in this step include: population size. Number of sub-models Length of binary string (Each sub-model corresponds to one binary bit), and the sampling distribution. These parameters directly affect the coverage of the initial population and the convergence efficiency of subsequent evolutionary processes. For example, larger... Values ​​that allow for exploring a wider space of weight combinations will increase computational resource consumption; while The symmetry of the initial weights ensures fairness and randomness.

[0029] At the application level, this step is suitable for scenarios that integrate multiple open-source large language models (such as ChatGLM, LLaMA, Flan-T5, etc.). By encoding the sub-model weights as binary genes, the performance of different ensemble strategies can be quickly evaluated in a proxy experimental environment, thus providing a starting point for subsequent evolutionary computation. This encoding method is particularly suitable for parallel computing architectures, facilitating the initialization and evaluation of large-scale populations in distributed systems.

[0030] From a technical perspective, this step, by introducing a binary gene encoding mechanism, transforms the continuous weight selection problem into a discrete evolutionary search problem, reducing computational complexity. Simultaneously, the Bernoulli-based random initialization strategy ensures population diversity, providing a solid foundation for subsequent crossover and mutation operations, thereby improving the global search capability and convergence speed of evolutionary computation. This step provides an efficient and scalable initial policy generation method for the entire ensemble learning system and is a key prerequisite for optimizing task-independent meta-learners.

[0031] Furthermore, S1 includes: S11, using Bernoulli distribution Independent sampling generates "0" or "1" in a binary string.

[0032] Specifically, in the technical solution of this invention, step one employs a Bernoulli distribution. The core purpose of independently sampling and generating "0" or "1" in a binary string is to provide a random and uniformly distributed encoding method for initializing the weights of the ensemble learning sub-models. This step is technically based on the Bernoulli process in probability theory, where the value of each binary bit independently follows a parameter... The Bernoulli distribution indicates that each sub-model has an equal probability of being selected or not selected in the initial ensemble weights.

[0033] From a technical implementation perspective, this process can be viewed as randomly initializing the "switch" states of sub-models participating in the ensemble. Specifically, the weight of each sub-model is encoded as a binary bit; a "1" indicates that the output of that sub-model will be fused by the meta-learner GENFUSER; a "0" indicates that the sub-model is excluded from the current ensemble. Independent sampling ensures the diversity of the initial population, avoids getting trapped in local optima, and provides a good starting point for subsequent evolutionary computations (such as crossover and mutation).

[0034] At the parameter level, the parameters of the Bernoulli distribution... This means that each binary bit has a 50% probability of being "1" and a 50% probability of being "0". This parameter choice aligns with the requirements of uniform initialization and helps explore a wider range of weight combinations in the early stages. Furthermore, the weights of each sub-model are encoded into a string of length [length missing]. The binary string, where The number of sub-models to be integrated, for example, in this scheme, might be 3 to 5 open-source large language models (such as Flan-T5, ChatGLM, etc.).

[0035] At the application level, this step is primarily used to build the initial population for ensemble learning, suitable for natural language processing tasks involving multi-model fusion, such as summarization, sentiment analysis, and natural language inference. By using randomly initialized sub-model weights in a proxy experimental environment, the performance of different combination strategies can be quickly evaluated, thus providing a feedback mechanism for evolutionary search.

[0036] The technical advantage of this step lies in ensuring the diversity of initial genes by introducing randomness, thus providing a good starting point for subsequent evolutionary calculations. Simultaneously, since the weight initialization of each sub-model is independent, the influence of human bias on the ensemble strategy is avoided, thereby improving the overall robustness and generalization ability of ensemble learning.

[0037] S12, encode the weight combination of each sub-model into a length of The binary string, where The number of sub-models, The weights are discretized into different levels.

[0038] Specifically, in the technical solution of this invention, step three involves encoding the weight combination of each sub-model into a length of... The binary string, where Indicates the number of sub-models. This indicates the number of weight discretization levels. This step is a crucial part of the sub-model weight optimization process in ensemble learning, and its technical implementation is based on the discretization of continuous weight values ​​and a binary encoding strategy.

[0039] From a technical implementation perspective, the weight values ​​of each sub-model were originally continuous floating-point numbers, typically within the range [0,1], representing their relative importance in the ensemble output. To facilitate processing by the gene algorithm, this scheme discretizes these weight values ​​into... Each tier is used to divide a continuous interval into tiers. Each sub-model has two discrete intervals, either equidistant or non-equidistant. The weight value of each sub-model is mapped to an integer index. This indicates its position in the discretized data. Subsequently, this integer index is converted to a length of... This allows for the binary encoding of weights. For example, if... ,but Each weight value will be encoded as a 3-bit binary string.

[0040] At the parameter level, This is typically the total number of sub-models in the ensemble model, such as 5 or 10 open-source large language models; The value of depends on the discretization precision requirement, and is generally between 2^3 and 2^8. This encoding method determines the total length of the gene string. For example, when and At that time, the total length is The position. This parameter setting directly affects the search efficiency and accuracy of the gene algorithm; a smaller value... It can reduce computational complexity, but may sacrifice the precision of weight representation.

[0041] At the application level, this step is primarily used to construct gene representations in the initial or intermediate generation population of the gene algorithm. Each gene represents a sub-model weight combination strategy, used to guide the training and evaluation of the meta-learner on the agent task. By encoding the weight combinations into binary strings, the gene algorithm can efficiently perform operations such as crossover and mutation, thereby quickly exploring the optimal weight configuration in the agent experimental environment.

[0042] From a technical perspective, this step achieves efficient representation and manipulation of sub-model weights, laying the foundation for subsequent evolutionary search. Through discretization and binary encoding, not only is computational resource consumption reduced, but the scalability and stability of the genetic algorithm in large-scale ensemble learning scenarios are also improved. Furthermore, this encoding method ensures the diversity of weight combinations, helping to find better-performing ensemble strategies within a limited search space.

[0043] S2. Construct a proxy experimental environment by randomly sampling the original pre-training corpus to generate proxy corpus, and select at least three representative natural language processing tasks as proxy tasks to evaluate the fitness value of the gene string.

[0044] Specifically, in the technical solution of this invention, constructing a proxy experimental environment is one of the key steps to achieve efficient ensemble learning. The core of this step lies in generating proxy corpora by randomly sampling the original pre-training corpora, and selecting at least three representative natural language processing tasks as proxy tasks to evaluate the fitness value of the gene string (i.e., the sub-model weight encoding).

[0045] At the technical implementation level, the original pre-training corpus is typically a large-scale text dataset, such as Common Crawl or Wikipedia. Its massive data volume means that directly using it for ensemble learning training would result in significant computational overhead. Therefore, this scheme employs a random sampling strategy, extracting a subset from the original corpus as the proxy corpus. Sampling rate Typically set to To achieve a balance between computational efficiency and model performance, the sampling process can employ stratified sampling or uniform sampling to ensure the diversity and representativeness of the corpus. The construction of the proxy corpus must meet the following criteria: corpus length... And the distribution of the corpus In order to preserve the statistical characteristics of the original corpus as much as possible.

[0046] In selecting proxy tasks, this approach prioritizes three representative NLP tasks: the SST-2 sentiment analysis task, the MNLI natural language inference task, and the SQUAD v1.1 extractive reading comprehension task. These tasks cover typical application scenarios such as text classification, logical reasoning, and information extraction, enabling a comprehensive evaluation of the meta-learner's generalization ability across different tasks. Evaluation metrics for each proxy task include accuracy, F1 score, and ROUGE-L, used to measure the meta-learner's performance on the proxy corpus.

[0047] In practical applications, this step is typically deployed in a distributed training framework, such as PyTorch or TensorFlow, to support parallel computing. Training and evaluation of the proxy task can be performed on GPU clusters or TPU devices to accelerate iteration. The evaluation results of the proxy task allow for the rapid selection of gene strings with high fitness values, thereby guiding subsequent evolutionary search processes.

[0048] The technical advantage of this step lies in significantly reducing the computational cost of ensemble learning while maintaining effective evaluation of the sub-model weight search strategy. By combining proxy corpora and proxy tasks, this scheme can efficiently optimize large-scale ensemble learning models with limited resources, providing a reliable weight configuration foundation for ultimately training the meta-learner on the full corpus.

[0049] Furthermore, S2 includes: S21, select SST-2, MNLI, and SQUAD v1.1 as the agent tasks.

[0050] S22, using a weighted evaluation formula Calculate the fitness value, where and This is the preset task weight coefficient.

[0051] Specifically, in the technical solution of this invention, "selecting SST-2 as one of the proxy tasks" is one of the key steps in constructing the proxy experimental environment. The technical implementation principle of this step is based on the principles of task diversity and representativeness, aiming to quickly evaluate the merits of the weights of the ensemble learning sub-models with limited computing resources, thereby accelerating the evolutionary computation process.

[0052] At the technical implementation level, SST-2 is a widely used dataset for sentiment analysis tasks. Its annotation granularity is binary classification (positive / negative), and it contains rich contextual semantic structures. This approach uses it as one of the proxy tasks because sentiment analysis tasks place high demands on the semantic understanding capabilities of language models, and its task structure is simple, facilitating rapid evaluation of model performance. In the proxy experimental environment, the SST-2 dataset is used to test the performance of the meta-learner under different sub-model weight configurations, thereby providing fitness values ​​for the genetic algorithm. The basis for the calculation.

[0053] At the parameter level, SST-2's proxy task evaluation metrics typically use accuracy or F1 score, which directly reflect the model's performance in sentiment classification tasks. In this approach, the average accuracy of SST-2 is incorporated into the comprehensive scoring system of the three proxy tasks to measure the fitness of the current generation of genes (i.e., sub-model weight configurations). Specifically, the meta-learner trained on the proxy corpus for each gene performs inference on SST-2, and its output is compared with the true labels to calculate the task performance metric. This metric is then weighted and averaged with the metrics of the other two proxy tasks (MNLI and SQUAD v1.1) to form the final fitness value. .

[0054] At the application level, this step is primarily used in the evolutionary search phase of sub-model weights in ensemble learning. By using tasks such as SST-2 in a proxy experimental environment, sub-model weight configurations with higher generalization ability can be quickly selected without consuming the training resources of the entire corpus. This method is particularly suitable for large-scale language model ensemble scenarios, where the number of parameters of each sub-model may reach billions, and performing multiple rounds of training and fine-tuning directly on the entire corpus would bring huge computational overhead.

[0055] From a technical perspective, choosing SST-2 as one of the surrogate tasks helps improve the search efficiency of the gene algorithm under limited computing resources. Because the SST-2 task has a clear structure and high-quality annotation, its evaluation results have high reliability and discriminative power, effectively guiding the gene algorithm towards a better sub-model weight configuration. Furthermore, this step enhances the simulation capability of the surrogate experimental environment to real-world task scenarios by introducing a representative NLP task, thereby improving the robustness and generalization performance of the final ensemble learning model in multi-task scenarios.

[0056] S3, based on the evaluation results of the agent task, uses the roulette wheel method to select gene strings with high fitness values, and generates the next generation gene strings through crossover and mutation operations.

[0057] Specifically, the technical solution of this invention involves gene selection and evolution operations based on the evaluation results of agent tasks, which is the core mechanism for achieving efficient weight search of ensemble learning sub-models. In some implementations, this step uses a roulette wheel method to select gene strings of the current generation, and the selection probability is directly proportional to the fitness value of the gene. Specifically, the fitness value of each gene string is... ,in Indicates the first One gene, Indicates the current generation. To prevent genes with excessively low fitness from being completely excluded, the selection probability is based on... Normalization was performed, where This represents the minimum fitness value of all genes in the current generation. This approach can effectively improve the stability and convergence efficiency of the search process.

[0058] Furthermore, after selecting genes with high fitness values, the next generation gene strings are generated through crossover and mutation operations. The crossover operation is performed probabilistically. The mutation operation involves randomly selecting an intersection point in two gene strings and swapping the corresponding binary elements to create a new gene. This operation preserves the superior components of the parent gene, helping to quickly approximate the optimal solution. The mutation operation, on the other hand, uses a lower probability... Randomly flipping the binary bits in the gene string (changing 0 to 1 or 1 to 0) is used to maintain population diversity and prevent getting trapped in local optima. In practice, It is usually set to 0.8~0.95, while The value is generally controlled between 0.01 and 0.05 to balance exploration and utilization.

[0059] In practical applications, especially in large language model ensemble learning scenarios, this step significantly reduces the computational cost of training on the entire corpus through rapid evaluation on proxy tasks (such as SST-2, MNLI, and SQUAD v1.1). Multiple meta-learners (such as Flan-T5-XL) are trained in parallel on the proxy corpus, and their average performance on the proxy task is used as the adaptation value, thereby achieving efficient search for sub-model weights. Finally, through iterative optimization, this step can automatically evolve the optimal sub-model weight configuration, providing high-quality meta-learner input for task-agnostic ensemble learning and improving the model's robustness and generalization ability.

[0060] Furthermore, S3 includes: S31, Set the crossover probability The sub-model weights are swapped at corresponding positions in the two gene tuples.

[0061] Specifically, in the technical solution of this invention, step five involves the crossover process in gene manipulation, the core of which lies in setting the crossover probability. This step involves swapping the sub-model weight encodings at corresponding positions in the two gene tuples. Based on the evolutionary mechanism of genetic algorithms (GA), this step aims to generate a new generation of genes with potentially better fitness values ​​by simulating crossover behavior in biological inheritance, thereby optimizing the combination strategy of sub-model weights in ensemble learning.

[0062] At the technical implementation level, each gene represents a binary code for a set of sub-model weights, where each bit or set of bits indicates the weight allocation state of a specific sub-model. The crossover operation involves randomly selecting two parent genes and applying a set crossover probability. The decision is made as to whether to exchange gene fragments at the corresponding positions. Specifically, a single-point crossover or multi-point crossover strategy can be adopted, with multi-point crossover being more suitable for the parallel optimization scenario of multiple sub-model weights in this invention. During the crossover process, if two genes are selected to be exchanged at a certain position, their corresponding weight codes will be swapped, thereby generating two new offspring genes.

[0063] From the perspective of parameter indicators, crossover probability It is a key parameter controlling population diversity and convergence speed in genetic algorithms. This invention will... Setting it to 0.8 indicates that there is an 80% probability of performing a crossover operation in each generation of gene manipulation, thereby accelerating the exploration of better weight combinations while maintaining population diversity. Experimental verification shows that this parameter setting can effectively improve the performance of the meta-learner within a limited number of iterations.

[0064] In application scenarios, this step is mainly used for the sub-model weight search process in surrogate experimental environments. Since ensemble learning of large language models involves the fusion of outputs from multiple models, the weight configuration directly affects the robustness and generalization ability of the final model. By evaluating the fitness values ​​of the progeny genes after crossover on surrogate tasks (such as SST-2, MNLI, and SQUAD v1.1), a better weight combination strategy can be quickly selected, avoiding costly enumeration searches on the entire corpus.

[0065] The technical advantage of this step lies in its ability to effectively preserve the superior characteristics of parent genes and introduce diversity into newly generated offspring genes by introducing a gene manipulation mechanism with high crossover probability, thereby improving search efficiency and model performance. Combined with a rapid evaluation mechanism using a surrogate experimental environment, this invention achieves efficient utilization of computational resources and a significant improvement in model output quality in large-scale language model ensemble learning.

[0066] S32, Configure mutation probability Each gene bit in the binary string is randomly flipped with that probability.

[0067] Specifically, in the technical solution of this invention, gene mutation is one of the key operations in the evolutionary computation process. Its core purpose is to enhance the diversity of the population by introducing random perturbations while maintaining excellent gene characteristics, thereby avoiding the algorithm from getting trapped in local optima and improving global search capabilities. Specifically, this step involves setting a relatively small mutation probability. Perform bit-by-bit mutation operations on the binary string corresponding to each gene individual in the current generation.

[0068] At the technical implementation level, each individual gene is represented as a string of length [length missing]. The binary string represents the weight state of a sub-model during the integration process (e.g., whether it is activated or participates in output fusion). Mutation operations are performed on each bit independently and identically distributed: for each bit in the string... With probability Randomly flip its value, that is, if Then The probability of turning it into Conversely, the same applies. This process can be formalized as:

[0069] in, Indicates the mutated gene location. Original gene locus, The mutation probability is set to . This is to ensure that the randomness of mutations does not undermine the excellent fitness characteristics already obtained in the current generation.

[0070] At the parameter level, the probability of mutation The settings need to balance the algorithm's convergence speed and population diversity. If... If the size is too large, it will cause the beneficial genes to be frequently destroyed, reducing the convergence efficiency; if If the size is too small, the population may experience premature convergence. In this scheme... The values ​​are reasonable and derived from experimental verification, which can maintain gene stability while appropriately introducing variations to maintain the population's exploratory ability.

[0071] At the application level, this step is mainly used for the sub-model weight search process in surrogate experimental environments. Since ensemble learning involves a large number of sub-models and a vast weight combination space, mutation operations help explore more potential weight configurations with limited computational resources, thereby improving the overall performance of the meta-learner GENFUSER on surrogate tasks (such as SST-2, MNLI, and SQUADv1.1).

[0072] From a technical perspective, this step effectively balances the algorithm's exploration and development capabilities by controlling the mutation probability, enabling the evolutionary process to converge to a sub-model weight configuration with superior performance in a relatively short time. Simultaneously, the mutation operation provides the algorithm with a mechanism to escape local optima, enhancing the robustness and generalization ability of the ensemble learning model, and is a crucial guarantee for achieving efficient and high-quality ensemble learning of large language models.

[0073] S4 iterates through S2 and S3 until convergence, obtains the optimal sub-model weight combination, and performs task-agnostic meta-learner fine-tuning on the original full corpus to obtain the final ensemble model.

[0074] Specifically, in the technical solution of this invention, iteratively executing steps S2 and S3 until convergence is a key optimization mechanism in the sub-model weight search process based on evolutionary computation. This step, through the iterative evolutionary process of the genetic algorithm (GA), gradually optimizes the weight combination of each sub-model in the ensemble learning, thereby improving the performance of the final ensemble model.

[0075] At the technical implementation level, S2 and S3 correspond to the execution of gene selection and gene manipulation (crossover and mutation), respectively. In each generation, the current gene set... Each gene in the dataset represents a set of sub-model weights, which are encoded as binary strings. In the proxy experiment environment, the weight combination corresponding to each gene is used to guide the training of the meta-learner, which is based on the Flan-T5-XL architecture and fuses the outputs of multiple sub-models. and its input This generates enhanced output results. The performance of the meta-learner is evaluated on surrogate tasks (SST-2, MNLI, SQUAD v1.1) to obtain the fitness value for each gene. This serves as a quantitative indicator of its merits or demerits.

[0076] Regarding parameter indicators, gene selection uses a roulette wheel method, and the probability of each gene being selected is the difference between its fitness value and the genetic parameters. The crossover operation is proportional to the genetic makeup, ensuring that genes with high fitness values ​​have a higher probability of being inherited. The mutation process involves exchanging tuple elements of two genes, preserving desirable traits and generating new solutions; the mutation operation is performed probabilistically. Randomly flipping bits in a binary string is used to maintain population diversity and prevent premature convergence. Typically... Set to a small value (e.g., 0.01~0.1) to avoid damaging existing excellent gene structures.

[0077] In the application scenario, this iterative process runs in a proxy experimental environment, using only a randomly sampled subset of the original corpus for training, significantly reducing computational overhead. Through performance feedback on the proxy task, the genetic algorithm can quickly converge to a better combination of sub-model weights, providing high-quality initial parameters for subsequent fine-tuning training on the full corpus.

[0078] In terms of technical effectiveness, this step, through evolutionary computation mechanism, effectively explores the weight space of large-scale sub-models, avoiding the high computational cost of exhaustive search. The resulting optimal weight combination significantly improves the generalization ability and robustness of the meta-learner in multi-task scenarios, providing an efficient and scalable optimization path for the ensemble learning of large language models.

[0079] The sub-model weight optimization method for large language model ensemble learning in this invention can effectively reduce the computational overhead of sub-model weight search in large language model ensemble learning and improve the training efficiency and model performance of the meta-learner.

[0080] Furthermore, S4 includes: S41, the number of iterations is limited to no more than 50.

[0081] Specifically, in the technical solution of this invention, "limiting the number of iterations to no more than 50" is a key control mechanism in the evolutionary computation (genetic algorithm) process for searching the optimal weights of the ensemble learning sub-model. This step involves setting a maximum number of iterations. This effectively controls the search process of the gene algorithm, avoiding the waste of computing resources and the decline in convergence efficiency caused by excessive iteration.

[0082] At the technical implementation level, this step, as one of the termination conditions of the genetic algorithm, is used to control the upper limit of the number of evolutionary generations. In each generation of evolution, the algorithm adjusts the algorithm based on the current genome. fitness value Selection, crossover, and mutation generate the next generation of genes. Limiting the number of iterations ensures the algorithm completes the search within limited computational resources, while avoiding getting trapped in local optima or lengthy convergence processes. In some implementations, the upper limit on the number of iterations can be set to... This value is a reasonable range based on experimental verification, and it can achieve stable convergence of the fitness value in most scenarios.

[0083] From the perspective of parameter metrics, number of iterations It is an important hyperparameter affecting search efficiency and model performance in gene algorithms. If If the value is too small, it may lead to insufficient search and failure to find the globally optimal solution; if... If the value is too large, it will significantly increase computational overhead. This solution addresses this by setting... Fast convergence was achieved in the surrogate experiment environment, while ensuring that the evaluation results of the meta-learner GENFUSER on the three surrogate tasks (SST-2, MNLI, SQUAD v1.1) are representative.

[0084] In application scenarios, this step is suitable for the meta-learner training stage of large language model ensemble learning, especially when the size of the proxy corpus is limited and computational resources are constrained. By controlling the number of iterations, training time can be significantly reduced while ensuring search quality, thereby improving the overall efficiency of ensemble learning.

[0085] In terms of technical effectiveness, limiting the number of iterations to no more than 50 helps to quickly select high-performance sub-model weight combinations in the proxy experiment environment, providing high-quality initial parameters for subsequent fine-tuning training on the full corpus, thereby improving the generalization ability and task-independent performance of the final meta-learner.

[0086] S42, set the sampling rate of the proxy corpus to be no less than 10% and no more than 20%.

[0087] Specifically, in the technical solution of this invention, the step of "setting the sampling rate of the proxy corpus to be no less than 10% and no more than 20%" is a key step in constructing the proxy experimental environment. Its technical implementation principle is based on the balance between statistical sampling and model training efficiency. This step aims to extract a representative subset from the original large-scale pre-training corpus to evaluate the fitness values ​​of different sub-model weight strategies in the proxy task, thereby accelerating the weight search efficiency in the evolutionary computation process.

[0088] In some implementations, the proxy corpus is sampled randomly to ensure that the samples are as close as possible to the original corpus in terms of semantic and length distribution. The sampling rate is set to [10%, 20%], meaning the size of the proxy corpus is a fraction of the original corpus. ,in This indicates the sampling ratio. The selection of this range is based on experimental validation to ensure that the proxy corpus still provides sufficient information to evaluate the performance of the meta-learner under computationally limited conditions.

[0089] From the perspective of parameter indicators, sampling rate The sampling rate setting directly affects the training time and model performance of the proxy experimental environment. If the sampling rate is too low (e.g., less than 10%), the proxy corpus may not be able to cover the diversity of the original corpus, resulting in a large bias in fitness evaluation and affecting the accuracy of evolutionary search. If the sampling rate is too high (e.g., greater than 20%), it will significantly increase computational overhead, which contradicts the original intention of the proxy experimental environment to "accelerate search". Therefore, limiting the sampling rate to [10%, 20%] is a compromise between model evaluation accuracy and computational efficiency.

[0090] In practical applications, this step is typically performed within a distributed training framework, such as using PyTorch or TensorFlow data loaders for hierarchical or chunked sampling to ensure corpus diversity. The surrogate corpus is used to train the GENFUSER meta-learner, with the input format being... ,in Indicates the first The weights of each sub-model are used to evaluate the fitness of the current gene by averaging the performance of proxy tasks (such as SST-2, MNLI, and SQUAD v1.1). .

[0091] The technical advantage of this step lies in the fact that, by reasonably controlling the size of the surrogate corpus, computational resource consumption can be significantly reduced while ensuring the quality of evolutionary search. Experiments show that when the sampling rate is controlled at [10%, 20%], the fitness evaluation under the surrogate experimental environment is highly consistent with the results under the full corpus, thus providing a reliable basis for weight selection in subsequent meta-learner fine-tuning.

[0092] The sub-model weight optimization method for large language model ensemble learning in this invention can effectively reduce the computational overhead of sub-model weight search in large language model ensemble learning and improve the training efficiency and model performance of the meta-learner.

[0093] S5 uses GENFUSER as the meta-learner, and inserts data into the sequence...<extra_id_i> Separator and append weight markers to the input The encoding is a fused input, where Indicates the first The weight parameters of each sub-model.

[0094] Specifically, in the technical solution of this invention, GENFUSER is used as a meta-learner to achieve the fusion and optimization of the outputs of multiple large language models. Specifically, the core of this step lies in encoding the outputs of multiple sub-models and their corresponding weight parameters in a specific sequence format to construct a fused input for the meta-learner to train and fine-tune.

[0095] At the technical implementation level, GENFUSER is a sequence-to-sequence architecture-based model designed to fuse the outputs of multiple sub-models to generate a higher-quality final output in task-agnostic ensemble learning scenarios. To achieve this goal, the input sequence is constructed as follows: ,in This represents the original input text. Indicates the first The output of each sub-model This represents the weight parameters of the sub-model.<extra_id_i> It is a special separator used to clearly identify the outputs and weights of different sub-models in a sequence, thereby facilitating the model to distinguish and process them during the decoding process.

[0096] At the parameter level, It is usually a real number, representing the first... The relative importance of each sub-model in the ensemble process is considered. These weight parameters are searched and optimized through evolutionary computation (such as genetic algorithms), with initial values ​​being randomly generated binary strings that are decoded and mapped to continuous weight values. In this scheme, the GENFUSER model uses Flan-T5-XL as the base model, with a parameter size of approximately 3B, exhibiting high generation capability and low computational overhead, making it suitable for rapid training and evaluation in surrogate experimental environments.

[0097] At the application level, this step is suitable for natural language processing tasks involving multi-model ensembles, such as text summarization, question answering systems, and text generation. By encoding the outputs and weight information of multiple sub-models into a unified input format, GENFUSER can learn during training how to dynamically adjust the contributions of each sub-model according to task characteristics, thereby improving the overall performance and robustness of the ensemble model.

[0098] In terms of technical effectiveness, this step, by introducing weight labels and delimiters, achieves structured fusion of the sub-model outputs, enabling the meta-learner to more effectively learn the weight allocation strategies among different sub-models. Furthermore, this method avoids the complexity of explicit weight modeling in traditional ensemble learning, improving the model's trainability and generalization ability. Finally, through fine-tuning training with GENFUSER, a task-agnostic and high-performance ensemble learning meta-model can be obtained, significantly improving the performance of large language models in multi-task scenarios.

[0099] The sub-model weight optimization method for large language model ensemble learning in this invention employs GENFUSER as a meta-learner and combines the sub-model output with weight information.<extra_id_i> The structured encoding of the delimiter as a fusion input further improves the accuracy and learnability of weight allocation during model ensemble, thereby enhancing overall prediction performance while reducing computational overhead.

[0100] To achieve the above embodiments, the present invention also proposes a sub-model weight optimization device for large language model ensemble learning. Figure 4 This is a schematic diagram of a sub-model weight optimization device for ensemble learning of a large language model, provided as an embodiment of the present invention. Figure 4 As shown, the device includes: The initial weight encoding module is used to generate the initial sub-model weight combination and encode it into a binary gene string, which serves as the initial population for evolutionary computation. The proxy environment construction module is used to build a proxy experimental environment. It generates proxy corpus by randomly sampling the original pre-training corpus and selects at least three representative natural language processing tasks as proxy tasks to evaluate the fitness value of the gene string. The evolutionary operation module is used to select gene strings with high fitness values ​​using a roulette wheel method based on the evaluation results of the agent task, and generate the next generation of gene strings through crossover and mutation operations; The iterative optimization and fine-tuning module is used to iteratively execute S2 and S3 until convergence, obtain the optimal sub-model weight combination, and perform task-independent meta-learner fine-tuning training on the original full corpus to obtain the final ensemble model.

[0101] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0102] To implement the above embodiments, the present invention also proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method provided in the foregoing embodiments.

[0103] To implement the above embodiments, the present invention also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the foregoing embodiments.

[0104] To implement the above embodiments, the present invention also proposes a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the foregoing embodiments.

[0105] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0106] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.

[0107] This invention is intended to provide implementation schemes for users to selectively prevent the use or access to personal information data. That is, this disclosure is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information is de-identified to protect user privacy.

[0108] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0109] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0110] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0111] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0112] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0113] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0114] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0115] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0116] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0117] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for optimizing sub-model weights in ensemble learning of a large language model, characterized in that, Includes the following steps: S1, generate the initial sub-model weight combination and encode it into a binary gene string, which serves as the initial population for evolutionary computation; S2. Construct a proxy experimental environment, generate proxy corpus by randomly sampling the original pre-training corpus, and select at least three representative natural language processing tasks as proxy tasks to evaluate the fitness value of the gene string. S3, based on the evaluation results of the agent task, uses the roulette wheel method to select gene strings with high fitness values, and generates the next generation gene strings through crossover and mutation operations; S4 iterates through S2 and S3 until convergence, obtains the optimal sub-model weight combination, and performs task-agnostic meta-learner fine-tuning on the original full corpus to obtain the final ensemble model.

2. The method according to claim 1, characterized in that, The step of generating the initial sub-model weight combination and encoding it into a binary gene string includes: S11, using Bernoulli distribution Independent sampling generates "0" or "1" in a binary string; S12, encode the weight combination of each sub-model into a length of binary strings, where The number of sub-models, The number of weight discretization levels.

3. The method according to claim 1, characterized in that, The steps for constructing the proxy experimental environment include: S21, select SST-2, MNLI, and SQUAD v1.1 as the agent tasks; S22, using a weighted evaluation formula Calculate the fitness value, where and This is the preset task weight coefficient.

4. The method according to claim 1, characterized in that, The steps based on the evaluation results of the agent task include: S31, Set the crossover probability To swap the sub-model weight encodings at corresponding positions in the two gene tuples; S32, Configure mutation probability Each gene bit in the binary string is randomly flipped with that probability.

5. The method according to claim 1, characterized in that, The steps of iteratively executing S2 and S3 until convergence include: S41, the number of iterations is limited to no more than 50; S42, set the sampling rate of the proxy corpus to be no less than 10% and no more than 20%.

6. The method according to claim 1, characterized in that, Also includes: S5 uses GENFUSER as the meta-learner, and inserts data into the sequence...<extra_id_i> Separator and append weight markers to the input The encoding is a fused input, where Indicates the first The weight parameters of each sub-model.

7. A sub-model weight optimization device for ensemble learning of a large language model, characterized in that, include: The initial weight encoding module is used to generate the initial sub-model weight combination and encode it into a binary gene string, which serves as the initial population for evolutionary computation. The proxy environment construction module is used to build a proxy experimental environment. It generates proxy corpus by randomly sampling the original pre-training corpus and selects at least three representative natural language processing tasks as proxy tasks to evaluate the fitness value of the gene string. The evolutionary operation module is used to select high-fitness gene strings based on the evaluation results of the agent task, and generate the next generation of gene strings through crossover and mutation operations; The iterative optimization and fine-tuning module is used to iteratively execute S2 and S3 until convergence, obtain the optimal sub-model weight combination, and perform task-independent meta-learner fine-tuning training on the original full corpus to obtain the final ensemble model.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-6.

Images