Method for improving training effect of large model based on pollution parameter and mixed expert model
By introducing contamination parameters and hybrid expert models (MOE), synthetic data is automatically generated and its quality is dynamically evaluated, solving the static problem of the proportion and weight of synthetic data in large model training and improving the stability and efficiency of training.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING KNOWLEDGE ATLAS TECHNOLOGY CO LTD
- Filing Date
- 2026-03-03
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies cannot flexibly control the quality and proportion of synthetic data in large model training, and lack dynamic adjustment mechanisms, resulting in unstable training results and low efficiency.
By introducing pollution parameters and hybrid expert models (MOE), dynamic scaling and weighting are achieved by automatically generating synthetic data, dynamically evaluating its quality, and adjusting its proportion and weight in training.
It improves the stability, efficiency, and reliability of large model training, ensures the efficient use of synthetic data, avoids interference from low-quality data, and adapts to different task requirements.
Smart Images
Figure CN122132838A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of artificial intelligence, and relates to a method for improving the training effect of a large model, in particular, a method for improving the training effect of a large model based on pollution parameters and a hybrid expert model. BACKGROUND
[0002] With the wide application of large language models (LLM) in complex tasks such as question and answer systems, knowledge reasoning, code generation, automatic decision-making, etc., how to ensure that the large model can utilize synthetic data to achieve efficient, stable and reliable training effect in the training process under the condition of insufficient or unbalanced data has become a core technical bottleneck for the deployment and application of current intelligent systems. Although the existing methods of training large models using synthetic data can compensate for the lack of data to some extent, they still face the following key problems in terms of how to dynamically adjust and weight synthetic data and real data according to data quality and training requirements: 1. Existing synthetic data optimization methods cannot flexibly control the quality and proportion of synthetic data.
[0003] In the existing technical system, when training a large model using synthetic data and real data, it relies on a fixed proportion of synthetic data and real data for mixed training. However, this static proportion setting does not take into account the dynamic changes in the quality of synthetic data. As the complexity of the task increases, the quality of synthetic data is often difficult to guarantee, and its contribution to large model training may also vary with different tasks. Existing systems usually use a fixed proportion of synthetic data and real data for mixed training, which makes it difficult to effectively reduce the negative impact of poor quality synthetic data on training results, leading to unstable reasoning, convergence difficulties and other problems.
[0004] 2. Lack of dynamic adjustment mechanism according to synthetic data quality and task requirements.
[0005] Currently, when training a large model using synthetic data and real data, the dynamic weighting of synthetic data and real data is not considered. During task execution, synthetic data may fluctuate in quality due to factors such as generation algorithm, task characteristics and data type, and existing methods usually use a fixed weighting strategy that cannot be adjusted in real time according to synthetic data quality and task state. The lack of this flexible adjustment mechanism makes it difficult to identify and reduce the weight of low-quality synthetic data in the training process, affecting the stability of the training and the performance of the model.
[0006] 3. Existing methods lack quality evaluation and feedback mechanisms for synthetic data.
[0007] While some studies have attempted to improve the quality of synthetic data through consistency checks or confidence assessments, these approaches are typically passive quality control methods, failing to proactively evaluate the quality of synthetic data and adjust it during training. Existing techniques often rely on heuristic rules or static weighting to evaluate the contribution of synthetic data, but these methods lack real-time monitoring and quality optimization mechanisms for synthetic data. Therefore, it is difficult for systems to effectively prevent poor-quality synthetic data from affecting the training of large models, leading to inconsistent and unstable inference results.
[0008] Based on the above analysis, there is an urgent need for a new method that uses synthetic and real data to improve the training effect of large models. Summary of the Invention
[0009] To address the problems existing in the prior art, this invention proposes a method for improving the training effect of large models based on contamination parameters and hybrid expert models (MOEs). By introducing contamination parameters and MOEs, the method effectively optimizes the proportion and quality of synthetic data in the training of large models and dynamically weights the influence of synthetic data on real data. This effectively solves the problems of static proportion control and low-quality data interference in the prior art, and improves the stability, efficiency and reliability of large model training.
[0010] To achieve the above objectives, the present invention provides the following technical solution: A method for improving the training performance of large models based on contamination parameters and hybrid expert models, characterized by the following steps: S1: Generate synthetic data automatically based on the context of the current task and existing data using a pre-trained large model; S2: Score and evaluate the quality of the synthesized data; S3: Based on the scoring and quality assessment results, the synthesized data is filtered and modified to obtain the modified synthesized data; S4: Train and iteratively optimize the large model using modified synthetic data and real data. During training and iterative optimization, the ratio of modified synthetic data to real data is dynamically adjusted based on dynamically updated contamination parameters, and the weights of modified synthetic data and real data are dynamically adjusted based on a hybrid expert model.
[0011] Preferably, step S2 specifically includes: S21: Use a pre-trained large model to score the semantic accuracy, syntactic correctness and fluency of the synthetic data to obtain semantic accuracy score, syntactic correctness score and fluency score; S22: Using a similarity matching method, the synthesized data is compared with documents in the knowledge base to obtain a factual consistency score.
[0012] Preferably, step S22 specifically includes: S221: Based on the synthesized data, a search is performed in the knowledge base to obtain multiple documents; S222: Encode the synthetic data and multiple documents into vectors using a vector model: S223: Calculate the cosine similarity between the vector of the synthesized data and the vectors of the multiple documents respectively; S224: Calculate a weighted average of the cosine similarity between the vector of the synthesized data and the vectors of multiple documents; S225: Map the results of the weighted average to To obtain a factual consistency score.
[0013] Preferably, in step S4, dynamically adjusting the ratio of the modified synthetic data to the real data based on dynamically updated pollution parameters specifically involves defining the ratio of the modified synthetic data to the real data as the pollution parameter. During the iteration process, the contamination parameter The dynamic update formula is:
[0014]
[0015]
[0016] In the formula, It is the first iteration in the process. Pollution parameters of the wheel; It is the first iteration in the process. Pollution parameters of the wheel; The update step size during the iteration process and ; The coefficient for the trade-off between quality and feedback and ; This serves as the target quality baseline for the synthetic data; It is the first iteration in the process. The quality score of the modified synthetic data used in the round; It is in the After the first round, the same validation set is used to measure the performance of the large model relative to the second round. The change in wheel performance; It is the minimum value of the pollution parameter; It is the maximum value of the pollution parameter; Indicates will Cut off to and between; It is the first iteration in the process. The first round adopted The quality score of the modified synthetic data; It is the first iteration in the process. The total number of modified synthetic data used in the round; , , and These are the iterations of the first... The first round adopted Semantic accuracy score of the modified synthetic data Grammar accuracy score Fluency score Consistency score with facts The weighting coefficients are all greater than or equal to 0; This is the normalization function.
[0017] Preferably, the step S4, which dynamically adjusts the weights of the modified synthetic data and the real data based on the hybrid expert model, specifically includes: S41: Combine the modified synthetic data and the real data to form a sample set, and construct the sample feature vector for each sample in the sample set:
[0018] In the formula, It is the first The sample feature vector of each sample; They are the first The semantic accuracy score, grammatical correctness score, fluency score, and factual consistency score of each sample; For the first Source identification of each sample; It is the first The loss value for each sample; It is the first Gradient norm of each sample; S42: If the hybrid expert model includes real data experts and synthetic data experts, then the gated output of the hybrid expert model is calculated as follows:
[0019]
[0020] In the formula, For the first The gated output vector of each sample; It is the weight matrix of the gated network; It is the bias vector of the gated network; and They are the first The probability of a sample being assigned to a real data expert and the probability of the first sample being assigned to a real data expert. Each sample is assigned a probability to a synthetic data expert, and the sum of the probabilities is 1. S43: Map the gating probabilities to training weights:
[0021]
[0022] The above formula means: if the first If each sample is real data, then its weight is... If the first If each sample is a modified synthetic data set, then its weight is... In the formula, It is a quality gating function.
[0023] Preferably, in step S43, the quality gate function It is a piecewise gated function or smooth gate function In the formula, It is the first The quality score of each sample It is the threshold of quality score. It is the weight scaling factor for samples with low-quality scores. It is the coefficient of the steepness of the quality gating function, σ( ) is the Sigmoid function.
[0024] Preferably, in S4, the loss during the iterative optimization process for:
[0025] In the formula, It is a collection of real data in the sample set; It is a collection of synthetic data modified from the sample set; It comes from the first The loss of real data; It comes from the first The loss of the modified synthetic data; These are the parameters of the large model.
[0026] Furthermore, this invention also provides a system for improving the training effect of large models based on contamination parameters and hybrid expert models, characterized in that it includes: The synthetic data generation module is used to automatically generate synthetic data based on the context of the current task and existing data using a pre-trained large model. A synthetic data scoring and quality assessment module is used to score and assess the quality of the synthetic data; The synthetic data filtering and modification module is used to filter and modify the synthetic data based on the scoring and quality assessment results to obtain the modified synthetic data. The training and iterative optimization module is used to train and iteratively optimize a large model using modified synthetic data and real data. During the training and iterative optimization process, the ratio of modified synthetic data to real data is dynamically adjusted based on dynamically updated contamination parameters, and the weights of modified synthetic data and real data are dynamically adjusted based on a hybrid expert model.
[0027] Furthermore, the present invention also provides a device for improving the training effect of large models based on contamination parameters and hybrid expert models, characterized in that it includes: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method described above for improving the training effect of large models based on contamination parameters and hybrid expert models.
[0028] Finally, the present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the program is executed by a processor, it implements the steps of the method described above for improving the training effect of large models based on contamination parameters and hybrid expert models.
[0029] Compared with existing methods of mixing synthetic data at a fixed ratio and fusing rules, the method proposed in this invention for improving the training effect of large models based on contamination parameters and hybrid expert models has significant advantages in the following aspects: (1) Upgrade from static data ratio control to dynamic ratio adjustment to achieve stable optimization of synthetic data for training large models.
[0030] In existing technologies, most rely on static ratio control to mix synthetic and real data, typically using a fixed 50 / 50 ratio. While this method alleviates the data shortage problem to some extent, it ignores the uncertainty and volatility of synthetic data quality, thus failing to ensure efficient use of synthetic data and potentially leading to instability and inconsistency in the inference process.
[0031] This invention introduces a contamination parameter, which allows the proportion of synthetic data to be dynamically adjusted according to the actual quality of the synthetic data during training, thereby ensuring the positive effect of synthetic data on the training of large models.
[0032] (2) Upgrade from a fixed weighting mechanism to a hybrid expert model MOE mechanism with dynamic weighting to achieve precise adjustment and efficient utilization of synthetic data.
[0033] Traditional methods typically employ fixed weighting or equal voting to fuse synthetic and real data. However, this approach fails to account for the different roles and quality fluctuations of synthetic and real data in the task. Consequently, when the quality of synthetic data is poor, its weights cannot be effectively identified and adjusted, leading to unstable training processes and misleading inference results.
[0034] This invention introduces the MOE (Mixture of Experts) mechanism, which dynamically adjusts the weights of synthetic and real data based on data quality and the performance of the synthetic data. The MOE mechanism can dynamically select appropriate expert models to weight different types of synthetic data, ensuring that the weighting ratio of synthetic to real data adapts to training needs in real time. This mechanism guarantees efficient utilization of synthetic data while avoiding interference from low-quality synthetic data in the training process, thus improving the stability and efficiency of large model training.
[0035] (3) Upgrade from passive quality control to proactive quality assessment and feedback mechanism to achieve intelligent optimization of synthetic data.
[0036] Existing methods often rely on simple quality control techniques, such as consistency checks or confidence level assessments. These methods are typically passive quality control and cannot dynamically evaluate and optimize synthetic data, making it difficult to adjust the quality of synthetic data in a timely manner during training.
[0037] This invention introduces a self-evaluation mechanism for synthetic data, utilizing a large model itself to assess the quality of the generated synthetic data. The large model scores the synthetic data in real time based on multiple dimensions, including semantic accuracy, syntactic correctness, and fluency, and dynamically adjusts the data's weights in training based on the scoring results. This mechanism optimizes the synthetic data through feedback, ensuring that the synthetic data used in training is consistently of high quality, thereby improving the model's training performance.
[0038] (4) Upgrade from static training mode to dynamic feedback mechanism to make the training process adaptive and continuously optimized.
[0039] Traditional multi-model collaborative training typically relies on a fixed collaborative process, making it impossible to optimize and adjust based on feedback during training. This static training model cannot cope with changes in synthetic data quality and task requirements, leading to instability in the inference process and making it difficult for the system to adapt to the needs of different tasks.
[0040] This invention introduces a dynamic feedback mechanism, enabling the training process to be adjusted in real time based on the quality assessment results of the synthetic data. When the system detects problems with the quality of the synthetic data, it automatically adjusts the proportion of synthetic data or optimizes the data quality, thereby avoiding unstable and non-convergent training processes caused by low-quality data. The feedback mechanism ensures the adaptive capability of the training process, enabling rapid adjustments to the synthetic data usage strategy in different tasks, thus improving the overall training efficiency and stability.
[0041] In summary, this invention, by combining contamination parameters with the MOE mechanism, effectively solves problems such as static scaling control and low-quality data interference in existing technologies, thereby improving the stability, efficiency, and reliability of large model training. It has broad application prospects, especially in task scenarios with insufficient or imbalanced data, providing a stable and efficient solution. Attached Figure Description
[0042] Figure 1 This is a flowchart of the method for improving the training effect of large models based on pollution parameters and hybrid expert models according to the present invention.
[0043] Figure 2 This is a schematic diagram of the system structure of the present invention, which improves the training effect of large models based on pollution parameters and hybrid expert models.
[0044] Figure 3 This is a structural block diagram of the device for improving the training effect of large models based on pollution parameters and hybrid expert models according to the present invention. Detailed Implementation
[0045] Before detailing any embodiment of the invention, it should be understood that the invention, in its application, is not limited to the details of the construction and arrangement of the components set forth in the following description or illustrated in the following figures. The invention can have other embodiments and can be practiced or carried out in various ways. Furthermore, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered limiting. The use of “comprising” or “having” and variations thereof in this invention is intended to cover the items set forth below and their equivalents, as well as any additional items. Unless otherwise specified or limited, the terms “installation,” “connection,” “support,” and “linkage,” and variations thereof are used broadly and cover both direct and indirect installation, connection, support, and linking. Moreover, “connection” and “linkage” are not limited to physical or mechanical connections or links.
[0046] Furthermore, firstly, in the disclosure of this invention, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limiting this invention. Secondly, the term "a" should be understood as "at least one" or "one or more," that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple. The term "a" should not be construed as a limitation on the quantity.
[0047] With the widespread application of Large Language Models (LLMs) in complex tasks such as code generation, text generation, and knowledge reasoning, effectively utilizing synthetic data to improve the training performance of large models has become a key technical bottleneck that urgently needs to be overcome. Existing large model training frameworks (such as GPT and BERT) typically rely on large amounts of real data, but in scenarios where data is missing or insufficient, synthetic data is becoming increasingly important as a supplement. However, controlling the quality of synthetic data and effectively utilizing synthetic data in training still present the following technical challenges: (1) The quality of synthetic data is unstable, making it difficult to guarantee a positive impact on the training of large models.
[0048] In training large models, the quality of synthetic data is often difficult to guarantee, especially when the generated text or data is of low quality. This can introduce errors or noise, leading to data inconsistencies during training and affecting the final model's performance. Existing methods for using synthetic data typically rely on mixing synthetic and real data in a fixed ratio, lacking effective monitoring and dynamic adjustment of the synthetic data's quality.
[0049] (2) Lack of dynamic weighting mechanism for synthetic data in large model training.
[0050] Synthetic and real data play different roles in the training of large models. Real data is usually a high-quality source of information, while synthetic data may contain a certain degree of uncertainty. Therefore, existing systems typically use a fixed ratio (e.g., 50 / 50) for synthetic and real data. This fixed strategy cannot be dynamically adjusted according to the quality of synthetic data during training, nor can it fully leverage the advantages of synthetic data; in fact, it may lead to performance degradation.
[0051] (3) Synthetic data lacks a quality assurance mechanism during training.
[0052] Monitoring and optimizing the quality of synthetic data has always been a challenge during the training of large models. Especially when data is insufficient, there is still a lack of theoretical and technical guarantees on how to effectively improve model capabilities through synthetic data without introducing unstable or inconsistent training results.
[0053] To address the aforementioned issues, this invention proposes for the first time a method for improving the training performance of large models based on contamination parameters and MOE mechanisms. By constructing a dynamic adjustment mechanism for the proportion of synthetic data, introducing a large model self-evaluation and dynamic weighting method, and optimizing the quality of synthetic data, the large model can automatically adjust the proportion of synthetic data in training, dynamically weight the impact of real data and synthetic data on the model, and provide stable consistency and convergence, ensuring that synthetic data can improve the training performance of large models.
[0054] Figure 1 A flowchart of the method for improving the training performance of large models based on contamination parameters and hybrid expert models according to the present invention is shown. Figure 1 As shown, the method of the present invention for improving the training effect of large models based on contamination parameters and hybrid expert models includes the following steps: S1: Synthetic data generation.
[0055] In this invention, a pre-trained large model (such as the GPT system, GLM system, etc.) is first used to automatically generate synthetic data based on the context of the current task and the limited amount of existing data. For example, in a text classification task, the large model can generate more similar text based on existing labels; in a sentiment analysis task, the large model can generate text with different sentiment labels.
[0056]
[0057] In the formula, It is a synthetic dataset generated by a large model. It is the first generated by the large model The large model generated a total of [number] synthetic data points. Synthetic data.
[0058] S2: Synthetic data scoring and quality assessment.
[0059] Considering the poor quality of some generated data, this invention requires scoring and quality assessment of the synthesized data to facilitate its selection and modification. The scoring and quality assessment of the synthesized data specifically includes: 1. Score for semantics, syntax, and fluency.
[0060] The generated synthetic data is scored, mainly including: (1) Is the semantic correct? Check whether the semantics of the generated synthetic data are reasonable and whether they meet the task requirements.
[0061] (2) Is the grammar correct? Check whether there are spelling errors or grammatical problems in the generated synthetic data.
[0062] (3) Fluency: Check whether the generated synthetic data conforms to the natural fluency of the language and whether the structure is clear.
[0063] In this invention, the scoring mechanism within a pre-trained large model can be directly used to score the semantic accuracy, syntactic correctness, and fluency of the synthesized data, and the scoring results can be normalized to obtain the semantic accuracy score. Grammar accuracy score and fluency score .
[0064] For example, the following prompt word template can be used to score using a pre-trained large model.
[0065] Please rate the following text comprehensively, and give a score for each aspect (from 1 to 10, where 1 is very poor and 10 is very good): 1. Semantic correctness: Check whether the text is logical and whether there are any semantic errors.
[0066] 2. Misspelling check: Check the text for spelling errors or grammatical problems.
[0067] 3. Fluency: Check whether the text flows naturally and whether it conforms to the structure of the language.
[0068] Please provide a rating for each aspect.
[0069] 2. Evaluation of the authenticity of synthetic data.
[0070] The authenticity of the generated data is verified using an external knowledge base (such as RAG, Retriever-Augmented Generation). In this invention, a similarity matching method can be used to compare the synthesized data with documents in the knowledge base to obtain a factual consistency score.
[0071] The evaluation of the authenticity of synthetic data specifically includes: (1) Based on the synthesized data, a search is performed in the knowledge base to obtain multiple documents related to it.
[0072] For example, for the generated data In the knowledge base The search retrieved the top-K documents:
[0073] In the formula, Based on generated data From the knowledge base The document collection was retrieved from the search. From the knowledge base The search yielded the first One document.
[0074] (2) Encode the synthetic data and multiple documents into vectors using a vector model.
[0075] Use pre-trained language models (such as SBERT / BGE) to encode synthetic data and multiple documents into vectors:
[0076] In the formula, It generates data The vector, It is the first A vector of documents.
[0077] (3) Calculate the cosine similarity between the vector of the synthesized data and the vectors of the multiple documents respectively.
[0078]
[0079] In the formula, yes and The cosine similarity.
[0080] (4) The cosine similarity between the vector of the synthesized data and the vectors of multiple documents is weighted and averaged.
[0081]
[0082] In the formula, It is the similarity after weighted average; yes The weighting coefficients can be simply taken as uniform weights, i.e. .
[0083] (5) Map the weighted average result to To obtain a factual consistency score.
[0084] Cosine similarity is generally in In order to achieve the semantic accuracy score after normalization Grammar accuracy score and fluency score Consistency requires mapping the weighted average result to... To obtain a factual consistency score .
[0085] For example, the following linear mapping method can be used for mapping:
[0086] In the formula, It is the factual consistency score after mapping.
[0087] S3: Filtering and modifying synthetic data.
[0088] The synthesized data is filtered and modified based on the scoring and quality assessment results to obtain the modified synthesized data.
[0089] In this invention, the synthesized data after scoring and quality assessment is filtered. The filtering criteria include: semantic accuracy score, grammatical correctness score, fluency score, and factual consistency score.
[0090] In the specific screening process, thresholds can be set for semantic accuracy, grammatical correctness, fluency, and factual consistency. For synthetic data with scores below the thresholds, pre-trained large models can be used to modify and refine it until the scores of the modified and refined synthetic data exceed the thresholds.
[0091] After filtering and modifying the synthetic data, the modified synthetic dataset is obtained:
[0092] In the formula, It is a high-quality synthetic dataset that has been screened and modified.
[0093] S4: Training and iterative optimization.
[0094] Training and iteratively optimizing large models using modified synthetic and real data can help improve their task performance when real data is insufficient. Furthermore, training with a mix of synthetic and real data helps large models achieve better generalization capabilities on tasks.
[0095] Unlike existing technologies, in this invention, during the training and iterative optimization process, the ratio of modified synthetic data to real data is dynamically adjusted based on dynamically updated pollution parameters, and the weights of modified synthetic data and real data are dynamically adjusted based on a hybrid expert model.
[0096] The dynamic adjustment of the ratio between modified synthetic and real data based on dynamically updated pollution parameters is specifically defined as the ratio of modified synthetic and real data to pollution parameters. Its value ranges between [0, 1], where, This indicates that the training sample set consists entirely of real data. This indicates that the training sample set consists entirely of modified synthetic data.
[0097] During the iteration process, pollution parameters The dynamic update formula is:
[0098] In the formula, It is the first iteration in the process. Pollution parameters of the wheel; It is the first iteration in the process. Pollution parameters of the wheel; The update step size during the iteration process and ; This is a tradeoff factor between quality and feedback, which can be preset as needed. ; The target quality baseline for the synthetic data can be set in advance as needed; It is the first iteration in the process. The quality score of the modified synthetic data used in the round; It is in the After the first round, the same validation set is used to measure the performance of the large model relative to the second round. The change in wheel performance; It is the minimum value of the pollution parameter; It is the maximum value of the pollution parameter; Indicates will Cut off to and between.
[0099] In this invention, It can be the first The average quality score of all modified synthetic data used in the round:
[0100]
[0101] In the formula, It is the first iteration in the process. The first round adopted The quality score of the modified synthetic data; It is the first iteration in the process. The total number of modified synthetic data used in the round; , , and These are the iterations of the first... The first round adopted Semantic accuracy score of the modified synthetic data Grammar accuracy score Fluency score Consistency score with facts The weighting coefficients are all greater than or equal to 0; This is the normalization function.
[0102] The The meaning is: in the first After each round, the same validation set is used to evaluate the large model "relative to the previous round," that is, the [number]th round. The change in wheel performance, that is, through the first... After rounds of training, did the large model improve or deteriorate?
[0103] To be more formal, let the verification metric be... (e.g., Accuracy / F1 / ROUGE, or negative loss on the validation set), the first Parameters obtained from round training The parameters obtained in the previous round If the metric is "the higher the better" (Accuracy / F1, etc.): If the metric is "the smaller the better" (verification loss) ): After this definition, It still indicates "things have gotten better." In other words, This indicates that the strategy improved the validation set performance, and the value could be appropriately increased. (More synthetic data is used); The worsening effect indicates that the synthesized data may be contaminated and needs to be reduced. (Less use of synthetic data).
[0104] Furthermore, in this invention, the MOE mechanism allows for the dynamic adjustment of the contribution of each training sample to the large model based on its characteristics. For example, for certain specific tasks, synthetic data may have a greater impact on the training of the large model, while for other tasks, real data may be more important. The core idea of the MOE mechanism is to select one or more expert networks based on the features of the input training samples. These expert networks process different parts of the real and synthetic data respectively and weight them for the final model output.
[0105] The specific aspects of dynamically adjusting the weights of the modified synthetic data and real data based on the hybrid expert model include: 1. Combine the modified synthetic data and real data to form a sample set, and construct the sample feature vector for each sample in the sample set:
[0106] In the formula, It is the first The sample feature vector of each sample; They are the first The semantic accuracy score, grammatical correctness score, fluency score, and factual consistency score of each sample; For the first Source identification of each sample; It is the first The loss value for each sample; It is the first Gradient norm of each sample.
[0107] Among them, if the first Each sample is the modified synthetic data, then The scoring can be performed using the synthetic data scoring and quality assessment method described in step S2; if the first If each sample is real data, then This can be a default constant and can be set as needed. Furthermore, For the first The source identifier of the first sample is used to identify the first... Is the sample data modified synthetic data or real data? It can be the first The historical mean loss of the nth sample is used to measure the loss of the nth sample. The difficulty or importance of individual samples in training large models. It can be the first The gradient norm of the nth sample, or its approximation, is used to reflect the gradient norm of the nth sample. The contribution strength of each sample to the parameter update of the large model.
[0108] 2. If the hybrid expert model includes real data experts and synthetic data experts, then the gated output of the hybrid expert model is calculated as follows:
[0109]
[0110] In the formula, For the first The gated output vector of each sample; It is the weight matrix of the gated network; It is the bias vector of the gated network; and They are the first The probability of a sample being assigned to a real data expert and the probability of the first sample being assigned to a real data expert. The probability that a sample is assigned to a synthetic data expert, and the sum of the two is 1.
[0111] 3. Map the gating probabilities to training weights:
[0112]
[0113] The above formula means: if the first If each sample is real data, then its weight is... If the first If each sample is a modified synthetic data set, then its weight is... .
[0114] in, It is a quality gating function. In this invention, the quality gating function... It can be a piecewise gated function .
[0115] The quality gate function It can also be a smooth gating function. .
[0116] In the formula, It is the first The quality score of each sample; It is a threshold for the quality score and it can be preset; It is the weight scaling factor for samples with low-quality scores; It is the coefficient of the steepness of the quality gating function; σ( ) is the Sigmoid function.
[0117] Therefore, the loss during the iterative optimization process for:
[0118] In the formula, It is a collection of real data in the sample set; It is a collection of synthetic data modified from the sample set; It comes from the first The loss of real data; It comes from the first The loss of the modified synthetic data; These are the parameters of the large model.
[0119] As training progresses, MOE dynamically adjusts the expert selection and weights for each sample, further optimizing the impact of synthetic and real data on the training results.
[0120] S5: Evaluation and Results Analysis.
[0121] In this invention, after training and iteratively optimizing a large model, its performance can be evaluated. For example, standard task evaluation metrics (such as accuracy, F1 score, perplexity, etc.) can be used to assess the large model's performance on the task. By evaluating the large model, the impact of synthetic data on task performance can be analyzed.
[0122] This invention combines innovative contamination parameters with the MOE mechanism to dynamically adjust the proportion and weight of synthetic data usage, significantly improving data utilization efficiency, training stability, and result reliability during large-scale model training. This method provides an innovative solution for large-scale, complex model optimization, with broad application prospects and engineering value, and has wide application value in the fields of Large Language Models (LLM) and multi-agent systems.
[0123] Figure 2 A schematic diagram of the system for improving the training performance of large models based on contamination parameters and hybrid expert models according to the present invention is shown. Figure 2 As shown, the system of the present invention for improving the training effect of large models based on contamination parameters and hybrid expert models includes: 1. Synthetic data generation module.
[0124] The synthetic data generation module is used to automatically generate synthetic data based on the context of the current task and existing data using a pre-trained large model.
[0125] 2. Synthetic data scoring and quality assessment module.
[0126] The synthetic data scoring and quality assessment module is used to score and assess the quality of the synthetic data.
[0127] 3. Synthetic data filtering and modification module.
[0128] The synthetic data filtering and modification module is used to filter and modify the synthetic data based on the scoring and quality assessment results to obtain the modified synthetic data.
[0129] 4. Training and Iterative Optimization Module.
[0130] The training and iterative optimization module is used to train and iteratively optimize a large model using modified synthetic data and real data. During the training and iterative optimization process, the ratio of modified synthetic data to real data is dynamically adjusted based on dynamically updated contamination parameters, and the weights of modified synthetic data and real data are dynamically adjusted based on a hybrid expert model.
[0131] Furthermore, this invention also provides a device for improving the training effect of large models based on contamination parameters and hybrid expert models. For example... Figure 3As shown, the device of the present invention for improving the training effect of large models based on contamination parameters and hybrid expert models includes: a memory 11 for storing one or more programs; one or more processors 12; when the one or more programs are executed by the one or more processors 12, the one or more processors 12 implement the method of the present invention for improving the training effect of large models based on contamination parameters and hybrid expert models.
[0132] Finally, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for improving the training effect of large models based on contamination parameters and hybrid expert models in the present invention.
[0133] The computer-readable storage medium includes both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. As defined in this invention, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0134] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0135] The steps of the methods or algorithms described in conjunction with the embodiments disclosed in this invention can be implemented in hardware, software modules executed by a processor, or a combination of both. The software modules can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art.
[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Those skilled in the art can modify or make equivalent substitutions to the technical solutions of the present invention based on the concept of the present invention, without departing from the essence and scope of the technical solutions of the present invention.
Claims
1. A method for improving the training effect of large models based on contamination parameters and hybrid expert models, characterized in that, Includes the following steps: S1: Generate synthetic data automatically based on the context of the current task and existing data using a pre-trained large model; S2: Score and evaluate the quality of the synthesized data; S3: Based on the scoring and quality assessment results, the synthesized data is filtered and modified to obtain the modified synthesized data; S4: Train and iteratively optimize the large model using modified synthetic data and real data. During training and iterative optimization, the ratio of modified synthetic data to real data is dynamically adjusted based on dynamically updated contamination parameters, and the weights of modified synthetic data and real data are dynamically adjusted based on a hybrid expert model.
2. The method for improving the training effect of large models based on contamination parameters and hybrid expert models according to claim 1, characterized in that, Step S2 specifically includes: S21: Use a pre-trained large model to score the semantic accuracy, syntactic correctness and fluency of the synthetic data to obtain semantic accuracy score, syntactic correctness score and fluency score; S22: Using a similarity matching method, the synthesized data is compared with documents in the knowledge base to obtain a factual consistency score.
3. The method for improving the training effect of large models based on contamination parameters and hybrid expert models according to claim 2, characterized in that, Step S22 specifically includes: S221: Based on the synthesized data, a search is performed in the knowledge base to obtain multiple documents; S222: Encode the synthetic data and multiple documents into vectors using a vector model: S223: Calculate the cosine similarity between the vector of the synthesized data and the vectors of the multiple documents respectively; S224: Calculate a weighted average of the cosine similarity between the vector of the synthesized data and the vectors of multiple documents; S225: Map the results of the weighted average to To obtain a factual consistency score.
4. The method for improving the training effect of large models based on contamination parameters and hybrid expert models according to claim 1, characterized in that, In step S4, dynamically adjusting the ratio of the modified synthetic data to the real data based on the dynamically updated pollution parameters specifically involves defining the ratio of the modified synthetic data to the real data as the pollution parameters. During the iteration process, the contamination parameter The dynamic update formula is: In the formula, It is the first iteration in the process. Pollution parameters of the wheel; It is the first iteration in the process. Pollution parameters of the wheel; The update step size during the iteration process and ; The coefficient for the trade-off between quality and feedback and ; The target quality baseline for synthetic data; It is the first iteration in the process. The quality score of the modified synthetic data used in the round; It is in the After the first round, the same validation set is used to measure the performance of the large model relative to the second round. The change in wheel performance; It is the minimum value of the pollution parameter; It is the maximum value of the pollution parameter; Indicates will Cut off to and between; It is the first iteration in the process. The first round adopted The quality score of the modified synthetic data; It is the first iteration in the process. The total number of modified synthetic data used in the round; , , and These are the iterations of the first... The first round adopted Semantic accuracy score of the modified synthetic data Grammar accuracy score Fluency score Consistency score with facts The weighting coefficients are all greater than or equal to 0; This is the normalization function.
5. The method for improving the training effect of large models based on pollution parameters and hybrid expert models according to claim 1, characterized in that, The specific steps in step S4, which involve dynamically adjusting the weights of the modified synthetic data and the real data based on a hybrid expert model, include: S41: Combine the modified synthetic data and the real data to form a sample set, and construct the sample feature vector for each sample in the sample set: In the formula, It is the first The sample feature vector of each sample; They are the first The semantic accuracy score, grammatical correctness score, fluency score, and factual consistency score of each sample; For the first Source identification of each sample; It is the first The loss value for each sample; It is the first Gradient norm of each sample; S42: If the hybrid expert model includes real data experts and synthetic data experts, then the gated output of the hybrid expert model is calculated as follows: In the formula, For the first The gated output vector of each sample; It is the weight matrix of the gated network; It is the bias vector of the gated network; and They are the first The probability of a sample being assigned to a real data expert and the probability of the first sample being assigned to a real data expert. Each sample is assigned a probability to a synthetic data expert, and the sum of the probabilities is 1. S43: Map the gating probabilities to training weights: The above formula means: if the first If each sample is real data, then its weight is... If the first If each sample is a modified synthetic data set, then its weight is... In the formula, It is a quality gating function.
6. The method for improving the training effect of large models based on pollution parameters and hybrid expert models as described in claim 5, characterized in that, In step S43, the quality gate function It is a piecewise gated function Or smooth gate function In the formula, It is the first The quality score of each sample It is the threshold of quality score. It is the weight scaling factor for samples with low-quality scores. It is the coefficient of the steepness of the quality gating function, σ( ) is the Sigmoid function.
7. The method for improving the training effect of large models based on pollution parameters and hybrid expert models as described in claim 6, characterized in that, In S4, the loss during the iterative optimization process for: In the formula, It is a collection of real data in the sample set; It is a collection of synthetic data modified from the sample set; It comes from the first The loss of real data; It comes from the first The loss of the modified synthetic data; These are the parameters of the large model.
8. A system for improving the training effect of large models based on contamination parameters and hybrid expert models, characterized in that, include: The synthetic data generation module is used to automatically generate synthetic data based on the context of the current task and existing data using a pre-trained large model. A synthetic data scoring and quality assessment module is used to score and assess the quality of the synthetic data; The synthetic data filtering and modification module is used to filter and modify the synthetic data based on the scoring and quality assessment results to obtain the modified synthetic data. The training and iterative optimization module is used to train and iteratively optimize a large model using modified synthetic data and real data. During the training and iterative optimization process, the ratio of modified synthetic data to real data is dynamically adjusted based on dynamically updated contamination parameters, and the weights of modified synthetic data and real data are dynamically adjusted based on a hybrid expert model.
9. A device for improving the training effect of large models based on pollution parameters and hybrid expert models, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for improving the training effect of large models based on contamination parameters and hybrid expert models as described in any one of claims 1-7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method for improving the training effect of large models based on contamination parameters and hybrid expert models as described in any one of claims 1-7.