Large model optimization method and device for education

Abstract

The invention belongs to the technical field of intelligent education, and relates to a large model optimization method and device for education, and the method comprises the steps: fitting real data based on a Gaussian model to generate synthetic data, evaluating the consistency of the synthetic data and the real data through distribution similarity measurement, obtaining the synthetic data meeting the consistency requirement, and forming a training set; and model parameters of the large model are updated by superposing Gaussian noise on a stochastic gradient through a Langhan dynamics mechanism, and the large model is trained in combination with the training set. The probability that a large model for education generates illusion problems can be reduced.

Description

Technical Field

[0001] This invention belongs to the field of intelligent education technology, and relates to a method and apparatus for optimizing large-scale models for education. Background Technology

[0002] With the rapid development of AI large-scale model technology, AI large-scale model products are also continuously growing, making life more convenient for the public. AI products are also increasingly entering all aspects of education, facilitating teaching, learning, examination, evaluation, and management in the education industry. This not only makes it easier for teachers and students to carry out their work, but also continuously improves the management level of schools and teachers.

[0003] However, large AI models suffer from a serious illusion problem for the following reasons: The multidimensional nature of educational data: Educational data has multiple dimensions, such as subjects, practices, diet, mental state, physical fitness, etc., which makes it necessary for large AI models to process complex and diverse knowledge systems. This can lead to knowledge confusion or fabrication when generating content, such as incorrectly combining different knowledge points to generate seemingly reasonable but actually wrong answers, creating illusion problems.

[0004] Differences in the intelligent development of educational institutions: The data quality, technical architecture and resource investment of each educational institution are different, and the data between the institutions is not interconnected. They lack adaptation to educational scenarios and real-time interactive correction, making it difficult to handle complex teaching needs and ultimately exacerbating the generation of illusions.

[0005] Domain limitations: Knowledge in professional fields such as education is updated rapidly, making it difficult for large models to keep up with the latest developments.

[0006] Training data issues: The accuracy of large AI models depends on model training, and the quality of training data directly determines the accuracy of the output of large AI models. However, training data may contain outdated, erroneous, contradictory, or even fictitious information. For example, there may be too few samples in the training data, resulting in insufficient coverage.

[0007] Model mechanism issues: Large AI models employ a regression-based generation framework, aiming to maximize the probability of the output sequence rather than ensuring factual accuracy. This results in the model outputting text that is statistically more reasonable but actually erroneous. Autoregressive generation prioritizes high-probability words over facts, lacking real-time verification capabilities.

[0008] Model training bias: The loss function of large AI models only measures the similarity between the generated text and the training data, rather than the accuracy of reality. This can lead the model to sacrifice realism in pursuit of fluency, thus reinforcing erroneous outputs. For example, large model training may use prompts to induce the model to generate incorrect content.

[0009] The illusion of large models causes them to output seemingly reasonable but actually erroneous content. This not only provides teachers and students with incorrect information, misleads teaching direction, and affects student enthusiasm, but may also lead to wrong decision-making solutions, seriously affecting school management.

[0010] Therefore, reducing the probability of large models used in education causing hallucinations is an important issue that the industry urgently needs to address. Summary of the Invention

[0011] To reduce the probability of large models used in education generating hallucination problems, this invention provides a method and apparatus for optimizing large models used in education.

[0012] According to a first aspect of the present invention, a method for optimizing large models for education is provided, comprising: Synthetic data generation steps: Generate synthetic data by fitting real data with a Gaussian model, evaluate the consistency between synthetic data and real data through distribution similarity measurement, obtain synthetic data that meets the consistency requirements, and form a training set; Large model training steps: Update the model parameters of the large model by superimposing Gaussian noise on the stochastic gradient through the Langevin dynamics mechanism, and train the large model in combination with the training set.

[0013] In one possible implementation, the large model optimization method for education further includes an iterative optimization step: repeating the synthetic data generation step and the large model training step until convergence is achieved.

[0014] In one possible implementation, the synthetic data generation step includes: Initialize the first parameter, which includes the number of clusters, cluster centers, cluster weights, and covariance matrix; Based on the initial parameters, synthetic data is generated by combining the probability density function of the Gaussian mixture model with real data. The distribution similarity between synthetic data and real data is assessed by using the posterior distribution KL divergence. Synthetic data with a distribution similarity greater than a similarity threshold are added to the training set.

[0015] In one possible implementation, the probability density function of the Gaussian mixture model is constructed by the following equation:

[0016] in, It is the first The weights of each cluster, ; The total number of clusters; and They are the first The mean vector and covariance matrix of each cluster; For data The probability density.

[0017] In one possible implementation, the posterior distribution KL divergence is constructed by the following formula:

[0018] in, The difference between the distribution of real data and the distribution of synthetic data; For the actual data belonging to the first The proportion of individual clusters , For the quantity of real data, For real data Belongs to the Cluster The posterior probability; For the synthetic data belonging to the first The proportion of individual clusters , For the amount of synthetic data, For synthetic data Belongs to the Cluster The posterior probability.

[0019] In one possible implementation, the large model training step includes: Parameter update steps: Update the model parameters of the large model by superimposing Gaussian noise on the stochastic gradient using the Langevin dynamics mechanism; First evaluation step: Randomly select one or more synthetic data sets from the training set as validation sets to input into the updated large model; Based on the output of the large model, evaluate the model through parameter updates to determine whether the parameter update convergence condition has been met; If the parameter update convergence condition has not been met, return to the parameter update step; If the parameter update convergence condition has been met, execute the training step. Training steps: Train a large model that has reached the convergence condition for parameter update using the training set; The third evaluation step is to determine whether the trained large model has reached the training convergence condition based on the training evaluation model. If the training convergence condition has not been reached, the parameter update step is returned. If the training convergence condition has been reached, the trained large model is output.

[0020] In one possible implementation, the large model training step further includes: Steps for building a multi-task generation and evaluation model: Construct a multi-task generation and evaluation model; The second evaluation step is to determine whether the output after training in the large model training step has reached the first evaluation convergence condition through the multi-task generation evaluation model. If the first evaluation convergence condition has not been reached, the parameter update step is returned. If the first evaluation convergence condition has been reached, the synthetic data in the training set that has reached the first evaluation convergence condition is used as the training set for the next iteration.

[0021] In one possible implementation, the parameter update step includes: Initialize the second parameter, which includes the model parameters, learning rate, and number of iterations of the large model; Multiple synthetic data points are randomly selected from the training set and input into the initialized large model to obtain gradient estimates of the model parameters of the large model. Gaussian noise is superimposed on the gradient of the model parameters of the large model, and discretized Langevin dynamic equations are introduced to update the model parameters of the large model.

[0022] In one possible implementation, the step of superimposing Gaussian noise on the gradient of the model parameters of the large model and introducing discretized Langevin dynamic equations to update the model parameters of the large model includes: The model parameters of the large model are updated using the following formula:

[0023] in, Index for iteration count; For the first The momentum term of the next iteration; For the first The second parameter of the next iteration; The momentum decay coefficient; The learning rate; For the loss function L in the current second parameter The gradient below; For the first Gaussian noise introduced in the next iteration , Noise intensity.

[0024] In one possible implementation, the parameter update step further includes: Gradient clipping is performed on the gradient of the second parameter.

[0025] In one possible implementation, the parameter update step further includes: Cosine annealing is introduced to adaptively adjust the learning rate.

[0026] In one possible implementation, the step of introducing cosine annealing to adaptively adjust the learning rate includes: The learning rate is adaptively adjusted using the following formula:

[0027] in, For the first The learning rate for each iteration; Minimum learning rate; The maximum learning rate; This represents the total number of iterations. These are the learning parameters.

[0028] In one possible implementation, the large model optimization method for education further includes a monitoring step to monitor the stability of the large model output; if the stability difference reaches a difference threshold, the process returns to the synthetic data generation step.

[0029] In one possible implementation, the large model optimization method for education further includes a monitoring step, which includes: Large-scale monitoring model; Construct an evaluation model; Determine whether the second evaluation convergence condition has been met; If the second evaluation convergence condition is not met, return to the parameter update step or the synthetic data generation step.

[0030] In one possible implementation, the large model optimization method for education further includes: Collect feedback information from user terminals; Adjust the model parameters of the large model based on the feedback information.

[0031] According to a second aspect of the present invention, a large model optimization apparatus for education is provided, comprising a synthetic data generation module and a training module: The synthetic data generation module is configured to generate synthetic data based on fitting real data using a Gaussian model, evaluate the consistency between the synthetic data and the real data through a distribution similarity metric, and obtain synthetic data that meets the consistency requirements to form a training set. The training module is configured to update the model parameters of the large model by superimposing Gaussian noise on the stochastic gradient through the Langevin dynamics mechanism, and to train the large model by combining the training set generated by the synthetic data generation module.

[0032] In one possible implementation, the large model optimization device for education further includes a monitoring module configured to monitor the stability of the trained large model and, when the stability difference reaches a difference threshold, send an instruction to generate synthetic data to the synthetic data generation module.

[0033] In one possible implementation, the large model optimization device for education further includes a feedback module configured to receive feedback information from a user terminal and transmit the feedback information to a synthetic data generation module and / or a training module.

[0034] According to a third aspect of the present invention, a large model optimization apparatus for education is provided, comprising a first cluster, a feature extraction module, and a second cluster, wherein the first cluster comprises a plurality of data processing nodes, and the second cluster comprises a plurality of computing nodes and at least one training node, wherein: The data processing node is configured to collect real data; The feature extraction module is configured to extract multiple features from the real data output by the data processing node and convert them into learnable feature vectors. The computing node is configured to generate synthetic data by fitting the feature vector of the real data extracted by the feature extraction module based on the Gaussian model. The consistency between the synthetic data and the real data is evaluated by the distribution similarity metric, and synthetic data that meets the consistency requirements is obtained to form a training set. The training nodes are configured to update the model parameters of the large model by superimposing Gaussian noise on the stochastic gradient through the Langevin dynamics mechanism, and to train the large model by combining the training set constructed by the computing nodes.

[0035] According to a fourth aspect of the present invention, a computing device is provided, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the large model optimization method for education described above.

[0036] According to a fifth aspect of the present invention, a computer-readable storage medium is provided that stores computer-executable instructions which, when executed by a processor, implement the steps of the large model optimization method for education described above.

[0037] According to a sixth aspect of the present invention, a computer program is provided, wherein when the computer program is executed in a computer, the computer performs the steps of the above-described large model optimization method for education.

[0038] The large-scale model optimization method for education described in this invention, based on Gaussian model fitting of real data and screening through distribution similarity measurement in the synthetic data generation step, can alleviate the fragmentation and unstructured problems caused by the multi-dimensional nature of educational data, reduce model overfitting caused by institutional data bias (such as sample imbalance, regional content limitations), ensure that training data covers more comprehensive educational scenarios (such as different subject knowledge points, teaching methods), and reduce fictitious associations caused by statistical regularity guessing.

[0039] The large-scale model optimization method for education described in this invention employs the Langevin dynamics mechanism to superimpose Gaussian noise during gradient updates in the large-scale model training step. This enhances the model's robustness to data disturbances, avoids over-reliance on specific data patterns from a single institution (such as a certain type of question or teaching case), reduces the "illusion of knowledge obsolescence" caused by differences in intelligence levels (such as solidifying old textbook content), and improves the model's logical reasoning ability in open educational tasks (such as interdisciplinary problem solving).

[0040] The large model optimization method for education described in this invention solves the problems of insufficient training data and domain limitations by synthesizing data through Gaussian models; it generates synthetic data through Gaussian processes to simulate the distribution of real data and reduce the risk of illusion; and it adds Gaussian noise during training to enhance the robustness of the model and make the output closer to reality.

[0041] This invention not only reduces the large model illusion problem by ensuring real-time and accurate data, but also detects, evaluates, and adjusts large model products at various stages of large model training and use, significantly reducing the large model illusion problem and thus better serving students, faculty, and the campus. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating an embodiment of the large model optimization method for education described in this invention; Figure 2 This is a flowchart illustrating a preferred embodiment of the large model optimization method for education described in this invention; Figure 3 This is a schematic block diagram of one embodiment of the large model optimization device for education described in this invention; Figure 4 This is a schematic block diagram of another embodiment of the large model optimization device for education described in this invention; Figure 5 This is a schematic block diagram of one embodiment of the computing device described in this invention; Figure 6 This is a schematic block diagram of another embodiment of the computing device described in this invention; The system comprises: 1. Synthetic data generation module; 2. Training module; 3. Monitoring module; 4. Feedback module; 10. First cluster; 11. Data processing node; 20. Feature extraction module; 30. Second cluster; 31. Computing node; 32. Training node; 100. Computing device; 101. Real data; 102. Synthetic data; 103. Training set; 104. Model parameters; 105. Large model; 110. Memory; 120. Processor; 130. Bus; 140. Access device; 150. Database; 160. Network. Detailed Implementation

[0043] Numerous specific details are set forth in the following description to provide a full understanding of the invention. However, the invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0044] The terminology used in one or more embodiments of the present invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of the invention. The singular forms “a” and “the” as used in one or more embodiments of the invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of the invention refers to and includes any or all possible combinations of one or more associated listed items.

[0045] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of the present invention, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of the present invention, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0046] The foregoing has described specific embodiments of the invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0047] Existing AI large models are prone to producing illusions due to problems with training data, model mechanisms, model training bias, and domain limitations. This can provide teachers and students with incorrect information, mislead teaching directions, affect student motivation, and even lead to incorrect decision-making solutions that seriously impact school management.

[0048] To overcome the negative impact of the illusion of large models on teaching, learning, and management in educational settings, such as... Figure 1 As shown, this invention provides a large-scale model optimization method for education, comprising: Step S1: Generate synthetic data based on the features of real data and construct a training set; Step S2 involves updating the model parameters of the large model by superimposing Gaussian noise on the stochastic gradient using the Langevin dynamics mechanism, and then iteratively training the large model using the training set.

[0049] The large-scale model optimization method for education described in this invention alleviates the hallucination problem at its root through data augmentation and noise injection training: Regarding the training data issue: Step S1 generates synthetic data, which essentially expands the high-quality training samples, covers more real-world scenarios, and reduces model illusions caused by insufficient or outdated data.

[0050] Regarding model mechanism and training bias: In step S2, Langevin dynamics introduces Gaussian noise to prevent the model parameters from overfitting the statistical bias or error patterns in the training data, thereby generating a more robust output that is less susceptible to being biased by erroneous data and improving generalization ability.

[0051] Addressing domain limitations: Synthetic data can simulate the latest domain knowledge, while noise injection enhances the model's adaptability to unknown or new data. The combination of the two helps the model adapt to knowledge updates more quickly and reduces the illusion caused by domain lag.

[0052] In one feasible embodiment, after step S2, the above-described large model optimization method for education further includes: Iterative optimization steps: Repeat steps S1 and S2 until a convergence condition is reached. This convergence condition can be reaching the maximum number of iterations, reaching the training time limit, the stability of cross-validation results meeting a preset threshold, meeting a set evaluation criterion, or any combination of the above convergence conditions. Iterative optimization of synthetic data and model training gradually reduces illusions, resulting in more stable model parameters and outputs closer to reality.

[0053] The large-scale model optimization method for education described in this invention involves repeated synthetic data generation and model training. This allows for dynamic correction of the matching degree between data distribution and model parameters, gradually narrowing the gap between synthetic data and real educational scenarios. It also alleviates the output ambiguity caused by the openness of educational goals, making the model-generated content more aligned with teaching logic and factual basis, and reducing the probability of illusion problems.

[0054] The following section provides a detailed explanation of each step in the large-scale model optimization method used in education: Step S1, Synthetic Data Generation Steps: In one feasible embodiment, step S1 includes a plurality of the following steps: Extract features from real data, including one or more of the mean, variance, and covariance matrix; Synthetic data is generated based on features from real data; Determine whether the synthesized data meets the preset evaluation criteria, which include statistical consistency and / or distribution similarity; If the synthetic data meets the preset evaluation criteria, the synthetic data will be included in the training set. If the synthesized data does not meet the preset evaluation criteria, the process returns to the step of generating synthesized data based on features from real data.

[0055] The synthetic data generation step described in this invention generates synthetic data based on features of real data. Through consistency verification, high-fidelity synthetic data is generated, preventing erroneous information from contaminating the model and fundamentally alleviating the model illusion problem. Synthetic data effectively supplements the lack of real samples, covers long-tail scenarios, and reduces the illusion caused by data sparsity. As additional training samples, synthetic data forces more robust model parameter updates, reducing overfitting to specific biases in the training data.

[0056] In one feasible embodiment, step S1 includes: Synthetic data is generated by fitting real data using a Gaussian model; The distribution similarity between synthetic data and real data is assessed using distribution similarity metrics. Determine whether the distribution similarity of the synthesized data reaches a similarity threshold; If the distribution similarity reaches the similarity threshold, the synthetic data will be included in the training set. If the distribution similarity does not reach the similarity threshold, the step of generating synthetic data based on features from real data is returned.

[0057] This invention generates synthetic data by fitting real data using a Gaussian model, preserving the real correlation between variables and preventing the model from learning false statistical patterns.

[0058] In a preferred embodiment, step S1 includes: Initialize the first parameter, which includes the number of clusters, cluster centers, cluster weights, and covariance matrix; Based on the initial parameters, synthetic data is generated by combining the probability density function of the Gaussian mixture model with real data. The distribution similarity between synthetic data and real data is assessed by using the posterior distribution KL divergence. Synthetic data with a distribution similarity greater than a similarity threshold are added to the training set.

[0059] This invention generates synthetic data based on Gaussian Mixture Models (GMMs) and uses the posterior distribution KL divergence to evaluate the distributional similarity between the synthetic data and the real data, ensuring that the generated data conforms to the true distribution. This effectively solves the problems of insufficient training data, model mechanism bias, and domain limitations, and reduces the risk of model illusion. It supports multi-dimensional data generation and is suitable for large-scale data expansion scenarios.

[0060] In one feasible embodiment, the probability density function of the Gaussian mixture model is constructed by the following equation (1): (1) in, It is the first The weights of each cluster, ; The total number of clusters; and They are the first The mean vector and covariance matrix of each cluster; For data The probability density; For the first The probability density function of each cluster represents a multidimensional normal distribution, which is an approximation of the distribution of the real data. , For data dimensions; In one feasible embodiment, the posterior distribution KL divergence is constructed by the following equation (2): (2) in, The difference between the distribution of real data and the distribution of synthetic data; For the actual data belonging to the first The proportion of clusters, scalar [0, 1]. , For the quantity of real data, For real data Belongs to the Cluster The posterior probability; For the synthetic data belonging to the first The proportion of clusters, scalar [0, 1]. , For the amount of synthetic data, For synthetic data Belongs to the Cluster The posterior probability; , For the dataset, For dataset One piece of data (real data or synthetic data) is used as a sample. Belongs to the Cluster posterior probability , and All are cluster indexes.

[0061] In one feasible embodiment, the step of generating synthetic data based on the initialized first parameters and the probability density function of the Gaussian mixture model combined with real data includes: The Gaussian mixture model randomly selects a cluster according to the cluster weights; A synthetic dataset is randomly generated from the cluster based on the Gaussian distribution of the real data. Repeat the above steps until the synthesized data reaches the data quantity threshold.

[0062] This invention improves data quality by dynamically adjusting the synthesized data through an iterative generation mechanism until a preset similarity threshold is reached.

[0063] In one feasible embodiment, the step of generating synthetic data based on the initialized first parameters and the probability density function of the Gaussian mixture model combined with real data includes: Based on the multiple clusters divided by the first parameter, analyze the posterior probability of each real data point belonging to each cluster; Update cluster weights, mean, and covariance matrix based on posterior probability; Repeat the iterations until convergence; Clusters are randomly selected based on cluster weights; Synthetic data is generated by randomly sampling real data from selected clusters; Repeat the iteration until the synthesized data reaches the data quantity threshold.

[0064] The synthetic data generation step of the large-scale model optimization method for education described in this invention is based on a Gaussian model. It identifies the distribution of real-world data features and fits the data to generate synthetic data to support the data requirements during large-scale model training. The generation criteria must meet evaluation standards, and iterative generation is performed to ensure the similarity between the generated data and real-world data, thus resolving problems in the large-scale model process caused by data quality issues.

[0065] In one feasible embodiment, in an application scenario with security compliance requirements, step S1 includes: Synthetic data is generated by fitting real data based on Gaussian mixture model. Preferably, the cluster weight is set by the contribution ratio of real data to synthetic data. The higher the contribution ratio, the higher the relative frequency of sampling synthetic data from the cluster. The distribution similarity between synthetic data and real data is assessed using distribution similarity metrics. Determine whether the distribution similarity of the synthesized data reaches a similarity threshold; If the distribution similarity does not reach the similarity threshold, then return to the step of generating synthetic data based on features from real data; If the distribution similarity reaches the similarity threshold, the synthetic data will be included in the training set. Desensitization of synthetic data included in the training set can be performed only on key synthetic data.

[0066] Preferably, the anonymization process for the synthetic data included in the training set includes: Modifying one or more of the cluster weights, mean vectors, and covariance matrices in the synthetic data generated from Gaussian mixture models fitted to real data allows for control over the proportion of different distribution features in the anonymized data, thereby altering the data distribution and protecting the privacy of critical data. Adjusting the mean generates new data, changing the data center along that dimension and hiding the true location information of the original data. Adjusting the covariance matrix alters the correlation and dispersion between data features, further disrupting the structural features of the original data and enhancing privacy protection. For example, reducing certain elements of the covariance matrix can decrease the correlation between corresponding features, causing the generated data to differ from the original data in terms of feature association.

[0067] In one feasible embodiment, in an application scenario lacking real data, step S1 includes: Combine expert intelligent agents to generate synthetic data by fitting real data based on Gaussian mixture models; The distribution similarity between synthetic data and real data is assessed using distribution similarity metrics. Determine whether the distribution similarity of the synthesized data reaches a similarity threshold; If the distribution similarity does not reach the similarity threshold, then return to the step of generating synthetic data based on features from real data; If the distribution similarity reaches the similarity threshold, the synthetic data will be included in the training set.

[0068] Preferably, the step of combining expert agents to fit real data based on Gaussian mixture models to generate synthetic data includes one or more of the following steps: Expert agents determine the proportion of different data features in the synthetic data from the real data, and thus set the cluster weights; The expert agent sets the mean based on the center position of each Gaussian distribution in different dimensions. For example, in image data generation, the expert agent sets the mean based on its understanding of image features to determine the average performance of the generated image in key feature dimensions (such as color, shape, etc.). Expert agents set the covariance matrix based on the correlation and dispersion between features of real data. For example, in speech data generation, experts set the covariance matrix according to the characteristics of the speech signal to control the correlation and fluctuation between different frequency features of the generated speech data, so that the generated data conforms to the expected feature distribution.

[0069] In the large model training phase, synthetic data is used to address the issue that real training data cannot be directly used in certain scenarios due to privacy and compliance concerns. This invention employs a Gaussian model to generate any number of new data sets, solving the data scarcity problem caused by security, compliance, and authenticity reasons. It also allows for diverse data generation, ensuring a uniform distribution of training data samples and improving the generalization ability of large models. For security and compliance requirements, real data can be fitted for data generation, and key data can be anonymized. In scenarios lacking real data, data features and evaluation models are developed by experts based on experience, and Gaussian models are used for data generation. This ensures both privacy protection (due to anonymization of key data, it is impossible to extrapolate to individuals) and data authenticity (consistent with statistical patterns), achieving training data that is essentially consistent with or similar to real data, thus avoiding the illusion of a large model due to insufficient data from the data source.

[0070] The synthetic data generation step described in this invention is based on a Gaussian model algorithm to generate synthetic data, identify the distribution of real data features, and fit the data to generate synthetic data to support the data requirements during large model training. The generation criteria must meet the evaluation criteria, and iterative generation is performed to ensure the similarity between the generated data and real data, thus solving problems in large model training caused by data quality issues.

[0071] In one feasible embodiment, step S1 further includes: By retrieving real data from the internet, databases, and knowledge bases using data acquisition tools, we can improve the accuracy and real-time performance of data used within our business scope and reduce the illusion of large models caused by data latency.

[0072] The synthetic data generation step described in this invention is based on a Gaussian model algorithm to generate synthetic data, identify the distribution of real data features, and fit the data to generate synthetic data to support the data requirements during large model training. The generation criteria must meet the evaluation criteria, and iterative generation is performed to ensure the similarity between the generated data and real data, thus solving problems in large model training caused by data quality issues.

[0073] Step S2, Large Model Training Steps: Large-scale model training and optimization are performed using SLGD (Stochastic Gradient Langevin Dynamics). By introducing Gaussian noise, the model parameters can be randomly searched while moving along the gradient direction, thus solving the problems of overfitting and local optima. By introducing discretized Langevin dynamic equations and reasonably designing the noise term, efficiency and model generalization ability are effectively balanced.

[0074] In one feasible embodiment, step S2 includes: Parameter update steps: Update the model parameters of the large model by superimposing Gaussian noise on the stochastic gradient using the Langevin dynamics mechanism; The first evaluation step is as follows: randomly select one or more synthetic data sets from the training set as validation sets and input them into the updated large model; based on the output of the large model, evaluate the model by updating the parameters to determine whether the parameter update convergence condition has been met; if the parameter update convergence condition has not been met, return to the parameter update step; if the parameter update convergence condition has been met, execute the training step; preferably, the parameter update evaluation model can be constructed by one or more linear combinations of accuracy and F1-Score, wherein the parameter update convergence condition is that the accuracy reaches the accuracy threshold requirement, the F1-Score reaches the F1 threshold requirement, or / and the combined value of accuracy and F1-Score reaches the combined threshold requirement; Training steps: Train a large model that has reached the convergence condition for parameter update using the training set; The third evaluation step is to determine whether the trained large model has reached the training convergence condition based on the training evaluation model. If the training convergence condition has not been reached, the parameter update step is returned. If the training convergence condition has been reached, the trained large model is output.

[0075] The above training and evaluation model can be constructed based on one or a combination of business metrics, such as click-through rate and conversion rate; the training convergence condition can be reaching the threshold requirement of the metric.

[0076] In one feasible embodiment, step S2 further includes: Steps for building a multi-task generation and evaluation model: Construct a multi-task generation and evaluation model; The second evaluation step is to determine whether the output after training in the large model training step has reached the first evaluation convergence condition through the multi-task generation evaluation model. If the first evaluation convergence condition has not been reached, the parameter update step is returned. If the first evaluation convergence condition has been reached, the synthetic data in the training set that has reached the first evaluation convergence condition is used as the training set for the next iteration.

[0077] The aforementioned multi-factor generative evaluation model can be constructed using model performance metrics (such as AUC, RMSE, etc.) of the evaluation model (classifier and / or regressor).

[0078] The above evaluation convergence condition can be reaching the evaluation index threshold.

[0079] In one feasible embodiment, the parameter update step includes: Initialization steps: Initialize the second parameter, which includes the model parameters, learning rate, and number of iterations of the large model; Gradient estimation steps: Randomly take multiple synthetic data from the training set and input them into the initialized large model to obtain gradient estimates of the model parameters of the large model; Noise injection and parameter update steps: Gaussian noise is superimposed on the gradient of the model parameters of the large model, and discretized Langevin dynamic equations are introduced to update the model parameters of the large model.

[0080] In each iteration, the parameter update steps described above randomly select a small batch of data from the training data to calculate the gradient estimate of the current parameters. This random sampling reduces computational cost, while the gradient information indicates the direction of parameter updates, guiding the model towards optimizing the loss function. Introducing Gaussian noise allows the model parameters to randomly search along the gradient direction, helping to address overfitting and local optima issues. The application of discretized Langevin dynamic equations effectively balances training efficiency and the model's generalization ability.

[0081] In one feasible embodiment, the parameter update step further includes: Repeat the gradient estimation step, noise injection, and parameter update step, gradually reducing the learning rate to stabilize training until the parameter distribution is stable or the preset number of iterations is reached.

[0082] In one feasible embodiment, the model parameters of the large model are updated using the following equations (3) and (4): (3) (4) in, Index for iteration count; For the first The momentum term of the next iteration; For the first The second parameter of the next iteration; The momentum decay coefficient; The learning rate; For the loss function L in the current second parameter The gradient below; For the first Gaussian noise introduced in the next iteration , This represents the noise intensity. In large models for classification tasks, the cross-entropy loss function can be chosen. In large models for regression tasks, the mean squared error loss function can be chosen. The learning rate is related to the noise level by shifting the model parameters in the direction that the loss function decreases. and To ensure sampling convergence, This represents the total number of iterations.

[0083] This invention combines the efficiency of stochastic gradient descent (SGD) with the noise injection mechanism of Langevin dynamics. By introducing Gaussian noise, it achieves random parameter search, solves the problems of overfitting and local optima, and alleviates the illusion of large models.

[0084] In one feasible embodiment, the parameter update step further includes: Gradient clipping is performed on the gradient of the second parameter.

[0085] In one embodiment, the step of gradient clipping of the gradient of the second parameter includes: Gradient clipping is performed using the following formula (5): (5) in, This is the gradient clipping threshold.

[0086] In one embodiment, the step of gradient clipping of the gradient of the second parameter includes: Set the gradient threshold; When the magnitude of the gradient exceeds the gradient threshold, the gradient ratio is reduced to not exceed the gradient threshold.

[0087] This invention limits the magnitude of the gradient through gradient clipping, preventing gradient explosion, stabilizing the training process, avoiding abnormal parameter updates due to excessively large gradients, and making model training more stable.

[0088] In one feasible embodiment, the parameter update step further includes: Cosine annealing is introduced to adaptively adjust the learning rate.

[0089] In one feasible embodiment, the step of introducing cosine annealing to adaptively adjust the learning rate includes: The learning rate is adaptively adjusted using the following formula (6): (6) in, For the first The learning rate for each iteration; Minimum learning rate; The maximum learning rate; This represents the total number of iterations. The learning parameters can be obtained through data training or set based on experience. In one feasible embodiment, It is 0.5.

[0090] This invention introduces cosine annealing to adaptively adjust the learning rate, gradually reducing the learning rate to stabilize training until the parameter distribution is stable or the preset number of iterations is reached. This helps the model to adjust parameters more finely in the later stages of training, thereby improving the model's performance.

[0091] In one feasible embodiment, step S2 further includes: Standardizing the training set makes the data in the training set have a similar scale, which helps the model learn patterns in the data better and improves the stability and efficiency of training.

[0092] In one embodiment, the step of standardizing the training set includes: The data in the training set is standardized using the following formula (7): (7) in, For the original data Standardized data; The mean of the data; denoted as the standard deviation of the data.

[0093] The large model training steps of this invention avoid the computational overhead of state switching by integrating adaptive learning rate adjustment, dynamic noise injection, and momentum acceleration mechanisms, while ensuring and enhancing the stability and generalization ability of model training until the large model training reaches the set evaluation criteria.

[0094] The large model training steps described in this invention first initialize parameters to provide starting conditions for subsequent training steps; stochastic gradient calculation provides directional information for parameter updates; noise injection and parameter updates introduce randomness based on stochastic gradients to optimize the parameter update process; gradient pruning and data standardization are implemented to stabilize the training process and ensure the smooth progress of the preceding steps; cosine annealing is introduced to adaptively adjust the learning rate, which dynamically adjusts the learning rate during training to further improve the training effect and performance of the model. These steps work together to complete the training optimization of the large model.

[0095] The large model training step described in this invention utilizes Langevin sampling based on stochastic gradients, combining the efficiency of stochastic gradient descent with the noise injection mechanism of Langevin dynamics. During the fine-tuning phase, it reduces reliance on random erroneous labels; during task generation, it improves output stability through sampling diversity, ensuring faster convergence of the model to the posterior distribution and effectively mitigating the illusion problem in large model training.

[0096] In one feasible embodiment, step S2 is followed by: Repeat steps S1 and S2, and by integrating adaptive learning rate adjustment, dynamic noise injection and momentum acceleration mechanisms, avoid the computational overhead of state switching, while ensuring and enhancing the stability and generalization ability of model training, until the large model training reaches the set evaluation criteria.

[0097] In one feasible embodiment, the above-described large model optimization method for education further includes;

[0098] Step S3, Monitoring Steps: Monitor the large model after training; determine whether the stability difference of the large model output reaches a threshold; if the stability difference does not reach the threshold, continue monitoring the large model after training; if the stability difference reaches the threshold, return to step S1 or / and step S2. Large model stability includes one or more of performance stability, output consistency, robustness, and load stability. Performance stability refers to whether the fluctuation range of model response time and throughput is within an acceptable threshold (e.g., response time standard deviation ≤ 20% of average response time) under fixed input complexity. Output consistency refers to whether the output of the same / similar input at different times and on different instances meets expectations (no logical contradictions, no deviation of key information). Robustness refers to whether the model can respond normally (without crashing, without outputting harmful content) when facing abnormal input (e.g., excessively long text, garbled characters, adversarial input). Load stability refers to whether the model service can maintain performance and output quality (without a surge in timeout rate or error rate) under scenarios of increased concurrent users and long-term continuous operation. Stability discrepancy refers to the deviation between the actual performance of a large model and its expected or normal performance across the various dimensions of stability mentioned above.

[0099] In one feasible embodiment, the above-described large model optimization method for education further includes: Step S4, User Feedback Step: Adjust the overall model based on user feedback information, specifically including: Collect feedback information from user terminals; The model parameters of the large model are adjusted based on the feedback information. For example, if the feedback information indicates a decrease in performance or accuracy, the process returns to step S2, triggering retraining of the large model and / or adjustment of the second parameter. If the feedback information is real data confirmed by the user terminal, the process returns to step S1 to trigger the generation of synthetic data. If the feedback information is incorrect labeling or abnormal results output by the user terminal, the error type (such as logical error, missing content) and specific problem (such as disjointed steps, insufficient data) can be identified through keyword matching. Synthetic data related to the feedback information used for training the large model is removed, new synthetic data is generated, and the large model is retrained. Dynamic parsing of user feedback information can be achieved through template error pattern matching or custom error pattern matching using regular expressions.

[0100] The above provides two feasible embodiments of large model optimization methods for education. However, the present invention is not limited to these embodiments. A combination of the two feasible embodiments can be used, for example: collecting feedback information from user terminals during the monitoring of the large model after training, and intervening or retraining the large model product based on the feedback issues to reduce the probability of large model illusion.

[0101] The large model optimization method for education described in this invention uses Gaussian models to generate synthetic data, fits it to real data, and desensitizes and encrypts confidential information. This effectively solves the problems of unusable or unavailable real data under compliant conditions, significantly improving the illusion problem of large models during training and use. In the fine-tuning stage of the large model and the generation task, Langevin sampling based on stochastic gradients is used. In the fine-tuning stage, Gaussian noise and discrete dynamic equations are introduced to prevent overfitting and reduce dependence on mislabeled data, effectively mitigating the impact of erroneous data on the illusion of large models. In the generation task, sampling diversity is improved to enhance output stability (e.g., inconsistencies and contradictions in text generation), further reducing the illusion problem of large model products and ensuring the effectiveness, accuracy, and real-time performance of the large model product output.

[0102] Figure 2 A flowchart illustrating a preferred embodiment of the large model optimization method for education described in this invention is shown, as follows: Figure 2 As shown, the large-scale model optimization method for education includes: Step S10: Generate composite data; Step S20: Fine-tuning of SLGD large model parameters; Step S30: Determine whether the parameter update convergence condition has been met; If the parameter update convergence condition is not met, return to step S20; If the parameter update convergence condition is met, then step S40 is executed, and the large model that has met the parameter update convergence condition is trained using the training set, and the trained output is input into the multi-task generation evaluation model. Step S50: Determine whether the first evaluation convergence condition has been met; If the first evaluation convergence condition is not met, return to step S20; If the first evaluation convergence condition is met, then step S60 is executed, and the synthetic data in the training set that has met the first evaluation convergence condition is used as the training set for the next iteration. Step S70: Determine whether the training convergence condition has been met; If the training convergence condition is not met, return to step S20; If the training convergence condition is met, proceed to step S80, output the trained large model, and export and deploy the large model product. Step S90: Monitor the large model; based on the evaluation model, collect feedback information from user terminals and adaptively adjust the large model. Step S100: Determine whether the second evaluation convergence condition has been met; If the second evaluation convergence condition is not met, return to step S20; If the second evaluation convergence condition is met, return to step S90.

[0103] The large-scale model optimization method for education described in this invention uses synthetic data generation to address the problem of insufficient real data or data security requirements that preclude the use of real data in the early stages of large-scale model training. It combines Langevin dynamics with stochastic gradient descent, injecting noise into parameter updates to cause the model parameter trajectory to converge to the posterior distribution rather than a single optimal solution. This increases the generalization ability of the large-scale model, improves the utilization efficiency of the large-scale model product, and thus better empowers business operations, providing support for business management, prediction, and decision-making. Through synthetic data generation, at each stage, it combines the experience of students, teachers, and campuses with expert assessments to determine whether each stage meets the evaluation criteria and whether the large-scale model product meets the acceptance and usage standards.

[0104] In one feasible embodiment, the evaluation model in step S90 can be constructed based on one or more of the following indicators: stability difference, accuracy, recall, F1 score, etc. The second evaluation convergence condition can be to reach the evaluation threshold requirement. The evaluation threshold requirement can be one or more of the following: stability difference is less than the difference threshold, accuracy reaches the accuracy threshold, recall is within the recall threshold range, F1 score reaches the score threshold, N consecutive feedback scores are higher than the threshold, error rate is lower than the set ratio, and user satisfaction is stable within the target range.

[0105] This invention evaluates metrics in stages such as generating synthetic data, pre-training, model fine-tuning, and generating tasks, improving the accuracy of the information and solutions provided by the model itself, solving most of the illusion problems of large models, and effectively improving the service level of large model products in teaching, learning, and management.

[0106] Figure 3 This is a schematic block diagram illustrating one embodiment of the large-scale model optimization device for education described in this invention, as shown below. Figure 3 As shown, the large model optimization device for education includes a synthetic data generation module 1 and a training module 2: The synthetic data generation module 1 is configured to generate synthetic data based on fitting real data using a Gaussian model, evaluate the statistical consistency between the synthetic data and the real data through a distribution similarity metric, obtain synthetic data that meets the consistency requirements, and form a training set. The training module 2 is configured to update the model parameters of the large model by superimposing Gaussian noise on the stochastic gradient through the Langevin dynamics mechanism, and to train the large model by combining the training set generated by the synthetic data generation module 1.

[0107] In one feasible embodiment, the large model optimization device for education further includes an iterative optimization module, which is configured to send iterative instructions to the synthetic data generation module 1 and the training module 2 to repeatedly perform synthetic data generation and model training until the convergence condition is met.

[0108] In one feasible embodiment, the large model optimization device for education further includes a monitoring module 3: The monitoring module 3 is configured to monitor the stability of the large model after training. When the stability difference reaches the difference threshold, it sends an instruction to generate synthetic data to the synthetic data generation module 1.

[0109] In one feasible embodiment, the large model optimization device for education further includes a feedback module 4: The feedback module 4 is configured to receive feedback information from the user terminal and transmit the feedback information to the synthetic data generation module 1 and / or the training module 2.

[0110] In a preferred embodiment, such as Figure 4 As shown, the large model optimization device for education is constructed using a distributed process framework (e.g., Flume or / and Flink framework). The large model optimization device for education includes: a first cluster 10, a feature extraction module 20, and a second cluster 30. The first cluster 10 includes multiple data processing nodes 11, and the second cluster 30 includes multiple computing nodes 31 and at least one training node 32, wherein: The data processing node 11, the feature extraction module 20, and the second cluster 30 work together to realize the function of the synthetic data generation module 1, specifically: The data processing node 11, as the "data entry point" of the first cluster 10, is configured to read real data from external data sources (such as databases and log files). It can also perform data preprocessing such as data cleaning and format conversion through parallel processing (multi-threaded concurrent computation processing of multi-threaded modules). The multi-threaded concurrent computation processing mechanism can further improve sampling efficiency and ensure the diversity and uniformity of the synthetic data. The feature extraction module 20 extracts multiple features from the real data output by the data processing node 11 and converts them into learnable feature vectors. The computing node 31 uses a parallel algorithm (such as the EM algorithm) to fit the feature vectors to a Gaussian Mixture Model (GMM) to determine the number of clusters, cluster centers, covariance matrix, and cluster weights. Based on the fitted GMM parameters, the computing node 31 generates synthetic data through random sampling. The steps performed by the computing node 31 include: randomly selecting clusters according to their cluster weights, sampling data points from the Gaussian distribution of those clusters, and iteratively generating an arbitrary number of new data points. The computing node 31 can calculate the statistical characteristics (mean, variance, correlation, etc.) of the synthetic data and the real data in parallel, and determine whether they meet the preset evaluation criteria (distribution similarity reaches a similarity threshold) through distribution similarity. If the verification fails, the data processing node 11 and the computing node 31 collaboratively perform parameter updates (such as adjusting the number of GMM clusters, optimizing feature extraction logic, etc.) and restart the synthetic data generation process until the synthetic data meets the preset evaluation criteria. The collaboration between computing node 31 and training node 32 enables the functionality of training module 2, specifically: Training node 32 is responsible for parameter initialization. Through multi-threaded concurrent computation, the initialization parameters are distributed to each computing node to achieve distributed parameter synchronization. Computation node 31 randomly samples mini-batch synthetic data and computes the gradient estimate of the loss function with respect to the parameters in parallel. The multi-threading mechanism supports processing multiple mini-batch datasets simultaneously, improving gradient computation efficiency. Training node 32 adds Gaussian noise to the gradient (the noise intensity σ is adjusted with iteration), and updates the parameters by discretizing the Langevin dynamic equation.

[0111] In one feasible embodiment, the first cluster 10 is a Flume cluster or a Flink cluster, and the second cluster 30 is a GPU cluster. Flink's distributed communication supports the synchronous parameter updates of multiple nodes in the GPU, ensuring global parameter consistency.

[0112] In one feasible embodiment, the training node 32 and the computing node 31 work together to achieve the functions of gradient clipping and / or learning rate annealing: the computing node 31 clips the gradients (to prevent gradient explosion), and the training node 32 dynamically adjusts the learning rate in conjunction with the cosine annealing strategy.

[0113] In one feasible embodiment, the training node 32, computing node 31, and data processing node 11 implement the functions of the feedback module 4 and / or the monitoring module 3: the training node 32 exports the optimized model parameters; the computing node 31 performs output detection on the large model through parallel inference tasks (multi-threaded concurrent computation processing), for example, by evaluating the model's stability in the generation task (such as sampling diversity and output consistency) through a validation set; the data processing node 11 collects feedback from the user terminal on the output of the large model (such as incorrect labeling and abnormal results) and synchronizes it to the training node in real time; if the user terminal feedback shows differences in the stability of the large model (such as hallucination problems), the computing node 31 triggers a retraining process: it retrieves real data and synthetic data from the data processing node 11, optimizes the synthetic data through Flink's dynamic resource scheduling, and repeats the function of the training module 2 until the stability of the large model meets the requirements.

[0114] The large-scale model optimization system for education described in this invention utilizes a distributed computing cluster to improve data processing efficiency and ensure the real-time performance of large-scale model training data, knowledge bases, and RAG data. For synthetic data generation, in scenarios where real data is lacking for large-scale model training or where data security requirements prevent the use of real data, a Gaussian model algorithm is used to simulate and generate real data. During large-scale model fine-tuning and generation tasks, Langevin sampling based on stochastic gradients is used. In the fine-tuning stage, discrete dynamic equations are introduced to prevent overfitting, reduce reliance on mislabeled data, and effectively mitigate the impact of erroneous data on the illusion of a large-scale model. During the generation task, sampling diversity is enhanced to improve output stability (e.g., inconsistencies and contradictions in generated text), further reducing the illusion problem of large-scale model products and ensuring the effectiveness, accuracy, and real-time performance of the large-scale model product output.

[0115] Figure 5 This diagram illustrates an application scenario of the large model optimization method for education described in this invention.

[0116] exist Figure 5In the application scenario, computing device 100 can acquire real data 101. Then, computing device 100 can generate synthetic data 102 based on the features of the real data, and can also obtain a training set 103 that meets the similarity requirements through similarity evaluation. Afterwards, computing device 100 can update the model parameters of the large model by superimposing Gaussian noise on the stochastic gradient using the Langevin dynamics mechanism 104, and train the large model in combination with the training set to obtain the trained large model 105. Finally, computing device 100 can monitor the stability of the output of the trained large model and / or adjust the synthetic data and / or model parameters based on feedback from the user terminal, retrain the large model, and continue until the stability of the large model reaches the stability requirements.

[0117] It should be noted that the aforementioned computing device 100 can be either hardware or software. When the computing device 100 is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device 100 is software, it can be installed in the hardware devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0118] Figure 6 This diagram illustrates another application scenario of the large model optimization method for education described in this invention.

[0119] exist Figure 6 In this application scenario, the components of the computing device 100 include, but are not limited to, a memory 110 and a processor 120. The processor 120 is connected to the memory 110 via a bus 130, and the database 150 is used to store data.

[0120] The computing device 100 also includes an access device 140, which enables the computing device 100 to communicate via one or more networks 160. Examples of such networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 140 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0121] In one embodiment of the present invention, the above-mentioned components of the computing device 100 and Figure 6 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 6 The illustrated block diagram of the computing device is for illustrative purposes only and is not intended to limit the scope of the invention. Those skilled in the art can add or replace other components as needed.

[0122] The computing device 100 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 100 can also be a mobile or stationary server.

[0123] The processor 120 executes computer-executable instructions, which, when executed by the processor, implement the steps of the aforementioned large model optimization method for education. The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the aforementioned large model optimization method for education belong to the same concept. Details not described in detail in the technical solution of the computing device can be found in the description of the technical solution of the aforementioned large model optimization method for education.

[0124] The present invention also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the large model optimization method for education described above.

[0125] The above is an illustrative scheme of the computer-readable storage medium described in this invention. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the large-scale model optimization method for education described above. Details not described in detail in the technical solution of the storage medium can be found in the description of the technical solution of the large-scale model optimization method for education described above.

[0126] The present invention also provides a computer program, wherein when the computer program is executed in a computer, the computer is instructed to perform the steps of the above-described large model optimization method for education.

[0127] The above is an illustrative scheme of the computer program described in this invention. It should be noted that the technical solution of this computer program belongs to the same concept as the technical solution of the aforementioned large-scale model optimization method for education. Details not described in detail in the computer program's technical solution can be found in the description of the aforementioned large-scale model optimization method for education.

[0128] The foregoing has described specific embodiments of the invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0129] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of the present invention.

[0130] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0131] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments of the present invention. These embodiments are selected and specifically described to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention.

Claims

1. A large-scale model optimization method for education, characterized in that, include: Synthetic data generation steps: Generate synthetic data by fitting real data with a Gaussian model, evaluate the consistency between synthetic data and real data through distribution similarity measurement, obtain synthetic data that meets the consistency requirements, and form a training set; Large model training steps: Update the model parameters of the large model by superimposing Gaussian noise on the stochastic gradient through the Langevin dynamics mechanism, and train the large model in combination with the training set.

2. The large-scale model optimization method for education according to claim 1, characterized in that, The synthetic data generation step includes: Initialize the first parameter, which includes the number of clusters, cluster centers, cluster weights, and covariance matrix; Based on the initial parameters, synthetic data is generated by combining the probability density function of the Gaussian mixture model with real data. The distribution similarity between synthetic data and real data is assessed by using the posterior distribution KL divergence. Synthetic data with a distribution similarity greater than a similarity threshold are added to the training set.

3. The large-scale model optimization method for education according to claim 2, characterized in that, The probability density function of the Gaussian mixture model is constructed by the following formula: in, It is the first The weights of each cluster, ; The total number of clusters; and They are the first The mean vector and covariance matrix of each cluster; For data The probability density; Or / and, the posterior distribution KL divergence is constructed by the following formula: in, The difference between the distribution of real data and the distribution of synthetic data; For the actual data belonging to the first The proportion of individual clusters , For the quantity of real data, For real data Belongs to the Cluster The posterior probability; For the synthetic data belonging to the first The proportion of individual clusters , For the amount of synthetic data, For synthetic data Belongs to the Cluster The posterior probability.

4. The large-scale model optimization method for education according to claim 1, characterized in that, The large model training steps include: Parameter update steps: Update the model parameters of the large model by superimposing Gaussian noise on the stochastic gradient using the Langevin dynamics mechanism; First evaluation step: Randomly select one or more synthetic data sets from the training set as validation sets to input into the updated large model; Based on the output of the large model, evaluate the model through parameter updates to determine whether the parameter update convergence condition has been met; If the parameter update convergence condition has not been met, return to the parameter update step; If the parameter update convergence condition has been met, execute the training step. Training steps: Train a large model that has reached the convergence condition for parameter update using the training set; The third evaluation step: Based on the training evaluation model, determine whether the trained large model has reached the training convergence condition; if the training convergence condition has not been reached, return to the parameter update step; if the training convergence condition has been reached, output the trained large model. Or / and, it also includes iterative optimization steps: repeating the synthetic data generation step and the large model training step until the convergence condition is met; Or / and, it also includes a monitoring step, which monitors the stability of the output of the large model, and if the stability difference reaches the difference threshold, returns to the synthetic data generation step; Or / and, it also includes a monitoring step, the monitoring step comprising: Large-scale monitoring model; Construct an evaluation model; Determine whether the second evaluation convergence condition has been met; If the second evaluation convergence condition is not met, return to the parameter update step or return to the synthetic data generation step; or / and, also includes: Collect feedback information from user terminals; Adjust the model parameters of the large model based on the feedback information.

5. The large-scale model optimization method for education according to claim 4, characterized in that, The large model training steps also include: Steps for building a multi-task generation and evaluation model: Construct a multi-task generation and evaluation model; The second evaluation step is to determine whether the output after training in the large model training step has reached the first evaluation convergence condition through the multi-task generation evaluation model. If the first evaluation convergence condition has not been reached, the parameter update step is returned. If the first evaluation convergence condition has been reached, the synthetic data in the training set that has reached the first evaluation convergence condition is used as the training set for the next iteration. Or / and, the parameter update step includes: Initialize the second parameter, which includes the model parameters, learning rate, and number of iterations of the large model; Multiple synthetic data points are randomly selected from the training set and input into the initialized large model to obtain gradient estimates of the model parameters of the large model. Gaussian noise is superimposed on the gradient of the model parameters of the large model, and discretized Langevin dynamic equations are introduced to update the model parameters of the large model.

6. The large-scale model optimization method for education according to claim 5, characterized in that, In the step of adding Gaussian noise to the gradient of the model parameters of the large model and introducing discretized Langevin dynamic equations to update the model parameters of the large model: The model parameters of the large model are updated using the following formula: in, Index for iteration count; For the first The momentum term of the next iteration; For the first The second parameter of the next iteration; The momentum decay coefficient; The learning rate; For the loss function L in the current second parameter The gradient below; For the first Gaussian noise introduced in the next iteration , Noise intensity; Or / and, the parameter update step further includes: Perform gradient clipping on the gradient of the second parameter; Or / and, the parameter update step further includes: Cosine annealing is introduced to adaptively adjust the learning rate.

7. The large-scale model optimization method for education according to claim 6, characterized in that, In the step of introducing cosine annealing to adaptively adjust the learning rate: The learning rate is adaptively adjusted using the following formula: in, For the first The learning rate for each iteration; Minimum learning rate; The maximum learning rate; This represents the total number of iterations. These are the learning parameters.

8. A large-scale model optimization device for education, characterized in that, Includes a synthetic data generation module and a training module: The synthetic data generation module is configured to generate synthetic data based on fitting real data using a Gaussian model, evaluate the consistency between the synthetic data and the real data through a distribution similarity metric, and obtain synthetic data that meets the consistency requirements to form a training set. The training module is configured to update the model parameters of the large model by superimposing Gaussian noise on the stochastic gradient through the Langevin dynamics mechanism, and to train the large model by combining the training set generated by the synthetic data generation module.

9. The large-scale model optimization device for education according to claim 8, characterized in that, It also includes a monitoring module and / or a feedback module: The monitoring module is configured to monitor the stability of the large model after training. When the stability difference reaches the difference threshold, it sends an instruction to generate synthetic data to the synthetic data generation module. The feedback module is configured to receive feedback information from the user terminal and transmit the feedback information to the synthetic data generation module and / or the training module.

10. A large-scale model optimization device for education, characterized in that, The system includes a first cluster, a feature extraction module, and a second cluster. The first cluster includes multiple data processing nodes, and the second cluster includes multiple computing nodes and at least one training node. The data processing node is configured to collect real data; The feature extraction module is configured to extract multiple features from the real data output by the data processing node and convert them into learnable feature vectors. The computing node is configured to generate synthetic data by fitting the feature vector of the real data extracted by the feature extraction module based on the Gaussian model. The consistency between the synthetic data and the real data is evaluated by the distribution similarity metric, and synthetic data that meets the consistency requirements is obtained to form a training set. The training nodes are configured to update the model parameters of the large model by superimposing Gaussian noise on the stochastic gradient through the Langevin dynamics mechanism, and to train the large model by combining the training set constructed by the computing nodes.

Images