Large language model self-training method and system based on GALA framework

Abstract

This invention relates to the field of language model training technology, and particularly to a self-training method and system for large-scale language models based on the GALA framework. The self-training method for large-scale language models based on the GALA framework includes the following steps: S10: For a preset seed dataset containing questions and corresponding true answers, guide a preset initial model to generate multiple candidate inference paths for each question, filter out candidate inference paths that can yield the true final answer, and combine them with the corresponding questions to form a candidate set. This invention solves the problems of insufficient strategy diversity and lack of depth of inference guidance in traditional self-training methods through a joint closed-loop design of rejection sampling, clustering screening, differentiated strategy generation, dual-dimensional scoring, and iterative optimization. It simultaneously improves the model's generalization ability, training efficiency, and inference logic, ultimately achieving a precise, stable, and efficient improvement in the inference ability of large-scale language models.

Description

Technical Field

[0001] This invention relates to the field of language model training technology, and in particular to a method and system for self-training large-scale language models based on the GALA framework. Background Technology

[0002] Self-improvement capability is a key frontier in the development of large language models (LLMs), and it holds promise for building intelligent systems that can autonomously improve their performance with minimal human intervention. Self-training, as a paradigm for models to autonomously generate training data, is the core path to achieving this goal, and it is particularly effective in fields where results can be verified, such as mathematical reasoning. The correctness of solutions in such fields can serve as a clear signal for reinforcement learning; by generating potential solutions in batches and selectively training based on successful cases, models can iteratively improve their problem-solving capabilities.

[0003] Compared to traditional active learning that relies on manual annotation, the core feature of self-training is the establishment of a fully automated data screening process, eliminating the need for manual annotation or sample selection and significantly reducing the cost of model iteration and optimization. However, existing research still has significant limitations. While pioneering works such as LIMA (Less Is More for Alignment) and FLAN (Finetuned Language Net) that focus on data efficiency reveal the crucial role of high-quality samples in performance improvement, their related technical solutions are mostly designed for static datasets. The data screening logic lacks dynamic adaptability and is difficult to match the dynamic iterative characteristics of generation, training, feedback, and regeneration in the self-improvement loop. Therefore, accurately identifying real, high-quality, and diverse sample examples from dynamic data streams, maximizing single-round training efficiency, and achieving continuous and stable performance improvement while avoiding model performance collapse due to noisy data has become the core challenge that the self-training paradigm urgently needs to overcome.

[0004] Currently, traditional rejection sampling remains the mainstream method for self-training large-scale language models. This method, tailored to a specific task, leverages cue engineering to drive the model to generate a large number of candidate reasoning paths, selecting only the paths that yield the correct answer as training data. While this approach can ensure the basic correctness of the training signal through binary decision-making, it overemphasizes outcome-oriented binary rewards, neglecting the quality of the reasoning process, strategy diversity, and guiding value, resulting in severe redundancy in the training corpus.

[0005] In practical applications, this method suffers from low data generation efficiency, with redundant data consuming significant computational and storage resources, leading to a rapid decay of the marginal benefits of fine-tuning. Furthermore, data quality is a weakness, with the selected correct solutions limited to a few dominant inference patterns, lacking strategy diversity and inference depth, thus restricting the model's generalization ability. It also lacks an efficient verification mechanism; standard training cycles rely on slow offline evaluation, making it impossible to determine the quality of generated data in real time. Evaluation can only be indirectly assessed through the performance of the fine-tuned model, resulting in significant latency and coarse granularity. These problems ultimately severely restrict the efficiency and effectiveness of self-training. Summary of the Invention

[0006] This invention provides a method and system for self-training large-scale language models based on the GALA framework, in order to solve the problem that traditional rejection sampling methods severely restrict the efficiency and effectiveness of self-training large-scale language models.

[0007] To solve the above-mentioned technical problems, this application provides the following technical solution: A self-training method for large-scale language models based on the GALA framework includes the following steps: S10: For a pre-set seed dataset containing questions and corresponding true answers, guide the pre-set initial model to generate multiple candidate reasoning paths for each question, filter out the candidate reasoning paths that can lead to the true final answer, and combine them with the corresponding questions to form a candidate set; S20: Use a preset sentence converter to perform semantic vector transformation on the candidate reasoning paths in the candidate set to obtain a set of semantic vectors; use a preset clustering algorithm to cluster the set of semantic vectors and calculate the average semantic vector of each cluster as the cluster center vector; select the reasoning path corresponding to the semantic vector closest to the center vector in each cluster as the core reasoning path and integrate them into a screening sample set; S30: Using the questions corresponding to the core reasoning paths in the selected sample set as input, guide the initial model to devise multiple differentiated strategies that can yield the true final answer, generate corresponding complete step-by-step solutions and form a set of candidate attempt solutions; guide the initial model to refine the core logic of each candidate attempt solution, identify and correct any logical gaps; combine the corrected reasoning paths with the corresponding questions to form an enhanced data set; S40: Heuristic scores are obtained based on the evaluation of the intrinsic attributes of the inference path; reasonableness scores are obtained based on the log probability evaluation of the inference path calculated based on the current parameters of the initial model; the heuristic scores and reasonableness scores are merged into a quality score through a preset quality scoring function; and the inference paths and corresponding questions with quality scores not lower than the preset verification threshold in the augmented dataset are integrated into the final training set. S50: Perform supervised fine-tuning on the initial model using the final training set, updating the parameters of the initial model by minimizing the negative log-likelihood loss; S60: Repeat steps S20-S50 until the quantization metrics and inference performance of the initial model meet the preset requirements, and output the initial model as the final large-scale language model.

[0008] The basic principle and beneficial effects of this invention are as follows: Based on the rejection sampling method, candidate reasoning paths that can yield the true final answer are screened from the seed dataset and combined with corresponding questions to construct a candidate set, ensuring the accuracy of the initial training data. Then, a pre-set sentence converter transforms the reasoning paths in the candidate set into semantic vectors. A pre-set clustering algorithm is used to cluster the semantic vector set, using the average semantic vector of each cluster as the cluster center vector. The core reasoning paths closest to the center vector are selected and integrated into a screening sample set, efficiently eliminating redundant data while accurately retaining key reasoning patterns. Subsequently, using the questions corresponding to the core reasoning paths as input, the initial model is guided to devise multiple differentiated strategies that can yield the true final answer, generating complete step-by-step solutions to form a candidate attempt set. Simultaneously, core logic is extracted and logical gaps are corrected, significantly improving the guidance depth and reliability of the training data. Heuristic scores are obtained by evaluating the inherent attributes of the reasoning paths, and a reasonableness score is obtained by combining the log probability of the reasoning paths calculated with the current parameters of the initial model. These are then fused into a two-dimensional quality score using a pre-set quality scoring function. Reasoning paths and corresponding questions not lower than a pre-set verification threshold are selected to form the final training set, ensuring that the training data input to the model is all high-value content. Supervised fine-tuning is performed on the initial model based on the final training set. The model parameters are updated with the goal of minimizing the negative log-likelihood loss. Through multiple rounds of iterative optimization, the initial model continuously absorbs diverse and high-quality reasoning logic, ultimately achieving a precise improvement in reasoning ability. This effectively solves the problems of insufficient strategy diversity and lack of depth of reasoning guidance in traditional single rejection sampling self-training methods.

[0009] This invention generates differentiated strategies and extracts core logic, enabling training data to comprehensively cover diverse problem-solving approaches. It breaks the limitation of traditional self-training methods where models rely on a single reasoning pattern, allowing the initial model to flexibly call different problem-solving logics when faced with new, unseen problems, thus significantly improving generalization ability. In contrast, traditional self-training methods, lacking guidance from diverse strategies, tend to solidify a single reasoning path, resulting in poor adaptability to unfamiliar scenarios.

[0010] The combined design of clustering screening core reasoning path and dual-dimensional scoring screening reduces the interference of redundant data on the training process, ensures the high quality and high value of training data, avoids invalid data occupying computing resources, and greatly improves the efficiency of model fine-tuning. Compared with traditional self-training methods that blindly use massive amounts of data for training, this invention can achieve the training goal faster with the same computing resources.

[0011] This invention specifically guides the model to identify and correct logical gaps in the reasoning path, ensuring that each reasoning path in the augmented dataset has a complete and rigorous logical chain. The reasoning process output by the initial model after learning is more organized and reliable, effectively reducing logical jumps or loopholes in the reasoning process. In contrast, traditional self-training methods often ignore logical defects in the data, which may result in the model still outputting logically inconsistent results after training.

[0012] The closed-loop design of multi-round iterative optimization allows the model parameters to be updated in a targeted manner based on high-quality data in each round of training, gradually consolidating the multivariate reasoning ability and logical rigor, avoiding parameter oscillation or local optimum problems caused by single training, so that the model performance can be steadily improved and eventually stabilized. Compared with traditional self-training methods that lack systematic iterative guidance, this invention can more efficiently drive the model to reach the preset performance standard.

[0013] In summary, this invention addresses the shortcomings of traditional self-training methods in terms of insufficient strategy diversity and lack of inference guidance depth through a joint closed-loop design that combines rejection sampling, clustering screening, differentiated strategy generation, dual-dimensional scoring, and iterative optimization. This simultaneously improves the model's generalization ability, training efficiency, and inference logic, ultimately achieving a precise, stable, and efficient enhancement of the reasoning capabilities of large-scale language models.

[0014] Furthermore, in step S20, a pre-trained semantic embedding model is used to convert the candidate inference paths in the candidate set into semantic vectors; then, an unsupervised clustering algorithm is used to divide the semantic vectors into multiple different clusters, and a semantic vector is selected from each spatially separated cluster. The average vector of the selected semantic vectors is calculated as the cluster center vector.

[0015] By accurately capturing the semantic features of reasoning paths through a pre-trained semantic embedding model, highly recognizable semantic vectors are generated. Combined with the clustering capabilities of unsupervised clustering algorithms, semantic clusters corresponding to different reasoning strategies are efficiently divided. The core intent of each reasoning strategy is accurately anchored through the calculation of cluster center vectors, laying the foundation for subsequent screening of non-redundant and diverse core reasoning paths, and effectively improving the targeting and efficiency of data selection.

[0016] Furthermore, in step S20, the semantic vector is normalized by unit and then clustered. After clustering, the redundancy of the inference path subsets corresponding to each cluster is not greater than a preset redundancy threshold. The redundancy is calculated by the average pairwise cosine similarity of the inference path embedding. The redundancy is determined based on the average pairwise cosine similarity of the inference path embeddings, where the embedding function is a preset semantic vector embedding function, and the number of semantic vectors in the inference path subset is the basic parameter for calculating the redundancy; the diversity score is determined based on the stable logarithmic determinant of the covariance matrix, where the covariance calculation function is used to solve the covariance matrix of the semantic vectors of the inference path, and the small ridge term is used to ensure that the covariance matrix is ​​a positive definite matrix.

[0017] Unit normalization of semantic vectors improves the numerical robustness of clustering computation; by constraining redundancy to be less than or equal to a preset redundancy threshold, redundant inference paths that are syntactically different but semantically equivalent are significantly reduced, avoiding gradient vanishing caused by the initial model stagnating on repetitive logic; using the diversity score defined by the stable logarithmic determinant of the covariance matrix as the optimization objective, the diversity of strategies for selecting sample sets is maximized, providing rich learning signals for the initial model and significantly enhancing the model's generalization ability.

[0018] Furthermore, during clustering, the data is divided into a number of clusters.

[0019] It avoids strategy omissions caused by too few clusters and prevents the introduction of sample noise caused by overly fine clustering; it makes the selected core inference paths both diverse and highly instructive, helping the initial model achieve the same accuracy as training with the full dataset with less data, and greatly improving data efficiency.

[0020] Furthermore, in step S30, when identifying and correcting logical gaps in the core logic of candidate attempts, common failure modes are corrected by bootstrapping. The common failure modes include incomplete enumeration, duplicate counting, ignoring reverses, and pre-set special cases. For the identified logical gaps, enhanced reasoning paths that explicitly handle different scrolling orders and the same scrolling situation are generated by bootstrapping, ultimately forming an enhanced data set with multiple deep reasoning trajectories for each problem.

[0021] By using the bootstrapping method to specifically correct common failure modes, accurately fill logical gaps such as incomplete enumeration and neglect of reverse cases, and effectively suppress error propagation in self-training; generate enhanced inference paths that clearly handle different scrolling orders and the same scrolling situation, so that each problem is equipped with multiple deep inference trajectories, significantly improving the completeness, accuracy and guidance depth of the inference path, and providing richer and higher quality learning signals for the initial model.

[0022] Furthermore, in step S40, the quality function is a weighted sum of heuristic scoring and reasonableness scoring; each sample can be evaluated in real time based on the quality score; during real-time evaluation, the augmented dataset is divided into several mini-batches, and gradient calculation is performed on the initial model and the parameters of the initial model are updated only when the mini-batches meet the quality screening conditions; the quality screening conditions are that the mean quality score of all samples in the target mini-batches is not lower than a preset quality threshold, and if the target mini-batches meet the quality screening conditions, the initial model is updated using mini-batches.

[0023] A quality assessment method that combines heuristic scoring with reasonableness scoring is adopted to achieve real-time and robust evaluation of sample quality. Through a mini-batch screening mechanism, gradient calculation and parameter updates are performed only on qualified batches, effectively blocking the interference of low-quality and noisy data on the initial model parameters. This dynamic screening process has low computational cost, which not only ensures the overall quality of training data, but also improves the stability and efficiency of model training.

[0024] Furthermore, in step S50, the parameter update of the initial model iteration is achieved by minimizing the SFT loss on the sample set consisting of all mini-batches that have passed the quality screening.

[0025] By limiting initial model parameter updates to all mini-batch sets that pass quality screening, precise parameter optimization is achieved by minimizing SFT loss, ensuring that initial model updates are entirely based on high-quality data from dynamic validation. This approach directly integrates data quality assurance into the optimization loop, strengthens the role of effective learning signals, further improves the inference performance and robustness of the initial model, and ensures the stability of training results.

[0026] Furthermore, the heuristic score is derived by evaluating the intrinsic attributes of the reasoning path. These intrinsic attributes include whether the length of the reasoning path meets the minimum length requirement for effective reasoning, whether there is a rejection statement, and whether the numerical calculation is stable and free from logical contradictions. Each attribute is quantified according to a preset weight and then summed to obtain a heuristic score in the range of 0-1. The rationality score is the log probability assigned to the reasoning path in the current state of the initial model. The quality score is determined based on a weighted combination of the heuristic score and the rationality score. The weight hyperparameter is used to adjust the proportion of the heuristic score in the quality score, and the preset verification threshold used to select the reasoning path is not lower than the preset quality verification threshold.

[0027] The heuristic score generated by multi-dimensional intrinsic attribute evaluation complements the rationality score given by the current state of the initial model. After weighted fusion, it achieves accurate quantification of the quality of the inference path. The strict screening with a preset verification threshold not lower than the preset quality verification threshold effectively filters out low-quality samples. This composite quality scoring mechanism balances the objectivity of the heuristic signal with the rationality of the model judgment, reduces training variance, and improves the consistency and reliability of the initial model performance.

[0028] Furthermore, in step S30, when guiding the initial model to conceive a differentiation strategy, the differentiation strategy is a non-overlapping reasoning paradigm, including case enumeration method, structured set reasoning method, and inclusion-exclusion principle method; the complete step-by-step solution corresponding to each differentiation strategy includes a clear logical starting point, intermediate derivation steps, and conclusion verification process.

[0029] By employing non-overlapping reasoning paradigms such as case enumeration, structured set reasoning, and inclusion-exclusion principle, the initial model's strategy coverage for problem-solving is enriched, avoiding the generation of semantically equivalent redundant reasoning paths. The complete step-by-step solutions corresponding to each strategy (including logical starting point, intermediate derivation, and conclusion verification) enhance the guidance depth and learnability of the reasoning path, thereby strengthening the initial model's strategic flexibility and generalization ability in dealing with complex problems. Attached Figure Description

[0030] Figure 1 A flowchart illustrating a self-training method for large-scale language models based on the GALA framework; Figure 2 This presents experimental results for GALA on non-mathematical, open-ended reasoning tasks. Figure 3 A graph showing the relationship between performance (accuracy) of GSM8K and ASDIV test sets on the LLaMA-3,1–8B model and the size of the training data. Figure 4 A graph showing the relationship between the performance (accuracy) of the MATH and MMLU-PRO test sets on the LLaMA-3,1–8B model and the size of the training data; Figure 5 A graph showing the relationship between the performance (accuracy) of the GSM8K and ASDIV test sets on the LLaMA-3,2–1B model and the size of the training data. Figure 6 A graph showing the relationship between the performance (accuracy) of the MATH and MMLU-PRO test sets on the LLaMA-3,2–1B model and the size of the training data; Detailed Implementation

[0031] The following will describe the concept and technical effects of the present invention clearly and completely with reference to embodiments, so as to fully understand the purpose, features and effects of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the scope of protection of the present invention. like Figure 1 As shown, a self-training method for large-scale language models based on the GALA framework includes the following steps: S10: For a pre-set seed dataset containing questions and corresponding true answers, guide the pre-set initial model to generate multiple candidate reasoning paths for each question, filter out the candidate reasoning paths that can lead to the true final answer, and combine them with the corresponding questions to form a candidate set.

[0032] Specifically, let's set For parameters The initial model is a large language model (LLM). In this embodiment, the model is improved through an iterative self-training process. Mathematical reasoning ability. Initially given a seed dataset. , As shown below: , In this implementation, the default is... For the i-th mathematical problem, For the i-th mathematical problem The corresponding true final answer. For a sample containing N samples It is the initial data source for self-training.

[0033] The standard self-training method based on the administrator's preset rejection sampling follows a simple "generate-filter" process, which is defined in this embodiment as a naive self-training loop (as the baseline method). This loop begins with the candidate generation phase, i.e., for the seed dataset... Each question in Initial model Prompted to generate containing A set of candidate reasoning paths : , in, For the i-th problem The generated candidate inference path set contains K paths. For the i-th question The Kth candidate inference path follows the initial model. Given The generation distribution ( ).

[0034] Then comes the validation and filtering phase. Specifically, this is done through a validation function. Evaluate each generated path If the path leads to the true answer If so, then return 1.

[0035] Collect all validated paths to form a new training dataset. , The calculation formula is as follows:

[0036] Finally, the parameters of the initial model are updated by fine-tuning. Specifically, this is done on the collected dataset. Supervised fine-tuning (SFT) is performed on the above, specifically by minimizing the negative log-likelihood loss:

[0037] , The above loop can be iterated, using the updated initial model. Proceed to the next generation step. To make the objective function (such as the loss function) Minimize the parameters of the initial model The operation is the core logic for updating parameters during the fine-tuning stage. For the expectation operator, on the dataset The average of all "problem-reasoning paths" (x,r) is calculated and used for global quantization of the loss function. The natural logarithm function is used to calculate the log probability of the inference path r generated by the initial model. This reflects the rationality of the path.

[0038] However, this naive cycle has obvious bottlenecks, namely, high verification cycle delay and coarse granularity.

[0039] Despite its simplicity, the naive self-training loop is extremely inefficient, often resulting in poor-performing training datasets. Specifically, it suffers from two major drawbacks: data redundancy and a lack of policy diversity and depth of guidance.

[0040] set up To allow arbitrary reasoning paths Mapping to semantic vector embedding The function. This is a semantic embedding function.

[0041] Redundancy The average pairwise cosine similarity is calculated based on the inference path embedding, where the embedding function is a pre-defined semantic vector embedding function, and the number of semantic vectors in the inference path subset is used to calculate redundancy. The basic parameters are: diversity score, which is determined by calculating the stable logarithmic determinant of the covariance matrix, where the covariance calculation function is used to solve for the covariance matrix of the semantic vector of the reasoning path, and the small ridge term is used to ensure that the covariance matrix is ​​a positive definite matrix.

[0042] Specifically, regarding the determination of data redundancy, the brute-force generation process often produces a large number of syntactically different but semantically equivalent reasoning paths. Define the dataset. The redundancy is the average pairwise cosine similarity of the inference path embeddings:

[0043] The redundancy index for dataset D is calculated using the average pairwise cosine similarity of the inference path embeddings. A higher value indicates more severe semantic redundancy (in GALA, J ≤ 0.3, where 0.3 is a preset redundancy threshold, which can be adjusted according to redundancy limit requirements).

[0044] High redundancy This means that the model is "standing still" (meaning that during the initial self-training process, the model only makes minor adjustments around reasoning paths that are syntactically different but semantically equivalent, and fails to learn new reasoning strategies or deeper logic, ultimately leading to diminishing learning gains and ineffective performance improvement), and only repeats the same logic with different expressions, resulting in vanishing gradients and insufficient guidance depth.

[0045] For datasets lacking policy diversity and guidance depth, correctness filtering alone cannot guarantee policy diversity; the initial model may only learn a few dominant inference patterns to solve problems. To quantify and improve the dataset... The diversity score is determined using an embedding-based generalized variance metric to measure the volume of inference paths within the semantic space. Specifically, a diversity score is defined using the stable logarithmic determinant of the covariance matrix. : , in To ensure the Xiaoling project in Zhengding. The geometric meaning is that the determinant is proportional to the square of the volume of the parallelepiped spanned by the embedding vectors. Maximizing... This can force the initial model to choose an "orthogonal" inference path (referring to the low correlation between inference paths in semantic embedding space and solution strategy). Specifically, if two paths use the same strategy, they will be collinear, resulting in volume collapse. A penalty will be applied to this set. All embeddings are unit-normalized before computation to improve numerical robustness.

[0046] For low relevance, from a semantic embedding perspective, it is through the embedding function After converting the reasoning paths into vectors, the vectors of the "orthogonal" paths have large angles and low cosine similarity (corresponding to low redundancy J(D)), indicating that their semantic expressions are very different. From the perspective of solution strategies, the "orthogonal" paths adopt completely different reasoning paradigms. For example, for the same mathematical problem, one path uses the "case enumeration method" to calculate each case, while the other uses the "inclusion-exclusion principle" to simplify the derivation. They are not paths that are only different in expression but have the same logical essence.

[0047] The main problem to be solved is geometric redundancy, while guidance depth (the solution is correct but superficial) is a qualitative problem that needs to be solved through semantic optimization rather than geometric selection. This goal is mainly achieved in the enhancement phase (step S30).

[0048] S20: Use a preset sentence converter to perform semantic vector transformation on the candidate inference paths in the candidate set to obtain a set of semantic vectors; use a preset clustering algorithm to cluster the set of semantic vectors and calculate the average semantic vector of each cluster as the cluster center vector; select the inference path corresponding to the semantic vector closest to the center vector in each cluster as the core inference path and integrate them into a screening sample set.

[0049] The core challenge of LLM self-training is to sift out a compact subset of samples with diverse strategies from massive amounts of redundant data. In this embodiment, it is transformed into an online maximum-volume subset selection problem: identifying the subset of inference paths with the most dispersed semantic embedding distribution, ensuring that subsequent gradient updates can incorporate diverse inference strategies, and maximizing the maximum volume defined by the formula. To efficiently address this issue during the training loop, a clustering-anchoring method is employed. This method first uses the all-mpnet-base-v2 model to convert inference paths into semantic vectors, then uses K-Means or DBSCAN clustering to identify diverse policy clusters. Finally, the true inference path closest to the center vector within each cluster is selected as the anchor point, reducing redundancy and ensuring policy diversity, thus providing high-quality learning samples for the initial model. This method identifies different inference policy clusters and selects a single high-quality example from each cluster as the main direction for a single round of learning.

[0050] During each training iteration, the embedding function is used. (The pre-trained semantic embedding model can use the all-mpnet-base-v2 sentence converter) to include all available inference paths. The embeddings are then converted into vector representations. For large-scale datasets, the clustering-anchor method avoids quadratic computation and is an efficient choice. Subsequently, unsupervised clustering algorithms (such as K-Means or DBSCAN) are applied to divide the embeddings into... This process effectively identifies diverse sets of learning signals by dividing the data into different clusters. By selecting a representative sample from each spatially separated cluster, it ensures that the samples guiding the initial model update are semantically free of redundancy, thus achieving highly diverse updates (in order to achieve high diversity). (A necessary condition for measurement).

[0051] Cluster center ( The vector represents the average intent of the clustering, but is a synthetic vector without a textual counterpart. Intuitively, training based on cluster centers is like attempting to learn from the fuzzy average of five different mathematical solutions. Therefore, GALA chooses anchor points, which are the embeddings of the real data points in the clusters that are closest to the cluster centers. The formal definition of this choice is as follows:

[0052] , This design ensures that the model learns from real, human-readable logic while capturing the core strategy direction of clustering.

[0053] Filtered sample set While offering diversity, the limited scale of these anchors can lead to insufficient gradient information. To enrich the learning signals, GALA shifts from passive selection to active exploration, transforming each anchor into a catalyst for generating high-quality inference. Intuitively, this design achieves a transformation from "what to learn" to "how to reason through diverse strategies and guided depth."

[0054] S30: Using the questions corresponding to the core reasoning paths in the selected sample set as input, guide the initial model to conceive multiple differentiated strategies that can obtain the true final answer, generate corresponding complete step-by-step solutions and form a set of candidate attempts; guide the initial model to refine the core logic of each candidate attempt, identify and correct the logical gaps; combine the corrected reasoning paths with the corresponding questions to form an enhanced data set.

[0055] Specifically, to encourage exploration of the solution space, the initial model is first prompted to devise multiple different solution strategies for a given problem in an expert's capacity, thereby stimulating multi-path reasoning. For each proposed strategy, the initial model is further guided to generate a complete step-by-step solution. For example, in the Melinda dice problem, this subroutine allows the initial model to move beyond simple case enumeration and explore structured set reasoning through the principle of inclusion-exclusion. This guided generation process, which includes self-verification and reflective prompts, ensures that each synthetic path is strategically unique, constituting a high-quality novel training signal.

[0056] To suppress error propagation inherent in self-training, an ensemble-reflection mechanism is employed. The initial model receives multiple potentially noisy proposed solutions, requiring the core logic to be extracted and the final solution regenerated. This process effectively corrects common failure modes (such as incomplete case enumeration). The initial model may initially fail due to ignoring reverse dice rolls (e.g., including (1,2) but omitting (2,1)). Through reflection, GALA can identify this logical gap and generate enhanced inference paths that explicitly handle different roll orders and identical roll cases via bootstrapping, improving the depth and clarity of guidance.

[0057] The final set A strategy-intensive training corpus is constructed, with each question accompanied by multiple deep reasoning trajectories.

[0058] S40: A heuristic score is obtained based on the evaluation of the intrinsic attributes of the inference path; a reasonableness score is obtained based on the log probability evaluation of the inference path calculated based on the current parameters of the initial model; the heuristic score and the reasonableness score are merged into a quality score through a preset quality scoring function; and the inference paths and corresponding questions with quality scores not lower than the preset verification threshold in the augmented dataset are integrated into the final training set.

[0059] In this embodiment, the problem of low verification efficiency is solved by directly integrating the quality control mechanism into the fine-tuning process.

[0060] A real-time mini-batch screening mechanism replaces the traditional, slow offline evaluation, providing immediate feedback. This screening mechanism ensures that the initial model calculates gradient updates only based on high-value data, avoiding the corruption of parameters by low-quality or noisy synthetic samples generated in the previous step.

[0061] The quality score is determined based on a weighted combination of heuristic scores and reasonableness scores, where the weight hyperparameter is used to adjust the proportion of heuristic scores in the quality score.

[0062] Specifically, the screening mechanism consists of a quality scoring function. Control, scoring function A comprehensive score is generated by combining heuristic signals with model-based evaluation. , in The weight hyperparameter has two components defined as follows: Heuristic scoring ( No model reasoning is required; the intrinsic properties of the reasoning path (such as length, presence of rejection statements, and numerical stability) can be directly evaluated. For a detailed breakdown, see D.7.

[0063] Reasonableness score ( The current model state (the teacher model in this round of training) represents the inference path. The logarithmic probability of the assignment. Intuitively, It serves as a rationality checker. Specifically, if the reasoning path is meaningless, even if the final answer is correct, the initial model's intrinsic probability of these tokens will be very low, thus preventing the initial model from learning flawed shortcuts that cannot generalize during fine-tuning. Robust real-time evaluation can be performed on each sample.

[0064] S50: Perform supervised fine-tuning on the initial model using the final training set, updating the parameters of the initial model by minimizing the negative log-likelihood loss.

[0065] Specifically, augmented data Classified as mini batch Only when mini batch Gradient calculation and parameter updates are only performed after the quality screening is passed. More specifically: .

[0066] This dynamic filtering process is computationally inexpensive and effectively prevents harmful data from affecting the parameters of the initial model. The final parameter update of the GALA iteration is performed only on the mini-batch set that passes the filtering. minimize on The SFT loss implementation ensures that the initial model update is entirely based on the dynamically validated sample set, directly integrating data quality assurance into the optimization loop.

[0067] S60: Repeat steps S20-S50 until the quantization metrics and inference performance of the initial model meet the preset requirements, and output the initial model as the final large-scale language model.

[0068] Each training iteration consists of collaborative steps S20-S50, dynamically generating the optimal data distribution for parameter updates. First, a set of high-impact core training samples is selected. Then, this sample set is expanded by exploring and synthesizing new inference trajectories. Finally, the generated mini-batches are filtered in real-time before gradient updates. This ensemble approach redefines self-training as a process of learning from a dynamically evolving stream of high-quality data.

[0069] In this embodiment, to rigorously evaluate the effectiveness of the proposed solution, a series of comprehensive experiments were conducted, covering performance, data efficiency, and robustness analysis.

[0070] Specifically, the initial model was evaluated on two representative open-source large language models in the LLaMA series (such as LLaMA-3.1-8B-Instruct and LLaMA-3.2-1B-Instruct), and the evaluation was extended to Qwen-2.5-7B-Instruct.

[0071] To demonstrate the effectiveness of GALA, this invention is compared with three baseline methods: full data reference baseline, data selection baseline, and state-of-the-art data-weighted baseline EAST. The full data reference baseline involves fine-tuning all validated generative inference paths. The data selection baselines were random sampling, the classic active learning strategy based on uncertainty sampling (selecting the data with the lowest probability in the initial model), and the NeMoCurator heuristic process. The size of the training subset for all selected strategies was consistent with GALA.

[0072] All models used the AdamW optimizer with bf16 accuracy to improve computational efficiency. For larger LLaMA-3.1-8B-Instruct models, low-rank adaptation (LoRA) was used to reduce the memory consumption of fine-tuning. General training and LoRA configurations are shown in Table 1 below:

[0073] Table 1 - General Experimental Setup and LoRA Configuration Where LoRA Configuration is the LoRA configuration, Rank(r) is the rank r of low-rank adaptation (LoRA), and Alpha(α) is the scaling factor α of LoRA.

[0074] All experiments were conducted on four NVIDIA A800 machines, and data generation followed the administrator's preset EAST workflow.

[0075] GALA was trained and validated using widely used mathematical reasoning benchmark datasets, and evaluated on tasks outside of mathematics, including Writingbench, CommonSenseQA, and TruthfulQA. The benchmark datasets are as follows: MATH (Mathematical Problems Dataset) contains 12,500 competition-level problems from competitions such as AMC and AIME, covering seven mathematical subjects and five difficulty levels. Training and evaluation are performed using complete training and test sets. GSM8K (Elementary School Math Word Problems Dataset) consists of 8,500 elementary school math problems collected by human question creators. Training and evaluation were performed using the complete training and test sets. MMLU-Pro-MATH is a mathematical subset of the MMLU-Pro dataset used for data augmentation. The MMLU-Pro dataset contains 12,000 complex cross-disciplinary problems, including 1,356 mathematical problems. 400 problems were randomly sampled for training, and the remainder were used for evaluation. ASDIV (a classic benchmark dataset for assessing mathematical reasoning ability) was used, containing 2,305 complex math problems. 230 problems were randomly sampled for training, and the remainder were used for evaluation. Writingbench (a comprehensive benchmark dataset for evaluating writing skills) is a comprehensive benchmark for evaluating LLM writing skills, containing 1,000 real-world queries. 200 questions are randomly sampled for training, and the remainder are used for evaluation. CommonSenseQA (a classic question-answering dataset for testing commonsense reasoning ability) requires different types of commonsense knowledge to predict the correct answer. It contains 12,102 questions, each with one correct answer and four distractor answers. 1,200 questions are randomly sampled for training, and the remainder are used for evaluation. TruthfulQA (a benchmark dataset for evaluating the authenticity and reliability of generated content) contains 817 questions, mainly in the categories of health, law, finance, and politics. 200 questions were randomly sampled for training, and the remainder were used for evaluation.

[0076] Experiments focusing on data selection and quality assessment rely on specific clustering and quality scoring hyperparameters, as detailed in Table 2.

[0077]

[0078] Table 2 - Hyperparameters of the data selection and quality scoring experiment Experiments show that GALA significantly improves data efficiency while maintaining competitive accuracy. As shown in Table 3, for SFT training of LLaMA-3.1-8B, GALA uses only slightly more than 1 / 10 of the data volume, and its accuracy is basically on par with the baseline of the full dataset.

[0079] The search spaces for Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) are shown in Table 3.

[0080]

[0081] Table 3 - Hyperparameter search space for different training methods and models In the hyperparameter sensitivity analysis of clustering methods, TF-IDF (Ramos et al., 2003) was used to generate the initial question embeddings due to its high efficiency. Subsequently, the range of values ​​for the number of clusters k was explored to find the optimal balance between data diversity and performance, and finally k=80 was chosen in the main experiments.

[0082] As shown in Table 9 below, in the ablation experiments of the quality function Q(r), the default quality acceptance threshold used by the complete model is not lower than the preset quality verification threshold (τ=0.8 in this embodiment). To demonstrate the importance of rigorous screening, the "relaxed threshold" configuration in the ablation experiments was specifically tested with a more lenient value τ=0.6.

[0083] Extending GALA to non-mathematical, open-ended reasoning tasks, the experimental results are as follows: Figure 2 As shown; where Full Data is random sampling, +Random Sampling is uncertain sampling, +Uncertainty Sampling +NoMoCurator, +GALA (Ours)GALA (the method of this embodiment), +EAST, +EAST +GALA (Ours) (the method of this embodiment); GSM8K (%), MATH (%), MMLU-PRO (%), ASDIV (%), AVG (%) represent the proportion of different data types in the benchmark dataset; LLaMA-3,2–1B and LLaMA-3,1–8B are the large language models used in the initial model.

[0084] according to Figure 2 Based on the data in the dataset, plot the relationship between performance (accuracy) and training data size on the GSM8K, MATH, ASDIV, and MMLU-PRO test sets. The plotting results are shown below. Figure 3-6 As shown in the figure. The X-axis represents the number of training samples used, and the Y-axis represents the accuracy. Figure 3-6 The overall trend clearly shows that GALA achieves high accuracy with superior data efficiency. Compared to the two baseline methods, the method presented in this embodiment achieves higher performance with fewer samples.

[0085] Figure 2 A quantitative summary of the main results is provided. To ensure statistical robustness, the average precision of three different randomized seed experiments is reported in this embodiment. The results show that the GALA method achieves sustained performance improvements across different model sizes and loss function baselines.

[0086] GALA's advantages are not limited to specific architectures or domains. As shown in Table 4 (extended experiments with Qwen-2.5-7B-Instruct) and Table 5 (extended experiments with tasks other than mathematics, including Writingbench, CommonSenseQA, and TruthfulQA), GALA consistently outperforms competing baselines in diverse models and non-mathematical reasoning scenarios, validating its broad applicability.

[0087] Specifically, the experiments used the LLaMA-3.1-8B-Instruct model to test GALA's generalization ability to non-mathematical tasks, evaluating it on tasks with subjective or fuzzy correctness criteria. As shown in Table 4, the evaluation was extended to the Qwen-2.5-7B-Instruct model. GALA improved the accuracy of the GSM8K and MATH datasets, confirming its effectiveness across different model architectures. As shown in Table 5, experiments on Writingbench (a benchmark dataset for evaluating the authenticity and reliability of generated answers), TruthfulQA (a benchmark dataset for evaluating the authenticity and reliability of content generated by large language models), and CommonSenseQA (an authoritative multiple-choice question-answering dataset for evaluating the commonsense reasoning ability of models in the field of natural language processing and large models) show that GALA achieves efficiency improvements comparable to those for mathematical tasks. The experiments used the LLaMA-3.1-8B-Instruct model.

[0088]

[0089] Table 4 - Extended experimental results of GALA on Qwen-2.5-7B-Instruct. Table 4 shows that both random sampling and uncertainty sampling extract 10% of the original data.

[0090]

[0091] Table 5 - Extended experimental results of GALA on non-mathematical, open-ended reasoning tasks. GALA observed comparable efficiency improvements to mathematical tasks in these non-mathematical, open-ended reasoning tasks; both random sampling and uncertain sampling extracted 10% of the original data.

[0092] To validate the contributions of each GALA component, numerous ablation experiments were conducted. Specifically, a component was systematically removed, and its impact on final performance was observed. Experimental configurations included: unfiltered GALA (no clustering or anchor selection, using all generated data); unenhanced GALA (omitting strategy brainstorming, reflection, and bootstrapping); and unvalidated GALA (removing the final real-time quality filter).

[0093]

[0094] Table 6 - Contribution of the Separation Selection Module when Selecting Ablation Experiments The enhancement and validation phases of GALA were fixed, with only the selection strategy in the first phase being changed. The proposed clustering-anchor method was compared with random sampling, uncertainty sampling, and K-Means cluster centers (synthetic mean versus true anchors). Table 6 shows that even using the same strong enhancement or validation backend, the clustering-anchor selection significantly outperforms the standard baseline, confirming the unique advantage of selecting true, high-quality geometric anchors over simple heuristic methods.

[0095] On the GSM8K dataset, the default embedding model (all-mpnet-base-v2) was compared with the statistical vectorized baseline (TF-IDF) and the state-of-the-art instruction fine-tuning model (BGE-Large). Different clustering algorithms (K-Means, DBSCAN (Deng, 2020), and hierarchical clustering) were also tested. The results are shown in Table 7. The results in Table 7 demonstrate that GALA's performance is robust to the choice of embedding model and clustering algorithm because geometric selection depends on relative semantic structure rather than absolute embedding values.

[0096]

[0097] Table 7 - Ablation Experiments of Embedded Models and Clustering Algorithms The frequency of failures in the reflection-guided bootstrapping method due to propagation error rather than correction error was analyzed using LLaMA-3.1-8B-Instruct on 100 GSM8K samples. Table 8 shows that the brainstorming phase introduced a moderate level of noise (17.8% error rate), while the correction rate of the reflection module was 1.7% (most solutions were correct). Dynamic validation filter ( The false positive rate remained at 20.0%, which effectively screened out incorrect solutions.

[0098]

[0099] Table 8 - Noise Quantification Results of Reflection-Guided Bootstrapping To verify the design of the composite mass function Q(r), ablation experiments were conducted to analyze the contribution of its various components, such as the mass scoring function. As shown in the controlled screening mechanism, the quality function is a heuristic scoring ( ) and reasonableness score ( The experiment systematically removes components of Q(r) to demonstrate the necessity of the complete formula for achieving optimal model performance and training stability. Four configurations are evaluated: the complete proposal function, a heuristic-only version, a reasonableness score-only version, and the complete function with a lenient acceptance threshold (τ).

[0100]

[0101] Table 9 - Ablation Experiment Results of the Mass Function Q(r) Component Table 9 illustrates the trade-offs among these configurations: the "heuristic-only" approach results in a significant drop in accuracy and a substantial increase in training variance, indicating that while heuristics are useful, they are insufficient to reliably identify high-quality data; conversely, the "reasonableness scoring only" configuration achieves higher accuracy but suffers from the worst stability, likely due to its sensitivity to noise signals during mini-batch validation; relaxing the quality threshold to τ=0.6 leads to a performance degradation in both metrics as low-quality data enters the training set. The proposed full Q(r) implementation significantly outperforms the ablation version, achieving the highest accuracy on GSM8K while maintaining the lowest training variance. This demonstrates that each component of the quality function is crucial for balancing performance and stability, justifying the composite design.

[0102]

[0103] Table 10 - Ablation experiments (SFT results) using the LLaMA-3.1-8B model on the GSM8K and MATH test sets.

[0104] Table 11 - Ablation experiments (DPO results) using the LLaMA-3.1-8B model on the GSM8K and MATH test sets. The results in Tables 10 and 11 show that each component is indispensable, meaning that removing any one stage leads to a significant performance degradation, confirming that the synergy of filtering, enhancement, and validation is key to the success of this method.

[0105] In self-training scenarios, each selected data point used for SFT can be considered a "labeled" sample, and GALA demonstrates excellent labeling efficiency. For example... Figure 3-6 As shown, GALA only requires a selection dataset of 80 samples to reach the target performance level, while the naive baseline needs fine-tuning on 2000 generation paths to approach similar performance, improving annotation efficiency by 24 times.

[0106] To gain a comprehensive understanding of the efficiency of the GALA training algorithm, the total computational cost (in GPU hours) of a single training iteration on the GSM8K dataset was analyzed. Table 12 compares the end-to-end iteration time of our proposed algorithm with the Naive SFT baseline. The analysis shows that GALA's ensemble design generally results in a more computationally efficient training process.

[0107]

[0108] Table 12 - Comparison of each stage and total GPU runtime of a single training iteration A single training iteration of the GALA algorithm consists of multiple computational stages: the initial geometry selection (embedding and clustering) is performed on the CPU at negligible cost; the main computational components come from model inference during dynamic data orchestration, which is accelerated by the vLLM framework.

[0109] The key advantage of this integrated approach becomes apparent in the final parameter update phase: although the GALA iteration includes intelligent sample generation and selection phases (approximately 3 hours), it significantly reduces the cost of expensive SFT training by over 90% (from 4.0 hours to 0.33 hours). The end result is a 16.8% reduction in total runtime, while data efficiency is improved tenfold. Therefore, based on the GSM8K dataset and the LLaMA-3.1-8B model, comparing the naive SFT baseline algorithm with the GALA training algorithm, GALA integrates data orchestration into the training loop, achieving a shorter overall iteration time. Regarding computational overhead, Table 12 confirms that GALA reduces total runtime by 16.8% by eliminating redundant parameter updates.

[0110] Beyond efficiency, this paper quantifies the depth of guidance provided by the data. Table 13 shows that GALA generates solutions with 14% more detail and higher clarity scores. To address the error propagation problem, the noise analysis in Table 8 confirms that the reflection and dynamic verification modules effectively suppress synthetic noise, ensuring high-quality learning signals. Qualitative analysis indicates that this process enhances the depth of guidance. Specifically, the GALA process learns from existing errors and generates correct inference paths through reflection and bootstrapping phases. For example, common failure modes of the baseline model include incomplete case enumeration or duplicate counting: the model might correctly identify that the tens digit needs to be "1" but fail to correctly handle the logic of the second die and reverse cases. Reflection and bootstrapping transform superficial or flawed inference paths into well-structured, deeply explained solutions, providing richer learning signals beyond mere correctness.

[0111]

[0112] Table 13 - Quantitative Indicators of Reasoning Depth

[0113] Table 14 - Output results of synthetic data evaluation using Self-BLEU scores In Table 14, a lower Self-BLEU score indicates better diversity.

[0114] When quantifying inference depth, the “depth” of the standard CoT baseline and GALA-enhanced data is compared using three specific metrics: average number of tokens (as a proxy for detail); explicit logical steps (the number of inference connection tokens); and blind teacher clarity scores (1-5 points) obtained by GPT-4o on 100 randomly sampled GSM8K instances.

[0115] As shown in Table 13 (4.7 and 4.5), this confirms that GALA can increase guidance value, rather than simply adding redundancy, even on existing high-quality baselines.

[0116] GALA's data augmentation method ensures that each solution path is strategically unique through strategic brainstorming. Table 14 directly verifies the effectiveness of this strategy: on the MATH, GSM8K, ASDIV, and MMLU_PRO_MATH datasets, the initial model generates 12 outputs for each question, and the final score is the average of the Self-BLEU scores for each question. The results show that the synthetic data selected by GALA has significantly lower Self-BLEU scores than the Naive QA synthetic and rewritten data, confirming that this strategy successfully constructs datasets with diversity at both the strategic and experimental levels.

[0117] When using the K-Means algorithm (instead of DBSCAN) in the anchor selection process, a sensitivity analysis of the number of clusters k was performed to explore the trade-off between the diversity of the selected data and the performance of the final large-scale language model task. k was varied across a range of values ​​(20, 40, 80, 160), and its impact on data diversity and GSM8K accuracy was measured. Data diversity was quantified as the average pairwise cosine distance of the selected anchor embeddings; a higher score indicates greater diversity of the selected data points.

[0118] The analysis in Table 15 reveals a clear trade-off: when k is small (e.g., k=20), the diversity score is high because the selected few anchors are necessarily far apart in the embedding space. However, this high diversity comes at the cost of performance, as the selected data points may be too sparse to cover the full distribution of the problem type. As k increases, more anchors are selected, naturally reducing the average distance between them and improving model accuracy. But when k is too large (k=160), this trend reverses, as the method begins to select anchors from smaller, less representative clusters, introducing noise and slightly reducing performance. The analysis confirms that choosing k=80 provides a robust balance, maximizing the final model accuracy by ensuring that the data selection is both diverse and representative. In the main experiments with GSM8K, k=80 was used.

[0119]

[0120] Table 15 - Sensitivity analysis data for the number of clusters (k) This embodiment also includes a large-scale language model self-training system based on the GALA framework, which implements a large-scale language model self-training method based on the GALA framework.

[0121] The above are merely embodiments of the present invention. The invention is not limited to the fields covered by these embodiments. Commonly known structures and characteristics in the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are able to access all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A self-training method for large-scale language models based on the GALA framework, characterized in that, Includes the following steps: S10: For a pre-set seed dataset containing questions and corresponding true answers, guide the pre-set initial model to generate multiple candidate reasoning paths for each question, filter out the candidate reasoning paths that can lead to the true final answer, and combine them with the corresponding questions to form a candidate set; S20: Use a preset sentence converter to perform semantic vector transformation on the candidate reasoning paths in the candidate set to obtain a set of semantic vectors; use a preset clustering algorithm to cluster the set of semantic vectors and calculate the average semantic vector of each cluster as the cluster center vector; select the reasoning path corresponding to the semantic vector closest to the center vector in each cluster as the core reasoning path and integrate them into a screening sample set; S30: Using the questions corresponding to the core reasoning paths in the selected sample set as input, guide the initial model to devise a variety of differentiated strategies that can yield the true final answer, generate corresponding complete step-by-step solutions, and form a set of candidate trial solutions; The initial model is guided to extract the core logic of each candidate solution and to identify and correct any logical gaps. The revised reasoning path is combined with the corresponding question to form an enhanced dataset; S40: Heuristic scores are obtained based on the evaluation of the intrinsic attributes of the reasoning path; The reasonableness score is obtained by evaluating the log probability of the inference path based on the current parameters of the initial model; the heuristic score and the reasonableness score are merged into a quality score through a preset quality scoring function; and the inference paths and corresponding questions with quality scores not lower than the preset verification threshold in the augmented dataset are integrated into the final training set. S50: Perform supervised fine-tuning on the initial model using the final training set, updating the parameters of the initial model by minimizing the negative log-likelihood loss; S60: Repeat steps S20-S50 until the quantization metrics and inference performance of the initial model meet the preset requirements, and output the initial model as the final large-scale language model.

2. The self-training method for large-scale language models based on the GALA framework according to claim 1, characterized in that: In step S20, a pre-trained semantic embedding model is used to convert the candidate inference paths in the candidate set into semantic vectors. Then, an unsupervised clustering algorithm is used to divide the semantic vectors into multiple different clusters. A semantic vector is selected from each spatially separated cluster, and the average vector of the selected semantic vectors is calculated as the cluster center vector.

3. The self-training method for large-scale language models based on the GALA framework according to claim 2, characterized in that: In step S20, the semantic vector is normalized by unit and then clustered. After clustering, the redundancy of the inference path subsets corresponding to each cluster is not greater than the preset redundancy threshold. The redundancy is calculated by the average pairwise cosine similarity of the inference path embedding; The redundancy is determined based on the average pairwise cosine similarity of the inference path embeddings, where the embedding function is a preset semantic vector embedding function, and the number of semantic vectors in the inference path subset is the basic parameter for calculating the redundancy; the diversity score is determined based on the stable logarithmic determinant of the covariance matrix, where the covariance calculation function is used to solve the covariance matrix of the semantic vectors of the inference path, and the small ridge term is used to ensure that the covariance matrix is ​​a positive definite matrix.

4. The self-training method for large-scale language models based on the GALA framework according to claim 3, characterized in that: When performing clustering, the data is divided into a number of clusters.

5. The self-training method for large-scale language models based on the GALA framework according to claim 1, characterized in that: In step S30, when identifying and correcting logical gaps in the core logic of candidate attempts, common failure modes are corrected by bootstrapping. The common failure modes include incomplete enumeration, duplicate counting, ignoring reverses, and pre-set special cases. For the identified logical gaps, enhanced reasoning paths that explicitly handle different scrolling orders and the same scrolling situation are generated by bootstrapping, and finally, an enhanced data set with multiple deep reasoning trajectories is formed for each problem.

6. The self-training method for large-scale language models based on the GALA framework according to claim 1, characterized in that: In step S40, the quality function is a weighted sum of heuristic scores and reasonableness scores; each sample can be evaluated in real time based on the quality scores; during real-time evaluation, the augmented dataset is divided into several mini-batches, and gradient calculation is performed on the initial model and the parameters of the initial model are updated only when the mini-batches meet the quality screening criteria. The quality screening condition is that the average quality score of all samples in the target mini-batch is not lower than a preset quality threshold. If the target mini-batch meets the quality screening condition, the initial model is updated using the mini-batch.

7. The self-training method for large-scale language models based on the GALA framework according to claim 6, characterized in that: In step S50, the parameter update of the initial model iteration is achieved by minimizing the SFT loss on the sample set consisting of all mini-batches that have passed the quality screening.

8. The self-training method for large-scale language models based on the GALA framework according to claim 6, characterized in that: The heuristic score is derived by evaluating the intrinsic properties of the reasoning path, including whether the length of the reasoning path meets the minimum length requirement for effective reasoning, whether there is a rejection statement, and whether the numerical calculation is stable and free from logical contradictions. Each property is quantified according to a preset weight and then weighted and summed to obtain a heuristic score in the 0-1 range. The rationality score is the log probability assigned to the inference path by the current state of the initial model. The quality score is determined based on a weighted combination of the heuristic score and the rationality score. The weight hyperparameter is used to adjust the proportion of the heuristic score in the quality score, and the preset verification threshold used to select the inference path is not lower than the preset quality verification threshold.

9. The self-training method for large-scale language models based on the GALA framework according to claim 1, characterized in that: When guiding the initial model to conceive a differentiation strategy in step S30, the differentiation strategy is a non-overlapping reasoning paradigm, including case enumeration method, structured set reasoning method, and inclusion-exclusion principle method; the complete step-by-step solution corresponding to each differentiation strategy includes a clear logical starting point, intermediate derivation steps and conclusion verification process.

10. A large-scale language model self-training system based on the GALA framework, characterized in that, The self-training method for large-scale language models based on the GALA framework, as described in any one of claims 1-9, was used.

Images