Large-scale model annealing training methods, apparatus and electronic equipment

Abstract

This invention relates to the field of artificial intelligence technology, providing a large-scale model annealing training method, apparatus, and electronic device. The method includes: extracting semantic features from text data in a target dataset and a candidate corpus based on a pre-configured and parameter-frozen lightweight proxy model to obtain a target feature vector set and a candidate feature vector set; constructing a target distribution fingerprint based on the target feature vector set, and determining sampling weights based on the distribution differences between the target distribution fingerprint and the subset distribution fingerprints after recombination of the candidate feature vector set; and sampling each candidate data subset in the candidate corpus according to the sampling weights to generate an annealing training dataset. This invention achieves accurate deep semantic alignment between training data and the target scene by introducing a lightweight proxy model to construct a unified feature space and objectively quantifying the optimal sampling weights based on distribution differences to generate the annealing dataset. This improves the efficiency of targeted optimization of the model in a specific domain with extremely low computational cost.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a large model annealing training method, apparatus, and electronic device. Background Technology

[0002] In the pre-training process of large models, the annealing stage is crucial and typically occurs in the final stage of training. During this stage, the model is continuously trained using a specific learning rate decay mechanism to achieve refined convergence and solidify its capabilities within a specific task space.

[0003] Currently, existing techniques for constructing annealing training datasets mainly employ manual mixing strategies or rule-based filtering strategies. The former relies on prior human experience to pre-determine the data mixing ratio and maintain it statically constant during training; the latter uses surface-level rules such as metadata tags, regular expressions, or keyword matching to perform coarse-grained filtering of the corpus.

[0004] However, these methods lack objective quantitative metrics and deep implicit semantic features, resulting in a significant discrepancy between the feature distribution of the training data and the target evaluation scenario—a misalignment. This not only leads to inefficient use of computing resources but also severely restricts the performance breakthroughs of models in complex cognitive tasks such as mathematical logic and specialized code. Summary of the Invention

[0005] This invention provides a large model annealing training method, apparatus, and electronic device to address the shortcomings of existing large model annealing training data construction that relies on human experience and shallow features, resulting in misaligned data distribution, difficulty in model convergence, and low computational efficiency.

[0006] This invention provides a large-model annealing training method, comprising: Obtain the target dataset and candidate corpus; Based on a pre-configured and parameter-frozen lightweight proxy model, semantic features are extracted from each text data in the target dataset and the candidate corpus, respectively, to obtain a target feature vector set and a candidate feature vector set mapped in a unified feature space. Based on the target feature vector set, a target distribution fingerprint is constructed, and the sampling weight of each candidate data subset in the candidate corpus is determined according to the distribution difference between the target distribution fingerprint and the subset distribution fingerprint after recombination of the candidate feature vector set. The candidate data subsets are sampled according to the sampling weights to generate an annealing training dataset, and the annealing training dataset is used to anneal the base model.

[0007] According to a large model annealing training method provided by the present invention, the lightweight proxy model based on a pre-configured and parameter-frozen system extracts semantic features from each text data in the target dataset and the candidate corpus, respectively, to obtain a target feature vector set and a candidate feature vector set mapped in a unified feature space, including: The text data in the target dataset and the text data in the candidate corpus are respectively input into the lightweight proxy model. The text data is multi-source heterogeneous text data including multiple languages ​​and / or various minority languages. The multiple languages ​​include at least one of Chinese and English. The various minority languages ​​include at least one of Tibetan, Uyghur and Mongolian. Extract the global mean pooling vector from the hidden layer at the end of the lightweight proxy model, and normalize the global mean pooling vector to obtain a normalized feature vector. Based on the normalized feature vectors of each text data in the target dataset and the normalized feature vectors of each text data in the candidate corpus, the target feature vector set and the candidate feature vector set are constructed respectively.

[0008] According to a large model annealing training method provided by the present invention, the step of inputting text data from the target dataset and text data from the candidate corpus into the lightweight proxy model respectively includes: When the current input text data is long text data, a sliding window strategy is used to segment the long text data into text segments that fit the context window length of the lightweight proxy model, and the text segments are input into the lightweight proxy model.

[0009] According to a large model annealing training method provided by the present invention, the step of constructing a target distribution fingerprint based on the target feature vector set, and determining the sampling weight of each candidate data subset in the candidate corpus based on the distribution difference between the target distribution fingerprint and the subset distribution fingerprint after recombination of the candidate feature vector set, includes: The probability density of each feature vector in the target feature vector set is fitted to generate the target distribution fingerprint; The candidate corpus is pre-divided into multiple pre-clustered clusters, which serve as subsets of each candidate data set; The objective function is to minimize the divergence between the target distribution fingerprint and the subset distribution fingerprint, and a regularization term is introduced as a constraint. The subset distribution fingerprint refers to the fitted distribution fingerprint obtained by weighting and recombining the probability densities of the feature vectors corresponding to each candidate data subset based on weights. The objective function is solved iteratively using the projective gradient descent method, and the weight vector is projected back to the set constraint space after the gradient is updated until the objective function converges, thereby obtaining the sampling weights of each candidate data subset.

[0010] According to a large model annealing training method provided by the present invention, the step of sampling each candidate data subset according to the sampling weight to generate an annealing training dataset includes: The number of samples for each candidate data subset is calculated based on the sampling weights and the preset total number of training samples. Based on the sampled data of any candidate data subset, samples are extracted from the candidate data subset to construct the training data subset corresponding to the candidate data subset; Calculate the similarity between the normalized feature vectors of each sample pair in any training data subset. If the similarity is greater than a preset semantic similarity threshold, then remove redundant samples from the training data subset. The training data subsets after removing redundant samples are cleaned to generate the annealed training dataset. The cleaning process includes at least one of formatting, global random shuffling, and serialization storage.

[0011] The large model annealing training method provided by the present invention further includes: During the annealing training process, training is paused at preset intervals and the output text of the large base model is sampled to obtain a text sequence; The text sequence is input into the lightweight proxy model for semantic feature extraction, and a real-time distributed fingerprint is constructed based on the extracted feature vectors; Calculate the local feature deviation vector between the real-time distributed fingerprint and the target distributed fingerprint, and update the current sampling weight based on the local feature deviation vector and a preset feedback coefficient, so as to perform the next round of sampling based on the updated sampling weight.

[0012] According to the present invention, a large model annealing training method is provided, wherein the annealing training dataset is used to perform annealing training on a large base model, comprising: When loading the base model weights of the base large model, the first and second moment state information of the optimizer in the pre-training stage is obtained and retained to initialize the training environment; Using the annealing training dataset, and employing a learning rate scheduling strategy with a decaying trend, the weighted pedestal model is subjected to gradient-guided training.

[0013] According to a large model annealing training method provided by the present invention, the step of performing gradient-guided training on the base large model after weight loading using a learning rate scheduling strategy with a decaying trend includes: Obtain the preset initial learning rate, minimum learning rate, and total number of annealing steps; During the gradient-guided training process, the real-time learning rate corresponding to the current training step is calculated based on the initial annealing learning rate, the minimum learning rate, and the total number of annealing steps. The parameters of the base model are updated using the real-time learning rate.

[0014] The present invention also provides a large-scale annealing training device, comprising: The data acquisition unit is used to acquire the target dataset and candidate corpus. The feature extraction unit is used to extract semantic features from each text data in the target dataset and the candidate corpus based on a pre-configured and parameter-frozen lightweight proxy model, respectively, to obtain a target feature vector set and a candidate feature vector set mapped in a unified feature space. The weight determination unit is used to construct a target distribution fingerprint based on the target feature vector set, and determine the sampling weight of each candidate data subset in the candidate corpus according to the distribution difference between the target distribution fingerprint and the subset distribution fingerprint after the candidate feature vector set is recombined. The annealing training unit is used to sample each candidate data subset according to the sampling weights to generate an annealing training dataset, and to use the annealing training dataset to perform annealing training on the base large model.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the large model annealing training method as described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the large model annealing training method as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the large model annealing training method as described above.

[0018] The large-scale model annealing training method, apparatus, and electronic device provided by this invention first acquires the target dataset and candidate corpus, and then uses a lightweight proxy model with frozen parameters to extract semantic features from large-scale text data. This process not only deeply mines the implicit semantic features of the text with extremely low computational cost, but also accurately maps data from different sources to a unified feature space, effectively overcoming the information perception deficiency caused by traditional methods that rely on shallow keyword or explicit label filtering. Second, a target distribution fingerprint is constructed based on the target feature vector set within the unified feature space, and the optimal sampling weight is determined by objectively quantifying the distribution difference between this target distribution fingerprint and the candidate subset distribution fingerprint. This eliminates the need for blind and rigid trial-and-error based on human experience to determine the data ratio, thus ensuring the scientific nature of the data ratio and the accuracy of the quantitative evaluation. Finally, a high-information-density annealing training dataset is generated by sampling based on the scientifically calculated sampling weights to train the base large model, fundamentally ensuring a high degree of alignment between the data feature distribution of the input large model and the target evaluation scenario. This training mechanism, driven by deep semantics and objective distribution differences, enables the model to converge quickly and precisely with very few iterations. It significantly reduces invalid tensor operations and memory overhead caused by data bias from the system's bottom layer, while effectively suppressing the occurrence of catastrophic forgetting. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the large model annealing training method provided by the present invention; Figure 2 This is a schematic diagram of the architecture of the large model low-resource annealing training system based on feature alignment of lightweight proxy model provided by the present invention; Figure 3 This is a schematic diagram of the structure of the large-scale annealing training device provided by the present invention; Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] With the rapid development of artificial intelligence technology, large language models (hereinafter referred to as "large models") are increasingly widely used in various vertical fields. In the pre-training process of large models, the annealing stage, as a key stage in the later stage of model training (usually in the last 5% to 10% of the total training steps), plays a crucial role in the final performance of the model. In this process, it is usually necessary to fix the initial parameter state and combine it with a specific learning rate decay mechanism for continuous model training to achieve fine convergence and capability solidification of the model in a specific task space (such as mathematical logic reasoning, professional code generation, etc.).

[0023] To achieve efficient annealing training, constructing a high-quality annealing training dataset is an essential prerequisite. Currently, annealing data construction typically employs two approaches: one is a manual allocation strategy, which relies on prior human experience to pre-determine the data mixing ratio, such as the proportion of general corpora to vertical corpora, and keeps it constant during model training; the other is a rule-based filtering strategy, which uses surface rules such as metadata tags, regular expressions, or keyword matching to filter the corpus, removing noisy text to construct the training dataset.

[0024] However, in practical applications, while the above solutions ensure data relevance to a certain extent, they still fall short of meeting the requirements of large-scale vertical models for data depth alignment and precise convergence. Firstly, the single dimension of data feature extraction leads to an increase in the ineffective computational load on computer systems. Existing rule-based filtering strategies only perform coarse-grained matching based on surface characters (such as keywords and explicit labels), failing to deeply extract and perceive the deep implicit features behind complex text data. This results in filtered data that, while seemingly relevant in terms of labels, actually contains a large number of inefficient or redundant samples. When computer equipment loads these low-quality datasets for model training, it not only fails to effectively improve model capabilities but also triggers a large amount of ineffective tensor operations and memory interactions, significantly wasting underlying computing resources.

[0025] Secondly, the lack of an objective quantitative evaluation mechanism leads to a blind and rigid data matching strategy. Traditional manual static matching is essentially an experience-based trial-and-error method, lacking systematic quantitative metrics. During data processing, the system cannot accurately calculate and evaluate the deviation between the feature distribution of the current training data and the feature distribution of the target application scenario, let alone dynamically and adaptively adjust according to the training status. This blind static matching results in severe misalignment of data distribution, reducing the accuracy of data processing and model optimization.

[0026] Furthermore, the model exhibits low convergence efficiency, resulting in low computational efficiency and system resource utilization. Due to the aforementioned inaccurate feature alignment and the lack of a quantization mechanism, the data distribution input into the large model suffers from severe bias. In actual computer system execution, this data bias forces the processor to perform numerous iterations during the annealing phase to achieve model convergence, significantly extending the model training cycle and drastically increasing computational resource consumption. Moreover, severe data distribution bias can easily trigger catastrophic model amnesia, destroying the original parameter features and potentially requiring a full retraining by reallocating computational resources, severely contradicting the initial goal of low-resource, fast convergence in the annealing phase.

[0027] To address the aforementioned issues, this invention provides a large-scale model annealing training method. By introducing a lightweight proxy model to construct a unified feature mapping space, using a distribution divergence minimization algorithm to automatically calculate the optimal data ratio, and introducing a dynamic feedback mechanism, the method achieves accurate alignment of training data with the deep semantics of the target scene. This avoids expensive full retraining while significantly improving the model's targeted optimization efficiency in specific domains with extremely low computational cost, thereby overcoming the aforementioned shortcomings.

[0028] It should be noted that the large model annealing training method provided by this invention can be executed by an electronic device with powerful data processing and model computation capabilities, such as a standalone server, server cluster, or cloud computing platform. This electronic device is typically equipped with a processor and memory. The processor can be a hardware computing unit such as a Central Processing Unit (CPU), Graphics Processing Unit (GPU), or Natural Processing Unit (NPU) to execute specific computation, feature extraction, and model scheduling instructions. The memory is used to cache massive amounts of corpus data, surrogate model parameters, and base large model parameters. In practical applications, such as developing large vertical domain models focused on complex cognitive tasks like mathematical logic reasoning and professional code generation, in order to guide the model's capabilities to converge accurately towards a specific domain during the pre-training annealing phase (usually occurring in the last 5%-10% of the total training steps), the electronic device can implement the method provided by this invention to achieve targeted optimization under low-resource conditions.

[0029] The technical solution provided by this invention will be described in detail below in conjunction with specific application scenarios. It should be understood that all actions of acquiring signals, information, or data in this invention are carried out in compliance with the relevant data protection laws and policies of the country where the invention is located, and with authorization from the owner of the relevant device.

[0030] Figure 1 This is a flowchart illustrating the large model annealing training method provided by the present invention, as shown below. Figure 1 As shown, the method includes: Step 110: Obtain the target dataset and candidate corpus.

[0031] Specifically, the method provided in this embodiment of the invention is mainly executed collaboratively by the main control processor and related hardware acceleration units of the aforementioned electronic device. Specifically, the processor first acquires the target dataset and the candidate corpus. Here, the target dataset represents the distribution of target capabilities that the base model needs to be targeted and enhanced; it is typically a small number of target evaluation datasets, such as the GSM8K dataset for evaluating mathematical capabilities or the MMLU dataset for evaluating comprehensive capabilities. The candidate corpus, on the other hand, is a massive amount of pre-trained text data. By establishing this dual-path data acquisition channel, the processor loads the target stream data (i.e., the target dataset) representing the characteristics of the target capabilities and the massive candidate stream data (i.e., the candidate corpus) into memory, providing a data foundation for subsequent feature space alignment.

[0032] Step 120: Based on the pre-configured and parameter-frozen lightweight proxy model, semantic features are extracted from each text data in the target dataset and the candidate corpus, respectively, to obtain a target feature vector set and a candidate feature vector set mapped in a unified feature space.

[0033] Specifically, after obtaining the target dataset and candidate corpus, the electronic device calls upon its internal computing hardware such as GPU or NPU to extract semantic features from each text data in the target dataset and candidate corpus based on a pre-configured and parameter-frozen lightweight proxy model, thereby obtaining the target feature vector set and candidate feature vector set mapped in a unified feature space.

[0034] Here, a lightweight surrogate model refers to a deep learning model with a parameter count much smaller than the large base model to be trained (such as the BERT-Large model or the Qwen-0.5B parameter model). Parameter freezing means that the network weights of the surrogate model remain fixed throughout the entire feature extraction process. The processor only controls its forward inference process without incurring the computational cost of backpropagation, thus treating it as a fixed high-dimensional semantic metric.

[0035] During extraction, the processor converts the input text data into a tensor readable by the model and inputs it into the surrogate model to extract the high-dimensional vectors output from its hidden layers. To eliminate the dimensional differences between data from different sources, the processor also performs a spatial projection transformation on the output feature vectors, mapping them all onto the same unit hypersphere to form a unified feature space. Since the data from both the target stream and the candidate stream are projected into this unified geometric space, text data from different sources and in different formats gain a measurable and computable basis, thus forming the target feature vector set and the candidate feature vector set.

[0036] Step 130: Construct a target distribution fingerprint based on the target feature vector set, and determine the sampling weight of each candidate data subset in the candidate corpus according to the distribution difference between the target distribution fingerprint and the subset distribution fingerprint after recombination of the candidate feature vector set.

[0037] Specifically, after extracting the target feature vector set and candidate feature vector sets, the processor constructs a target distribution fingerprint based on the target feature vector set. Then, based on the distribution difference between the target distribution fingerprint and the subset distribution fingerprints after recombination of the candidate feature vector sets, it determines the sampling weight of each candidate data subset in the candidate corpus. Here, the distribution fingerprint is an abstract and quantitative representation of the probability density distribution of data in a unified feature space, which can accurately reflect the semantic category and clustering degree of the text data.

[0038] Specifically, the processor first fits the spatial probability density of the target feature vector set to generate a target distribution fingerprint representing standard capability characteristics. Simultaneously, the candidate corpus can be viewed in the feature space as a collection of multiple different semantic clusters (i.e., candidate data subsets). The processor performs weighted recombination by assigning different weight coefficients to these subsets, obtaining a virtual subset distribution fingerprint. Subsequently, the processor quantifies the difference between the two fingerprints by calculating mathematical divergence, and with the optimization objective of reducing this distribution difference, it uses an algorithm to solve for the weight allocation scheme that best approximates the two, thereby determining the optimal sampling weights for each subset in the candidate corpus.

[0039] Step 140: Sample each candidate data subset according to the sampling weight to generate an annealing training dataset, and use the annealing training dataset to perform annealing training on the base large model.

[0040] Specifically, after obtaining the optimal sampling weights, the processor can extract real samples from a massive subset of candidate data according to the weight ratio, assemble and format the data to form a training dataset specifically for the annealing stage. Subsequently, the electronic device loads this annealed training dataset into the video memory, and under fixed initial parameters, in conjunction with a specific learning rate scheduling mechanism, drives the base model to perform incremental targeted training. This allows the model to achieve deep and refined convergence and capability solidification within the target domain task space while retaining its general capabilities.

[0041] The method provided in this invention first acquires the target dataset and candidate corpus, and then uses a lightweight proxy model with frozen parameters to extract semantic features from large-scale text data. This process not only deeply mines the implicit semantic features of the text with extremely low computational cost, but also accurately maps data from different sources to a unified feature space, effectively overcoming the information perception deficiency caused by traditional methods that rely on shallow keywords or explicit labels for filtering. Secondly, a target distribution fingerprint is constructed based on the target feature vector set within the unified feature space, and the optimal sampling weight is determined by objectively quantifying the distribution difference between this target distribution fingerprint and the candidate subset distribution fingerprint. This eliminates the need for blind and rigid trial-and-error based on human experience to determine data matching, thus ensuring the scientific nature of data matching and the accuracy of quantitative evaluation. Finally, a high-information-density annealed training dataset is generated by sampling based on the scientifically calculated sampling weights to train the base model, fundamentally ensuring a high degree of alignment between the data feature distribution input to the large model and the target evaluation scenario. This training mechanism, driven by deep semantics and objective distribution differences, enables the model to converge quickly and precisely with very few iterations. It significantly reduces invalid tensor operations and memory overhead caused by data bias from the system's bottom layer, while effectively suppressing the occurrence of catastrophic forgetting.

[0042] Based on the above embodiments, step 120 specifically includes: Step 121: Input the text data in the target dataset and the text data in the candidate corpus into the lightweight proxy model respectively. The text data is multi-source heterogeneous text data including multiple languages ​​and / or various minority languages. The multiple languages ​​include at least one of Chinese and English. The various minority languages ​​include at least one of Tibetan, Uyghur and Mongolian. Step 122: Extract the global mean pooling vector of the hidden layer at the end of the lightweight proxy model, and normalize the global mean pooling vector to obtain a normalized feature vector. Step 123: Based on the normalized feature vectors of each text data in the target dataset and the normalized feature vectors of each text data in the candidate corpus, construct the target feature vector set and the candidate feature vector set respectively.

[0043] It should be noted that the embodiments of the present invention mainly focus on the detailed process of how to use a lightweight proxy model to extract semantic features and construct a feature vector set. In the development scenarios of cross-language government service large-scale models or multilingual legal consultation large-scale models, the system needs to process extremely complex mixed corpora. The method of the embodiments of the present invention can be executed by the processor of an electronic device (such as a CPU or GPU) by calling the data and proxy model in the memory.

[0044] Specifically, the processor first inputs the text data from the target dataset and the text data from the candidate corpus into the lightweight proxy model. To ensure the model's broad applicability in multicultural and cross-linguistic scenarios, the processor retrieves multi-source heterogeneous text data from the storage channel, including multiple languages ​​and / or various minority languages. For example, it can include multi-source heterogeneous text data from multiple languages ​​such as Chinese and English, as well as various minority languages ​​such as Tibetan, Uyghur, and Mongolian. The processor converts this heterogeneous text data from different sources and in different formats into tensor form and sequentially feeds it into the lightweight proxy model (e.g., the BERT-Large model or the Qwen-0.5B parameter model) with its parameters frozen for forward computation.

[0045] Subsequently, the processor extracts the global mean pooling vector from the last hidden layer of the lightweight proxy model and normalizes it to obtain a normalized feature vector. Specifically, since the text sequence is processed through multiple neural networks after being input into the proxy model, the processor locates the last layer (i.e., the last hidden layer) of the proxy model, extracts the high-dimensional hidden layer vector corresponding to each character (Token), and calculates the average of all hidden layer vectors within the length of the input text sequence to obtain the global mean pooling vector representing the global semantic information of the entire text.

[0046] The formula for calculating global mean pooling is as follows: In the above formula (1), The original feature vectors are not normalized; The length of the input text sequence (i.e., the total number of tokens). The output of the proxy model The high-dimensional hidden layer vector corresponding to each token.

[0047] To eliminate potential dimensional differences between data sources of different languages ​​and texts of varying lengths, and to ensure the mathematical validity of subsequent distance or divergence calculations, the processor further performs an L2 norm normalized projection operation on the original pooled vector (i.e., dividing the original vector by its L2 norm magnitude). This ensures that all feature vectors are strictly constrained and mapped to a unified unit hypersphere, generating the final normalized feature vector. The normalized projection formula is as follows: In the above formula (2), This is the normalized eigenvector that is ultimately mapped onto the unit hypersphere; For the original vector Norm (modulus).

[0048] Finally, the processor combines all target flow feature vectors after the above-mentioned standardized spatial projection processing to form a target feature vector set, and combines all candidate flow feature vectors to form a candidate feature vector set, and stores them in a cache, providing a standardized underlying data structure for subsequent distribution divergence calculation and weight allocation.

[0049] The method provided in this invention constructs a unified high-dimensional semantic metric space that is not limited by language or dimension by defining a global mean pooling and normalized projection feature extraction paradigm. This not only achieves compatibility and deep implicit semantic extraction of multi-source heterogeneous data including Chinese, English and various minority languages, but also eliminates spatial bias when comparing cross-language data, laying a solid quantitative computational foundation for subsequent accurate evaluation of the distribution differences between training data and target scenes.

[0050] Based on any of the above embodiments, step 121 specifically includes: When the current input text data is long text data, a sliding window strategy is used to segment the long text data into text segments that fit the context window length of the lightweight proxy model, and the text segments are input into the lightweight proxy model.

[0051] It should be noted that in practical professional fields (such as understanding long medical literature) large model annealing training scenarios, the candidate corpus often contains a large number of long documents. In this regard, the embodiments of the present invention use the processor of the electronic device to perform the following processing steps to support long text data.

[0052] Specifically, when the current input text data is long text data, the processor can use a sliding window strategy to segment the long text data into text fragments that fit the context window length of the lightweight proxy model, and then input the text fragments into the lightweight proxy model. Here, long text data usually refers to text that contains a total number of tokens that exceeds the maximum processing capacity of the currently selected lightweight proxy model (i.e., the context window limit, such as 512 or 1024 tokens).

[0053] To avoid losing crucial semantics due to direct truncation, the processor executes a sliding window strategy in memory. This involves setting a fixed-length data reading window (e.g., equal to the maximum window length of the proxy model) and sliding it across the long text sequence at preset step sizes. To ensure the coherence and integrity of the context, a certain percentage of overlap is typically maintained between adjacent sliding windows. The processor treats the content extracted from each window as an independent text segment and sequentially inputs it into the lightweight proxy model for subsequent high-dimensional hidden vector extraction.

[0054] The method provided in this invention introduces a sliding window segmentation mechanism for long text data, which effectively breaks through the bottleneck of context window length caused by the underlying architecture of the lightweight proxy model. While avoiding semantic gaps or loss of contextual information caused by direct truncation of long texts, it ensures the semantic integrity of local text fragments in the process of mapping in a unified feature space, thereby improving the model's ability to perceive and learn the complex logical depth of long texts.

[0055] Based on any of the above embodiments, step 130 specifically includes: Step 131: Fit the probability density of each feature vector in the target feature vector set to generate the target distribution fingerprint; Step 132: The candidate corpus is pre-divided into multiple pre-clustered clusters as subsets of each candidate data set; Step 133: Minimize the divergence between the target distribution fingerprint and the subset distribution fingerprint as the optimization objective, and introduce a regularization term as a constraint to generate an objective function. The subset distribution fingerprint refers to the fitted distribution fingerprint obtained by weighting and recombining the probability densities of the feature vectors corresponding to each candidate data subset based on weights. Step 134: The objective function is solved iteratively using the projected gradient descent method, and the weight vector is projected back to the set constraint space after the gradient is updated, until the objective function converges, and the sampling weights of each candidate data subset are obtained.

[0056] It should be noted that in the annealing training scenario of large mathematical logic reasoning models or large professional code generation models, in order to maximize the fit of the data extracted from the massive generalized corpus to the knowledge distribution of the target professional domain, the embodiments of the present invention convert the annealing data allocation into a constraint optimization problem of minimizing divergence, and solve the optimal sampling weight under the premise of introducing regularization constraints, thereby maximizing the fit to the target domain capability distribution under the condition of extremely low training data volume.

[0057] Specifically, based on the target feature vector set obtained from the output The processor first fits the probability density of each feature vector in the set to generate the target distribution fingerprint. Here, This represents the L2-normalized feature vector generated for each text data point in the target dataset. The target distribution fingerprint refers to the probability density function that accurately characterizes the semantic properties of the target domain.

[0058] Since the feature vectors have been L2 normalized and projected onto the unit hypersphere, their spatial distribution characteristics can directly reflect the semantic category and degree of clustering. Therefore, the processor can use a Gaussian Mixture Model (GMM) to process these feature vectors mapped onto the hypersphere. The probability density is mathematically fitted to obtain a continuous and smooth target distribution fingerprint. Here, the target distribution probability density function is as follows: In the above formula (3), The normalized high-dimensional semantic feature vector serves as the input variable for the probability density function; It represents the distribution fingerprint of the target dataset as a whole at a macroscopic level, and the feature vector at the specific numerical calculation level. The probability density under this fingerprint model; The number of sub-clusters for the target domain can be preset based on the size and semantic complexity of the target corpus; For vectors From the The prior probabilities of the Gaussian components satisfy the following condition: ; For the first There are Gaussian distribution components, among which Let be the mean vector of this component. Let be the covariance matrix. Through the rigorous mathematical fitting described above, the abstract target domain capability distribution can be transformed into a computable probability density fingerprint model, thus providing a quantitative benchmark for subsequent divergence comparison with candidate data.

[0059] At the same time, the processor pre-divides the candidate corpus into multiple (e.g.) (Number) pre-clustered clusters are used as subsets of each candidate data. During the massive data loading stage, the processor has already clustered the entire candidate corpus into multiple clusters (i.e., candidate data subsets) with known internal feature distributions based on the underlying feature distribution of the text, in order to facilitate subsequent hierarchical processing.

[0060] Subsequently, the processor optimizes by minimizing the divergence between the target distribution fingerprint and the subset distribution fingerprint, and introduces a regularization term as a constraint to generate the objective function. Here, the subset distribution fingerprint refers to the fitted distribution fingerprint obtained by weighted recombination of the probability densities of the feature vectors corresponding to each candidate data subset.

[0061] Specifically, the processor abstracts the data matching problem into a probability distribution approximation optimization problem under constraints. The system defines that the entire candidate corpus has been pre-clustered into... There are several clusters, and the internal feature distribution of each cluster is known. The goal of this invention is to solve for an optimal sampling weight vector. The objective function is as follows: This minimizes the divergence between the recombined subset fingerprint and the target fingerprint. In the above formula (4), For the adaptive sampling weight vector, in At time t, it is the initial optimal solution calculated based on the static distribution; at Moment is a dynamic variable that incorporates feedback signals; This represents the objective function constructed, which is used to quantitatively evaluate the current sampling weight allocation scheme (i.e., the weight vector). Under this condition, the degree of difference between the generated subset distribution fingerprint and the target distribution fingerprint is determined. This indicates the method for obtaining the objective function. The minimum value is the optimization objective, and the adaptive sampling weight vector that minimizes the objective function is used to solve for it. .when The minimum value is obtained when the subset distribution fingerprint obtained through weighted recombination most closely approximates the target distribution fingerprint; the solution obtained at this point is the target fingerprint. Optimal sampling weights.

[0062] The first in the full candidate stream The characteristic probability density function of each pre-clustered cluster; The generated target domain semantic distribution fingerprint (i.e., target distribution fingerprint); Current weight The subset distribution fingerprint after hybrid recombination (i.e., the fitted distribution fingerprint); This is the Kullback-Leibler divergence (KL divergence), used to quantify the difference between the two probability distributions: the target distribution fingerprint and the subset distribution fingerprint. for Regularization term, where The adjustment coefficient (empirical value is 0.01) is used to prevent the weights from being overly concentrated in a single pre-cluster, thus ensuring the diversity of the sampled data.

[0063] In the above formula (5), "Subject to" is an abbreviation for "subject to" (subjected to or constrained by conditions), which guides the optimization variables (i.e., weight vectors) in formula (4). The elements in The constraints are defined by the weight vector projection constraint space.

[0064] Finally, for the aforementioned constrained optimization problem, the processor, in conjunction with the dynamic policy adjustment signal from subsequent embodiments, uses the projected gradient descent method to iteratively solve the objective function. Specifically, after each gradient update step, the weight vector... Projection Back This defined constraint space extends to the objective function. The process converges, and the optimal mixing ratio matrix is ​​finally output as the sampling weight for each candidate data subset.

[0065] It is understandable that the aforementioned constraint space refers to all The sum of the sampled weights of each pre-cluster (i.e., a subset of candidate data) must be equal to 1, and the sampled weights of each pre-cluster must be greater than or equal to 0 in a mathematical space. After each gradient descent iteration updates the weights, the processor will forcibly project the out-of-bounds weight vectors back into the legal constraint space. This process is repeated until the value of the objective function no longer decreases significantly (i.e., convergence is achieved). At this point, the output weight vector is the optimal mixing ratio matrix (i.e., sampled weights) at that moment.

[0066] The method provided in this invention transforms the annealing data matching problem into a probability distribution divergence approximation optimization problem with regularization constraints, and uses the projection gradient descent method to solve for the optimal mixing weights within a strict constraint space. This method abandons the traditional static matching strategy that relies on blind trial and error based on human experience. While ensuring the diversity of sampled data, it achieves accurate approximation and alignment of the candidate data subset distribution to the target scene capability distribution.

[0067] Based on any of the above embodiments, step 140, which involves sampling each candidate data subset according to the sampling weight to generate an annealing training dataset, includes: Step 141: Calculate the number of samples for each candidate data subset based on the sampling weights and the preset total number of training samples; Step 142: Based on the sampling data of any candidate data subset, extract samples from the candidate data subset to construct the training data subset corresponding to the candidate data subset; Step 143: Calculate the similarity between the normalized feature vectors of each sample pair in any training data subset. If the similarity is greater than the preset semantic similarity threshold, then remove redundant samples from the training data subset. Step 144: Clean the subsets of training data after removing redundant samples to generate the annealing training dataset. The cleaning process includes at least one of formatting, global random shuffling, and serialization storage.

[0068] It should be noted that the embodiments of this invention mainly introduce how to construct and clean the annealing training dataset based on the calculated sampling weights. In application scenarios that require high information density and high coverage data, such as large professional code generation models, the embodiments of this invention adopt a hierarchical assembly method based on semantic density. While maintaining distribution alignment, the sampled data is semantically deduplicated and cleaned, and packaged into a dedicated annealing dataset in a unified format, thereby ensuring that small-scale data has high information density and high coverage.

[0069] Specifically, the processor first uses the solved optimal sampling weight vector Stratified sampling is performed on the entire candidate corpus. The specific sampling formula is as follows: In the above formula (6), The first in the full candidate corpus The first pre-cluster (i.e., the...) The number of physical samples (of candidate data subsets); The target total number of rows in the preset annealing training dataset (also known as the total number of training samples, for example, 100,000 rows). For the current iteration period, the th The adaptive sampling weights for each pre-clustered cluster are determined by the optimization objective function. Calculations show that the following conditions are met. .

[0070] Next, the processor extracts samples from any candidate data subset based on the sampled data of that subset, constructing the corresponding training data subset. Specifically, it iterates through the entire candidate corpus... A pre-clustered group, calculated based on the current time. Value, from the first A corresponding number of samples are randomly selected from each pre-clustered cluster, and the extracted text fragments are loaded into memory to form an initial subset of training data. This process ensures the distribution of physically sampled data. It always closely follows the optimal path determined by mathematical optimization and dynamic feedback.

[0071] Subsequently, for each subset of training data, the processor calculates the cosine similarity between the normalized feature vectors of each sample pair in that subset. If the similarity is greater than a preset semantic similarity threshold, redundant samples are removed from that subset. It should be understood that since the initial sampling may introduce redundant data with repeated or highly similar content, this embodiment of the invention employs semantic comparison based on feature vectors for refined cleaning to improve the information density per unit amount of data.

[0072] The formula for cosine similarity is as follows: In the above formula (7), The normalized feature vectors corresponding to two samples A and B in the training data subset (generated by the lightweight proxy model; since they have been normalized, their dot product is equal to the cosine similarity). The semantic similarity score has a range of values. .

[0073] The processor sets a strict semantic similarity threshold. (For example, 0.95), when the similarity between two samples exceeds this threshold (i.e. When a data item is found to be semantically redundant, the processor will use a strategy of retaining long text or randomly retaining redundant items to ensure that each piece of data can provide an independent feature contribution to the model annealing.

[0074] Finally, the processor cleans each subset of training data after removing redundant samples to generate the annealing training dataset. This cleaning process includes: ① Formatting: Tokenizing the text using a tokenizer and truncating or concatenating it according to the maximum context length of the base model; ② Global shuffling: Performing global random shuffling on all samples to eliminate order bias in the data sources and prevent local overfitting during model training; ③ Serialization and storage: Serializing the processed data into a binary format (such as Bin format) to generate the final annealing-specific dataset for direct loading by subsequent modules.

[0075] The method provided in this invention, while maintaining macroscopic alignment of data distribution, introduces a semantic-level deduplication mechanism based on the cosine similarity of high-dimensional feature vectors and a standardized cleaning and encapsulation process. This effectively filters highly homogeneous redundant noise, improves information density per unit amount of data, and ensures that the dedicated dataset input to the final large model annealing stage is not only lightweight but also has extremely high data quality and logical depth.

[0076] Based on any of the above embodiments, step 140, which involves annealing the large base model using the annealing training dataset, includes: Step 145: When loading the base model weights of the base large model, obtain and retain the first and second moment state information of the optimizer in the pre-training stage to initialize the training environment. Step 146: Using the annealing training dataset, and employing a learning rate scheduling strategy with a decaying trend, perform gradient-guided training on the weighted pedestal model.

[0077] Specifically, in application scenarios such as building large models for mathematical logic reasoning and professional code generation, in order to smoothly transition and make full use of the existing general knowledge base of the model, the processor can obtain and retain the first and second moment state information of the optimizer in the pre-training stage to initialize the training environment when loading the weights of the base model of the base large model.

[0078] In the traditional model fine-tuning phase, only the network parameters of the model are loaded and the optimizer state is reset, which often leads to severe oscillations in the early stages of training. In order to maximize the inheritance of gradient inertia in the pre-training phase, the processor, while loading the weights of the base model into GPU memory, strictly preserves the first moment (momentum, reflecting the historical direction of gradient movement) and second moment (variance, reflecting the historical amplitude of gradient fluctuations) state information accumulated by the optimizer (such as AdamW) at the end of pre-training. This information is used as non-reset initialization parameters to build a more stable annealing training environment.

[0079] Subsequently, the processor utilizes the annealed training dataset and employs a decaying learning rate scheduling strategy to perform gradient-guided training on the weighted pedestal model. After smooth initialization, the processor batches the previously highly cleaned and precisely proportioned annealed dataset into the pedestal model, performing high-intensity backpropagation. During this process, the processor abandons a constant learning rate and instead invokes a decaying scheduling strategy that decreases with each training step, guiding the model to perform small-step, high-precision gradient updates and parameter adjustments in the defined target domain direction.

[0080] The method provided in this invention adopts a non-reset loading strategy to initialize the annealing training environment. By fully inheriting the gradient history information of the optimizer in the pre-training stage, it smooths the transition from massive general pre-training to targeted annealing training. This not only effectively avoids training oscillations caused by optimizer re-initialization and sudden changes in data distribution, but also fundamentally suppresses the significant decline in the general capability of the model (i.e., catastrophic forgetting), laying a stable foundation for subsequent targeted optimization.

[0081] Based on any of the above embodiments, step 146 specifically includes: Obtain the preset initial learning rate, minimum learning rate, and total number of annealing steps; During the gradient-guided training process, the real-time learning rate corresponding to the current training step is calculated based on the initial annealing learning rate, the minimum learning rate, and the total number of annealing steps. The parameters of the base model are updated using the real-time learning rate.

[0082] Specifically, in low-resource, fast convergence scenarios for large models in complex industrial-grade cognitive tasks (such as multimodal mathematical logic deduction), the processor first obtains the preset initial learning rate, minimum learning rate, and total number of annealing steps. These parameters constitute the computational benchmark for the scheduling strategy. The learning rate scheduling formula using the cosine decay mechanism is as follows: In the above formula (8), Current training steps The real-time learning rate; The initial learning rate for the annealing phase is usually set to a value much lower than that in the early stages of pre-training (e.g., 10% of the peak value during pre-training, such as 1e-5) to ensure the smoothness of fine-tuning. The minimum learning rate, which usually approaches 0, represents the minimum update step size at the end of training; This represents the total number of annealing steps, typically only accounting for the last 1%-5% of the total pre-training steps. The processor reads these hyperparameters and caches them in registers.

[0083] Next, after each gradient calculation, the processor reads the current training step and substitutes it into the preset cosine function mathematical model (i.e., the above formula (8)). Since the cosine curve has the characteristic of smooth descent, the processor can calculate a real-time learning rate that smoothly transitions from the initial value to the minimum value, so that the update step size of the model is relatively large in the early stage of annealing to quickly adapt to the new distribution, while the step size is extremely small in the late stage of annealing to achieve fine optimization.

[0084] Finally, after calculating the real-time learning rate for the current training step, the processor sends it to the underlying optimizer engine. Combining the previously retained first and second moment states, it calculates the specific adjustment amount of the weights of each node in the network layer and writes it into the GPU memory, thus completing the targeted update of the base model parameters.

[0085] The method provided in this invention introduces a rapidly decaying cosine learning rate scheduling strategy into annealing training with extremely short cycles. By dynamically and smoothly compressing the model parameter update step size, the model stops exploring a wide range of features within limited computing resources and a very small number of training steps. This allows its existing vast general knowledge to rapidly crystallize in a specific target domain task space, achieving refined convergence and solidification of the model's vertical professional capabilities with extremely low data and computing costs.

[0086] Based on any of the above embodiments, during the annealing training process, the processor constructs a real-time distributed fingerprint based on the text sequence generated by the base large model, and dynamically updates the sampling weights according to the feature deviation between the real-time distributed fingerprint and the target distributed fingerprint.

[0087] It should be noted that in continuous training scenarios such as high-precision medical question-answering models or complex mathematical logic reasoning models, the ability distribution of the model in the annealing stage will dynamically drift as the number of training steps increases. In order to correct this deviation, this embodiment of the invention introduces a dynamic closed-loop feedback training architecture based on online distribution monitoring.

[0088] Specifically, unlike traditional one-way static input training, this embodiment of the invention introduces a dynamic update mechanism. While the base model performs gradient updates and parameter optimization using the data from the current batch, the processor uses the text sequence generated by the base model's real-time inference to again call the aforementioned lightweight proxy model with frozen parameters for mapping, thereby characterizing a real-time distribution fingerprint that reflects the model's current true convergence state. Subsequently, the processor compares this real-time state with the initially set target distribution fingerprint, quantifies the feature deviation between the two in different semantic dimensions, i.e., the gap between the model's current capability and its expected capability, and generates dynamic optimization instructions based on this deviation signal to adaptively adjust the old sampling weights used in the previous round.

[0089] Next, the processor resamples each candidate data subset according to the updated sampling weights to generate a new annealing training dataset. Specifically, after obtaining the new sampling weight matrix, the processor issues a new data retrieval instruction to the memory, extracts samples from each pre-cluster of the full candidate corpus according to the corrected proportions, assembles them into the next stage of the annealing training dataset, and uses this dataset to continue annealing training on the base model.

[0090] The method provided in this invention constructs a dynamic closed-loop feedback training architecture based on online feature distribution monitoring. By quantifying the deviation between the capability distribution fingerprint generated in real time during model training and the target domain fingerprint, and directly mapping the deviation to the update signal of the sampling weights, it breaks the rigid mode of unchanging data ratio in traditional annealing training. It realizes real-time adaptive correction of the sampling strategy as the model's cognitive state evolves, thereby avoiding catastrophic forgetting caused by static data bias and ensuring the accurate and stable convergence of the model to the target domain capabilities.

[0091] Based on any of the above embodiments, the method further includes: Step 150: During the annealing training process, training is paused at preset intervals and the output text of the large base model is sampled to obtain a text sequence. Step 160: Input the text sequence into the lightweight proxy model for semantic feature extraction, and construct a real-time distributed fingerprint based on the extracted feature vector; Step 170: Calculate the local feature deviation vector between the real-time distributed fingerprint and the target distributed fingerprint, and update the current sampling weight based on the local feature deviation vector and the preset feedback coefficient, so as to perform the next round of sampling based on the updated sampling weight.

[0092] It should be noted that the embodiments of the present invention mainly focus on further refining the step of how to specifically quantify feature deviations and complete dynamic weight updates.

[0093] Specifically, in the annealing optimization scenario of industrial-grade automatic code generation large model, in order to achieve millisecond-level real-time effect monitoring and to balance training efficiency and monitoring accuracy, the processor will set a high-frequency evaluation interval (e.g., every 50 training steps). When the preset interval is reached, the system will automatically issue an interrupt command to briefly pause the backpropagation training of the base large model and trigger its forward inference module to collect the text output by it for a specific prompt word, forming the text sequence of the current stage.

[0094] The generated text sequence is input into a lightweight proxy model for semantic feature extraction, and a real-time distribution fingerprint representing the current model's capabilities is constructed based on the extracted feature vectors. Specifically, the processor feeds the newly sampled text sequence into the aforementioned parameter-frozen lightweight proxy model, extracts its hidden layer vectors, performs high-dimensional projection, and then fits a real-time distribution fingerprint representing the true capability distribution of the current model. .

[0095] Subsequently, the processor calculates the local feature deviation vector between the real-time distributed fingerprint and the target distributed fingerprint. The formula for calculating the local feature deviation vector is as follows: In the above formula (9), For the first The deviation value of each cluster; Indicates the current real-time distributed fingerprint In the middle, belonging to the first Real-time percentage of class-specific capabilities; Target distribution fingerprint for the target evaluation set In the middle, the first The standard proportion of class-specific ability characteristics.

[0096] Understandable This can be achieved by calculating the target evaluation dataset (such as MMLU) that falls into the first position. The number of samples in each cluster is obtained by dividing the total number of samples in the target dataset. For example, if 30% of the MMLU question bank consists of logic reasoning questions, then the cluster corresponding to the logic reasoning question bank would be... , .

[0097] for In the current training step, model training is paused, and the sampled text is input into the proxy model to extract features and determine which feature cluster they belong to. This is done by calculating the cluster to which the feature cluster belongs. The percentage of samples belonging to a particular cluster relative to the total number of samples currently being sampled is used to determine the number of samples. For example, if 20 out of the 100 results generated by the model are logic questions, then... .

[0098] Because the proportion of generated logic reasoning questions (0.2) is lower than the target requirement (0.3), a deviation has occurred. The system updates the current sampling weights based on this local feature deviation vector and a preset feedback coefficient, obtaining the sampling weights for the next round of sampling. The dynamic weight update formula is as follows: In the above formula (10), For the next round of sampling, the first New weights for class data; The old weights currently in use; The calculated deviation value; This is the feedback coefficient (e.g., 0.5).

[0099] The method provided in this invention achieves a leap from offline lag evaluation to online real-time monitoring by establishing a millisecond-level online proxy evaluation mechanism based on high-frequency preset intervals. Furthermore, by accurately calculating the local feature deviation vector and introducing feedback coefficients for formulaic weight adjustment, a directional correction signal with rigorous mathematical logic is provided for data reorganization. This ensures that each gradient descent during the annealing process of the large model evolves along the optimal path that maximizes the compensation of capability shortcomings, thereby improving computational efficiency and the model's final performance on complex cognitive tasks.

[0100] Based on any of the above embodiments Figure 2 This is a schematic diagram of the architecture of the large model low-resource annealing training system based on feature alignment of a lightweight proxy model provided by the present invention, as shown below. Figure 2 As shown, the system can be used to implement the large model annealing training method provided in any of the above embodiments. Its overall architecture is divided into four core logic layers, which are tightly coupled through data flow and control flow. It also includes a bottom-up dynamic feedback link, realizing a fully automated closed loop from feature perception to model convergence.

[0101] Specifically, the system utilizes a lightweight proxy model with frozen parameters to map the implicit capability distribution of the target evaluation dataset into the feature space of a massive candidate corpus. Furthermore, it abstracts the data matching into a constrained optimization problem of minimizing divergence, solving for the optimal sampling weights. Finally, during the annealing phase, it performs incremental directional training on the base model and generates dynamic feedback signals through online proxy evaluation to drive continuous correction of the sampling strategy. Each module will be described in detail below.

[0102] First layer: Multi-source feature perception and heterogeneous mapping module.

[0103] This module is primarily responsible for accessing the underlying data and mapping features in a unified spatial space. As the perception front-end of the entire system, it employs a lightweight proxy model with frozen parameters as a unified measurement tool. This projects data sources from different sources and in different formats onto the same high-dimensional semantic feature space, thereby achieving comparability, measurability, and computation across data sources. The specific process is as follows: S11, dual-source data acquisition.

[0104] The system establishes two independent data channels for accessing and processing multi-source heterogeneous text data covering Chinese, English, multiple languages, and various minority languages ​​(such as Tibetan, Uyghur, and Mongolian). These two data channels are used to access a small target evaluation dataset (target stream) representing the model's desired enhancement capabilities, and a massive full-scale pre-training corpus (candidate stream), respectively. Furthermore, for long text data, the system automatically employs a sliding window strategy for segmentation, dividing the text into fragments that fit the proxy model's context window, ensuring the integrity of the context.

[0105] S12, Lightweight proxy model loading.

[0106] A lightweight model with a significantly smaller number of parameters than the base model (such as BERT-Large or Qwen-0.5B parameter models) is selected as the feature extractor. Throughout the process, the weights of this surrogate model remain frozen, serving only as a fixed metric for forward inference without consuming backpropagation computational power.

[0107] S13, High-dimensional semantic feature vectorization and spatial projection.

[0108] To construct a unified metric space, this invention defines a strict feature extraction paradigm. Target and candidate stream text data are input into a proxy model, and the global mean pooling vector of the model's final hidden layer is extracted. L2 norm normalized projection processing is then enforced to project all multi-source heterogeneous texts indiscriminately into a unified unit hyperspherical feature space, outputting a target feature vector set and a candidate feature vector set, thereby eliminating potential dimensional differences between different data sources.

[0109] Second layer: Distribution alignment and optimal sampling calculation module.

[0110] This module is responsible for quantifying feature differences and calculating the optimal data allocation. It transforms the annealed data allocation into a constrained optimization problem of minimizing divergence, and solves for the optimal sampling weight vector under the premise of introducing regularization constraints. This maximizes the fit to the target domain's capability distribution under conditions of extremely low training data volume. The specific process is as follows: S21, Target domain distributed fingerprint generation.

[0111] Based on the target feature vector set output from the first layer, algorithms such as Gaussian Mixture Model (GMM) are used to fit the probability density of the feature vectors to generate a target distribution fingerprint that can accurately characterize the semantic properties of the target domain.

[0112] S22, divergence minimization solver.

[0113] The candidate corpus is pre-divided into multiple pre-clusters, and subset distribution fingerprints are generated by weighted recombination based on the probability density of feature vectors. The solver aims to minimize the KL divergence between the target distribution fingerprint and the subset distribution fingerprints. Combined with regularization constraints, it uses the projective gradient descent method for iterative solving, ultimately outputting the current optimal sampling weights for each pre-cluster. Furthermore, the system reserves a receiving port to receive dynamic feedback signals from the fourth layer, thereby triggering secondary calculations and updates of the weights.

[0114] The third layer: Data genealogy construction and assembly module.

[0115] This module is responsible for generating the final training files based on the calculated weights. It employs a semantic density-based hierarchical assembly method to perform semantic-level deduplication and cleaning of the sampled data while maintaining distribution alignment. The data is then packaged into a unified format for annealing, ensuring high information density and coverage even with small-scale datasets. The specific process is as follows: S31, weighted full sampling.

[0116] Based on the sampling weights output by the second-layer solver and the preset total number of target annealing training datasets, the entire pre-clustered clusters of the candidate corpus are traversed, and physical-level hierarchical random sampling is performed to construct the initial training data subset.

[0117] S32, semantic-level deduplication and cleaning.

[0118] Since the initial sampling may introduce redundant data with duplicate or highly similar content, this step uses semantic comparison based on feature vectors for fine-grained cleaning to improve the information density per unit of data. Specifically, this step calculates the cosine similarity of the sampled data pairs. Once the similarity exceeds a set semantic similarity threshold (e.g., 0.95), deduplication logic is triggered to accurately remove highly homogeneous redundant samples.

[0119] S33, Assembly of annealing-specific dataset.

[0120] The cleaned high-quality data fragments are formatted by token segmentation, truncation or concatenation, etc., and global random shuffling is performed to break up the data distribution order. Finally, the data is serialized into binary format and assembled to generate an annealing-specific dataset that is adapted to the training framework of the large-scale model.

[0121] Fourth layer: Incremental annealing training and closed-loop feedback module.

[0122] This module serves as the system's execution and monitoring terminal. It is responsible not only for enhancing the orientation capabilities of large models but also for real-time evaluation and transmitting correction signals upwards. Specifically, during annealing training, this module generates capability distribution fingerprints in real time, quantifies the deviation between the fingerprint and the target distribution, maps the deviation to dynamic sampling weight update signals, and feeds them back to the second layer to achieve real-time correction of the sampling strategy. The specific process is as follows: S41, Base model weight loading.

[0123] A non-reset loading strategy is adopted to load the parameters of the large model of the base into the video memory, while fully acquiring and inheriting the first and second moment state information of the optimizer in the pre-training stage, thereby initializing a smooth and oscillating training environment.

[0124] S42, Gradient-guided annealing training.

[0125] By loading the annealing-specific dataset generated in the third layer and combining it with a rapidly decaying cosine learning rate scheduling strategy, the base model is driven to perform high-intensity directional gradient descent and parameter updates, enabling the model's existing general capabilities to be crystallized in the target domain.

[0126] S43, Online Agent Evaluation.

[0127] During the annealing training process, training is paused at preset high-frequency intervals (e.g., every 50 steps), the current output text of the base model is sampled, the lightweight proxy model with frozen parameters in the first layer is reused, features are quickly extracted and a real-time distributed fingerprint representing the current model's capabilities is constructed, and the local feature deviation vector between the real-time distributed fingerprint and the target distributed fingerprint in the second layer is calculated.

[0128] S44, Dynamic Strategy Adjustment Signal.

[0129] This step is responsible for converting the monitored deviations into specific optimization instructions and transmitting them back. Specifically, based on the calculated deviations and a preset feedback coefficient, a quantified weight update instruction (i.e., a dynamic strategy adjustment signal) is generated. This signal will serve as a dynamic feedback signal. Figure 2 The feedback link shown by the dashed line is directly fed back to the divergence minimization solver in the second layer, driving it to adaptively adjust the sampling weights for the next round.

[0130] The system provided in this invention introduces a lightweight proxy model with parameter freezing as a unified metric tool, projecting data from different sources onto the same high-dimensional semantic feature space. This overcomes the shortcomings of traditional methods that rely on expert experience or explicit labels for coarse-grained data matching. By transforming annealed data matching into a quantitative solution process for the differences in distributed fingerprint divergence, it achieves accurate alignment between candidate training data and the deep implicit semantics of the target scene. Thus, while avoiding expensive full retraining, it improves the efficiency of targeted optimization and cognitive convergence of large models in specific vertical domains with extremely low computing power and high-quality, small-quantity data.

[0131] The large-scale annealing training apparatus provided by the present invention is described below. The large-scale annealing training apparatus described below can be referred to in correspondence with the large-scale annealing training method described above.

[0132] Based on any of the above embodiments Figure 3 This is a schematic diagram of the structure of the large-scale annealing training device provided by the present invention, as shown below. Figure 3 As shown, the device includes: Data acquisition unit 310 is used to acquire the target dataset and candidate corpus; The feature extraction unit 320 is used to extract semantic features from each text data in the target dataset and the candidate corpus based on a pre-configured and parameter-frozen lightweight proxy model, to obtain a target feature vector set and a candidate feature vector set mapped in a unified feature space. The weight determination unit 330 is used to construct a target distribution fingerprint based on the target feature vector set, and determine the sampling weight of each candidate data subset in the candidate corpus according to the distribution difference between the target distribution fingerprint and the subset distribution fingerprint after the candidate feature vector set is recombined. Annealing training unit 340 is used to sample each candidate data subset according to the sampling weight, generate an annealing training dataset, and use the annealing training dataset to perform annealing training on the base large model.

[0133] The apparatus provided in this invention first acquires a target dataset and a candidate corpus, and then uses a lightweight proxy model with frozen parameters to extract semantic features from large-scale text data. This process not only deeply mines the implicit semantic features of the text with extremely low computational cost, but also accurately maps data from different sources to a unified feature space, effectively overcoming the information perception deficiency caused by traditional methods that rely on shallow keywords or explicit labels for filtering. Secondly, a target distribution fingerprint is constructed based on the target feature vector set within the unified feature space, and the optimal sampling weight is determined by objectively quantifying the distribution difference between this target distribution fingerprint and the candidate subset distribution fingerprint. This eliminates the need for blind and rigid trial-and-error based on human experience to determine data matching, thus ensuring the scientific nature of data matching and the accuracy of quantitative evaluation. Finally, a high-information-density annealed training dataset is generated by sampling based on the scientifically calculated sampling weights to train the base model, fundamentally ensuring a high degree of alignment between the data feature distribution input to the large model and the target evaluation scenario. This training mechanism, driven by deep semantics and objective distribution differences, enables the model to converge quickly and precisely with very few iterations. It significantly reduces invalid tensor operations and memory overhead caused by data bias from the system's bottom layer, while effectively suppressing the occurrence of catastrophic forgetting.

[0134] Based on any of the above embodiments, the feature extraction unit is specifically used for: The text data in the target dataset and the text data in the candidate corpus are respectively input into the lightweight proxy model. The text data is multi-source heterogeneous text data including multiple languages ​​and / or various minority languages. The multiple languages ​​include at least one of Chinese and English. The various minority languages ​​include at least one of Tibetan, Uyghur and Mongolian. Extract the global mean pooling vector from the hidden layer at the end of the lightweight proxy model, and normalize the global mean pooling vector to obtain a normalized feature vector. Based on the normalized feature vectors of each text data in the target dataset and the normalized feature vectors of each text data in the candidate corpus, the target feature vector set and the candidate feature vector set are constructed respectively.

[0135] Based on any of the above embodiments, the feature extraction unit is further configured to: When the current input text data is long text data, a sliding window strategy is used to segment the long text data into text segments that fit the context window length of the lightweight proxy model, and the text segments are input into the lightweight proxy model.

[0136] Based on any of the above embodiments, the weight determination unit is specifically used for: The probability density of each feature vector in the target feature vector set is fitted to generate the target distribution fingerprint; The candidate corpus is pre-divided into multiple pre-clustered clusters, which serve as subsets of each candidate data set; The objective function is to minimize the divergence between the target distribution fingerprint and the subset distribution fingerprint, and a regularization term is introduced as a constraint. The subset distribution fingerprint refers to the fitted distribution fingerprint obtained by weighting and recombining the probability densities of the feature vectors corresponding to each candidate data subset based on weights. The objective function is solved iteratively using the projective gradient descent method, and the weight vector is projected back to the set constraint space after the gradient is updated until the objective function converges, thereby obtaining the sampling weights of each candidate data subset.

[0137] Based on any of the above embodiments, the annealing training unit includes a dataset construction subunit, which is specifically used for: The number of samples for each candidate data subset is calculated based on the sampling weights and the preset total number of training samples. Based on the sampled data of any candidate data subset, samples are extracted from the candidate data subset to construct the training data subset corresponding to the candidate data subset; Calculate the similarity between the normalized feature vectors of each sample pair in any training data subset. If the similarity is greater than a preset semantic similarity threshold, then remove redundant samples from the training data subset. The training data subsets after removing redundant samples are cleaned to generate the annealed training dataset. The cleaning process includes at least one of formatting, global random shuffling, and serialization storage.

[0138] Based on any of the above embodiments, the device further includes a closed-loop feedback unit, the closed-loop feedback unit being used for: During the annealing training process, training is paused at preset intervals and the output text of the large base model is sampled to obtain a text sequence; The text sequence is input into the lightweight proxy model for semantic feature extraction, and a real-time distributed fingerprint is constructed based on the extracted feature vectors; Calculate the local feature deviation vector between the real-time distributed fingerprint and the target distributed fingerprint, and update the current sampling weight based on the local feature deviation vector and a preset feedback coefficient, so as to perform the next round of sampling based on the updated sampling weight.

[0139] Based on any of the above embodiments, the annealing training unit includes a training subunit, the training subunit being used for: When loading the base model weights of the base large model, the first and second moment state information of the optimizer in the pre-training stage is obtained and retained to initialize the training environment; Using the annealing training dataset, and employing a learning rate scheduling strategy with a decaying trend, the weighted pedestal model is subjected to gradient-guided training.

[0140] Based on any of the above embodiments, the training subunit is specifically used for: Obtain the preset initial learning rate, minimum learning rate, and total number of annealing steps; During the gradient-guided training process, the real-time learning rate corresponding to the current training step is calculated based on the initial annealing learning rate, the minimum learning rate, and the total number of annealing steps. The parameters of the base model are updated using the real-time learning rate.

[0141] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a large model annealing training method, which includes: acquiring a target dataset and a candidate corpus; extracting semantic features from each text data in the target dataset and the candidate corpus based on a pre-configured and parameter-frozen lightweight proxy model, respectively, to obtain a target feature vector set and a candidate feature vector set mapped in a unified feature space; constructing a target distribution fingerprint based on the target feature vector set, and determining the sampling weight of each candidate data subset in the candidate corpus according to the distribution difference between the target distribution fingerprint and the subset distribution fingerprint after recombination of the candidate feature vector set; sampling each candidate data subset according to the sampling weight to generate an annealing training dataset, and using the annealing training dataset to perform annealing training on the base large model.

[0142] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0143] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the large model annealing training method provided by the above methods. The method includes: acquiring a target dataset and a candidate corpus; extracting semantic features from each text data in the target dataset and the candidate corpus based on a pre-configured and parameter-frozen lightweight proxy model to obtain a target feature vector set and a candidate feature vector set mapped in a unified feature space; constructing a target distribution fingerprint based on the target feature vector set, and determining the sampling weight of each candidate data subset in the candidate corpus according to the distribution difference between the target distribution fingerprint and the subset distribution fingerprint after recombination of the candidate feature vector set; sampling each candidate data subset according to the sampling weight to generate an annealing training dataset, and using the annealing training dataset to perform annealing training on the base large model.

[0144] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the large model annealing training method provided by the above methods. This method includes: acquiring a target dataset and a candidate corpus; extracting semantic features from each text data in the target dataset and the candidate corpus based on a pre-configured and parameter-frozen lightweight proxy model, respectively, to obtain a target feature vector set and a candidate feature vector set mapped in a unified feature space; constructing a target distribution fingerprint based on the target feature vector set, and determining the sampling weight of each candidate data subset in the candidate corpus according to the distribution difference between the target distribution fingerprint and the subset distribution fingerprint after recombination of the candidate feature vector set; sampling each candidate data subset according to the sampling weight to generate an annealing training dataset, and using the annealing training dataset to perform annealing training on a base large model.

[0145] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A large-scale model annealing training method, characterized in that, include: Obtain the target dataset and candidate corpus; Based on a pre-configured and parameter-frozen lightweight proxy model, semantic features are extracted from each text data in the target dataset and the candidate corpus, respectively, to obtain a target feature vector set and a candidate feature vector set mapped in a unified feature space. A target distribution fingerprint is constructed based on the target feature vector set, and the sampling weight of each candidate data subset in the candidate corpus is determined according to the distribution difference between the target distribution fingerprint and the subset distribution fingerprint after recombination of the candidate feature vector set. Both the target distribution fingerprint and the subset distribution fingerprint are probability density distributions generated by fitting the probability density of the feature vectors. The candidate data subsets are sampled according to the sampling weights to generate an annealing training dataset, and the annealing training dataset is used to anneal the large base model. During the annealing training process, a real-time distributed fingerprint is constructed based on the text sequence generated by the base large model, and the sampling weights are dynamically updated according to the feature deviation between the real-time distributed fingerprints and the target distributed fingerprints, so as to perform the next round of sampling based on the updated sampling weights.

2. The large model annealing training method according to claim 1, characterized in that, The lightweight proxy model based on pre-configuration and parameter freezing extracts semantic features from each text data in the target dataset and the candidate corpus, respectively, to obtain a target feature vector set and a candidate feature vector set mapped in a unified feature space, including: The text data in the target dataset and the text data in the candidate corpus are respectively input into the lightweight proxy model. The text data is multi-source heterogeneous text data including multiple languages ​​and / or various minority languages. The multiple languages ​​include at least one of Chinese and English. The various minority languages ​​include at least one of Tibetan, Uyghur and Mongolian. Extract the global mean pooling vector from the hidden layer at the end of the lightweight proxy model, and normalize the global mean pooling vector to obtain a normalized feature vector. Based on the normalized feature vectors of each text data in the target dataset and the normalized feature vectors of each text data in the candidate corpus, the target feature vector set and the candidate feature vector set are constructed respectively.

3. The large model annealing training method according to claim 2, characterized in that, The step of inputting the text data from the target dataset and the text data from the candidate corpus into the lightweight proxy model includes: When the current input text data is long text data, a sliding window strategy is used to segment the long text data into text segments that fit the context window length of the lightweight proxy model, and the text segments are input into the lightweight proxy model.

4. The large model annealing training method according to claim 1, characterized in that, The step of constructing a target distribution fingerprint based on the target feature vector set, and determining the sampling weight of each candidate data subset in the candidate corpus based on the distribution difference between the target distribution fingerprint and the subset distribution fingerprint after recombination of the candidate feature vector set, includes: The probability density of each feature vector in the target feature vector set is fitted to generate the target distribution fingerprint; The candidate corpus is pre-divided into multiple pre-clustered clusters, which serve as subsets of each candidate data set; The objective function is to minimize the divergence between the target distribution fingerprint and the subset distribution fingerprint, and a regularization term is introduced as a constraint. The subset distribution fingerprint refers to the fitted distribution fingerprint obtained by weighting and recombining the probability densities of the feature vectors corresponding to each candidate data subset based on weights. The objective function is solved iteratively using the projective gradient descent method, and the weight vector is projected back to the set constraint space after the gradient is updated until the objective function converges, thereby obtaining the sampling weights of each candidate data subset.

5. The large model annealing training method according to claim 1, characterized in that, The step of sampling each candidate data subset according to the sampling weight to generate an annealing training dataset includes: The number of samples for each candidate data subset is calculated based on the sampling weights and the preset total number of training samples. Based on the sampled data of any candidate data subset, samples are extracted from the candidate data subset to construct the training data subset corresponding to the candidate data subset; Calculate the similarity between the normalized feature vectors of each sample pair in any training data subset. If the similarity is greater than a preset semantic similarity threshold, then remove redundant samples from the training data subset. The training data subsets after removing redundant samples are cleaned to generate the annealed training dataset. The cleaning process includes at least one of formatting, global random shuffling, and serialization storage.

6. The large model annealing training method according to any one of claims 1 to 5, characterized in that, During the annealing training process, a real-time distributed fingerprint is constructed based on the text sequence generated by the base large model, and the sampling weights are dynamically updated according to the feature deviation between the real-time distributed fingerprint and the target distributed fingerprint, so as to perform the next round of sampling based on the updated sampling weights, including: During the annealing training process, training is paused at preset intervals and the output text of the large base model is sampled to obtain a text sequence; The text sequence is input into the lightweight proxy model for semantic feature extraction, and a real-time distributed fingerprint is constructed based on the extracted feature vectors; Calculate the local feature deviation vector between the real-time distributed fingerprint and the target distributed fingerprint, and update the current sampling weight based on the local feature deviation vector and a preset feedback coefficient, so as to perform the next round of sampling based on the updated sampling weight.

7. The large model annealing training method according to any one of claims 1 to 5, characterized in that, The step of using the annealing training dataset to anneal the large base model includes: When loading the base model weights of the base large model, the first and second moment state information of the optimizer in the pre-training stage is obtained and retained to initialize the training environment; Using the annealing training dataset, and employing a learning rate scheduling strategy with a decaying trend, the weighted pedestal model is subjected to gradient-guided training.

8. The large model annealing training method according to claim 7, characterized in that, The gradient-guided training of the base model after weight loading, employing a learning rate scheduling strategy with a decaying trend, includes: Obtain the preset initial learning rate, minimum learning rate, and total number of annealing steps; During the gradient-guided training process, the real-time learning rate corresponding to the current training step is calculated based on the initial annealing learning rate, the minimum learning rate, and the total number of annealing steps. The parameters of the base model are updated using the real-time learning rate.

9. A large-scale annealing training device, characterized in that, include: The data acquisition unit is used to acquire the target dataset and candidate corpus. The feature extraction unit is used to extract semantic features from each text data in the target dataset and the candidate corpus based on a pre-configured and parameter-frozen lightweight proxy model, respectively, to obtain a target feature vector set and a candidate feature vector set mapped in a unified feature space. The weight determination unit is used to construct a target distribution fingerprint based on the target feature vector set, and determine the sampling weight of each candidate data subset in the candidate corpus according to the distribution difference between the target distribution fingerprint and the subset distribution fingerprint after the candidate feature vector set is recombined. Both the target distribution fingerprint and the subset distribution fingerprint are probability density distributions generated by fitting the probability density of the feature vectors. The annealing training unit is used to sample each candidate data subset according to the sampling weight, generate an annealing training dataset, and use the annealing training dataset to perform annealing training on the base large model. During the annealing training process, a real-time distributed fingerprint is constructed based on the text sequence generated by the base large model, and the sampling weights are dynamically updated according to the feature deviation between the real-time distributed fingerprints and the target distributed fingerprints, so as to perform the next round of sampling based on the updated sampling weights.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the large model annealing training method as described in any one of claims 1 to 8.

Images