Predictor joint training method and system based on approximate reduction mechanism

Abstract

The present invention provides a predictor joint training method and system based on approximate reduction, which relates to the field of predictor training technology, including: modifying the original model step: modifying the feedforward layer structure of the original model, embedding the predictor before the feedforward layer and performing weight initialization; differentiable step: making the model structure of the embedded predictor differentiable as a whole through a binary conversion mechanism; freezing parameter step: freezing all parameters of the original model, leaving only the predictor parameters trainable; training step: collecting and cleaning general corpus, training the model on the general corpus, and using multivariate data to make the predictor fit the activation features of the model. The present invention has the characteristics of high efficiency, safety, automation, etc., and is very suitable for the prediction module of the sparsity of the training model.

Description

Technical Field

[0001] The present invention relates to the technical field of predictor training, and in particular to a predictor joint training method and system based on approximate reduction. Background Art

[0002] In recent years, artificial intelligence has made remarkable progress in the field of natural language processing (NLP) with the development of large-scale pre-trained language models such as GPT-3, BERT, LLaMA, etc. These models have performed well in various tasks such as machine translation, text generation, question-answering systems, etc. However, as the scale of the model continues to increase, the consumption of computing resources and the demand for deployment have also increased dramatically, resulting in high computing costs.

[0003] To address this challenge, leveraging the built-in activation sparsity of large language models is a promising approach because it can improve inference efficiency by skipping useless computations by leveraging the zero elements in the output of the activation function of the large language model. Before skipping computations, a prediction module is required to predict which part of the computation can be skipped. Existing methods use the method of training the predictor separately after model training, which has two major defects:

[0004] High training cost: Post-training requires collecting the sparse activation information of the model in a general text, and then training the sparse conditions of each layer separately. In addition, after each model fine-tuning, the predictor needs to be retrained, which makes the predictor training too complex.

[0005] Low predictor accuracy: Due to the huge overhead of collecting sparse activation information of the model, post-training can only be collected on a small piece of text, which leads to low predictor accuracy and poor generalization. Summary of the invention

[0006] In view of the defects in the prior art, the present invention provides a predictor joint training method and system based on approximate reduction.

[0007] According to a predictor joint training method and system based on approximate reduction provided by the present invention, the scheme is as follows:

[0008] In a first aspect, a predictor joint training method based on approximate reduction is provided, the method comprising:

[0009] Modify the original model steps: modify the feedforward layer structure of the original language model, embed the predictor before the feedforward layer and initialize the weights;

[0010] Differentiable step: The language model structure after embedding the predictor is made differentiable as a whole through the binary conversion mechanism;

[0011] Freeze parameter step: freeze all parameter weights of the original language model, leaving only the predictor parameters trainable;

[0012] Training steps: Collect and clean general corpus, train language model on general corpus, and use multivariate data to make the predictor fit the activation features of the language model.

[0013] Preferably, the model in the step of modifying the original model is applicable to LLaMA structure, Mistral structure dense language model, and mixed expert language model based on Mixtral and Deepseek;

[0014] The specific process of the embedding process is to modify the feedforward layer structure of the original language, add a prediction module with a two-layer multi-layer perceptron (MLP) structure before the original feedforward layer, and randomly initialize the weights of the prediction module.

[0015] Preferably, the differentiable step comprises:

[0016] A sigmoid function is added after the output of the predictor so that the output value is between 0 and 1, thus achieving soft gating;

[0017] During forward propagation, the output of the sigmoid function is binarized, that is, the output value greater than 0.5 is set to 1, and the output value less than or equal to 0.5 is set to 0 to ensure the correctness of forward reasoning;

[0018] During back propagation, the gradient of the prediction module is approximately reduced to the gradient of the sigmoid function to ensure the differentiability of the gradient.

[0019] Preferably, the parameter gradient is always set to be empty, thereby freezing all pre-trained parameters in the original model in the parameter freezing step, and the frozen parameters specifically include: an embedding layer, an attention layer, and a feedforward layer, and only the parameters of the newly added predictor are retained for training.

[0020] Preferably, in the training step, the loss value of the model is restored to the loss value of the original language model by performing unsupervised training in the corpus.

[0021] In a second aspect, a predictor joint training system based on approximate reduction is provided, comprising:

[0022] Modify the original model module: modify the feedforward layer structure of the original language model, embed the predictor before the feedforward layer and initialize the weights;

[0023] Differentiable module: The language model structure after embedding the predictor is made differentiable as a whole through the binary conversion mechanism;

[0024] Freeze parameter module: freezes all parameter weights of the original language model, leaving only the predictor parameters trainable;

[0025] Training module: Collect and clean general corpus, train language model on general corpus, and use multivariate data to make the predictor fit the activation features of the language model.

[0026] Preferably, the model of the modified original model module is applicable to LLaMA structure, Mistral structure dense language model, and mixed expert language model based on Mixtral and Deepseek;

[0027] The specific process of the embedding process is to modify the feedforward layer structure of the original language, add a prediction module with a two-layer multi-layer perceptron (MLP) structure before the original feedforward layer, and randomly initialize the weights of the prediction module.

[0028] Preferably, the differentiable module includes:

[0029] A sigmoid function is added after the output of the predictor so that the output value is between 0 and 1, thus achieving soft gating;

[0030] During forward propagation, the output of the sigmoid function is binarized, that is, the output value greater than 0.5 is set to 1, and the output value less than or equal to 0.5 is set to 0 to ensure the correctness of forward reasoning;

[0031] During back propagation, the gradient of the prediction module is approximately reduced to the gradient of the sigmoid function to ensure the differentiability of the gradient.

[0032] Preferably, the parameter gradient is always set to be empty, thereby freezing all pre-trained parameters in the original model in the parameter freezing step, and the frozen parameters specifically include: an embedding layer, an attention layer, and a feedforward layer, and only the parameters of the newly added predictor are retained for training.

[0033] Preferably, in the training module, the loss value of the model is restored to the loss value of the original language model by performing unsupervised training in the corpus.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. Significantly reduce training costs: By embedding the predictor module in the original model and freezing the original model parameters, only the parameters of the predictor module need to be trained, reducing the consumption of computing resources during the training process. Compared with traditional methods, the method of the present invention only requires 5% of the cost to complete the training of the sparse model prediction module, greatly reducing the training cost;

[0036] 2. Improve the accuracy and generalization ability of the predictor: By training the predictor on a diverse general corpus, it can better learn and fit the activation features of the model. Diverse data sources (such as news text, social media, encyclopedias, technical documents, etc.) ensure the breadth and diversity of training data, thereby improving the generalization ability and adaptability of the predictor;

[0037] 3. Accelerate the training process: Since only the parameters of the predictor module need to be trained, and the differentiability of the model structure is guaranteed by the binary conversion mechanism, the parameter update amount during the training process is significantly reduced, thereby accelerating the convergence speed of the predictor. In addition, the Straight-Through Estimator mechanism is used to ensure the correctness of forward reasoning while avoiding the gradient disconnection problem;

[0038] 4. Maintain the performance and stability of the original model: By freezing all pre-trained parameters of the original model, the performance and stability of the original model are ensured to be unaffected. Only the parameters of the newly added predictor module are retained for training, avoiding destructive modifications to the original model and ensuring the reliability of the model in practical applications;

[0039] 5. Wide applicability: The method of the present invention is applicable to a variety of model structures, including but not limited to dense models such as LLaMA and Mistral, and expert models such as Mixtral and Deepseek. The diverse model structures enable the method of the present invention to be applied to different scenarios and needs, and it has strong versatility.

[0040] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0042] Figure 1 It is a schematic diagram of the overall process of the present invention. DETAILED DESCRIPTION

[0043] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0044] The embodiment of the present invention provides a predictor joint training method based on approximate reduction, which modifies the structure of the original model, adds the predictor module to the model itself, freezes the model parameters on the selected corpus and then performs training, thereby significantly accelerating the convergence speed of the predictor. Figure 1 As shown, the method specifically includes the following contents:

[0045] Steps to modify the original model: It is known that the original language model is formed by stacking self-attention layer and feedforward layer structure. In order to predict the sparse distribution of the activation of the feedforward layer, the feedforward layer structure of the original language model is modified, and the predictor module is embedded before all feedforward layers and the weights are initialized.

[0046] The language models in this step include but are not limited to dense large language models of LLaMA structure and Mistral structure, and hybrid expert language models represented by Mixtral and Deepseek. These models are representative model structures of large language models, and the present invention can be used for all the above language models.

[0047] The specific process of the embedding process is to modify the feedforward layer structure of the original language, add a prediction module with a two-layer multi-layer perceptron (MLP) structure before the original feedforward layer, and randomly initialize the weights of the prediction module.

[0048] Differentiable step: The model structure embedded in the predictor is made differentiable through the binary conversion mechanism. Soft gating is implemented through the sigmoid function after the output of the predictor, and then the approximate algorithm reduction mechanism is used to ensure the correctness of the model's forward reasoning. During back propagation, the gradient is converted to the gradient calculation method of the sigmoid function to ensure differentiability.

[0049] This step specifically includes: adding a sigmoid function after the output of the predictor module so that the output value is between 0 and 1, thereby realizing soft gating (in this embodiment, soft gating assigns an activation weight to each neuron, and if the activation weight is greater than 0.5, the neuron is considered to be activated); using an approximate reduction mechanism, binarizing the output of the sigmoid function during forward propagation, that is, setting output values ​​greater than 0.5 to 1, and setting output values ​​less than or equal to 0.5 to 0, to ensure the correctness of forward reasoning; during back propagation, the predictor parameter gradient is approximately reduced to the gradient of the sigmoid function, thereby ensuring the differentiability of the gradient and avoiding gradient disconnection.

[0050] Freeze parameter step: Freeze all parameters of the original model and only keep the predictor parameters trainable; speed up the predictor training.

[0051] Freeze all pre-trained parameters in the original model, including but not limited to the embedding layer, attention layer, feedforward layer, etc., and only keep the parameters of the newly added predictor module trainable. In this way, the amount of parameter updates during training is reduced, significantly accelerating the convergence speed of the predictor while maintaining the performance and stability of the original model.

[0052] Training steps: Collect and clean common corpus, train the model on the common corpus, and use multivariate data to fit the predictor to the activation features of the model.

[0053] The specific implementation method of training the model on the general corpus includes selecting a diversified general corpus, including but not limited to news text, social media, encyclopedia, technical documents, etc., to ensure the diversity and extensiveness of the training data. Using this diverse data, the predictor can learn and fit the activation features of the model faster, improving the generalization ability and adaptability of the predictor.

[0054] The present invention also provides a predictor joint training system based on approximate regulations, which can be implemented by executing the process steps of the predictor joint training method based on approximate regulations, that is, those skilled in the art can understand the predictor joint training method based on approximate regulations as a preferred implementation of the predictor joint training system based on approximate regulations. The system specifically includes:

[0055] Modify the original model module: It is known that the original language model is formed by stacking self-attention layer and feedforward layer structure. In order to predict the sparse distribution of the activation of the feedforward layer of the predictor, the feedforward layer structure of the original language model is modified, and the predictor module is embedded before all feedforward layers and the weights are initialized.

[0056] The language models in this module include but are not limited to dense large language models of LLaMA structure and Mistral structure, and mixed expert language models represented by Mixtral and Deepseek. These models are representative model structures of large language models, and the present invention can be used for all the above language models.

[0057] The specific process of the embedding process is to modify the feedforward layer structure of the original language, add a prediction module with a two-layer multi-layer perceptron (MLP) structure before the original feedforward layer, and randomly initialize the weights of the prediction module.

[0058] Differentiable module: The model structure embedded in the predictor is made differentiable through the binary conversion mechanism. Soft gating is implemented through the sigmoid function after the output of the predictor, and the approximate algorithm reduction mechanism is used to ensure the correctness of the model's forward reasoning. During back propagation, the gradient is converted to the gradient calculation method of the sigmoid function to ensure differentiability.

[0059] The module specifically includes: adding a sigmoid function after the output of the predictor module so that the output value is between 0 and 1, thereby realizing soft gating (in this embodiment, soft gating assigns an activation weight to each neuron, and if the activation weight is greater than 0.5, the neuron is considered to be activated); using an approximate reduction mechanism, binarizing the output of the sigmoid function during forward propagation, that is, setting the output value greater than 0.5 to 1, and setting the output value less than or equal to 0.5 to 0, so as to ensure the correctness of forward reasoning; during back propagation, the gradient of the predictor parameter is approximately reduced to the gradient of the sigmoid function, so as to ensure the differentiability of the gradient and avoid gradient disconnection.

[0060] Freeze parameter module: Freeze all parameters of the original model and only keep the predictor parameters trainable; speed up the predictor training.

[0061] Freeze all pre-trained parameters in the original model, including but not limited to the embedding layer, attention layer, feedforward layer, etc., and only keep the parameters of the newly added predictor module trainable. In this way, the amount of parameter updates during training is reduced, significantly accelerating the convergence speed of the predictor while maintaining the performance and stability of the original model.

[0062] Training module: Collect and clean common corpus, train the model on the common corpus, and use multivariate data to fit the predictor to the activation features of the model.

[0063] The specific implementation method of training the model on the general corpus includes selecting a diversified general corpus, including but not limited to news text, social media, encyclopedia, technical documents, etc., to ensure the diversity and extensiveness of the training data. Using this diverse data, the predictor can learn and fit the activation features of the model faster, improving the generalization ability and adaptability of the predictor.

[0064] Next, the present invention will be described in more detail.

[0065] The present invention provides a predictor joint training method based on approximate reduction, which is described in detail as follows:

[0066] Modify the feedforward layer structure of the large language model: embed the predictor module before the feedforward layer and initialize the weights. Specifically, a predictor module can be added before each feedforward layer, which outputs a value between 0 and 1 through the sigmoid function, indicating the activation sparsity of the layer, and then executes the next step.

[0067] The model structure is made differentiable as a whole through the binarization conversion mechanism. After the output of the predictor module is processed by the sigmoid function, the sigmoid output is binarized during forward propagation, that is, the output value greater than 0.5 is set to 1, and the output value less than or equal to 0.5 is set to 0 to ensure the correctness of forward reasoning; during back propagation, the gradient is approximately reduced to the gradient of sigmoid, thereby ensuring the differentiability of the gradient and avoiding gradient disconnection, and then executing the next step.

[0068] Freeze all parameters of the original model, leaving only the predictor parameters trainable, and speeding up the predictor training. In specific implementation, you can freeze the pre-trained parameters of the original model by setting the frozen parameter component, including the embedding layer, attention layer, feedforward layer, etc., leaving only the parameters of the newly added predictor module trainable, and then proceed to the next step.

[0069] Select the optimal training hyperparameters according to the hyperparameter scaling law. The selection process includes adjusting the learning rate, batch size, weight decay coefficient, etc., and then proceed to the next step.

[0070] Train the model on a general corpus and use multivariate data to enable the predictor to quickly fit the activation features of the model. Select a diverse general corpus, including news text, social media, encyclopedias, technical documents, etc., to ensure the diversity and breadth of the training data, so that the predictor can learn and fit the activation features of the model faster, and improve the generalization and adaptability of the predictor.

[0071] The embodiment of the present invention provides a predictor joint training method and system based on approximate reduction, which realizes a fast model and predictor joint training method by optimizing the model algorithm and adjusting the training process. Compared with the existing method, it only uses less than 5% of the training cost and can complete the training of the predictor in less than two hours. Compared with the existing technology, the present invention has the characteristics of high efficiency, safety, automation, etc., and is very suitable for training the prediction module of model sparsity.

[0072] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0073] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A predictor joint training method based on approximate reduction, characterized in that: include: Modify the original model steps: modify the feedforward layer structure of the original language model, embed the predictor before the feedforward layer and initialize the weights; The specific process of the embedding process is to modify the feedforward layer structure of the original language, add a prediction module with a two-layer multi-layer perceptron MLP structure before the original feedforward layer, and randomly initialize the weight of the prediction module; Differentiable step: The language model structure after embedding the predictor is made differentiable as a whole through the binary conversion mechanism; Freeze parameter step: Freeze all parameter weights of the original language model, leaving only the predictor parameters trainable; By setting the gradient of the parameters to always be empty, all pre-trained parameters in the original model are frozen in the parameter freezing step; Training steps: Collect and clean common corpora, train language models on common corpora, and use multivariate data to fit the predictor to the activation features of the language model; common corpora include news texts, social media, encyclopedias, and technical documents; The differentiable steps include: A sigmoid function is added after the output of the predictor so that the output value is between 0 and 1, thus achieving soft gating; During forward propagation, the output of the sigmoid function is binarized, that is, the output value greater than 0.5 is set to 1, and the output value less than or equal to 0.5 is set to 0 to ensure the correctness of forward reasoning; During back propagation, the gradient of the prediction module is approximately reduced to the gradient of the sigmoid function to ensure the differentiability of the gradient.

2. The predictor joint training method based on approximate reduction according to claim 1, characterized in that: The model in the step of modifying the original model is applicable to the LLaMA structure, the Mistral structure dense language model, and the mixed expert language model of Mixtral and Deepseek.

3. The predictor joint training method based on approximate reduction according to claim 1, characterized in that: The frozen parameters specifically include: embedding layer, attention layer and feedforward layer, and only the parameters of the newly added predictor are retained for training.

4. The predictor joint training method based on approximate reduction according to claim 1, characterized in that: In the training step, the loss value of the model is restored to the loss value of the original language model by performing unsupervised training in the corpus.

5. A predictor joint training system based on approximate reduction, characterized in that: include: Modify the original model module: modify the feedforward layer structure of the original language model, embed the predictor before the feedforward layer and initialize the weights; The specific process of the embedding process is to modify the feedforward layer structure of the original language, add a prediction module with a two-layer multi-layer perceptron MLP structure before the original feedforward layer, and randomly initialize the weight of the prediction module; Differentiable module: The language model structure after embedding the predictor is made differentiable as a whole through the binary conversion mechanism; Freeze parameter module: freezes all parameter weights of the original language model, leaving only the predictor parameters trainable; By setting the gradient of the parameters to always be empty, all pre-trained parameters in the original model are frozen in the parameter freezing step; Training module: Collect and clean general corpus, train language model on general corpus, and use multivariate data to fit the predictor to the activation features of the language model; general corpus includes news text, social media, encyclopedia, and technical documents; The differentiable module includes: A sigmoid function is added after the output of the predictor so that the output value is between 0 and 1, thus achieving soft gating; During forward propagation, the output of the sigmoid function is binarized, that is, the output value greater than 0.5 is set to 1, and the output value less than or equal to 0.5 is set to 0 to ensure the correctness of forward reasoning; During back propagation, the gradient of the prediction module is approximately reduced to the gradient of the sigmoid function to ensure the differentiability of the gradient.

6. The predictor joint training system based on approximate reduction according to claim 5, characterized in that: The model of the modified original model module is applicable to the LLaMA structure, the Mistral structure dense language model, and the mixed expert language model of Mixtral and Deepseek.

7. The predictor joint training system based on approximate reduction according to claim 5, characterized in that: The frozen parameters specifically include: embedding layer, attention layer and feedforward layer, and only the parameters of the newly added predictor are retained for training.

8. The predictor joint training system based on approximate reduction according to claim 5, characterized in that: In the training module, the loss value of the model is restored to the loss value of the original language model by performing unsupervised training in the corpus.

Images