Continuous learning method and system for anti-forgetting large language model

Abstract

The invention discloses an anti-forgetting large language model continuous learning method and system, and the method comprises the steps: selecting samples which can represent the universal capability of a model or samples generated through an initial large language model, and forming a universal anti-forgetting sample set; inputting each sample in the universal anti-forgetting sample set into the initial large language model, and calculating a hidden state value of each sample; counting distribution of all hidden state value sets, and calculating related statistical values; calculating a positive activation threshold and a negative activation threshold for threshold edge loss calculation according to the related statistical value; and fusing the universal anti-forgetting sample set with an original continuous learning data set, and performing continuous learning of the initial large language model by adopting a hidden state threshold constraint algorithm. According to the large language model continuous pre-training and fine tuning method based on universal anti-forgetting sample hidden state threshold constraint optimization, downstream new tasks can be effectively learned in an anti-forgetting mode, and the universal capacity of the model is kept.

Description

Technical Field

[0001] The present invention relates to the field of continuous learning of large language models, and in particular to a method and system for continuous learning of large language models that resist forgetting. Background Art

[0002] With the significant increase in the computing power of high-performance computers and the continuous accumulation of internet data resources, artificial intelligence (AI) technology has rapidly developed and been widely applied in the computer field. In particular, large language models, which have emerged in recent years, are gaining increasing attention from academia and industry due to their high intelligence and powerful general capabilities. Currently, large language models can provide basic services such as intelligent question-answering, information retrieval, document writing assistance, and interactive conversations, and they demonstrate significant application potential in vertical sectors such as finance, education, and healthcare. As demand for AI technology continues to grow, there is increasing emphasis on how to enable large language models to continuously learn new knowledge to adapt to the needs of diverse application scenarios.

[0003] The "big" in the large language model not only reflects its powerful capabilities, but also means high training costs. Building a usable large language model requires massive data resources and computing resources, and is accompanied by huge time costs and hardware investment. When faced with new data and new application scenarios, the large language model is required to continuously update its internal knowledge. Compared with retraining from scratch, continuous pre-training or fine-tuning based on the existing model is a more economical and efficient method. However, this continuous learning process will face the "catastrophic forgetting" problem inherent in gradient optimization-based neural networks. That is, when the model learns a new task, its performance on old tasks (or its original general capabilities for large language models) will drop significantly, eventually leading to the loss of its original general capabilities. This poses substantial challenges to the application of large language models to specific vertical fields such as finance and healthcare.

[0004] While various approaches exist for continuous learning of large language models, including solutions similar to the present invention that utilize common samples as playback samples for fusion learning, these traditional methods often simply mix samples for training and lack further optimization mechanisms, resulting in limited effectiveness in combating forgetting. Therefore, existing technologies still require improvement and development. Summary of the Invention

[0005] In response to the above-mentioned defects of the prior art, the purpose of the present invention is to provide a continuous learning method and system for a large language model that is resistant to forgetting. It is a continuous pre-training and fine-tuning method for a large language model based on the optimization of the hidden state threshold constraint of a general anti-forgetting sample. This method can effectively learn new downstream tasks in a forgetting-resistant manner and maintain the general capabilities of the model.

[0006] The present invention solves the technical problem by adopting the following technical solutions:

[0007] First, a method for continuous learning of a large language model that resists forgetting includes the following steps:

[0008] Step S1, constructing a general anti-forgetting sample set: selecting samples that can represent the general capabilities of the model or samples generated by the initial large language model to form a general anti-forgetting sample set;

[0009] Step S2, calculation of the hidden state value of the universal anti-forgetting sample: input each universal anti-forgetting sample in the universal anti-forgetting sample set into the initial large language model, and calculate the hidden state value of each universal anti-forgetting sample;

[0010] Step S3, distribution statistics of hidden state value sets: statistics the distribution of all hidden state value sets and calculate relevant statistical values;

[0011] Step S4, calculation of activation threshold: calculating the positive activation threshold and the negative activation threshold for threshold edge loss calculation based on the relevant statistical values;

[0012] Step S5, continuous learning of the large language model with hidden state activation preservation: the general anti-forgetting sample set is integrated with the original continuous learning data set, and the hidden state threshold constraint algorithm is used to continuously learn the initial large language model, thereby improving the anti-forgetting ability of the initial large language model during the continuous learning process and maintaining its general capabilities.

[0013] Furthermore, in step S1, the universal anti-forgetting sample set is composed of a large number of corresponding universal samples, each universal sample is a natural language sentence, and the construction method includes directly randomly extracting or carefully selecting from the training corpus of the initial large language model, or automatically generating the corpus using the initial large language model; during the construction process, data cleaning, length restriction, individual sample rewriting optimization and other post-processing are performed according to quality requirements.

[0014] Furthermore, in step S2, each universal anti-forgetting sample is input into the initial large language model for forward calculation of the neural network, and the neuron activation value of each sub-network hidden layer is obtained as the hidden state value.

[0015] Furthermore, in step S3, the distribution statistics of the hidden state value set are calculated, and each layer of the sub-network and the hidden layer dimension of the sub-network are used as statistical calculation units to obtain the corresponding statistical values of the hidden state of each layer of the sub-network and each neuron, including the mean, variance, standard deviation, median, maximum value, and minimum value.

[0016] Furthermore, in step S4, the statistical value of the hidden state is converted into an activation threshold to obtain a positive activation threshold and a negative activation threshold respectively; the positive activation threshold is the mean obtained in step S3 shifted in the positive direction by n times the standard deviation, and the negative activation threshold is the mean obtained in step S3 shifted in the negative direction by n times the standard deviation.

[0017] Furthermore, in step S5, continuous learning is applicable to learning a single or multiple tasks, and is also applicable to full parameter learning or efficient parameter learning; during continuous learning, universal anti-forgetting samples and original continuous learning training samples are integrated; during integration, each batch of training is used as a unit to ensure that a specific number of universal anti-forgetting samples are inserted into each batch of training data; universal anti-forgetting samples are randomly extracted or carefully selected from the universal anti-forgetting corpus during training; in terms of quantity selection, it includes adopting a number of universal anti-forgetting samples inserted that is a fixed ratio to the batch data volume, or specifying a fixed number of universal anti-forgetting samples inserted that is independent of the batch data volume; or using a direct mixing method to directly mix the universal anti-forgetting samples with the task training samples that need to be continuously learned and randomly shuffle them.

[0018] Furthermore, in step S5, during continuous learning, a threshold marginal loss of the universal anti-forgetting sample is calculated; a threshold marginal loss calculation method is used to calculate the corresponding loss of the universal anti-forgetting sample; and a joint optimization is performed together with the loss of the downstream task of continuous learning; the objects of the threshold marginal loss calculation include considering the hidden state values of all layers of the model, or only the hidden state values of the last layer; the threshold marginal loss calculation will ensure that the hidden state values of the universal anti-forgetting sample are consistent with the hidden state values based on the initial large language model obtained in step S2 during the training process, with respect to whether they are activated or not, positively activated or negatively activated under a specific threshold constraint;

[0019] Dynamically balance the threshold edge loss and the language model loss during normal continuous learning; use dynamic loss weight calculation method to calculate the total loss of the two to avoid the optimization of the neural network being biased towards either side during continuous learning.

[0020] The second aspect is a continuous learning system for large language models that resists forgetting, including:

[0021] The universal anti-forgetting corpus construction module is used to obtain samples and construct a universal anti-forgetting sample set as the anti-forgetting corpus; and preprocess the universal anti-forgetting samples, including corpus data cleaning, length restriction, error sample correction, sample optimization, sample encoding and indexing;

[0022] The hidden state value distribution statistics calculation module is used to load the initial large language model and input all common anti-forgetting samples for forward calculation to obtain all corresponding hidden state values. Statistics are then performed to obtain relevant statistical values including mean, variance, standard deviation, median, maximum, and minimum values, and to calculate the positive and negative activation thresholds used to calculate the threshold edge loss.

[0023] Hidden state value set storage and indexing module, used to store and index the hidden state values of all samples of the universal anti-forgetting corpus obtained by the above-mentioned hidden state value distribution statistical calculation module, so as to be read by the subsequent continuous learning module;

[0024] The continuous learning module is used to read samples of the universal anti-forgetting corpus and the corresponding stored hidden state values. It uses the obtained positive and negative activation thresholds to calculate the threshold edge loss of the universal anti-forgetting samples during training and jointly optimizes it with the existing language model loss.

[0025] In a third aspect, a terminal device includes a memory, a processor, and a program of a method for continuously learning a large language model that resists forgetting, which is stored in the memory and can be run on the processor. When the processor executes the program of the method for continuously learning a large language model that resists forgetting, the steps of the method for continuously learning a large language model that resists forgetting described in any one of the above items are implemented.

[0026] In a fourth aspect, a computer-readable storage medium stores a program for a method for continuous learning of a large language model that resists forgetting. When the program for the method for continuous learning of a large language model that resists forgetting is executed by a processor, the steps of any of the above-mentioned methods for continuous learning of a large language model that resists forgetting are implemented.

[0027] Beneficial effects:

[0028] Compared with the existing technology, the present invention provides a method and system for continuous learning of a large language model that is resistant to forgetting. The present invention can allow the large language model to maintain the original general capabilities of the model with high performance during continuous learning, and reduce catastrophic forgetting after fully learning new downstream tasks. The anti-forgetting training method of the present invention can ensure that any large language model can learn new tasks at a low cost, without the need to start training from scratch, and without the need to save a huge amount of original general training data. Therefore, the demand for large amounts of data and computing power is reduced, and good economic benefits are achieved. At the same time, the present invention also has a promoting effect on the continuous learning of large language models in terms of model effect. After maintaining the general capabilities, it can even improve the learning effect of the final downstream tasks. Therefore, it is a key technology for building a powerful general artificial intelligence in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1This is a flowchart of a method for continuous learning of a large language model that resists forgetting according to this embodiment.

[0030] Figure 2 This is a principle block diagram of a continuous learning system for a large language model that resists forgetting provided by this embodiment.

[0031] Figure 3 This is a block diagram of the internal structure of the terminal device provided in this embodiment. DETAILED DESCRIPTION

[0032] To make the purpose, technical solution, and advantages of this embodiment more clear, the technical solution of this embodiment will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0033] In order to make it easier for readers to understand the entire content of this disclosure, the following terms appearing in this disclosure are now explained. It should be noted that the term explanations herein are only to assist readers in understanding and do not constitute a limitation on the technical solutions of this disclosure;

[0034] Large language model: refers to a deep learning model trained using large amounts of text data. It can generate natural language text or understand the meaning of language text. It usually refers to a language model with more than billions of parameters.

[0035] The technical problem to be solved by the present invention is that the traditional continuous learning (including pre-training and fine-tuning) for large language models is only to organize some general samples for mixing and then conduct regular training. For the continuous learning (including pre-training and fine-tuning) of large language models, the general ability of the large language model cannot be maintained, resulting in catastrophic forgetting, which is not effective in anti-forgetting performance, reduces the performance of the downstream task model obtained by continuous learning, and limits the application of large language models. The present invention provides a continuous learning method and system for a large language model with anti-forgetting, which uses the positive and negative activation thresholds of the hidden state of the anti-forgetting samples obtained by statistics, designs a new threshold edge loss and combines it with the original loss in a fixed ratio insertion and dynamic loss weighting method in batches for joint optimization. It ensures that there is sufficient anti-forgetting label information in the process of continuous learning (including pre-training and fine-tuning). The method can maintain the maximum general ability without affecting the ability to learn new tasks, and further promotes the learning of new tasks with the help of the maintained general ability.

[0036] refer to Figure 1 The present invention discloses a continuous learning method for a large language model that resists forgetting, comprising the following steps:

[0037] Step S1, constructing a general anti-forgetting sample set: selecting samples that can represent the general capabilities of the model or samples generated by the initial large language model to form a general anti-forgetting sample set;

[0038] Step S2, calculation of the hidden state value of the universal anti-forgetting sample: input each universal anti-forgetting sample in the universal anti-forgetting sample set into the initial large language model, and calculate the hidden state value of each universal anti-forgetting sample;

[0039] Step S3, distribution statistics of hidden state value sets: statistics the distribution of all hidden state value sets and calculate relevant statistical values;

[0040] Step S4, calculation of activation threshold: calculating the positive activation threshold and the negative activation threshold for threshold edge loss calculation based on the relevant statistical values;

[0041] Step S5, continuous learning of the large language model with hidden state activation preservation: the general anti-forgetting sample set is integrated with the original continuous learning data set, and the hidden state threshold constraint algorithm is used to continuously learn the initial large language model, thereby improving the anti-forgetting ability of the initial large language model during the continuous learning process and maintaining its general capabilities.

[0042] To further optimize the technical solution, in step S1, the universal anti-forgetting sample set refers to corpus that matches the domain of the initial large language model and reflects the source of its general ability learning. The universal anti-forgetting sample set is composed of a large number of corresponding universal samples, each of which is a natural language sentence. The construction method includes directly randomly extracting or carefully selecting the training corpus of the initial large language model, or automatically generating the corpus using the initial large language model. During the construction process, post-processing such as data cleaning, length restriction, and individual sample rewriting optimization is performed according to quality requirements.

[0043] To further optimize the technical solution, in step S2, each universal anti-forgetting example is input into the initial large language model for forward computation of the neural network. The hidden state value refers to the neuron activation value of each hidden layer of the sub-network in the neural network of the initial large language model during the forward computation of each universal anti-forgetting example.

[0044] To further optimize the technical solution, in step S3, the hidden state value set refers to the corresponding hidden state value set obtained in step S2 for all or part of the samples in the universal anti-forgetting sample set. A distribution statistical calculation is performed on the hidden state value set, using each subnetwork layer and the hidden layer dimension of the subnetwork as the statistical calculation unit to obtain the corresponding hidden state statistical values of each subnetwork layer and each neuron, including relevant statistical values such as mean, variance, standard deviation, median, maximum value, and minimum value.

[0045] To further optimize the technical solution, in step S4, the statistical value of the hidden state is converted into an activation threshold to obtain a positive activation threshold and a negative activation threshold respectively; the positive activation threshold is the mean obtained in step S3 offset in the positive direction by n times the standard deviation, and the negative activation threshold is the mean obtained in step S3 offset in the negative direction by n times the standard deviation.

[0046] To further optimize the technical solution, in step S5, the continuous learning refers to using the initial large language model to perform continuous pre-training or continuous fine-tuning after collecting downstream task training samples that need to be continuously learned. The downstream tasks of continuous learning can be single or multiple. Continuous learning can use full parameter learning or efficient parameter learning, as well as various learning methods derived therefrom. During continuous learning, the general anti-forgetting samples and the original continuous learning training samples are integrated; during integration, each batch of training is used as a unit to ensure that a specific number of general anti-forgetting samples are inserted into each batch of training data; the general anti-forgetting samples are randomly extracted or carefully selected from the general anti-forgetting corpus during training; in terms of quantity selection, it includes adopting a fixed ratio of general anti-forgetting sample insertion number to the batch data volume, or specifying a fixed general anti-forgetting sample insertion number that is independent of the batch data volume; in addition, the general anti-forgetting samples can also be directly mixed with the task training samples that need to be continuously learned and randomly shuffled by direct mixing. This is suitable for situations where the task training samples that need to be continuously learned are of the same order of magnitude as the general anti-forgetting samples.

[0047] Further optimizing the technical solution, in step S5, during continuous learning, the threshold marginal loss of the universal anti-forgetting sample is calculated; the threshold marginal loss calculation method proposed in the present invention is used to calculate the corresponding loss of the universal anti-forgetting sample; and the threshold marginal loss calculation method is used to calculate the corresponding loss of the universal anti-forgetting sample together with the loss of the downstream task of continuous learning; the threshold marginal loss calculation object includes considering the hidden state values of all layers of the model, or only the hidden state values of the last layer; the threshold marginal loss calculation will ensure that the hidden state values of the universal anti-forgetting sample are consistent with the hidden state values based on the initial large language model obtained in step S2 during the training process, whether they are activated or not, positively activated or negatively activated under a specific threshold constraint;

[0048] Dynamically balance the threshold edge loss and the language model loss during normal continuous learning; use dynamic loss weight calculation method to calculate the total loss of the two to avoid the optimization of the neural network being biased towards either side during continuous learning.

[0049] The second aspect is a continuous learning system for large language models that resists forgetting, including:

[0050] The universal anti-forgetting corpus construction module is used to obtain samples and construct a universal anti-forgetting sample set as the anti-forgetting corpus; and preprocess the universal anti-forgetting samples, including corpus data cleaning, length restriction, error sample correction, sample optimization, sample encoding and indexing;

[0051] The hidden state value distribution statistics calculation module is used to load the initial large language model and input all common anti-forgetting samples for forward calculation to obtain all corresponding hidden state values. Statistics are then performed to obtain relevant statistical values including mean, variance, standard deviation, median, maximum, and minimum values. The positive and negative activation thresholds used to calculate the threshold edge loss are further calculated.

[0052] Hidden state value set storage and indexing module, used to store and index the hidden state values of all samples of the universal anti-forgetting corpus obtained by the above-mentioned hidden state value distribution statistical calculation module, so as to be read by the subsequent continuous learning module;

[0053] The continuous learning module is used to read samples of the universal anti-forgetting corpus and the corresponding stored hidden state values. It uses the obtained positive and negative activation thresholds to calculate the threshold edge loss of the universal anti-forgetting samples during training and jointly optimizes it with the existing language model loss.

[0054] In a third aspect, a terminal device includes a memory, a processor, and a program of a continuous learning method for a large language model that is resistant to forgetting, which is stored in the memory and can be run on the processor. When the processor executes the program of the continuous learning method for a large language model that is resistant to forgetting, the steps of the continuous learning method for a large language model that is resistant to forgetting are implemented.

[0055] In a fourth aspect, a computer-readable storage medium stores a program for a method for continuous learning of a large language model that is resistant to forgetting. When the program for the method for continuous learning of a large language model that is resistant to forgetting is executed by a processor, the steps of the method for continuous learning of a large language model that is resistant to forgetting are implemented.

[0056] This constrained optimization method of the present invention enables the large language model to maintain its original general capabilities during continuous learning (including pre-training and fine-tuning) without affecting the learning effect of new tasks, thereby improving the anti-forgetting performance. The technical solution of the present invention is adopted, and the threshold edge loss is used to ensure that the model can maintain its general capabilities in the process of learning new tasks, avoiding catastrophic forgetting. At the same time, it can also better learn downstream tasks, and improve the application effect of the large language model in continuous learning of various vertical fields. The present invention reduces the cost of retraining large models with mixed data in continuous learning, including continuous pre-training and fine-tuning scenarios, and the maintenance of general capabilities can also promote the learning effect of downstream tasks. Ultimately, a large language model with strong general capabilities can be created to meet the needs of continuous iterative progress in artificial intelligence applications in real scenarios.

[0057] Example

[0058] This embodiment provides a method for continuous learning of a large language model that resists forgetting, including the following steps:

[0059] Step S1, construction of a general anti-forgetting sample set: Select samples that can represent the general capabilities of the model or samples generated by the initial large language model to form a general anti-forgetting sample set.

[0060] In this embodiment, the universal anti-forgetting sample set refers to the corpus that matches the domain to which the initial large language model belongs and can reflect the source of its universal ability learning. The universal anti-forgetting sample set is composed of a large number of corresponding universal samples, each of which is a natural language sentence.

[0061] In specific implementation, the definition of "universal" depends on the field that the initial large language model itself is targeting. For example: for the initial large language model in the general field, the general corpus can be any sentence obtained from Wikipedia; for the initial large language model in the financial field, it can be any sentence obtained from financial research reports and financial reports; for the initial large language model in the medical field, it can be any sentence obtained from medical textbooks. The most direct construction method is to randomly extract or carefully select from the training corpus of the initial large language model. The initial large language model can also be used to automatically generate corpus. During the construction process, data cleaning, length restriction, individual sample rewriting optimization and other post-processing are performed according to quality requirements. For example, when the length is restricted, the length of each sentence can be truncated according to the maximum length supported by the model for subsequent continuous pre-training or continuous fine-tuning. Use D f ={s1,s2,…,s n} represents the universal anti-forgetting corpus finally obtained, where s represents each sample sentence and n represents the total number of samples in the corpus.

[0062] Step S2, calculation of the hidden state value of the universal anti-forgetting sample: input each sample of the universal anti-forgetting sample set into the initial large language model, and calculate the hidden state value of each universal anti-forgetting sample.

[0063] In this embodiment, each universal anti-forgetting sample is input into the initial large language model for forward computation of the neural network. The hidden state value refers to the neuron activation value of each hidden layer of the sub-network in the neural network of the initial large language model during the forward computation of each universal anti-forgetting sample.

[0064] In specific implementation, the initial large language model structure is not limited to the Encoder, Decoder, Encoder-Decoder model, nor is it limited to the Self-attention network structure using Transformer, or network structures such as RNN and CNN. As long as it is a structure that conforms to the language model for sentence modeling. Taking Transformer as an example, the hidden state value refers to the network neuron activation value output by the last Feed-Forward network in each layer of the sub-network, which can be obtained by tensor Represents the hidden state tensor value obtained after each sample sentence is input, where n layer Indicates the number of layers of the neural network, n length Indicates the length of the sentence (ie, the number of tokens), d hidden Represents the hidden state dimension of the network. The general anti-forgetting corpus D in S1 f All n universal anti-forgetting samples get a total of n h f A collection of hidden state tensor values.

[0065] Step S3, distribution statistics of hidden state value sets: statistics the distribution of all hidden state value sets and calculate relevant statistical values.

[0066] In this embodiment, the hidden state value set refers to the corresponding hidden state value set obtained in step S2 for all or part of the samples in the universal anti-forgetting sample set. Statistical calculations are performed on the hidden state value set, using each sub-network layer and the hidden layer dimension of the sub-network as statistical calculation units to obtain statistical values of the hidden state of each sub-network layer and each neuron, including relevant statistics such as mean, variance, standard deviation, median, maximum value, and minimum value.

[0067] In specific implementation, given all n h obtained in S2 f A set of hidden state tensor values, one way to calculate the statistical value is: for each layer, with d hidden Dimension is the unit of calculation, in n, n length The two dimensions are combined to calculate the mean, variance, and standard deviation. The jth component value of the hidden state corresponding to the i-th token in the l-th layer is represented by the mean calculation formula of each layer as follows (where m1, var1, std1, etc. are omitted to indicate the subscript l of the layer):

[0068]

[0069] Similarly, the calculation formula of variance is as follows:

[0070]

[0071] Similarly, the calculation formula for standard deviation is as follows:

[0072]

[0073] Step S4, calculation of activation threshold: calculating a positive activation threshold and a negative activation threshold for threshold edge loss calculation according to the relevant statistical value.

[0074] In this embodiment, the statistics of the hidden state are converted into thresholds to obtain positive activation thresholds and negative activation thresholds. One implementation method for calculating the positive and negative activation thresholds is: the mean obtained in step S3 is shifted in the positive axis direction by 1 times the standard deviation to obtain the positive activation threshold. The negative activation threshold is the mean obtained in step S3 shifted in the negative axis direction by 1 times the standard deviation to obtain the negative activation threshold. Specifically, given the mean and standard deviation obtained in S3, for the general corpus D f Corresponding to d in each layer hidden The positive activation threshold and negative activation threshold of the hidden state of each component of the dimension are calculated as follows:

[0075]

[0076] in, is the positive activation threshold of the hidden state of each component, is the negative activation threshold of the hidden state of each component, thres j is the set of positive and negative activation thresholds for each component, For all the common d hidden The activation thresholds of the components.

[0077] It should be understood that the threshold calculation method described herein may try multiple standard deviations, such as 2, 3, etc., depending on the situation. The median may be used to replace the mean, and the maximum and minimum values may be used to calculate the numerical range.

[0078] Step S5, continuous learning of the large language model with hidden state activation preservation: the general anti-forgetting sample set is integrated with the original continuous learning data set, and the hidden state threshold constraint algorithm is used to perform continuous learning of the initial large language model, thereby improving the anti-forgetting ability of the initial large language model during the continuous learning process and maintaining general capabilities.

[0079] In this embodiment, during continuous learning, the general anti-forgetting samples and the original continuous learning training samples are integrated. First, the general anti-forgetting samples to be used are selected, including but not limited to directly using the full amount of general anti-forgetting corpus D f , or use a part that is carefully selected manually or randomly extracted. It is worth mentioning that this means that the full amount of universal anti-forgetting corpus is only needed when calculating the threshold information in S2. After calculating the threshold information, only part of the samples can be saved according to the specific situation to reduce storage overhead. Here, a part of 500 samples is randomly extracted as the implementation method for subsequent operations. Then, when mixing, each batch of training is used as a unit to ensure that a specific number of the universal anti-forgetting samples are inserted into each batch of training data. In terms of the selection of the number of insertions in each batch, one possible implementation method is to use a number that is a fixed ratio to the amount of batch data. For example: when the ratio r = 1 / 16 is selected, if the current batch size batch_size = 64, it means that batch_size × r = 4 universal anti-forgetting corpora will be inserted into each batch, and the remaining batch_size-4 = 60 are the original downstream task corpora. The 4 universal anti-forgetting samples inserted in each batch are further randomly sampled from the 500 samples. This implementation method is suitable for situations where the universal anti-forgetting corpus is several orders of magnitude less than the downstream task corpus. Another possible implementation method is to directly mix the 500 general anti-forgetting corpora with the downstream task corpora and disrupt their order. This implementation method is suitable for the case where the number of general anti-forgetting corpora and downstream task corpora is of the same order of magnitude.

[0080] In this embodiment, the threshold marginal loss of the universal anti-forgetting sample is calculated during continuous learning. Specifically, during the training process, the hidden state value obtained by forward calculation of the universal anti-forgetting sample appearing in each batch is extracted and used To represent the corresponding hidden state tensor of each sample. and h f The threshold edge loss proposed by the present invention is calculated In particular, the threshold edge loss calculation is not limited to considering the hidden state values of all layers of the model, i.e. l = 1, 2, ... n layer ; You can also consider only some of the layers, or the hidden state value of the last layer, that is, k = n layer For example, if only the last layer is selected, the following corresponding formula will use l = n layerThat is, the last layer is used and the identification of the number of layers is omitted. The specific calculation method of the threshold edge loss is as follows:

[0081]

[0082] Among them, tml i,j Represents the lth layer (l=n layer ) The threshold marginal loss of the jth component in the latent vector of the i-th token. Finally, the overall threshold marginal loss of each universal anti-forgetting sample is

[0083]

[0084] It should be understood that the threshold edge loss described here can be used in different ways depending on the situation, such as square edge threshold loss, as long as the calculation method ensures that the inactivation, positive activation, and negative activation of the hidden state value are consistent under a specific threshold constraint.

[0085] In this embodiment, during continuous learning, a normal continuous learning language model loss can use cross entropy loss. And the cross entropy loss when continuing to learn downstream tasks normally Perform dynamic balance. Use dynamic loss weight calculation to calculate the total loss of the two, balance the magnitude of the two losses, and avoid the optimization of the neural network being biased towards either side during continuous learning. The specific calculation formula is as follows:

[0086]

[0087] Here, α is the dynamic loss weight, which dynamically scales the magnitude of the threshold edge loss up or down to optimize it together with the magnitude of the cross entropy. detach(·) truncates the weight value during the model's backward gradient propagation, making it no longer part of the model's optimization target. is the final total loss of training.

[0088] This embodiment also discloses a large language model continuous learning system that resists forgetting, such as Figure 2 As shown, it includes:

[0089] The general anti-forgetting corpus construction module S100 is used to obtain and construct the general anti-forgetting sample set as the anti-forgetting corpus and pre-process the samples, including but not limited to corpus data cleaning, length restriction, error sample correction, sample optimization, sample encoding and indexing, etc.

[0090] Hidden state value distribution statistics calculation module S200 is used to load the initial large language model and input all common anti-forgetting samples for forward calculation to obtain all corresponding hidden state values, and perform statistics to obtain relevant statistical values including mean, variance, standard deviation, median, maximum value, minimum value, etc., and further calculate the positive and negative activation thresholds used to calculate the threshold edge loss.

[0091] The hidden state value set storage and indexing module S300 is used to store and index the hidden state values of all samples of the universal anti-forgetting corpus obtained by the hidden state value distribution statistics calculation module, so as to be read by the subsequent continuous learning module.

[0092] The continuous learning module S400 is used to read samples of the universal anti-forgetting corpus and the corresponding stored hidden state values, and use the obtained positive and negative activation thresholds to calculate the threshold edge loss of the universal anti-forgetting samples during the training process, and jointly optimize it with the existing language model loss.

[0093] In this embodiment, a flowchart of a large language model continuous learning system that resists forgetting is as follows: Figure 3 As shown. First, a universal anti-forgetting corpus is constructed and the corpus samples are preprocessed. Secondly, the initial large language model is loaded and the universal anti-forgetting corpus is input for forward calculation to obtain the corresponding set of all hidden state values. Then, the hidden state value set is statistically calculated to obtain relevant statistics. Further, the positive and negative activation thresholds for calculating the threshold edge loss are calculated based on the relevant statistics. Subsequently, all the above hidden state sets are stored and indexed. Finally, in the continuous learning process, the hidden state values are read and the positive and negative activation thresholds are used to calculate the threshold edge loss of the universal anti-forgetting samples during training, and jointly optimized with the existing cross entropy loss.

[0094] The method and system for continuous learning of a large language model that is resistant to forgetting in this embodiment can be deployed on various platforms or devices, and support continuous pre-training and fine-tuning of the large language model that is resistant to forgetting for downstream tasks.

[0095] This embodiment provides a computer-readable storage medium, which includes a computer program. When the computer program runs on an electronic device, the electronic device executes a large language model continuous learning method with anti-forgetting as described in any one of the above items.

[0096] This embodiment provides a terminal device, which includes a connected processor and memory, wherein the processor is used to execute a computer program stored in the memory to perform an anti-forgetting large language model continuous learning method as described in any of the above items.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for continuous learning of a large language model that resists forgetting, characterized in that: The steps include: Step S1, constructing a general anti-forgetting sample set: selecting samples that can represent the general capabilities of the model or samples generated by the initial large language model to form a general anti-forgetting sample set; Step S2, calculation of the hidden state value of the universal anti-forgetting sample: input each universal anti-forgetting sample in the universal anti-forgetting sample set into the initial large language model, and calculate the hidden state value of each universal anti-forgetting sample; Step S3, distribution statistics of hidden state value sets: statistics the distribution of all hidden state value sets and calculate relevant statistical values; Step S4, calculation of activation threshold: calculating the positive activation threshold and the negative activation threshold for threshold edge loss calculation based on the relevant statistical values; Step S5, continuous learning of the large language model with hidden state activation preservation: the general anti-forgetting sample set is integrated with the original continuous learning data set, and the hidden state threshold constraint algorithm is used to continuously learn the initial large language model, thereby improving the anti-forgetting ability of the initial large language model during the continuous learning process and maintaining its general capabilities.

2. The method for continuous learning of a large language model that resists forgetting according to claim 1, characterized in that: In step S1, the universal anti-forgetting sample set is composed of a large number of corresponding universal samples, each universal sample is a natural language sentence, and the construction method includes directly randomly extracting or carefully selecting from the training corpus of the initial large language model, or automatically generating the corpus using the initial large language model; during the construction process, data cleaning, length restriction, individual sample rewriting optimization and other post-processing are performed according to quality requirements.

3. The method for continuous learning of a large language model that resists forgetting according to claim 2, characterized in that: In step S2, each universal anti-forgetting sample is input into the initial large language model for forward calculation of the neural network, and the neuron activation value of each sub-network hidden layer is obtained as the hidden state value.

4. The method for continuous learning of a large language model that resists forgetting according to claim 3, characterized in that: In step S3, the distribution statistics of the hidden state value set are calculated, and each layer of the network sub-network and the hidden layer dimension of the sub-network are used as statistical calculation units to obtain the corresponding statistical values of the hidden state of each layer of the sub-network and each neuron, including the mean, variance, standard deviation, median, maximum value, and minimum value.

5. The method for continuous learning of a large language model that resists forgetting according to claim 4, characterized in that: In step S4, the statistical value of the hidden state is converted into an activation threshold to obtain a positive activation threshold and a negative activation threshold respectively; the positive activation threshold is the mean obtained in step S3 shifted in the positive direction by n times the standard deviation, and the negative activation threshold is the mean obtained in step S3 shifted in the negative direction by n times the standard deviation.

6. The method for continuous learning of a large language model that resists forgetting according to claim 5, characterized in that: In step S5, continuous learning is applicable to learning a single or multiple tasks, and is also applicable to full parameter learning or efficient parameter learning; during continuous learning, universal anti-forgetting samples and original continuous learning training samples are integrated; during integration, each batch of training is used as a unit to ensure that a specific number of universal anti-forgetting samples are inserted into each batch of training data; universal anti-forgetting samples are randomly extracted or carefully selected from the universal anti-forgetting corpus during training; in terms of quantity selection, it includes adopting a number of universal anti-forgetting samples inserted that is a fixed ratio to the batch data volume, or specifying a fixed number of universal anti-forgetting samples inserted that is independent of the batch data volume; or using a direct mixing method to directly mix the universal anti-forgetting samples with the task training samples that need to be continuously learned and randomly shuffle them.

7. The method for continuous learning of a large language model that resists forgetting according to claim 6, characterized in that: In step S5, during continuous learning, a threshold marginal loss is calculated for the universal anti-forgetting sample; a threshold marginal loss calculation method is used to calculate the corresponding loss for the universal anti-forgetting sample; and this loss is jointly optimized with the loss of the downstream task of continuous learning; the threshold marginal loss calculation object includes considering the hidden state values of all layers of the model, or only the hidden state values of the last layer; the threshold marginal loss calculation ensures that the hidden state values of the universal anti-forgetting sample are consistent with the hidden state values based on the initial large language model obtained in step S2 during the training process, with respect to whether they are activated or not, positively activated or negatively activated under a specific threshold constraint; Dynamically balance the threshold edge loss and the language model loss during normal continuous learning; use dynamic loss weight calculation method to calculate the total loss of the two to avoid the optimization of the neural network being biased towards either side during continuous learning.

8. A continuous learning system for a large language model that resists forgetting, characterized by: include: The universal anti-forgetting corpus construction module is used to obtain samples and construct a universal anti-forgetting sample set as the anti-forgetting corpus; and preprocess the universal anti-forgetting samples, including corpus data cleaning, length restriction, error sample correction, sample optimization, sample encoding and indexing; The hidden state value distribution statistics calculation module is used to load the initial large language model and input all common anti-forgetting samples for forward calculation to obtain all corresponding hidden state values. Statistics are then performed to obtain relevant statistical values including mean, variance, standard deviation, median, maximum, and minimum values, and to calculate the positive and negative activation thresholds used to calculate the threshold edge loss. Hidden state value set storage and indexing module, used to store and index the hidden state values of all samples of the universal anti-forgetting corpus obtained by the above-mentioned hidden state value distribution statistical calculation module, so as to be read by the subsequent continuous learning module; The continuous learning module is used to read samples of the universal anti-forgetting corpus and the corresponding stored hidden state values. It uses the obtained positive and negative activation thresholds to calculate the threshold edge loss of the universal anti-forgetting samples during training and jointly optimizes it with the existing language model loss.

9. A terminal device, characterized in that: The terminal device includes a memory, a processor, and a program of a continuous learning method for a large language model that is resistant to forgetting, which is stored in the memory and can be run on the processor. When the processor executes the program of the continuous learning method for a large language model that is resistant to forgetting, the steps of the continuous learning method for a large language model that is resistant to forgetting are implemented as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program of a continuous learning method for a large language model that is resistant to forgetting. When the program of the continuous learning method for a large language model that is resistant to forgetting is executed by a processor, the steps of the continuous learning method for a large language model that is resistant to forgetting as described in any one of claims 1 to 8 are implemented.

Images