Large model continuous learning method, device and equipment based on representation offset constraint

Abstract

The present application relates to a method, device and equipment for continuous learning of a large model based on characterization of offset constraints. The method comprises: obtaining a pre-trained large model and current newly added domain data; using a preset continuous learning strategy, original domain data and newly added domain data to continuously learn the pre-trained large model, and obtaining a large model trained in each preset stage; obtaining a set of detection task sets containing multiple detection task sets, using a pre-trained probe classifier to detect the large model trained in the current stage on each detection task set, calculating the offset loss based on the detection results of the current stage and the previous stage, and obtaining the total loss based on the offset loss and continuous learning loss between the preset stages. When the total loss converges, stop continuous learning to obtain a large model with domain enhancement; and use the domain enhanced large model to generate content in the domain. The use of this method can improve the reliability of model-generated content.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a large model continuous learning method, device and equipment based on characterization offset constraints. Background Art

[0002] Although general language models have achieved impressive results in commonsense question answering and dialogue, there are still significant challenges in curbing hallucinations and aligning models with human values. In particular, in specific application areas such as healthcare, law, and education, it is difficult to avoid the situation where the model has misperceptions. Early methods usually involve fine-tuning pre-trained language models using domain-specific datasets to enable the models to solve specific downstream tasks. However, since the emergence of ChatGPT, generative architectures have become the first choice for pre-trained models. These models are usually well pre-trained and contain billions of parameters, which ensures that they can generalize well to a variety of unseen tasks through natural language instructions. Directly fine-tuning these models usually leads to catastrophic forgetting or even performance degradation. Continual Learning (CL) is becoming an effective means to solve these problems. More and more research focuses on efficient parameter training to supplement domain knowledge and diachronic language changes and overcome catastrophic forgetting. Another advantage is that CL costs relatively small instruction datasets and computing resources.

[0003] As language models become too large to be interpreted, probes have become an important way to understand their internal vector representations. By hierarchically combining labeled vectors and then training them with a classifier, probes can be used to evaluate the model's fit to the data distribution. Due to its simplicity and intuitive nature, it has been widely used in many studies to explore the mechanics of how attention works. Recent studies have begun to use probes to explain why models forget during learning, thereby improving training procedures.

[0004] However, only a few studies have explored CL methods for recent large-scale language models (LLMs), including simple data mixing, instruction fine-tuning, and building reading comprehension data. Previous studies have mainly focused on CL methods for BERT series models, where the model size is 2 to 3 orders of magnitude smaller and is usually targeted at traditional natural language processing (NLP) tasks. Traditional continuous learning methods are mainly divided into memory replay-based methods, regularization-based methods, and structure-based methods according to their implementation methods. Among them, memory replay-based methods (MR) approximate or restore the distribution of historical task training data to replay historical task samples (such as original samples, generated pseudo samples, sample prototypes / features, etc.) in the training of new tasks to overcome catastrophic forgetting (CF). Regularization-based methods avoid large updates by adding penalty terms to the loss function to protect parameters that are important to historical tasks. Among regularization methods, knowledge distillation (KD) is particularly prominent. The distillation method adds similarity constraints on the output of the teacher model and the student model in the historical task data samples in the loss function to ensure that the functions of the student model and the teacher model are consistent. The structure-based method (Dynamic Structure, DS) dynamically expands the model structure to assign specific parameter blocks to each task to achieve parameter isolation and suppress catastrophic forgetting. Due to the huge number of parameters of large models, most of the continuous training methods of large models are inseparable from the structure-based method, that is, adding adapters. MBPA++ (Memory Based Parameter Adaption) proposes a situational memory model that continuously learns and reuses previously acquired knowledge through sparse experience replay and local adaptation. O-LoRA (Orthogonal Low-Rank Adaptation) is based on the assumption that the gradient subspace of historical tasks is concisely represented by LoRA parameters, and trains adapters separately for each task and keeps them orthogonal in the gradient subspace. LwF (Learning Without Forgetting) uses new task training data to perform knowledge distillation on model training and constrain important parameter updates.

[0005] At the same time, the exploration methods used in CL mainly evaluate the learning effect of the model and do not directly participate in the calibration of model parameters. That is, they often identify the stages and levels where forgetting occurs, but their effectiveness in mitigating model degradation is limited.

[0006] These methods are only incremental learning for tasks of traditional language models, such as enabling document classification models to learn entity extraction, document summarization, or Chinese-English translation tasks. However, current large-scale language models already have a large amount of general knowledge and generalization capabilities for common natural language processing tasks. The focus of continuous learning for large-scale language models is to supplement the model's knowledge in specific fields and update temporal knowledge while maintaining the general knowledge and generalization capabilities of large models. However, the above methods do not have ideal learning effects in such scenarios, and the model suffers from obvious catastrophic forgetting, which limits their application in updating knowledge of large-scale language models. Summary of the invention

[0007] Based on this, it is necessary to provide a large model continuous learning method, device and equipment based on characterization offset constraints to address the above technical problems.

[0008] A large model continuous learning method based on representation offset constraints, the method comprising:

[0009] Obtaining a pre-trained large model and current newly added domain data; the pre-trained large model is obtained by training with original domain data; the original domain data involves at least one domain;

[0010] The pre-trained large model is continuously learned using a preset continuous learning strategy, the original domain data, and the newly added domain data to obtain a large model trained in each preset stage;

[0011] A detection task set set including multiple detection task sets is obtained, and a pre-trained probe classifier is used to detect the large model trained in the current stage on each detection task set. The offset loss is calculated according to the detection results of the current stage and the previous stage, and the total loss is obtained according to the offset loss and the continuous learning loss between the preset stages. When the total loss converges, the continuous learning is stopped to obtain the domain-enhanced large model;

[0012] Content generation is performed in the domain using a large model enhanced by the domain.

[0013] A large model continuous learning device based on characterization deviation constraint, the device comprising:

[0014] A model acquisition module is used to acquire a pre-trained large model and current newly added domain data; the pre-trained large model is obtained by training with original domain data; the original domain data involves at least one domain;

[0015] A continuous learning module, used to continuously learn the pre-trained large model using a preset continuous learning strategy, the original domain data, and the newly added domain data, to obtain the large model trained in each preset stage;

[0016] An offset detection module is used to obtain a detection task set including multiple detection task sets, use a pre-trained probe classifier to detect the large model trained in the current stage on each detection task set, calculate the offset loss according to the detection results of the current stage and the previous stage, and obtain the total loss according to the offset loss and the continuous learning loss between the preset stages. When the total loss converges, stop the continuous learning to obtain the domain-enhanced large model;

[0017] The content generation module is used to generate content in the domain by using the large model enhanced by the domain.

[0018] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0019] Obtaining a pre-trained large model and current newly added domain data; the pre-trained large model is obtained by training with original domain data; the original domain data involves at least one domain;

[0020] The pre-trained large model is continuously learned using a preset continuous learning strategy, the original domain data, and the newly added domain data to obtain a large model trained in each preset stage;

[0021] A detection task set set including multiple detection task sets is obtained, and a pre-trained probe classifier is used to detect the large model trained in the current stage on each detection task set. The offset loss is calculated according to the detection results of the current stage and the previous stage, and the total loss is obtained according to the offset loss and the continuous learning loss between the preset stages. When the total loss converges, the continuous learning is stopped to obtain the domain-enhanced large model;

[0022] Content generation is performed in the domain using a large model enhanced by the domain.

[0023] The above-mentioned large model continuous learning method, device and equipment based on representation shift constraint, by obtaining the pre-trained large model and the current newly added domain data, using the preset continuous learning strategy, the original domain data and the newly added domain data to continuously learn the pre-trained large model, obtain the large model trained in each preset stage, obtain the detection task set set containing multiple detection task sets, use the pre-trained probe classifier to detect the large model trained in the current stage on each detection task set, calculate the shift loss according to the detection results of the current stage and the previous stage, and obtain the total loss according to the shift loss and the continuous learning loss between the preset stages. When the total loss converges, stop the continuous learning and obtain the domain-enhanced large model. The probe classifier detects the shift in the model vector representation, which can limit the shift of the internal representation of the large model, effectively retains the general knowledge learned during the model pre-training, thereby effectively alleviating the catastrophic forgetting of the model's universality and generalization, better realizes the domain knowledge supplementation of LLM, and uses the domain-enhanced large model to generate content in the domain. The embodiment of the present invention can improve the reliability of the model output content, generate high-quality and accurate content in a specific field, and meet the actual application needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of a flow chart of a large model continuous learning method based on characterization offset constraints in one embodiment;

[0025] Figure 2 A schematic diagram of a domain enhancement training framework in one embodiment;

[0026] Figure 3 A schematic diagram of a domain enhancement training process in an embodiment;

[0027] Figure 4 A schematic diagram of changes in three evaluation indicators after the detection loss is introduced in one embodiment;

[0028] Figure 5 It is a structural block diagram of a large model continuous learning device based on characterization offset constraint in one embodiment;

[0029] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0031] In one embodiment, Figure 1As shown, a large model continuous learning method based on representation offset constraints is provided, including the following steps:

[0032] Step 102, obtain the pre-trained large model and the current new domain data.

[0033] The pre-trained large model is obtained by training with original domain data. The original domain data involves at least one domain. When multiple domains are involved, the involved domains may include healthcare, law, and education. The domain data respectively contain a large amount of dialogue and question-answering data, knowledge retrieval data, and natural language processing task data including text classification, information extraction, and summarization. Experiments can verify that the method of the present invention still has good performance on cross-domain combined data sets.

[0034] Step 104, using a preset continuous learning strategy, original domain data, and newly added domain data to continuously learn the pre-trained large model, to obtain the large model trained in each preset stage.

[0035] Based on the characteristics of LLM, the present invention improves the traditional continuous learning method to adapt to the needs of continuous learning of large models, and based on this, sets a variety of continuous learning strategies. Each preset stage refers to each training stage with a preset training step length, and the interval step length between each training stage is designed according to actual needs. It can be understood that after obtaining the evaluation results of each training stage, the interval step length can be adjusted according to the evaluation results so that the training is carried out in a direction that is conducive to improving the evaluation results.

[0036] Step 106, obtain a detection task set set including multiple detection task sets, use a pre-trained probe classifier to detect the large model trained in the current stage on each detection task set, calculate the offset loss based on the detection results of the current stage and the previous stage, and obtain the total loss based on the offset loss and continuous learning loss between each preset stage. When the total loss converges, stop continuous learning to obtain a domain-enhanced large model.

[0037] We propose a simple yet effective probe method to constrain representational shifts in model parameters - the probe, which can reduce catastrophic forgetting and achieve better knowledge transfer by combining continuous learning loss and shift loss.

[0038] Representation shift focuses on the loss of general language capabilities of the test model, including common sense, grammatical knowledge, semantic parsing, sentiment analysis, and multilingual understanding. The detection task set includes two types of tasks: grammar and semantics. In an embodiment of the present invention, grammar tasks include bigram word order change (BShift), tree depth detection (TreeDepth) and subject count (SubjNum), and semantic tasks include tense detection (Tense), typo recognition (SOMO) and clause reversal (CoordInv).

[0039] Step 108, using the domain-enhanced big model to generate content in the domain.

[0040] Before applying the domain-enhanced large model, the method of the present invention uses improved evaluation indicators to thoroughly evaluate the accuracy changes during continuous learning, which is conducive to adjusting the learning parameters and further optimizing the training process.

[0041] In the above-mentioned large model continuous learning method based on representation shift constraint, by obtaining the pre-trained large model and the current newly added domain data, the pre-set continuous learning strategy, the original domain data and the newly added domain data are used to continuously learn the pre-trained large model, obtain the large model trained in each preset stage, obtain the detection task set set containing multiple detection task sets, use the pre-trained probe classifier to detect the large model trained in the current stage on each detection task set, calculate the shift loss according to the detection results of the current stage and the previous stage, and obtain the total loss according to the shift loss and the continuous learning loss between the preset stages. When the total loss converges, stop the continuous learning and obtain the domain-enhanced large model. The probe classifier detects the shift in the model vector representation, which can limit the shift of the internal representation of the large model, effectively retain the general knowledge learned during the model pre-training, thereby effectively alleviating the catastrophic forgetting of the model's universality and generalization, better realizes the domain knowledge supplementation of LLM, and uses the domain-enhanced large model to generate content in the domain. The embodiment of the present invention can improve the reliability of the output content of the model, generate high-quality and accurate content in a specific field, and meet the actual application needs.

[0042] In one embodiment, obtaining a pre-trained large model includes: obtaining a task sequence to be learned, and obtaining original domain data corresponding to each task in the task sequence to be learned; the original domain data includes multiple samples, and each sample is marked with a label; the large model is trained according to the task sequence to be learned and the corresponding original domain data to obtain a pre-trained large model.

[0043] In this embodiment, we first define a mathematical model for continuous learning of a large model. Given a series of task sequences to be learned , each task The corresponding training dataset is , in, Task Sample space, Task Label space, in Domain Incremental Learning (DIL), the data distribution between domains is independent but the label space is consistent, that is, CL's task is to Train a language model on , the model can correctly predict the label of any sample belonging to DT, that is, .

[0044] In one embodiment, Figure 2 Schematic diagram of the domain enhancement training framework shown, continuous learning strategies include memory replay-based methods, distillation methods, or structure-based methods.

[0045] In this embodiment, if Figure 3 The domain enhancement training process diagram shown in Figure 1 shows that continuous learning based on memory replay includes collecting samples from each domain corpus and replaying them at a predetermined frequency in the subsequent learning process. There are two ratios that need to be defined: the sampling ratio and playback ratio . Given the current dataset size is , you need to select The samples in , the samples selected from them are In the training process of the current dataset, each Replay the selected sample once.

[0046] Continuous learning using the distillation method involves: The model is used as the teacher model to complete the model of the current stage The distillation dataset consists of the same number of samples from the training sets of the two stages. Both models will provide predictions for the samples in the current batch, which are used to calculate the distillation loss. The training loss consists of two parts. One part is the traditional next word segmentation prediction loss. , and the other part is distillation loss , which is used to penalize the deviation of the model from the previous stage.

[0047] The structure-based approach to continuous learning includes: while the backbone structure is frozen, the model is extended with trainable adapters. In order to better overcome the forgetting of parameter gradient descent, more constraints are introduced. The present invention adopts the O-LoRA structure as a trainable adapter, trains adapters for each task separately, and keeps them orthogonal in the gradient subspace. For each domain learning, LoRA adds attention matrix to the pre-trained model. Add an edge matrix ,in , , In the inference process, we can directly use is added to the original parameters without incurring additional delay, i.e. At the same time, the present invention introduces the EWC algorithm, which adds an L2 penalty to the parameter of ΔW and updates the parameter according to the severity of the parameter. This achieves the isolation of the structural parameters added at each stage and reduces the modification of previously important parameters.

[0048] In one embodiment, when the pre-trained large model is continuously learned using the distillation method, the continuous learning loss used is:

[0049]

[0050] in, To continuously learn the loss, is the word segmentation prediction loss, is the weight factor of the distillation error, is the distillation loss, , is an empirical parameter used to control the proportion of accuracy and recall. and are the lengths of the output sentences of the two-stage models, Output the longest common subsequence of sentences to the model.

[0051] In this embodiment, due to It is already the maximum likelihood estimate of the generated word, so the cross entropy and KL divergence commonly used in early distillation learning will not introduce new error terms, especially when the teacher model and the standard output are consistent, both will degenerate into Therefore, the ROUGE-L metric is selected as the distillation error, which is used to reduce the overall deviation between the output sentences of the two models and pays more attention to the content and completeness of the information between the compared items than the BLEU metric.

[0052] In one embodiment, using a pre-trained probe classifier to detect the large model trained in the current stage on each detection task set includes: freezing the parameters of the large model trained in the current stage, connecting the pre-trained detection classifier to each layer of the large model to be detected; extracting the representation vector of each layer, inputting the representation vector into the corresponding connected detection classifier, and the probe classifier performs classification prediction based on the input representation vector to obtain the detection result on the current detection task set; the detection task set includes semantic tasks and grammatical tasks.

[0053] In this embodiment, model degradation is an overall process. The representation changes at different levels of the model will accumulate a significant impact on the output layer. The method of the present invention integrates the vector representation shift of the middle layer of the model as part of the loss function to prevent excessive deviation of the weights from causing catastrophic forgetting in the model. The first step in constructing a vector representation loss function is to design a corresponding detection method. Due to the sparsity of high-dimensional vector space, it is difficult to directly compare two vectors. The proxy task is designed as Conduct detection. After training, the corresponding model is obtained , by freezing the parameters and connecting a classifier composed of MLP, training and obtaining the prediction of the detection classifier. The change in classification prediction accuracy is used to measure the degree of forgetting of the current model. The detection result on is defined as:

[0054] in, Representation Model In order to comprehensively consider the influence of displacement errors at different levels, the linear regression method is used for weighted calculation. The weight parameter of the Nth layer vector is defined as , and calculate the SoftMax probability of these feature vectors as output. The same method is also applied to the models trained in the previous stage to calculate their MSE loss (Mean Squared Error). Then, through learning, the task set can be obtained The optimal weights of the upper layers:

[0055] .

[0056] In addition, in the The representation vector extracted by the layer is Since LLMs are all autoregressive generative models, they are different from encoder language models that use the [CLS] token inserted at the beginning of the sentence as the representation vector of the input sentence. The generative model uses left padding and causal attention mechanisms, and the token at the beginning of the sentence cannot capture the semantics of the entire sentence. Therefore, we choose to use the average of all word segmentation vectors in the current layer as the feature vector.

[0057] In one embodiment, calculating the offset loss according to the detection results of the current stage and the previous stage includes: calculating the offset loss according to the detection results of the current stage and the previous stage; the offset loss is:

[0058]

[0059] in, is the offset loss, is the number of detection task sets, is the transpose operation, For the detection task set The optimal weights of the upper layers, For the detection task set Upper model The detection results, For Model The number of layers to be detected, The multi-layer perceptron classifier used for detection is the one that uses the The model trained in the stage is used for the detection task set Make a prediction, in The output vector of the layer, is the L2 norm.

[0060] In this embodiment, considering that there is no direct relationship between the data distributions of a single task set, the present invention selects the root mean square error (RMSE) as the evaluation result of the detection between multiple task sets. , that is, the offset loss.

[0061] In one embodiment, the total loss is:

[0062]

[0063] in, is the total loss, To continuously learn the loss, is the scaling empirical parameter, is the offset loss.

[0064] In this embodiment, is the cross entropy loss function used in the CL process. As mentioned above, when the distillation method is used to continuously learn the pre-trained large model, .

[0065] In one embodiment, the method also includes: constructing evaluation data sets for the large models trained in each preset stage respectively, testing the large models trained in the corresponding stage on each evaluation data set, evaluating the test results using pre-set evaluation indicators to obtain corresponding evaluation results; and adjusting the learning parameters of continuous learning according to the evaluation results.

[0066] In this embodiment, for each learning stage, there is a corresponding evaluation data set , each time the model completes a stage of training, it is tested on each evaluation dataset. Definition After the model is trained on the tth task, the test set of the i-th task The accuracy score on It is the accuracy score of the model on the i-th task before starting CL. There are three specific evaluation indicators:

[0067]

[0068]

[0069]

[0070] BWT examines the degree of forgetting of the model during the learning process, and FWT examines the degree of knowledge inheritance and generalization of the model. FWT is defined as the change in the prediction accuracy of the currently learned model on unknown tasks to the model that has not undergone CL during the sequential learning process. Since it is a forward transfer to a non-learning task, the final stage T is not considered. The larger the FWT value, the higher the knowledge transfer efficiency; the negative FWT value inhibits the knowledge transfer efficiency. Similarly, BWT represents the prediction accuracy of the model for the i-th task that has been learned in the current j-th stage, compared with the change in prediction accuracy immediately after the learning task is completed. The larger the BWT value, the more it indicates that the current stage of learning has a beneficial effect on past knowledge, and a negative value indicates catastrophic forgetting. The third indicator is the average accuracy. After the model completes learning on each incremental task in turn, the average accuracy of the model on all test sets is calculated to evaluate the overall performance of the CL algorithm.

[0071] In order to achieve a higher average accuracy in all evaluation tasks, a relatively balanced learning strategy is required. Traditional methods only consider the bidirectional knowledge transfer after the model completes all CL tasks, and only retain , where the change in model speed during the CL process is ignored. Such a setting also makes it difficult to observe the impact of different learning tasks on model performance at each stage. Therefore, the present invention proposes to calculate the average accuracy of each stage, taking into account the speed of change of model accuracy and the importance of different learning tasks. In addition, the present invention also adopts two metrics to evaluate the representation shift of the model using the detection method. By dividing the detection task into grammatical tasks and semantic tasks, these two indicators are used to evaluate the corresponding functions respectively:

[0072]

[0073]

[0074] in, Represents the final hidden layer feature vector of the initial model without CL for the detection task These two indicators calculate the average proportion of the accuracy drop of the detection task during the CL process, which can measure the degree and speed of the model capability drop under different CL algorithms.

[0075] In a specific embodiment, in order to verify the effectiveness of the continuous learning strategy on large language models (LLMs), the present invention selects three fine-tuning datasets from the fields of law, finance, and medicine as continuous learning corpora. These are the DISC-Law (403k), DISC-Med (470k), and DISCFin (250k) datasets. Due to the span of the domains, there is no overlap between the three datasets {D1, D2, D3} and they can be used directly for training without the need for a data screening step. In order to construct the test data, 1,000 natural language processing task sets and question-answering datasets were randomly selected from each original dataset as domain task datasets. In order to keep the knowledge learned from the previous stage, A subset was extracted from the training set and added to the semantic task. Since the SentEval dataset is relatively large, a small number of samples is sufficient to meet the detection needs. Considering computational efficiency, 1200 samples of each task type were randomly extracted and translated, accounting for only 1% of the original SentEval dataset size. The optimizer used for model training is AdamW, the batch size is 32, the learning rate is set to 2e-5, the warm-up ratio is 0.1, and a linear scheduler is used. The maximum length of the context window is 2048, and each dataset is trained for 3 cycles. The total loss is set = 0.05. The representation shift loss of the model under different continuous learning strategies is shown in Table 1:

[0076] Table 1 Representation shift loss of the model under different continuous learning strategies

[0077]

[0078] SynF and SemF represent the decline ratio of the model in syntactic and semantic tasks, respectively, compared with before continuous learning. The larger the value, the greater the decline in accuracy and the more serious the deviation. D1, D2 and D3 in the upper part of the table represent three training stages. It can be seen that the deviation of the model in semantic tasks is significantly lower than that in syntactic tasks. This is because the continuous learning tasks of the model are all knowledge data in the field, involving less grammatical text corpus, which leads to a significant decline in the accuracy of syntactic tasks when the model is evaluated at each stage. In particular, sequential learning , performing the worst on both tasks. In contrast, the MR strategy performs the best among all methods. The differences between the other methods are not large, with the syntactic task losing nearly 20% in the final stage, while the semantic task is close to 10%. This is because the test task setting of the present invention includes a portion of replay data, which is naturally suitable for MR, allowing the model to maintain the minimum vector representation shift. Finally, it can also be seen that as the learning stage deepens, the overall degree of deviation of the model gradually increases, but the only exception is the D3 stage, where some methods such as , MR, and DS show a downward trend. This is because the financial domain knowledge used in this stage includes some mathematical calculations and time series analysis corpus, which is similar to some of the content in the semantic task. This also verifies the generalization ability of the language model, that is, learning domain knowledge will enhance the reasoning performance of similar tasks.

[0079] Next, the present invention examines the representation vectors of different combinations. The results in the upper half of Table 1 are calculated using the average vector of all word segmentations. In the lower half, the [EOS] tag vector and the maximum pooling of the tag vector are added for comparison, denoted as Avg, EOS, and Max, respectively. The data shown in each row is the average score calculated in three stages. The reason for these two enhancements is that the [EOS] tag is usually considered to focus on all tags of the entire input, while the maximum pooling stores the maximum value from each dimension, which can capture the most significant features. It can be seen that EOS has the worst performance, while Max has the best performance. The results show that the feature vector extraction method has a significant impact on the accuracy of the representation shift. Among them, the methods that consider all tag vectors (Avg and Max) are better than the methods that use only a single tag vector, and the maximum pooling can avoid the feature attenuation caused by the average method. Therefore, it can lead to lower accuracy loss.

[0080] The schematic diagram of the changes of the three evaluation indicators after the introduction of detection loss is as follows Figure 4 Each subfigure shows four methods in turn: , MR, KD and DS before and after. However, the other two single-step models and It is essentially a one-time training method, so it is impossible to integrate multi-stage representation shift loss. In the three-stage learning process, there is no BWT in the first stage and no FWT in the last stage. It can be observed that after the introduction of the detection constraint, the accuracy and knowledge transfer ability of the model in each stage have been improved, ranging from 0.99% to 6.85%. The degree of gain of the representation shift on different continuous learning algorithms is different. It can be seen that KD has the best gain effect, especially in FWT, where the improvement is usually more than 6%. However, the improvement obtained by DS by introducing detection is relatively limited, with an improvement of less than 2% in the three indicators. This is because the way the representation shift constrains the model weights is similar to MR, which mainly updates the model through historical data. Therefore, it is equivalent to superimposing the effect of MR, while DS itself limits the range of parameters that can be updated, resulting in limited improvement. It is worth noting that the improvement effect of introducing detection is also related to the upper limit of the model's ability. For example, errors caused by some overly difficult problems or knowledge that the model has not learned are inevitable, which will make the improvement of detection smaller when it is close to the upper limit and more significant at a lower level.

[0081] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0082] In one embodiment, Figure 5 As shown, a large model continuous learning device based on characterization offset constraint is provided, comprising:

[0083] Model acquisition module 502, used to acquire a pre-trained large model and current newly added domain data; the pre-trained large model is obtained by training with original domain data; the original domain data involves at least one domain;

[0084] The continuous learning module 504 is used to continuously learn the pre-trained large model using a preset continuous learning strategy, original domain data, and newly added domain data to obtain the large model trained in each preset stage;

[0085] The offset detection module 506 is used to obtain a detection task set including multiple detection task sets, use a pre-trained probe classifier to detect the large model trained in the current stage on each detection task set, calculate the offset loss according to the detection results of the current stage and the previous stage, and obtain the total loss according to the offset loss and the continuous learning loss between each preset stage. When the total loss converges, stop the continuous learning to obtain the domain enhanced large model;

[0086] The content generation module 508 is used to generate content in the domain using the domain-enhanced big model.

[0087] In one of the embodiments, it is also used to freeze the parameters of the large model trained in the current stage, connect the pre-trained detection classifiers to each layer to be detected in the large model; extract the representation vector of each layer, input the representation vector into the corresponding connected detection classifier, and the probe classifier performs classification prediction based on the input representation vector to obtain the detection result on the current detection task set; the detection task set includes semantic tasks and grammatical tasks.

[0088] In one embodiment, it is also used to calculate the offset loss according to the detection results of the current stage and the previous stage; the offset loss is:

[0089]

[0090] in, is the offset loss, is the number of detection task sets, is the transpose operation, For the detection task set The optimal weights of the upper layers, For the detection task set Upper model The detection results, For Model The number of layers to be detected, The multi-layer perceptron classifier used for detection is the one that uses the The model trained in the stage is used for the detection task set Make a prediction, in The output vector of the layer, is the L2 norm.

[0091] In one embodiment, the total loss is:

[0092]

[0093] in, is the total loss, To continuously learn the loss, is the scaling empirical parameter, is the offset loss.

[0094] In one of the embodiments, the continuous learning strategy includes a memory replay-based method, a distillation method, or a structure-based method.

[0095] In one embodiment, when continuously learning a pre-trained large model using a distillation method, the continuous learning loss used is:

[0096]

[0097] in, To continuously learn the loss, is the word segmentation prediction loss, is the weight factor of the distillation error, is the distillation loss, , is an empirical parameter used to control the proportion of accuracy and recall. and are the lengths of the output sentences of the two-stage models, Output the longest common subsequence of sentences to the model.

[0098] In one of the embodiments, it is also used to construct evaluation data sets for the large models trained in each preset stage, test the large models trained in the corresponding stage on each evaluation data set, evaluate the test results using pre-set evaluation indicators, and obtain corresponding evaluation results; and adjust the learning parameters of continuous learning according to the evaluation results.

[0099] In one of the embodiments, it is also used to obtain a sequence of tasks to be learned, and to obtain original domain data corresponding to each task in the sequence of tasks to be learned; the original domain data includes multiple samples, and each sample is marked with a label; the large model is trained according to the sequence of tasks to be learned and the corresponding original domain data to obtain a pre-trained large model.

[0100] For the specific limitations of the large-model continuous learning device based on the representation offset constraint, please refer to the limitations of the large-model continuous learning method based on the representation offset constraint above, which will not be repeated here. Each module in the above-mentioned large-model continuous learning device based on the representation offset constraint can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0101] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a large model continuous learning method based on characterizing offset constraints is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0102] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0103] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0104] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0105] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A large model continuous learning method based on representation offset constraints, characterized in that: The method comprises: Obtaining a pre-trained large language model and current newly added domain data; the pre-trained large language model is obtained by training with original domain data; the original domain data involves at least one domain; the domain data includes dialogue and question-answering data, knowledge retrieval data, and natural language processing task data; the natural language processing task data includes text classification, information extraction, and summarization; Using a preset continuous learning strategy, the original domain data, and the newly added domain data to continuously learn the pre-trained large language model, to obtain the large language model trained in each preset stage; A detection task set set including multiple detection task sets is obtained, and a pre-trained probe classifier is used to detect the large language model trained in the current stage on each detection task set, and the offset loss is calculated according to the detection results of the current stage and the previous stage, and the total loss is obtained according to the offset loss and the continuous learning loss between the preset stages. When the total loss converges, the continuous learning is stopped to obtain a domain-enhanced large language model; the detection task set includes semantic tasks and grammatical tasks; the grammatical tasks include binary word order change detection, tree depth detection and subject counting; the semantic tasks include tense detection, typo recognition and clause reversal detection; Using the domain-enhanced large language model to generate content in the domain; The use of the pre-trained probe classifier to detect the large language model trained in the current stage on each detection task set includes: Freeze the parameters of the large language model trained in the current stage, and connect the pre-trained detection classifier to each layer of the large language model to be detected; The representation vector of each layer is extracted, and the representation vector is input into the corresponding connected detection classifier. The probe classifier performs classification prediction according to the input representation vector to obtain the detection result on the current detection task set.

2. The method according to claim 1, characterized in that The offset loss calculated based on the detection results of the current stage and the previous stage includes: The offset loss is calculated based on the detection results of the current stage and the previous stage; the offset loss is: in, is the offset loss, is the number of detection task sets, is the transpose operation, For the detection task set The optimal weights of the upper layers, For the detection task set Upper model The detection results, For Model The number of layers to be detected, The multi-layer perceptron classifier used for detection is the one that uses the The model trained in the stage is used for the detection task set Make a prediction, in The output vector of the layer, is the L2 norm.

3. The method according to claim 1, characterized in that The total loss is: in, is the total loss, To continuously learn the loss, is the scaling empirical parameter, is the offset loss.

4. The method according to claim 1, characterized in that: Continuous learning strategies include memory replay-based methods, distillation methods, or structure-based methods.

5. The method according to any one of claims 1 or 4, characterized in that: When the pre-trained large language model is continuously learned using the distillation method, the continuous learning loss used is: in, To continuously learn the loss, is the word segmentation prediction loss, is the weight factor of the distillation error, is the distillation loss, , is an empirical parameter used to control the proportion of accuracy and recall. and are the lengths of the output sentences of the two-stage models, Output the longest common subsequence of sentences to the model.

6. The method according to claim 1, characterized in that The method further comprises: Construct evaluation data sets for the large language models trained in each preset stage, test the large language models trained in the corresponding stage on each evaluation data set, evaluate the test results using the preset evaluation indicators, and obtain corresponding evaluation results; The learning parameters of the continuous learning are adjusted according to the evaluation results.

7. The method according to claim 1, characterized in that Get the pre-trained large language model including: Obtain a sequence of tasks to be learned, and obtain original domain data corresponding to each task in the sequence of tasks to be learned; the original domain data includes multiple samples, and each sample is marked by a label; The large language model is trained according to the task sequence to be learned and the corresponding original domain data to obtain a pre-trained large language model.

8. A large model continuous learning device based on representation offset constraints, characterized in that: The device comprises: A model acquisition module is used to acquire a pre-trained large language model and current newly added domain data; the pre-trained large language model is obtained by training with original domain data; the original domain data involves at least one domain; the domain data includes dialogue and question-answering data, knowledge retrieval data, and natural language processing task data; the natural language processing task data includes text classification, information extraction, and summarization; A continuous learning module, used to continuously learn the pre-trained large language model using a preset continuous learning strategy, the original domain data, and the newly added domain data, to obtain the large language model trained in each preset stage; An offset detection module is used to obtain a detection task set set including multiple detection task sets, use a pre-trained probe classifier to detect the large language model trained in the current stage on each detection task set, calculate the offset loss according to the detection results of the current stage and the previous stage, and obtain the total loss according to the offset loss and the continuous learning loss between the preset stages. When the total loss converges, stop the continuous learning to obtain a domain-enhanced large language model; the detection task set includes semantic tasks and grammatical tasks; the grammatical tasks include binary word order change detection, tree depth detection and subject counting; the semantic tasks include tense detection, typo recognition and clause reversal detection; A content generation module, configured to generate content in the domain using the domain-enhanced large language model; The offset detection module is also used to freeze the parameters of the large language model trained in the current stage, connect the pre-trained detection classifier to each layer of the large language model to be detected; extract the representation vector of each layer, input the representation vector into the corresponding connected detection classifier, and the probe classifier performs classification prediction based on the input representation vector to obtain the detection result on the current detection task set.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images