Response method and device of large language model based on incremental learning and program product

Abstract

The invention provides a large language model response method and device based on incremental learning and a program product, and the method comprises the steps: searching a corresponding knowledge sample in a vector database according to an obtained user input text; wherein the vector database is firstly constructed according to historical question and answer pair knowledge samples of the large language model and text knowledge samples of other sources; comprehensively scoring the text input by the user based on the unmatched corresponding knowledge sample to obtain an importance score, creating an adapter based on the fact that the importance score is greater than a first preset threshold value, and training the adapter in combination with a pre-trained large language model; and according to the text input by the user, determining a corresponding reply by utilizing the trained adapter and combining with the large language model. According to the method, the smoothness and the high efficiency of answering are ensured, the latest and most accurate knowledge points are injected, the calculation cost is greatly reduced, high-quality and personalized answering is provided, and personalized knowledge service is realized.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a response method and program product for a large language model based on incremental learning. Background Technology

[0002] Large language models represent a core technological achievement in the field of artificial intelligence, based on the Transformer architecture built upon deep neural networks containing hundreds of billions of parameters. Through pre-training on massive amounts of text and code data, these models learn rich linguistic patterns, world knowledge, and logical reasoning abilities, enabling them to perform complex tasks such as text generation, summarization and translation, and coding. However, the pre-trained model is a general-purpose foundational model; further fine-tuning is necessary to make it proficient in specific domains or adapt to the style of specific users. How to efficiently and economically update and optimize these models, enabling them to continuously learn new knowledge without forgetting old skills, has become a key challenge determining the breadth and depth of their applications.

[0003] To address the need for model updates, the current main approach is full retraining. The core idea is that whenever new knowledge or data needs to be incorporated into the model, historical data and new data are merged into a completely new dataset, and then the entire large language model is retrained from scratch using this merged dataset.

[0004] However, the aforementioned full retraining method requires retraining the entire model on the complete dataset for each update. For large language models with hundreds of billions or even trillions of parameters, a single training session can take several days to weeks, consuming a large amount of GPU computing resources and incurring extremely high computational costs. It cannot quickly respond to the addition of new data and is unsuitable for applications requiring near real-time updates. Furthermore, it requires permanently storing all historical training data, and storage costs will increase linearly over time. For continuously operating systems, the data volume may reach TB or even PB levels, resulting in a heavy storage burden. In addition, the full retraining process for large-scale models is extremely unstable, prone to convergence issues or performance fluctuations. It requires significant manpower for repeated hyperparameter adjustments and experiments, further increasing time and economic costs, making the entire update process inefficient and risky. Summary of the Invention

[0005] This invention provides a response method, device, and program product for a large language model based on incremental learning, which solves the shortcomings of existing technologies where full retraining of large language models cannot respond quickly and in real time to the addition of new data and has a high computational cost. It ensures the fluency and efficiency of the response, while injecting the latest and most accurate knowledge points, significantly reducing the computational cost, providing high-quality and personalized responses, and realizing personalized knowledge services.

[0006] This invention provides a response method for a large language model based on incremental learning, comprising: searching for corresponding knowledge samples in a vector database based on the acquired user input text; wherein the vector database is constructed in advance based on the historical question-and-answer knowledge samples of the large language model and text knowledge samples from other sources; based on the absence of a matching knowledge sample, comprehensively scoring the user input text to obtain an importance score; based on the importance score being greater than a first preset threshold, creating an adapter and training the adapter in conjunction with a pre-trained large language model; and determining the corresponding response based on the user input text, using the trained adapter, and in conjunction with the large language model.

[0007] According to the present invention, a response method for a large language model based on incremental learning includes creating an adapter based on an importance score greater than a first preset threshold, and training the adapter in conjunction with the large language model. The method comprises: creating an adapter based on an importance score greater than the first preset threshold and according to a preset adapter rank, and encapsulating the loading, inference, and fine-tuning interfaces of the large language model; loading the large language model according to the adapter, injecting the adapter into the target layer of the large language model according to a preset scaling factor, and freezing the original model parameters; constructing a training sample dataset from samples extracted from a data buffer using a preset sampling algorithm, and incrementally learning the adapter in conjunction with the large language model and the injected adapter; wherein the data buffer is constructed prior to samples and their corresponding real answers using experience replay technology, and the samples include historical questions or previously obtained knowledge text.

[0008] According to the present invention, a response method for a large language model based on incremental learning is provided. This method utilizes a preset sampling algorithm to extract samples from a data buffer to construct a fine-tuning dataset. It then combines the large language model and the large language model of the injected adapter to perform incremental learning on the adapter. The method includes: constructing a training sample dataset using samples extracted from the data buffer using the preset sampling algorithm; and performing incremental learning on the adapter according to a first preset iteration strategy, based on the training sample dataset and combining the large language model of the large language model and the large language model of the injected adapter. The first preset iteration strategy is used to: for each iteration, based on the training sample dataset, obtain the predicted response of the original model and the predicted response of the updated model using the large language model and the large language model of the injected adapter, respectively; construct a task loss function based on the predicted response of the updated model and the true answer of the corresponding sample, and construct a distillation loss function based on the predicted response of the original model and the predicted response of the updated model; obtain a first total loss function based on the task loss function and the distillation loss function, and perform backward gradient updates on the adapter based on the first total loss function and iterate again until the maximum number of iterations is reached, at which point training ends.

[0009] According to the response method of a large language model based on incremental learning provided by the present invention, after obtaining the importance score, the method further includes: obtaining the pre-trained adapter corresponding to the knowledge matched in the vector database based on the importance score being less than or equal to a first preset threshold and greater than a second preset threshold; loading the corresponding large language model; injecting the adapter into the target layer of the large language model according to a preset scaling factor; and freezing the original model parameters; constructing a fine-tuning sample dataset by using samples extracted from the data buffer using a preset sampling algorithm; and updating the adapter by combining the large language model and the large language model into which the adapter is injected; wherein, the data buffer is constructed in advance based on samples and the real answers corresponding to the samples using experience replay technology, and the samples include historical questions or previously obtained knowledge text; and delaying learning based on the importance score being less than or equal to the second preset threshold.

[0010] According to the present invention, a response method for a large language model based on incremental learning is provided. This method utilizes a preset sampling algorithm to extract samples from a data buffer, constructs a fine-tuning sample dataset, and updates the adapter by combining the large language model and the large language model of the injection adapter. The method includes: constructing a fine-tuning sample dataset by extracting samples from a data buffer using a preset sampling algorithm; updating the adapter according to the fine-tuning sample dataset and the large language model of the injection adapter, following a second preset iteration strategy; wherein the second preset iteration strategy is used to: for each iteration, based on the fine-tuning sample dataset, utilize the large language model... The large language model with the injected adapter is used to obtain the predicted response of the original model and the predicted response of the updated model. Based on the predicted response of the updated model and the true answer of the corresponding sample, a task loss function is constructed. Based on the predicted response of the original model and the predicted response of the updated model, a distillation loss function is constructed. Based on the current adapter parameters and the parameters of the previously trained version of the adapter, a regularization term is constructed. Based on the task loss function, the distillation loss function and the regularization term, a second total loss function is obtained. The adapter is then updated by backpropagation gradient based on the second total loss function and iterated again until the maximum number of iterations is reached, at which point the training ends.

[0011] According to the response method of a large language model based on incremental learning provided by the present invention, before constructing a corresponding training sample dataset or fine-tuning sample dataset by sampling samples extracted from a data buffer using a preset sampling algorithm, the method includes: constructing new samples based on previously acquired user input text, combined with the response obtained by the large language model after the corresponding injection adapter and the corresponding task identifier; adding new samples to the data buffer when it is determined that the data buffer capacity is not full; wherein, the data buffer is constructed based on a preset capacity; when it is determined that the data buffer capacity is full, determining the similarity between the new samples and each sample stored in the data buffer, selecting the sample with the highest similarity for removal, and adding the new samples to the data buffer.

[0012] According to the present invention, a response method for a large language model based on incremental learning, after determining the corresponding response based on user input text using a trained adapter and a large language model, includes: updating the vector database with the user input text and the corresponding response as new knowledge samples based on the lack of matching knowledge samples in the vector database; determining the relative entropy KL divergence between the predicted response of the original model and the predicted response of the updated model based on the matching knowledge samples in the vector database; determining a conflict between the new knowledge samples composed of the user input text and the corresponding response and the corresponding matching knowledge samples when the KL divergence is greater than a preset divergence threshold; classifying the conflict between the new knowledge samples and the corresponding matching knowledge samples to determine the conflict type and corresponding confidence level; fusing the new knowledge samples and the corresponding matching knowledge samples according to the conflict type and corresponding confidence level, and updating the vector database according to a preset knowledge fusion strategy; wherein, the preset knowledge fusion strategy is configured in advance based on different conflict types and confidence levels; and updating the vector database with new knowledge samples when the KL divergence is less than or equal to the preset divergence threshold.

[0013] According to the present invention, a response method for a large language model based on incremental learning is provided. Based on the conflict type and corresponding confidence level, and following a preset knowledge fusion strategy, the method fuses newly added knowledge samples with corresponding matching knowledge samples to update a vector database. The method includes: aligning the newly added knowledge samples and corresponding matching knowledge samples in the representation space when the conflict type is complementary; updating the vector database with the newly added knowledge samples when the conflict type is contradictory, while retaining the corresponding matching knowledge samples in the vector database; replacing the corresponding matching knowledge samples in the vector database with the newly added knowledge samples when the corresponding confidence level is greater than a first preset confidence threshold when the conflict type is substitutional; updating the vector database with the newly added knowledge samples when the confidence level is less than or equal to the first preset confidence threshold and greater than a second preset confidence threshold, while retaining the corresponding matching knowledge samples in the vector database; and retaining the corresponding matching knowledge samples in the vector database when the confidence level is less than or equal to the second preset confidence threshold.

[0014] This invention also provides a response device for a large language model based on incremental learning, comprising: a knowledge addition judgment module, which searches for corresponding knowledge samples in a vector database based on the acquired user input text; wherein the vector database is constructed in advance based on the historical question-and-answer pairs of the large language model and text knowledge samples from other sources; an adapter configuration module, which performs a comprehensive score on the user input text based on the absence of a matching knowledge sample to obtain an importance score, creates an adapter based on the importance score being greater than a first preset threshold, and trains the adapter in conjunction with the pre-trained large language model; and a response module, which determines the corresponding response based on the user input text, using the trained adapter and the large language model.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a response method for a large language model based on incremental learning as described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the response method of a large language model based on incremental learning as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a response method for a large language model based on incremental learning as described above.

[0018] The present invention provides a response method, apparatus, and program product based on an incremental learning-based large language model. By utilizing the acquired user input text, a vector database is searched to determine whether the user input text represents new knowledge. If no corresponding knowledge sample is found, the corresponding user input text is comprehensively scored. Based on the obtained importance score, a new adapter is created for the user input text. This new adapter is then trained in conjunction with the large language model to freeze the large language model parameters. Training only the adapter significantly reduces computational costs, protects the general capabilities and existing knowledge already learned by the large language model, prevents catastrophic forgetting, and allows new knowledge to be integrated in a non-intrusive manner through the adapter, thereby ensuring the overall stability and reliability of the system. Using the trained adapter, combined with the large language model, a rapid response to user input text is generated. This ensures that the final response combines the general capabilities of the base model with the professional knowledge of the newly trained adapter, guaranteeing both fluency and efficiency while incorporating the latest and most accurate knowledge points, providing high-quality, personalized responses, and achieving personalized knowledge services. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is one of the flowcharts illustrating the response method of a large language model based on incremental learning provided by this invention; Figure 2 This is the second flowchart of the response method for a large language model based on incremental learning provided by this invention; Figure 3 This is the third flowchart of the response method for a large language model based on incremental learning provided by this invention; Figure 4 This is a schematic diagram of the structure of the response device of the large language model based on incremental learning provided by the present invention; Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] Figure 1 This is a flowchart illustrating the response method of a large language model based on incremental learning provided by the present invention, as shown below. Figure 1 As shown, the method includes: S11, based on the obtained user input text, search for the corresponding knowledge sample in the vector database; wherein, the vector database is constructed in advance based on the historical question-and-answer knowledge samples of the large language model and text knowledge samples from other sources; S12, based on the lack of matching knowledge samples, the user input text is comprehensively scored to obtain an importance score. Based on the importance score being greater than the first preset threshold, an adapter is created and trained in conjunction with the pre-trained large language model. S13. Based on the user's input text, the corresponding response is determined using the trained adapter and in conjunction with the large language model.

[0023] It should be noted that the step numbers "S1N" in this manual do not represent the order of the response methods for large language models based on incremental learning. The following will explain in detail... Figures 2-3 The present invention describes the response method of a large language model based on incremental learning.

[0024] Step S11: Based on the obtained user input text, search for the corresponding knowledge sample in the vector database; wherein, the vector database is constructed in advance based on the historical question-and-answer knowledge samples of the large language model and text knowledge samples from other sources.

[0025] In this embodiment, the process of searching for corresponding knowledge samples in a vector database based on the acquired user input text includes: retrieving the target number of most similar knowledge samples from the vector database based on the acquired user input text, and determining the cosine similarity between each target number of knowledge samples and the user input text; determining that when the highest cosine similarity is less than a preset similarity threshold, the user input text is new knowledge and no corresponding knowledge sample is found; and determining that when the highest cosine similarity is greater than or equal to the preset similarity threshold, the user input text is old knowledge and a corresponding knowledge sample is found.

[0026] It should be noted that the number of targets and the preset similarity threshold can be set according to actual design requirements and prior experience. For example, the number of targets can be 3-5, and the preset similarity threshold can be 0.85; no further limitations are made here. Additionally, the user input text can be a question or related knowledge text entered by the user, determined specifically based on the user's actual input content; no further limitations are made here.

[0027] It is worth noting that in vector databases, the inverted index and product quantization (IVF-PQ) index of the vector search engine FAISS can be used to retrieve the top-5 in 1 million vectors in just 1-2ms with a recall rate of >95%, thereby significantly improving retrieval efficiency.

[0028] In addition, before searching for corresponding knowledge samples in the vector database based on the acquired user input text, the process includes: acquiring historical question-and-answer pairs of knowledge samples and text knowledge samples from other sources; representing the corresponding knowledge samples as high-dimensional vectors and storing them to obtain the corresponding vector database.

[0029] It should be noted that each knowledge sample is represented as a vector according to the hidden layer dimension of the corresponding large language model to support efficient similarity retrieval. In addition, model state snapshots are periodically saved and stored in a preset file format. The model state snapshots include Low-Rank Adaptation (LoRA) adapter parameters, importance weights, metadata, etc. The metadata includes version number, timestamp, updated content summary, performance metrics, etc. The metadata is stored in JSON format to support version comparison and rollback.

[0030] Accordingly, based on the acquired user input text, the most similar target number of knowledge samples are retrieved from the vector database, including: converting the acquired user input text into a vector representation and retrieving the most similar target number of knowledge samples from the vector database.

[0031] Step S12: Based on the lack of matching knowledge samples, the user input text is comprehensively scored to obtain an importance score. Based on the importance score being greater than a first preset threshold, an adapter is created and trained in conjunction with a pre-trained large language model.

[0032] In this embodiment, based on the lack of a matching knowledge sample, a comprehensive score is given to the user input text to obtain an importance score. This includes: determining the novelty score based on the highest cosine similarity between the user input text and the corresponding knowledge sample in the vector database; extracting keywords or topics from the user input text, and based on the keywords or topics, counting the number of queries in the user query log within the target time window, as well as the maximum and minimum number of queries for all query keywords or topics; normalizing the number of queries based on the maximum and minimum number of queries to obtain a frequency score; scoring the user input text using a pre-trained quality scoring model to obtain a corresponding quality score; and weighted summing the novelty score, frequency score, and quality score to obtain the importance score.

[0033] It should be added that novelty and cosine similarity are negatively correlated. If the user input text finds a very similar knowledge sample in the vector database, it means that the user input text is not new and the novelty score is close to 0. If the user input text is very dissimilar to all knowledge samples in the vector database, it means that the user input text is very novel and the novelty score is close to 1.

[0034] In addition, when extracting keywords or topics, you can use term frequency-inverse document frequency (TF-IDF), text ranking (TextRank), or keyword BERT (KeyBERT). The specific method can be chosen based on actual usage needs and prior experience. No further restrictions are made here. If a keyword or topic is frequently asked, it indicates that user demand is strong and the frequency score will be high. If a keyword or topic is rarely asked, it indicates that it may be a personalized or niche demand and the frequency score will be low.

[0035] Furthermore, the quality scoring model can be determined based on actual design requirements and prior experience, such as GPT-4. No further restrictions are made here. When using large language models such as GPT-4, a prompt needs to be constructed first so that the corresponding quality score can be obtained directly from the model. A clear, complete, and valuable text will have a higher quality score, while a vague, incomplete, or meaningless text will have a lower quality score.

[0036] Furthermore, when weighting and summing the novelty score, frequency score, and quality score, the weights can be set according to actual design needs. If the goal is for the system to discover and include entirely new knowledge, the novelty weight can be set higher, for example, a novelty weight of 0.5, a frequency weight of 0.3, and a quality weight of 0.2. If the goal is for the system to be more responsive to user trends, the frequency weight can be set higher, for example, a novelty weight of 0.2, a frequency weight of 0.6, and a quality weight of 0.2. If the goal is for the system to highly value the standardization of knowledge, the quality weight can be set higher, for example, a novelty weight of 0.25, a frequency weight of 0.25, and a quality weight of 0.5. No further limitations are imposed here.

[0037] In addition, based on the importance score being greater than a first preset threshold, an adapter is created and trained in conjunction with the large language model. This includes: creating an adapter based on the importance score being greater than the first preset threshold and according to a preset adapter rank, and encapsulating the loading, inference, and fine-tuning interfaces of the large language model; loading the large language model according to the adapter, injecting the adapter into the target layer of the large language model according to a preset scaling factor, and freezing the original model parameters; using a preset sampling algorithm, constructing a training sample dataset from samples extracted from the data buffer, and performing incremental learning on the adapter in conjunction with the large language model and the large language model injected with the adapter; wherein, the data buffer is constructed in advance based on samples and the real answers corresponding to the samples using experience replay technology, and the samples include historical questions or previously obtained knowledge text.

[0038] It should be added that the target layer can be determined according to the actual large language model involved, and the corresponding attention layer can be selected. The adapter can be a LoRA adapter, and the preset adapter rank and preset scaling factor can be determined according to the actual design requirements. For example, the preset adapter rank can be 8 or 16, and the preset scaling factor can be 16 or 32. No further restrictions are made here.

[0039] Furthermore, after injecting the LoRA adapter into the attention layer, the projection of the query becomes Query = W. q ×X+(α / r)×B q ×A q ×X, W q X represents the frozen parameters of the large language model, i.e., the original query projection weight matrix; X represents the input to the attention layer; α represents the preset scaling factor; A q B represents the dimensionality reduction matrix, used to project a high-dimensional input X onto a lower-dimensional matrix. q B represents the dimension-up matrix, used to project a low-dimensional structure back to the original dimension. q ×A q The rank is at most r, where r represents the preset adapter rank, A q and Bq Together, they form the core of the LoRA adapter, representing the adapter parameters, which are the parameters that need to be trained. Through the two small matrices of the learner, the model can simulate the response to W. q In an incremental update, the parameter size comparison is as follows: the original was about 16.8M, while LoRA is only about 65K (0.39%).

[0040] Additionally, the large language model representation of the injected adapter is: Output = BaseModel(Q) + Σw i ×Adapter i (Q), where Output represents the output of the large language model injected into the adapter; BaseModel(Q) represents the response obtained by the large language model based on the user input text Q; Adapter_i(Q) represents the processing result of the i-th adapter on the input Q; w i This represents the weight of the i-th adapter.

[0041] Specifically, a fine-tuning dataset is constructed using samples extracted from a data buffer using a pre-defined sampling algorithm. This dataset is then combined with the large language model of the large language model and the large language model of the injected adapter to perform incremental learning on the adapter. This includes: constructing a training sample dataset using samples extracted from the data buffer using the pre-defined sampling algorithm; and performing incremental learning on the adapter according to a first pre-defined iteration strategy, based on the training sample dataset and the large language models of the large language model and the injected adapter. The first pre-defined iteration strategy is used to: for each iteration, obtain the predicted response of the original model and the predicted response of the updated model using the large language model of the large language model and the large language model of the injected adapter, respectively; construct a task loss function based on the predicted response of the updated model and the true answer of the corresponding sample, and construct a distillation loss function based on the predicted response of the original model and the predicted response of the updated model; obtain a first total loss function based on the task loss function and the distillation loss function, and perform backward gradient updates on the adapter based on the first total loss function and iterate again until the maximum number of iterations is reached, at which point training ends.

[0042] Furthermore, the adapter is updated with backward gradients based on the first total loss function, including: determining the backward update gradient based on the first total loss function; freezing the gradient to update the parameters when the adapter parameters are determined to be core parameters, otherwise performing gradient scaling to update the parameters.

[0043] In addition, after reaching the maximum number of iterations, the iteration is completed, and the trained adapter is accepted when the performance of the validation model drops below a preset performance threshold.

[0044] It should be added that the distillation loss can be determined by calculating the difference between the predicted response of the original model and the predicted response of the updated model. For example, KL divergence can be used for calculation. The specific design depends on the actual design requirements and prior experience, and no further restrictions are made here.

[0045] In one alternative embodiment, reference Figure 2 After obtaining the importance score, the process includes: based on the importance score being less than or equal to a first preset threshold and greater than a second preset threshold, obtaining the previously trained adapter corresponding to the knowledge in the vector database, loading the corresponding large language model, injecting the adapter into the target layer of the large language model according to a preset scaling factor, and freezing the original model parameters; using a preset sampling algorithm, extracting samples from the data buffer to construct a fine-tuning sample dataset, and updating the adapter by combining the large language model and the large language model into which the adapter is injected; wherein, the data buffer is constructed in advance based on samples and the real answers corresponding to the samples using experience replay technology, and the samples include historical questions or previously obtained knowledge text; and delayed learning is performed based on the importance score being less than or equal to the second preset threshold.

[0046] It should be added that the first and second preset thresholds can be set according to actual design requirements and prior experience. For example, the first preset threshold can be 0.8 and the second preset threshold can be 0.5, which facilitates determining whether to create a new adapter, update an existing adapter, or delay learning based on the importance score. No further limitations are made here. In addition, the adapter is created and trained based on different knowledge domains or learning batches. The base model is frozen by updating only the adapter parameters (0.8-2% of the total number of parameters) to achieve efficient incremental learning.

[0047] Specifically, a fine-tuning sample dataset is constructed by sampling samples from the data buffer using a pre-defined sampling algorithm. This dataset is then combined with the large language model of the large language model and the large language model of the injected adapter to update the adapter. This includes: constructing the fine-tuning sample dataset by sampling samples from the data buffer using the pre-defined sampling algorithm; updating the adapter according to the fine-tuning sample dataset and the large language model of the large language model of the large language model and the injected adapter, following a second pre-defined iteration strategy. The second pre-defined iteration strategy is used to: for each iteration, obtain the predicted response of the original model and the predicted response of the updated model based on the fine-tuning sample dataset, using both the large language model and the large language model of the injected adapter; construct a task loss function based on the predicted response of the updated model and the true answer of the corresponding sample; construct a distillation loss function based on the predicted response of the original model and the predicted response of the updated model; and construct a regularization term based on the current adapter parameters and the parameters of the previously trained version of the adapter; obtain a second total loss function based on the task loss function, the distillation loss function, and the regularization term; and perform backward gradient updates on the adapter based on the second total loss function and iterate again until the maximum number of iterations is reached, at which point training ends.

[0048] It should be added that, based on the current adapter parameters and the adapter parameters of the previously trained version, a regularization term is constructed, including: using the log-likelihood function to determine the expectation of the second derivative with respect to the parameters based on the adapter parameters of the previously trained version, to obtain the Fisher information matrix; and using elastic weights to consolidate the regularization term based on the diagonal elements of the Fisher information matrix, the current adapter parameters, and the adapter parameters of the previously trained version.

[0049] It's worth noting that by calculating Fisher values ​​(approximately 65K parameters) on the LoRA adapter parameters, the storage requirement is only about 0.5MB, significantly reducing storage costs. Furthermore, through experience replay to review old knowledge, Elastic Weight Consolidation (EWC) to solidify the core, and knowledge distillation to impart experience, a comprehensive, multi-layered defense is formed from data and parameters to model output. This greatly enhances the model's robustness in continuous learning. This collaborative protection mechanism effectively suppresses catastrophic forgetting, allowing the model to adapt to new tasks while its overall knowledge system evolves smoothly and orderly, rather than passively compensating for losses.

[0050] Furthermore, the regularization term = Σ (λ / 2) * F i * (θ i -θ* i) 2 ;θ i θ*i represents the value of the i-th parameter of the current adapter; θ*i represents the i-th parameter of the previously trained version of the adapter; λ represents the hyperparameter used to control the strength of the regularization term; F iF represents the diagonal elements of the Fisher information matrix. i The larger the value, the stronger the parameter θ. i The greater the impact on the performance of older tasks, the more critical they are, requiring strong protection and incurring significant penalties. i A small value indicates that the parameter θ i It has little impact on old tasks, is not important, can be adjusted freely, and imposes minor penalties.

[0051] In addition, the second total loss function L total Represented as: L total =L task +λ EWC ×L EWC +λ distill ×L distill Among them, L task λ represents the task loss function. EWC L represents the weight of the regularization term; EWC λ represents the regularization term. distill L represents the weight of distillation loss. distill This indicates distillation loss.

[0052] In an optional embodiment, before constructing the corresponding training sample dataset or fine-tuning sample dataset from samples extracted from the data buffer using a preset sampling algorithm, the method includes: constructing new samples based on previously acquired user input text, combined with the responses obtained from the large language model after the corresponding injection adapter and the corresponding task identifier; adding new samples to the data buffer when the data buffer capacity is not full; wherein the data buffer is constructed based on a preset capacity; when the data buffer capacity is full, determining the similarity between the new samples and each sample stored in the data buffer, selecting the sample with the highest similarity for removal, and adding the new samples to the data buffer.

[0053] It should be added that new samples can also include metadata, as described above, and will not be repeated here.

[0054] Furthermore, using a pre-defined sampling algorithm, samples are drawn from the data buffer to construct the corresponding training sample dataset or fine-tuning sample dataset. This includes: using a pre-defined sampling algorithm to construct the corresponding training sample dataset or fine-tuning sample dataset from the data buffer through uniform random sampling, stratified sampling, or importance-weighted sampling. It should be noted that uniform random sampling involves completely randomly selecting a batch of samples from the data buffer without considering any other factors; it is simple, unbiased, and provides a general review of historical knowledge. Stratified sampling involves sampling from the data buffer according to tasks, selecting a certain number of samples from the samples corresponding to the chosen tasks to construct the corresponding sample dataset. This effectively prevents tasks with excessively large datasets from overwhelming tasks with smaller datasets, ensuring the balance of learning. Importance-weighted sampling assigns a higher probability of selection to more important samples, for example, by constructing a probability distribution based on importance scores and sampling according to the corresponding probability distribution, so that the model focuses its review on the knowledge points that most need reinforcement. The specific pre-defined sampling algorithm can be selected according to actual design requirements, and no further limitations are made here.

[0055] Step S13: Based on the user's input text, use the trained adapter and combine it with the large language model to determine the corresponding response.

[0056] It is worth noting that after training the created adapter or updating the existing adapter, the corresponding adapter and the large language model can be used to predict the response to the user's input text. Furthermore, after searching for corresponding knowledge samples in the vector database based on the acquired user input text, the process includes: based on the matched knowledge samples, calling the corresponding adapter and its pre-trained weights; if only one adapter is called, directly using the corresponding adapter and combining it with the large language model to determine the response to the user input text; if at least two adapters are called, each adapter is used separately, combined with the large language model, to determine the response to the user input text, and the results are aggregated using strategies such as weighted averaging, voting, or ensemble learning for output.

[0057] In one alternative embodiment, reference Figure 3After determining the corresponding response based on the user input text, using the trained adapter, and combining it with a large language model, the process includes: updating the vector database with the user input text and corresponding response as new knowledge samples based on the lack of matching knowledge samples in the vector database; determining the relative entropy KL divergence between the predicted response of the original model and the predicted response of the updated model based on the matching knowledge samples in the vector database; determining a conflict between the new knowledge sample composed of the user input text and corresponding response and the corresponding matching knowledge sample when the KL divergence is greater than a preset divergence threshold; classifying the conflict between the new knowledge sample and the corresponding matching knowledge sample to determine the conflict type and corresponding confidence level; fusing the new knowledge sample and the corresponding matching knowledge sample according to the conflict type and corresponding confidence level, and updating the vector database according to a preset knowledge fusion strategy; wherein, the preset knowledge fusion strategy is configured in advance based on different conflict types and confidence levels; and updating the vector database with new knowledge samples when the KL divergence is less than or equal to the preset divergence threshold.

[0058] It should be noted that conflict classification between newly added knowledge samples and their corresponding matching knowledge samples can be achieved by using a pre-trained classification model, or by using predefined conflict types and matching the newly added knowledge samples and their corresponding matching knowledge samples with each conflict type to determine the corresponding conflict type. The specific settings can be set according to actual design requirements and prior experience, and no further restrictions are made here.

[0059] Furthermore, based on the conflict type and corresponding confidence level, and according to a preset knowledge fusion strategy, the newly added knowledge samples and their corresponding matching knowledge samples are fused to update the vector database. This includes: aligning the newly added knowledge samples and their corresponding matching knowledge samples in the representation space when the conflict type is complementary; updating the vector database with the newly added knowledge samples when the conflict type is contradictory, while retaining the corresponding matching knowledge samples in the vector database; replacing the corresponding matching knowledge samples in the vector database with the newly added knowledge samples when the conflict type is substitutional, and when the corresponding confidence level is greater than a first preset confidence threshold; updating the vector database with the newly added knowledge samples when the confidence level is less than or equal to the first preset confidence threshold but greater than a second preset confidence threshold, while retaining the corresponding matching knowledge samples in the vector database; and retaining the corresponding matching knowledge samples in the vector database when the confidence level is less than or equal to the second preset confidence threshold.

[0060] In an optional embodiment, for the full retraining method, the LoRA adapter technology described above only updates 0.8-2% of the parameters. Using a parameter-efficient fine-tuning method, under the same hardware conditions, training time is reduced from 120 hours to 8-10 hours, GPU resource consumption is reduced by approximately 92%, and it does not require permanently storing all historical data, only retaining representative samples (1-2%) in the replay buffer. This means that 1TB of historical data only requires 10GB of representative samples, and the total storage requirement is approximately 10-20GB, a reduction of over 95%. In addition, a single incremental learning session only takes a few hours and can be completed during off-peak hours; it supports a scheduled automatic update mechanism, enabling daily updates, reducing response time from weeks to hours, meeting the needs of rapid iteration. Furthermore, incremental training only updates a small portion of parameters, having minimal impact on the base model. The anti-forgetting mechanism ensures that performance does not fluctuate drastically, resulting in relatively stable performance for both new and old tasks. Repeated hyperparameter adjustments are unnecessary, and the training success rate is close to 100%.

[0061] In summary, this invention utilizes the acquired user input text to search a vector database to determine whether the user input text represents new knowledge. If no matching knowledge sample is found, the corresponding user input text is comprehensively scored. Based on the obtained importance score, a new adapter is created for the user input text. This new adapter is then trained using a large language model, freezing the large language model parameters and training only the adapter. This significantly reduces computational costs, protects the general capabilities and existing knowledge already learned by the large language model, prevents catastrophic forgetting, and allows new knowledge to be integrated non-intrusively, thus ensuring the overall stability and reliability of the system. The trained adapter, combined with the large language model, quickly responds to user input text, ensuring that the final response combines the general capabilities of the base model with the professional knowledge of the newly trained adapter. This guarantees both fluency and efficiency in the response while injecting the latest and most accurate knowledge points, providing high-quality, personalized answers and achieving personalized knowledge services.

[0062] The response device of the large language model based on incremental learning provided by the present invention will be described below. The response device of the large language model based on incremental learning described below can be referred to in correspondence with the response method of the large language model based on incremental learning described above.

[0063] Figure 4 A schematic diagram of a response device based on an incremental learning-based large language model is shown. The device includes: The knowledge addition judgment module 41 searches for corresponding knowledge in the vector database based on the acquired user input text; the vector database is constructed in advance based on the historical question-and-answer pairs of knowledge from the large language model and knowledge text from other sources. The adapter configuration module 42 performs a comprehensive score on the user input text based on the lack of matching knowledge, obtains an importance score, creates an adapter based on the importance score being greater than a first preset threshold, and trains the adapter in conjunction with a pre-trained large language model. The response module 43 determines the corresponding response based on the user's input text, using the trained adapter and combining it with the large language model.

[0064] It should be noted that the specific principles of the embodiments of the present invention are the same as those of the method embodiments described above. For details, please refer to the method embodiments above. More detailed explanations will not be repeated here.

[0065] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a response method based on an incremental learning-based large language model. This method includes: searching for corresponding knowledge samples in a vector database based on the acquired user input text; wherein the vector database is constructed in advance based on historical question-and-answer pairs of knowledge samples from the large language model and text knowledge samples from other sources; comprehensively scoring the user input text based on the absence of a matching knowledge sample to obtain an importance score; creating an adapter based on an importance score greater than a first preset threshold, and training the adapter in conjunction with a pre-trained large language model; and determining the corresponding response based on the user input text, using the trained adapter, and in conjunction with the large language model.

[0066] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the response method based on the incremental learning large language model provided by the above methods. The method includes: searching for corresponding knowledge samples in a vector database based on the acquired user input text; wherein the vector database is constructed in advance based on the historical question-and-answer pairs of the large language model and text knowledge samples from other sources; comprehensively scoring the user input text based on the lack of a matching knowledge sample to obtain an importance score; creating an adapter based on the importance score being greater than a first preset threshold, and training the adapter in conjunction with a pre-trained large language model; and determining the corresponding response based on the user input text, using the trained adapter, and in conjunction with the large language model.

[0068] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a response method based on an incremental learning large language model provided by the methods described above. This method includes: searching for corresponding knowledge samples in a vector database based on acquired user input text; wherein the vector database is constructed prior to the use of historical question-and-answer pairs of knowledge samples from the large language model and text knowledge samples from other sources; comprehensively scoring the user input text based on the absence of a matching knowledge sample to obtain an importance score; creating an adapter based on an importance score greater than a first preset threshold, and training the adapter in conjunction with a pre-trained large language model; and determining the corresponding response based on the user input text, using the trained adapter, and in conjunction with the large language model.

[0069] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0070] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A response method for a large language model based on incremental learning, characterized in that, include: Based on the acquired user input text, the corresponding knowledge sample is searched in the vector database; wherein, the vector database is constructed in advance based on the historical question-and-answer pairs of the large language model and text knowledge samples from other sources; Based on the lack of matching knowledge samples, the user input text is comprehensively scored to obtain an importance score. If the importance score is greater than a first preset threshold, an adapter is created and trained in conjunction with a pre-trained large language model. Based on the user's input text, the corresponding response is determined using the trained adapter and the large language model.

2. The response method for a large language model based on incremental learning according to claim 1, characterized in that, Based on the importance score being greater than a first preset threshold, an adapter is created, and the adapter is trained in conjunction with a large language model, including: Based on the importance score being greater than a first preset threshold, an adapter is created according to the preset adapter rank, and the loading, inference, and fine-tuning interfaces of the large language model are encapsulated. According to the adapter, the large language model is loaded, and the adapter is injected into the target layer of the large language model according to a preset scaling factor, and the original model parameters are frozen; Using a preset sampling algorithm, samples are extracted from the data buffer to construct a training sample dataset. This dataset is then combined with the large language model and the large language model injected into the adapter to perform incremental learning on the adapter. The data buffer is constructed in advance based on samples and the real answers corresponding to those samples, using experience replay technology. The samples include historical questions or previously obtained knowledge text.

3. The response method for a large language model based on incremental learning according to claim 2, characterized in that, Using a preset sampling algorithm, samples are extracted from the data buffer to construct a fine-tuning dataset. This dataset is then combined with the large language model and the large language model injected into the adapter to perform incremental learning on the adapter, including: Using a pre-defined sampling algorithm, samples are extracted from the data buffer to construct a training sample dataset; Based on the training sample dataset, and combining the large language model and the large language model injected into the adapter, incremental learning is performed on the adapter according to a first preset iteration strategy; wherein, the first preset iteration strategy is used for: For each iteration, based on the training sample dataset, the large language model and the large language model injected with the adapter are used respectively to obtain the predicted response of the original model and the predicted response of the updated model. Based on the predicted response of the updated model and the actual answer of the corresponding sample, a task loss function is constructed, and based on the predicted response of the original model and the predicted response of the updated model, a distillation loss function is constructed. Based on the task loss function and the distillation loss function, a first total loss function is obtained. The adapter is then updated with backward gradients based on the first total loss function and iterated again until the maximum number of iterations is reached, at which point the training ends.

4. The response method for a large language model based on incremental learning according to claim 1, characterized in that, After obtaining the importance score, the following is also included: Based on the importance score being less than or equal to the first preset threshold and greater than the second preset threshold, the adapter corresponding to the knowledge matched in the vector database is obtained and the corresponding large language model is loaded. The adapter is then injected into the target layer of the large language model according to a preset scaling factor, and the original model parameters are frozen. Using a preset sampling algorithm, samples are extracted from the data buffer to construct a fine-tuning sample dataset. This dataset is then combined with the large language model and the large language model injected into the adapter to update the adapter. The data buffer is constructed prior to the samples and the corresponding real answers using experience replay technology. The samples include historical questions or previously obtained knowledge text. Learning is delayed if the importance score is less than or equal to the second preset threshold.

5. The response method for a large language model based on incremental learning according to claim 4, characterized in that, Using a preset sampling algorithm, samples are extracted from the data buffer to construct a fine-tuning sample dataset. This dataset is then combined with the large language model and the large language model injected into the adapter to update the adapter, including: Using a pre-defined sampling algorithm, samples are extracted from the data buffer to construct a fine-tuning sample dataset; Based on the fine-tuned sample dataset, combined with the large language model and the large language model injected into the adapter, the adapter is updated according to a second preset iteration strategy; wherein, the second preset iteration strategy is used for: For each iteration, based on the fine-tuned sample dataset, the large language model and the large language model injected into the adapter are used respectively to obtain the predicted response of the original model and the predicted response of the updated model. Based on the predicted response of the updated model and the true answer of the corresponding sample, a task loss function is constructed; based on the predicted response of the original model and the predicted response of the updated model, a distillation loss function is constructed; and based on the current adapter parameters and the adapter parameters of the previously trained version, a regularization term is constructed. Based on the task loss function, the distillation loss function, and the regularization term, a second total loss function is obtained. The adapter is then updated using backward gradients based on the second total loss function and iterated again until the maximum number of iterations is reached, at which point training ends.

6. The response method for a large language model based on incremental learning according to claim 2 or 4, characterized in that, Before constructing the corresponding training sample dataset or fine-tuning sample dataset from samples drawn from the data buffer using a preset sampling algorithm, the process includes: Based on the user input text obtained earlier, combined with the response obtained from the large language model after the corresponding adapter is injected and the corresponding task identifier, a new sample is constructed. If it is determined that the data buffer capacity is not full, the new sample is added to the data buffer; wherein the data buffer is constructed based on a preset capacity. When it is determined that the data buffer capacity is full, the similarity between the new sample and each sample stored in the data buffer is determined, the sample with the highest similarity is removed, and the new sample is added to the data buffer.

7. The response method for a large language model based on incremental learning according to claim 3 or 5, characterized in that, After determining the corresponding response based on the user input text, using the trained adapter and the large language model, the process includes: Since no corresponding knowledge sample was found in the vector database, the user input text and the corresponding reply were updated to the vector database as new knowledge samples. Based on the corresponding knowledge samples matched in the vector database, the relative entropy KL divergence between the predicted response of the original model and the predicted response of the updated model is determined. When the KL divergence is determined to be greater than a preset divergence threshold, it is determined that the newly added knowledge sample consisting of the user input text and the corresponding response conflicts with the corresponding matching knowledge sample. The newly added knowledge samples and their corresponding matching knowledge samples are classified into conflict types to determine the conflict type and corresponding confidence level. Based on the conflict type and corresponding confidence level, the newly added knowledge sample and the corresponding matching knowledge sample are fused according to a preset knowledge fusion strategy to update the vector database; wherein, the preset knowledge fusion strategy is configured in advance based on different conflict types and confidence levels; When the KL divergence is determined to be less than or equal to the preset divergence threshold, the newly added knowledge sample is updated to the vector database.

8. The response method for a large language model based on incremental learning according to claim 7, characterized in that, Based on the conflict type and corresponding confidence level, the newly added knowledge sample is fused with the corresponding matching knowledge sample according to a preset knowledge fusion strategy to update the vector database, including: Based on the fact that the conflict type is complementary conflict, the newly added knowledge sample and the corresponding matching knowledge sample are aligned in the representation space. When the conflict type is a contradiction, the newly added knowledge sample is updated to the vector database, and the corresponding matching knowledge sample in the vector database is retained. Based on the conflict type being a substitution conflict, when the corresponding confidence level is greater than the first preset confidence threshold, the newly added knowledge sample replaces the corresponding matching knowledge sample in the vector database. When it is determined that the confidence level is less than or equal to the first preset confidence threshold and greater than the second preset confidence threshold, the newly added knowledge sample is updated to the vector database, and the corresponding matching knowledge sample in the vector database is retained; When the confidence level is determined to be less than or equal to the second preset confidence threshold, the corresponding matching knowledge sample in the vector database is retained.

9. A response device for a large language model based on incremental learning, characterized in that, include: The knowledge addition judgment module searches for corresponding knowledge samples in the vector database based on the acquired user input text; wherein, the vector database is constructed in advance based on the historical question-and-answer pairs of knowledge samples from the large language model and text knowledge samples from other sources; The adapter configuration module performs a comprehensive score on the user input text based on the absence of a matching knowledge sample, obtains an importance score, creates an adapter based on the importance score being greater than a first preset threshold, and trains the adapter in conjunction with a pre-trained large language model. The response module determines the corresponding response based on the user's input text, using the trained adapter and the large language model.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the response method of a large language model based on incremental learning as described in any one of claims 1 to 7.

Images