Large language model incremental learning and text processing method and device

Abstract

The application discloses a kind of big language model's incremental learning and text processing method and device, it is related to natural language processing technical field.The method includes: obtaining vertical field text data as training sample;Output text is obtained by retrieval enhancement generation system;Based on the semantic similarity of output text and reference document and the source authority degree, calculate feedback confidence;Convert confidence into dynamic penalty coefficient by nonlinear mapping function, construct total loss function containing elastic weight solidification regularization term;Accumulate parameter gradient in training period, whether update parameter is decided based on average confidence when period ends;Big language model is processed target text data with training completion.The application realizes the self-adaptive learning of data quality perception, enhances regularization under noisy data to suppress catastrophic forgetting, weakens regularization under high-quality data to accelerate convergence, and introduces period-level knowledge gate, significantly improves the robustness and efficiency of incremental learning.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to an incremental learning and text processing method and apparatus for a large language model. Background Technology

[0002] With the increasing application of large language models in numerous professional fields, enabling them to learn new knowledge from the continuous influx of domain data while retaining old skills has become a key technological requirement. However, existing technologies face two core contradictions in achieving this goal: First, when large language models are trained on new data, parameter updates overwrite knowledge encoded for older tasks, leading to a sharp performance drop and catastrophic forgetting. Mainstream solutions, such as elastic weight fixation, constrain changes in important parameters by introducing a regularization term based on parameter importance into the loss function. However, its penalty coefficient is fixed, and this one-size-fits-all static strategy has a fundamental flaw: it cannot distinguish between high-quality new knowledge and low-quality noise in the current input data. With noisy data, too small a penalty leads to incorrect model learning; while with high-quality data, too large a penalty hinders the model from effectively absorbing new knowledge, leading to knowledge stagnation.

[0003] Secondly, the existing continuous learning process is an open-loop system, where the model's optimizer updates parameters according to pre-defined rules independent of the data content. The system lacks an embedded quality sensor capable of real-time evaluation of the input data's reliability, thus failing to dynamically adjust the learning strategy based on data quality. It often trains high-quality data and low-quality data equally, which is the main reason for the model's unstable performance and low efficiency.

[0004] Therefore, there is an urgent need for an endogenous intelligent mechanism that can evaluate the quality of training data in real time and adjust the learning intensity automatically and accurately accordingly, so as to dynamically adjust the regularization constraints in the process of updating model parameters, thereby achieving an adaptive balance between suppressing catastrophic forgetting and efficiently absorbing new knowledge. Summary of the Invention

[0005] In view of the above-mentioned defects or deficiencies, the present invention provides an incremental learning method and apparatus for large language models and text processing, which can effectively solve the above-mentioned technical problems mentioned in the background art.

[0006] One aspect of the present invention provides an incremental learning and text processing method for a large language model, comprising the following steps: Acquire text data from vertical domains as training samples for pre-trained large language models; The training samples are input into the retrieval enhancement and generation system, which triggers the system to retrieve reference documents related to the training samples from the knowledge base. The training samples and the reference documents are then input into the pre-trained large language model to obtain the output text. Calculate the semantic similarity between the output text and the reference document, and obtain the source authority of the reference document; calculate the feedback confidence of the training sample based on the semantic similarity and the source authority. The feedback confidence is converted into a dynamic penalty coefficient based on a preset nonlinear mapping function. The dynamic penalty coefficient is used to construct a regularization term in the total loss function. The regularization term is used to constrain the degree to which the parameters of the large language model deviate from the old parameter benchmark during the update process. The total loss function is constructed using the dynamic penalty coefficient. The parameter gradient corresponding to each training sample is calculated based on the total loss function. The parameter gradient of each training sample is accumulated within a training cycle to obtain the periodic cumulative gradient. After the training period ends, the average confidence score of all training samples in the training period is calculated; if the average confidence score is greater than or equal to a preset threshold, the model parameters are formally updated according to the periodic cumulative gradient, and the updated model parameters are saved as the new old parameter benchmark; if the average confidence score is less than the preset threshold, the periodic cumulative gradient is discarded and the model parameters remain unchanged. The trained large language model is used to process target text data from the vertical domain, generating output results related to the vertical domain.

[0007] Another aspect of the present invention provides an incremental learning and text processing apparatus for a large language model, comprising: The data acquisition module is used to acquire text data from vertical domains as training samples for pre-trained large language models. The retrieval enhancement generation module is used to input the training samples into the retrieval enhancement generation system, trigger the retrieval enhancement generation system to retrieve reference documents related to the training samples from the knowledge base, and input the training samples and the reference documents together into the pre-trained large language model to obtain the output text; The confidence calculation module is used to calculate the semantic similarity between the output text and the reference document, and to obtain the source authority of the reference document; based on the semantic similarity and the source authority, the feedback confidence of the training sample is calculated. The dynamic adjustment module is used to convert the feedback confidence into a dynamic penalty coefficient based on a preset nonlinear mapping function. The dynamic penalty coefficient is used to construct the regularization term in the total loss function. The regularization term is used to constrain the degree to which the parameters of the large language model deviate from the old parameter benchmark during the update process. The gradient accumulation module is used to construct a total loss function using the dynamic penalty coefficient, calculate the parameter gradient corresponding to each training sample based on the total loss function, and accumulate the parameter gradient of each training sample within a training cycle to obtain the periodic accumulated gradient. The parameter update module is used to calculate the average confidence of all training samples during the training period after the training period ends; if the average confidence is greater than or equal to a preset threshold, the model parameters are formally updated according to the periodic cumulative gradient, and the updated model parameters are saved as the new old parameter benchmark; if the average confidence is less than the preset threshold, the periodic cumulative gradient is discarded and the model parameters remain unchanged. The text processing module uses a trained large language model to process target text data from the vertical domain and generate output results related to the vertical domain.

[0008] The present invention provides an incremental learning method and apparatus for large language models and text processing, which has the following beneficial effects: (1) It realizes automated learning decision-making with data quality awareness. This invention enables the model to have the intrinsic intelligence to adjust the learning strategy in real time according to the quality of the input data itself, overcoming the inherent limitations of traditional continuous learning as an optimization process that lacks data quality guidance.

[0009] (2) Significantly improves the balance between noise resistance and learning efficiency. For low-quality data, the system automatically increases the regularization penalty strength to effectively suppress catastrophic forgetting; for high-quality data, the system automatically weakens the regularization constraint to accelerate model convergence. This achieves an adaptive match between regularization strength and data quality, and a dynamic balance between learning stability and agility.

[0010] (3) A multi-layered security protection system has been constructed. This invention forms a defense-in-depth architecture from the sample-level dynamic penalty mechanism, the task-level quality verification and gradient rollback mechanism, to the system-level security mode switching mechanism, which significantly improves the system robustness and reliability of the model in the open and uncontrollable data flow environment for continuous learning. Attached Figure Description

[0011] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating an incremental learning and text processing method for a large language model provided in one embodiment of this application; Figure 2 This is a schematic diagram of the structure of an incremental learning and text processing apparatus for a large language model provided in another embodiment of this application; Figure 3This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] Example 1 This embodiment details an incremental learning and text processing method for a large language model. For example... Figure 1 As shown, the method mainly includes the following steps.

[0014] Step S101: Obtain text data from the vertical domain as training samples for the pre-trained large language model.

[0015] This step aims to provide raw data input for the incremental learning process of the large language model. The large language model refers to an artificial neural network model with billions of parameters, pre-trained on massive amounts of data, capable of understanding and generating natural language text. Incremental learning refers to the process by which the model continuously learns new knowledge from new data, building upon its existing knowledge while minimizing the forgetting of old knowledge.

[0016] Specifically, for a particular vertical application domain, such as finance, law, or healthcare, the system acquires text data to be learned from business databases, real-time document streams, web resources, or other data sources within that domain. This text data constitutes one or more training samples, each of which can be a user query, a draft legal provision, or a recent medical research report. These training samples are then used to update the pre-trained large language model, enabling it to grasp the latest knowledge and specific expression habits of that vertical domain.

[0017] Step S102: Input the training sample into the retrieval enhancement generation system, trigger the retrieval enhancement generation system to retrieve reference documents related to the training sample from the knowledge base, and input the training sample and the reference document together into the pre-trained large language model to obtain the output text.

[0018] This step aims to leverage a Retrieval-Augmented Generation (RAG) system to provide factual basis for the content generated by the model, thereby quantifying the reliability of the generated results. A RAG system is a technical framework combining information retrieval and text generation. Before generating text, the system retrieves relevant reference documents from a knowledge base. Then, it provides both the input and the retrieved documents to the large language model to guide it in generating more accurate and factually grounded output. This step addresses the disconnect between model parameter updates and data quality awareness in existing technologies; that is, the model itself cannot determine whether the current learning sample is truth or noise. By introducing a RAG system, it's equivalent to equipping the model with an external knowledge judge. Its technical principle is as follows: The system first inputs the acquired text data (e.g., the user question "What is portfolio investment?") into the RAG system's retrieval tool. The retrieval tool quickly retrieves the top-K most relevant reference documents from a pre-built, high-quality knowledge base (e.g., a vector database containing official regulations, academic papers, and authoritative encyclopedias) by calculating the vector similarity (e.g., cosine similarity) between the query and documents in the database. These reference documents are considered the authoritative factual basis for answering the question. Subsequently, the RAG system concatenates the original input with the retrieved reference documents to form a new, information-enhanced prompt, which is then input into a pre-trained large language model. The large language model generates the final output text based on this enhanced prompt. Because this output text is guided by authoritative documents, its reliability is significantly improved compared to direct generation, laying the foundation for subsequent quality assessment.

[0019] Step S103: Calculate the semantic similarity between the output text and the reference document, and obtain the source authority of the reference document; calculate the feedback confidence of the training sample based on the semantic similarity and the source authority.

[0020] This step aims to quantify the inherent generation quality assessment signal of the RAG system into a specific numerical value that can be used to control the learning process: feedback confidence. This step is a key component of the present invention for achieving data quality awareness.

[0021] Specifically, the feedback confidence level of this invention integrates information from two dimensions: The first dimension is semantic similarity, which measures the semantic consistency between the content generated by the model and the retrieved authoritative reference documents. Ideally, the model output on high-quality data should be highly consistent with the reference documents. A preferred implementation is to first use a sentence embedding model (such as Sentence-BERT) to convert the output text and the reference documents into high-dimensional vectors, and then calculate the cosine similarity between these two vectors, using this cosine similarity value as the semantic similarity. The closer the cosine similarity value is to 1, the more semantically similar the texts are. Alternatively, a more accurate but computationally more expensive cross-encoder can be used to directly calculate the semantic relevance score between the two texts.

[0022] The second dimension is source authority, used to measure the credibility of the reference document's source. The authority of information from different sources varies greatly; for example, documents published on government websites or by standards bodies are far more authoritative than those from personal blogs or forum posts. The system can pre-build an authority mapping table, assigning authority weights to different types of sources. For example, reference documents published by government or standards bodies correspond to the highest authority value, while reference documents from other sources correspond to a lower authority value (secondary authority value). Finally, a weighted fusion method is used to calculate the overall feedback confidence level. : ; in, It is semantic similarity. It's about the authority of the source. It is a preset fusion coefficient used to adjust the relative importance of the two in the confidence assessment. The higher the value, the more the system values ​​the degree of matching between the generated content and the facts; The lower the value, the more the system trusts the information source itself.

[0023] This step transforms the originally abstract concept of data quality into a standardized, computable score, enabling the model to perceive the quality of each training sample and providing a quantitative basis for subsequent refined control of learning intensity.

[0024] Step S104: Based on a preset nonlinear mapping function, the feedback confidence is converted into a dynamic penalty coefficient. The dynamic penalty coefficient is used to construct a regularization term in the total loss function. The regularization term is used to constrain the degree to which the parameters of the large language model deviate from the old parameter benchmark during the update process.

[0025] This step is the core of the invention to achieve adaptive adjustment of learning intensity. By establishing an intelligent mapping from data quality perception to learning behavior control, it solves the inherent defect of the existing elastic weight solidification algorithm, which cannot dynamically adjust the balance between knowledge stability and learning agility in the process of updating model parameters based on data quality due to the use of static penalty coefficients.

[0026] Specifically, this invention designs a nonlinear mapping function that maps continuous value confidence to a dynamic penalty coefficient, which is a regularization strength parameter that changes in real time during model training. The design goal of the nonlinear mapping function is clear and explicit: the lower the continuous value confidence (i.e., the less reliable the data), the larger the dynamic penalty coefficient should be (i.e., the stronger the penalty, the more the model tends to maintain the original parameters); the higher the continuous value confidence (i.e., the more reliable the data), the smaller the dynamic penalty coefficient should be (i.e., the weaker the penalty, the more the model tends to learn new parameters).

[0027] To achieve the above objectives and address the inherent limitation of existing elastic weight fixation algorithms, which, due to their use of static penalty coefficients, cannot dynamically adjust the balance between knowledge stability and learning agility during model parameter updates based on data quality, this embodiment proposes a preferred piecewise mapping function, the calculation expression of which is: in, This is a dynamic penalty coefficient; To provide feedback confidence level; The base penalty coefficient is a baseline value preset by the user based on the characteristics of the task; Where is the noise amplification factor, and > 0, used to significantly increase penalty intensity in low confidence regions, when At extremely low temperatures, Promoted to This creates strong constraints to prevent parameter updates; Let be the signal attenuation coefficient, and This is used to linearly reduce the penalty in the high-confidence region, as... from Increase to 1, from linearly decreasing to The severity of punishment has been gradually relaxed; The low confidence threshold is used to delineate the noise suppression region. < ) and signal learning area ( ≥ ).

[0028] The beneficial effects of this piecewise function are: First, it outputs a constant high value in the low confidence region, avoiding model mislearning due to fluctuations in individual scores and providing a stable robust interval; second, although it... While the function is segmented, parameter design ensures its approximate continuity, preventing drastic shifts in the learning strategy. Furthermore, by adjusting... and Users can independently control the model's degree of suppression of low-quality data and its sensitivity to learning high-quality data, thus achieving first-order control over the model's learning behavior.

[0029] Furthermore, in obtaining the dynamic penalty coefficient Subsequently, this invention uses it to construct a regularization term for elastic weight solidification. Specifically, let all the parameters of the model form a vector θ, where The model represents the first One parameter. A quadratic penalty term is introduced into the total loss function: in: For parameters The diagonal elements of the Fisher information matrix quantify this parameter. The importance of old tasks. This is the old parameter baseline, which is a knowledge snapshot of the model before it learns a new task.

[0030] The function of the aforementioned quadratic penalty term, also known as the regularization term, is: when the model attempts to update its parameters, if the current parameters... Deviating from the old benchmark This will produce a result similar to The penalty cost is proportional to the penalty. In this way, the dynamic penalty coefficient... It directly controls the extent to which model parameters are allowed to deviate from the old parameter baseline during the update process. The larger the value, the higher the cost of deviation, and the more the model tends to maintain the original parameters, thereby suppressing catastrophic forgetting; The smaller the value, the lower the deviation cost, and the more freely the model can absorb new knowledge. Therefore, this invention uses the result of data quality perception, i.e., feedback confidence,... This transforms into real-time control of the degree of freedom for parameter updates, achieving an adaptive match between regularization strength and data quality.

[0031] Step S105: Construct a total loss function using the dynamic penalty coefficient, calculate the parameter gradient corresponding to each training sample based on the total loss function, and accumulate the parameter gradient of each training sample within a training cycle to obtain the periodic cumulative gradient.

[0032] This step aims to apply the dynamic penalty coefficients generated in the previous step to the actual optimization process of the model, guiding the update direction of the large language model parameters.

[0033] Specifically, in each training iteration (processing one or a small batch of training samples), the system uses the dynamic penalty coefficient calculated in step S104 for that sample (or the average confidence of that batch of samples). To construct the total loss function: in: This is the task loss function (e.g., cross-entropy loss for text generation tasks), used to measure the model's predictive performance on the current training samples.

[0034] For the model number Parameters The corresponding diagonal elements of the Fisher information matrix quantify the importance of this parameter in completing the old task. The larger the value, the more important the parameter is, and the greater the impact of its changes on the old task. This matrix can be calculated and fixed after the model learns the old task.

[0035] For the current model number Each parameter value.

[0036] The first parameter snapshot of the model before it begins learning a new task (or after successfully solidifying the previous iteration). The values ​​of these parameters represent validated baseline knowledge. To solidify the regularization term for the elastic weights, a quadratic penalty function is formed by summing the weighted squared differences of the importance of all parameters.

[0037] By minimizing the total loss function During training, the model not only needs to reduce the error on the current task. It also requires the payment of dynamic penalty coefficients. The cost is proportional to the change in important old parameters. When the dynamic penalty coefficient... When the value is large, the model tends to avoid changing old parameters; when the dynamic penalty coefficient is large... When the value is smaller, the model can learn new knowledge more freely.

[0038] Furthermore, after calculating the total loss for the current sample (or mini-batch samples), the system uses the backpropagation algorithm to calculate the gradient of each model parameter. To achieve differentiated processing at the sample level—fully learning from high-quality data and effectively skipping low-quality data—this invention accumulates gradients at the training epoch level. Specifically, the system initializes the epoch-accumulated gradient as a zero vector. For each sample within the current training epoch, the system calculates its corresponding parameter gradient, but does not immediately update the model parameters; instead, it adds the gradient of that sample to the epoch-accumulated gradient. This design allows the model to comprehensively evaluate the quality of all samples within a complete epoch, thus avoiding being dominated by the gradient of a single noisy sample.

[0039] Step S106: After the training period ends, calculate the average confidence of all training samples in the training period; if the average confidence is greater than or equal to a preset threshold, then update the model parameters according to the periodic cumulative gradient and save the updated model parameters as the new old parameter benchmark; if the average confidence is less than the preset threshold, then discard the periodic cumulative gradient and keep the model parameters unchanged.

[0040] This step establishes a task-level knowledge protection mechanism designed to prevent model parameters from being contaminated by unreliable gradients during periods of poor overall data quality, thereby addressing the lack of high-level, backtrackable defense mechanisms in existing technologies.

[0041] Specifically, after a training cycle ends, the system first calculates the arithmetic mean of the feedback confidence scores of all training samples within that cycle. Then, the system compares this average with a preset update threshold. If the average is greater than or equal to the update threshold, the system determines that the knowledge learned in this cycle is generally reliable. The system then uses the accumulated gradients from the previous cycle to formally execute one or more steps of model parameter updates. After the parameter updates are complete, the model changes from its current parameter values ​​to the updated parameter values, and these updated parameter values ​​are saved as the new old parameter baseline for use in the elastic weight solidification regularization calculation of the next cycle, completing the learning-solidification closed loop. If the arithmetic mean of the feedback confidence scores of all training samples is less than the update threshold, the system determines that the data quality in this cycle is unreliable. The system discards all accumulated parameter gradients from the current cycle, keeps the model parameters unchanged, and reverts to the parameter state before the start of the cycle. Simultaneously, the old parameter baseline also remains unchanged, thus preventing noisy knowledge from contaminating the model's long-term memory.

[0042] Furthermore, the system includes a safety mode: when the arithmetic mean of the feedback confidence scores of training samples for N consecutive training epochs (e.g., N=3) is lower than the update threshold, the system automatically pauses the update of the pre-trained large language model parameters, only cumulatively updating the diagonal elements of the Fisher information matrix to continuously reinforce the memory of old knowledge, until the arithmetic mean of the feedback confidence scores of training samples in a subsequent training epoch is greater than or equal to the update threshold, at which point the parameter update of the pre-trained large language model resumes. This safety mode ensures the absolute stability of the system under the impact of continuous low-quality data streams.

[0043] Step S107: Process the target text data from the vertical domain using the trained large language model to generate output results related to the vertical domain.

[0044] This step is the application phase of the large language model after incremental training. After one or more rounds of incremental learning in steps S101 to S106 above, the large language model has absorbed new knowledge from the vertical domain while retaining its memory of old knowledge. At this point, the system receives text data actually input by users from that vertical domain, such as a financial client's inquiry, a lawyer's case inquiry, or a medical report to be processed. The model will understand, reason, and generate from the input target text data based on its acquired knowledge, outputting high-quality, accurate text results that meet the professional requirements of that vertical domain.

[0045] The following example, using an incremental learning scenario in the financial field, serves as a complete embodiment to demonstrate the specific application process of the method of the present invention: A financial institution uses the large language model of this invention as its intelligent customer service question-and-answer system. The model has already learned financial regulations up to the end of 2025. Now, the regulatory agency has released a new "XXX Guidance (2026)," and the model needs to continuously learn from this new document. Meanwhile, the internet is flooded with interpretations of old regulations and outdated information, creating noisy data.

[0046] Step S1: High-quality sample processing (knowledge absorption stage) The input sample is a user question, "What are the principles of portfolio investment in asset management?" The system retrieves the most relevant document from its knowledge base: the full text of the "XXX Guidance (2026)" published on the XXX official website. According to a predefined authority mapping table, documents published by government agencies or standards bodies correspond to a first authority value of 1.0. The model generates interpretations based on the new regulations, calculating the similarity between the generated text and the reference document using a semantic similarity model. =0.92. Taking γ=0.7, then... = 0.7 0.92 + 0.3 1.0 = 0.944. This value is much higher than the low confidence threshold. =0.3. Enter the signal learning area and set... =1000, β=0.5, then λ= 1000 (1 - 0.5 0.944) = 528. This penalty coefficient is much smaller than the baseline value of 1000. The system calculates the gradient of this sample and adds it to the periodic cumulative gradient. Due to the reduced penalty, the model gradient direction tends to reduce the task loss more, effectively learning the new definition.

[0047] Step S2: Low-quality sample processing (noise filtering stage) The input sample is a user question, "Please explain the concept of 'rigid redemption' in the old version." This question implicitly refers to the repealed old regulations. The enhanced search generation system retrieves currently effective regulatory documents (which do not support the concept of rigid redemption), and may also retrieve an outdated analysis article from a personal blog. According to the authority mapping table, personal blogs and other sources correspond to a second authority value of 0.2, therefore... =0.2. The model strives to generate based on the new rules, but it has semantic conflicts with the reference document (the new rules), resulting in a semantic similarity score of 0.2. =0.3. Therefore, the feedback confidence level is... =0.7 0.3+ 0.3 0.2 = 0.27. Because... < (0.3), entering the noise suppression region, setting α=1.0, then λ= 1000 (1+1.0) = 2000. The penalty coefficient is significantly increased to twice the baseline value. The system calculates the gradient of this sample, but at this time the elastic weights are fixed and the regularization term weights are extremely high. The main direction of this gradient is "change of constraint parameters" rather than "learning". Therefore, the overall impact on the periodic cumulative gradient is minimal, effectively preventing the model from learning outdated rigid redemption concepts.

[0048] Step S3: Knowledge consolidation at the end of the training cycle Assuming that high-quality samples (such as interpretations of new regulations) account for 80% during a training cycle, The average score is approximately 0.94; low-quality samples (such as outdated information) account for 20%, and their... The average is approximately 0.2. The calculated average confidence level is 0.8. 0.94 + 0.2 0.2 = 0.792. The preset update threshold is 0.6. Since 0.792 > 0.6, the system determines that the data for this period is generally reliable. The system uses the gradient accumulated in this period to formally update the model parameters. The model successfully grasped the new definitions such as "portfolio investment" and did not learn incorrect knowledge such as "rigid redemption". After the update, the system saves the current model parameters as the new old parameter benchmark, completing the knowledge solidification.

[0049] Step S4: Combat persistent noise (safe mode triggered) The system was attacked by malicious data streams for three consecutive training epochs, with the average confidence level in each epoch falling below 0.6. Upon detecting this three-epoch failure, the system automatically triggered a safety mode. All model parameter updates were paused, and gradient accumulation ceased, but the system continued to run in the background, continuously updating the Fisher information matrix to reinforce the model's memory of old knowledge. Once the subsequent data streams returned to normal and the average confidence level in one epoch reached above 0.6 again, the system automatically exited safety mode and resumed normal training, demonstrating extremely strong robustness.

[0050] As can be seen from this embodiment, the method of the present invention can achieve dual quality control at the sample level and the task level, ensuring that the model can efficiently absorb high-quality data and be robustly immune to low-quality data, and ultimately achieve safe and efficient continuous learning in open and uncontrollable data streams.

[0051] Example 2 See Figure 2 Another embodiment of the present invention provides an incremental learning and text processing apparatus 200 for a large language model, comprising: a data acquisition module 201, a retrieval enhancement generation module 202, a confidence calculation module 203, a dynamic adjustment module 204, a gradient accumulation module 205, a parameter update module 206, and a text processing module 207. The incremental learning and text processing apparatus 200 for the large language model is capable of executing the incremental learning and text processing method for the large language model in the method embodiment.

[0052] Specifically, the incremental learning and text processing device 200 for the large language model includes: The data acquisition module 201 is used to acquire text data in the vertical domain as training samples for the pre-trained large language model. The retrieval enhancement generation module 202 is used to input the training samples into the retrieval enhancement generation system, trigger the retrieval enhancement generation system to retrieve reference documents related to the training samples from the knowledge base, and input the training samples and the reference documents together into the pre-trained large language model to obtain the output text. The confidence calculation module 203 is used to calculate the semantic similarity between the output text and the reference document, and obtain the source authority of the reference document; and calculate the feedback confidence of the training sample based on the semantic similarity and the source authority. The dynamic adjustment module 204 is used to convert the feedback confidence into a dynamic penalty coefficient based on a preset nonlinear mapping function. The dynamic penalty coefficient is used to construct a regularization term in the total loss function. The regularization term is used to constrain the degree to which the parameters of the large language model deviate from the old parameter benchmark during the update process. The gradient accumulation module 205 is used to construct a total loss function using the dynamic penalty coefficient, calculate the parameter gradient corresponding to each training sample based on the total loss function, and accumulate the parameter gradient of each training sample within a training cycle to obtain the periodic accumulated gradient. The parameter update module 206 is used to calculate the average confidence of all training samples in the training period after the training period ends; if the average confidence is greater than or equal to a preset threshold, the model parameters are formally updated according to the periodic cumulative gradient, and the updated model parameters are saved as the new old parameter benchmark; if the average confidence is less than the preset threshold, the periodic cumulative gradient is discarded and the model parameters remain unchanged. The text processing module 207 uses a trained large language model to process target text data from the vertical domain and generates output results related to the vertical domain.

[0053] It should be noted that the incremental learning and text processing device 200 for the large language model provided in this embodiment, which can be used to execute the technical solutions of each method embodiment, has similar implementation principles and technical effects to the methods, and will not be repeated here.

[0054] Example 3 Figure 3 In another embodiment of the present invention, a schematic diagram of an electronic device 300 is provided. This electronic device 300 is used to implement the incremental learning and text processing methods of the large language model in the method embodiment. The electronic device 300 in the embodiments of the present invention may include, but is not limited to, smartphones, tablet computers, PCs, laptops, etc. Figure 3 The electronic device 300 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0055] like Figure 3As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes to implement the methods of the embodiments described herein, based on a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing device 301, ROM 302, and RAM 303 are interconnected via a bus 305. An input / output (I / O) interface 304 is also connected to the bus 305.

[0056] Typically, the following devices can be connected to I / O interface 304: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0057] The above description is merely a preferred embodiment of the present invention. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to the specific combination of the above-described technical features, but should also cover other technical solutions formed by any combination of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

Claims

1. An incremental learning and text processing method for a large language model, characterized in that, Includes the following steps: Acquire text data from vertical domains as training samples for pre-trained large language models; The training samples are input into the retrieval enhancement and generation system, which triggers the system to retrieve reference documents related to the training samples from the knowledge base. The training samples and the reference documents are then input into the pre-trained large language model to obtain the output text. Calculate the semantic similarity between the output text and the reference document, and obtain the source authority of the reference document; calculate the feedback confidence of the training sample based on the semantic similarity and the source authority. The feedback confidence is converted into a dynamic penalty coefficient based on a preset nonlinear mapping function. The dynamic penalty coefficient is used to construct a regularization term in the total loss function. The regularization term is used to constrain the degree to which the parameters of the large language model deviate from the old parameter benchmark during the update process. The total loss function is constructed using the dynamic penalty coefficient. The parameter gradient corresponding to each training sample is calculated based on the total loss function. The parameter gradient of each training sample is accumulated within a training cycle to obtain the periodic cumulative gradient. After the training period ends, the average confidence score of all training samples in the training period is calculated; if the average confidence score is greater than or equal to a preset threshold, the model parameters are formally updated according to the periodic cumulative gradient, and the updated model parameters are saved as the new old parameter benchmark; if the average confidence score is less than the preset threshold, the periodic cumulative gradient is discarded and the model parameters remain unchanged. The trained large language model is used to process target text data from the vertical domain, generating output results related to the vertical domain.

2. The incremental learning and text processing method for a large language model according to claim 1, characterized in that, The feedback confidence level of the training samples is calculated using the following formula: ； in, To provide feedback on confidence level, For semantic similarity, To ensure the authority of the source, This is the preset fusion coefficient.

3. The incremental learning and text processing method for a large language model according to claim 1, characterized in that, The preset nonlinear mapping function is the following piecewise function: in, This is a dynamic penalty coefficient; To provide feedback confidence level; Basic penalty coefficient; Where is the noise amplification factor, and ; Let be the signal attenuation coefficient, and ; This is the low confidence threshold.

4. The incremental learning and text processing method for a large language model according to claim 1, characterized in that, It also includes the steps for switching to safe mode: When the average confidence of all training samples in N consecutive training periods is lower than the preset threshold, the update of the pre-trained large language model parameters is paused, and only the diagonal elements of the Fisher information matrix are cumulatively updated until the average confidence of all training samples in a subsequent training period is greater than or equal to the preset threshold, at which point the update of the pre-trained large language model parameters is resumed.

5. The incremental learning and text processing method for a large language model according to claim 1, characterized in that, The step of calculating the semantic similarity between the output text and the reference document includes: Calculate the cosine similarity between the output text and the vector representation of the reference document, and use the cosine similarity as the semantic similarity; wherein the vector representation of the reference document is generated by a sentence embedding model.

6. The incremental learning and text processing method for a large language model according to claim 1, characterized in that: The authority of the referenced document is obtained from a predefined authority mapping table based on the document's source. In the authority mapping table, reference documents published by governments or standards organizations correspond to a first authority value, while reference documents from other sources correspond to a second authority value that is lower than the first authority value.

7. An incremental learning and text processing device for a large language model, characterized in that, include: The data acquisition module is used to acquire text data from vertical domains as training samples for pre-trained large language models. The retrieval enhancement generation module is used to input the training samples into the retrieval enhancement generation system, trigger the retrieval enhancement generation system to retrieve reference documents related to the training samples from the knowledge base, and input the training samples and the reference documents together into the pre-trained large language model to obtain the output text; The confidence calculation module is used to calculate the semantic similarity between the output text and the reference document, and to obtain the source authority of the reference document; based on the semantic similarity and the source authority, the feedback confidence of the training sample is calculated. The dynamic adjustment module is used to convert the feedback confidence into a dynamic penalty coefficient based on a preset nonlinear mapping function. The dynamic penalty coefficient is used to construct the regularization term in the total loss function. The regularization term is used to constrain the degree to which the parameters of the large language model deviate from the old parameter benchmark during the update process. The gradient accumulation module is used to construct a total loss function using the dynamic penalty coefficient, calculate the parameter gradient corresponding to each training sample based on the total loss function, and accumulate the parameter gradient of each training sample within a training cycle to obtain the periodic accumulated gradient. The parameter update module is used to calculate the average confidence of all training samples during the training period after the training period ends; if the average confidence is greater than or equal to a preset threshold, the model parameters are formally updated according to the periodic cumulative gradient, and the updated model parameters are saved as the new old parameter benchmark; if the average confidence is less than the preset threshold, the periodic cumulative gradient is discarded and the model parameters remain unchanged. The text processing module uses a trained large language model to process target text data from the vertical domain and generate output results related to the vertical domain.

8. The incremental learning and text processing apparatus for a large language model according to claim 7, characterized in that, The confidence calculation module calculates the feedback confidence of the training samples according to the following formula: ； in, To provide feedback on confidence level, For semantic similarity, To ensure the authority of the source, This is the preset fusion coefficient.

9. The incremental learning and text processing apparatus for a large language model according to claim 7, characterized in that, The nonlinear mapping function in the dynamic control module is the following piecewise function: in, This is a dynamic penalty coefficient; To provide feedback confidence level; Basic penalty coefficient; Where is the noise amplification factor, and ; Let be the signal attenuation coefficient, and ; This is the low confidence threshold.

10. The incremental learning and text processing apparatus for a large language model according to claim 7, characterized in that, The parameter update module is further used for: When the average confidence of all training samples in N consecutive training periods is lower than the preset threshold, the update of the pre-trained large language model parameters is paused, and only the diagonal elements of the Fisher information matrix are cumulatively updated until the average confidence of all training samples in a subsequent training period is greater than or equal to the preset threshold, at which point the update of the pre-trained large language model parameters is resumed.

Images