Big language model knowledge ability continuous expansion method based on dual-mode dynamic loop

Abstract

The invention relates to the technical field of natural language processing, and discloses a method for continuously expanding knowledge ability of a large language model based on dual-mode dynamic circulation, and the method comprises the steps: controlling the LLM to be circularly switched between a waking mode and a sleep mode through a mode conversion mechanism unit: in the waking mode, collecting and cleaning new data by an RAG module to construct a knowledge base, and in the sleep mode, collecting and cleaning new data; based on user questions, knowledge is retrieved, and VertiLLM is called to generate answers; when the data volume reaches a threshold value and resources are idle, switching to a sleep mode, starting a continuous training module, generating training data by using the cleaning data to carry out parameter adjustment on LLM, generating a new VertiLLM model subjected to test verification, updating the new VertiLLM model to an RAG module, and deleting trained data at the same time; the RAG module improves instant knowledge through a retrieval enhancement technology, and the continuous training module consolidates professional knowledge through continuous training. According to the method, dynamic continuous expansion of large language model knowledge ability is realized, the problem of knowledge updating lag of a traditional model is solved, and the method has the characteristics of high automation degree, high resource utilization rate and the like.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of natural language processing, and particularly relates to a large language model knowledge ability continuous expansion method based on a dual-mode dynamic cycle. BACKGROUND

[0002] With the release of ChatGPT model by OpenAI in December 2022, the application of large language models (LLM) has rapidly expanded in various fields, from daily communication to enterprise operations and even scientific research. However, despite the significant progress made by existing large language models (LLM), there are limitations in specific domains or knowledge-intensive tasks. Specifically, LLMs are prone to "hallucinations" when generating inaccurate or fabricated information in situations requiring up-to-date or specialized knowledge. This is mainly because LLMs draw their knowledge from pre-training data, which cannot be updated in a timely manner, and it is difficult to deeply understand specialized knowledge in specific domains.

[0003] To address these issues, researchers have introduced RAG technology and continued training (such as fine-tuning and continued pre-training): RAG enhances the performance of LLM by combining external knowledge bases to reduce the occurrence of factual errors; while continued training adjusts parameters based on existing pre-trained models to adapt to specific domain data and requirements. However, RAG technology, although not changing the LLM itself, relies heavily on the retrieval stage, and its cost, speed, and response quality are heavily dependent on vector databases; continued training, while improving the model's understanding of specialized fields, requires a large amount of carefully planned data sets and extensive experiments to verify parameter configurations, with uncertain time consumption.

[0004] Therefore, a method is needed that combines the advantages of RAG technology and continued training while overcoming their respective shortcomings to improve the performance and knowledge updating ability of LLM in specific domains. SUMMARY

[0005] The present application addresses the deficiencies of the prior art by providing a large language model knowledge ability continuous expansion method based on a dual-mode dynamic cycle to address the slow knowledge update and low response efficiency of LLM in specific domain applications, and to achieve continuous expansion of LLM knowledge ability.

[0006] The present application proposes a large language model knowledge ability continuous expansion method based on a dual-mode dynamic cycle, which comprises:

[0007] Step 1: Control LLM to cycle between wakefulness and sleep modes through a modal conversion mechanism unit, RAG module collects and cleans new data, constructs a knowledge base, retrieves the knowledge base based on user questions, assembles Prompt, and calls VertiLLM model to generate answers to user questions;

[0008] Step 2: When the new data collected by the RAG module reaches the preset threshold and the computing resources are idle, the modal conversion mechanism unit switches the LLM to the sleep mode, the RAG module stops collecting new data, and the retraining module is started;

[0009] Step 3: The retraining module generates training data using the cleaned data, adjusts the parameters of the LLM using the training method to generate a new model VertiLLM, updates the new model VertiLLM to the RAG module after testing and verification, and deletes the trained data in the RAG knowledge base;

[0010] Step 4: After the RAG module data is updated, the modal conversion mechanism unit switches the LLM to the awake mode, and the above operations are repeated to realize the continuous expansion of the knowledge and ability of the large language model.

[0011] Further, in the large language model knowledge and ability continuous expansion method based on the dual-mode dynamic cycle, the RAG module runs in the LLM awake mode, improves the instant knowledge of the LLM through the RAG technology, and enhances the short-term memory;

[0012] The retraining module runs in the LLM sleep mode, consolidates the professional field knowledge of the LLM through continuous training, and deepens the long-term memory;

[0013] The modal conversion mechanism unit is used to control the cycle conversion between the LLM awake mode and the LLM sleep mode.

[0014] Further, in step 1, the modal conversion mechanism unit controls the LLM to be in the awake mode, the RAG module collects and cleanses new data, constructs a knowledge base, retrieves the knowledge base based on user questions, assembles a Prompt, and calls the VertiLLM model to generate an answer to the user question, and the specific process includes:

[0015] Step S101: Collect new data, and use the automatic tool pandas to clean the collected new data;

[0016] Step S102: Organize the cleaned data into a structured form and construct a knowledge base for subsequent retrieval;

[0017] Step S103: Perform fusion retrieval on the knowledge base based on user questions, and return the retrieval result;

[0018] Step S104: Assemble a Prompt based on the returned retrieval result;

[0019] Step S105: Deploy the model, start and call the model to generate an answer to the user question according to the assembled Prompt.

[0020] Further, in step S101, the new data includes word, excel, pdf file data; the collected new data is cleaned using the automation tool pandas, including identifying and removing duplicate data entries, identifying and filling missing values, identifying and modifying errors, inconsistent data entries.

[0021] Further, in step S102, the cleaned data is organized into a structured form and constructed into a knowledge base for subsequent retrieval, the specific process including the following steps:

[0022] Step S102-1: Use the langchain tool to extract text data from the cleaned data and store it as a markdown format document, then split it according to the markdown title to get the split document fragments;

[0023] Step S102-2: Use one of the BGE-M3 vector model or the Text2vec vector model to vectorize the split document fragments and store them in the Faiss vector database to form the knowledge base.

[0024] Further, in step S103, the knowledge base is fused and retrieved based on the user question, and the retrieval result is returned, specifically: using one of the BGE-M3 vector model or the Text2vec vector model to vectorize the user question, the RAG module performs fusion retrieval in the knowledge base based on the vectorized user question, obtains the most relevant document fragments and provides them to the large model; wherein the fusion retrieval is performed using the BM25 model and the semantic retrieval based on the vector model, specifically including the following steps:

[0025] Step S103-1: preprocessing stage, first preprocessing the user question, i.e. word segmentation, removing stop words;

[0026] Step S103-2: BM25 scoring, using the BM25 algorithm to score each document according to the keyword matching degree;

[0027] Step S103-3: semantic encoding, converting the user question and the document into a vector representation generated by the vector model;

[0028] Step S103-4: semantic scoring, calculating the similarity score between the user query vector and each document vector as a measure of semantic relevance;

[0029] Step S103-5: comprehensive scoring, combining the BM25 score and the semantic similarity score according to the pre-set 6:4 ratio to get the final comprehensive score;

[0030] Step S103-6: Sort and return the results, sort all documents according to the comprehensive score, and select the top 3 documents with the highest score as the search results.

[0031] Further, in step S104, the Prompt is a designed input sequence, including three main contents: user questions, role positioning in professional fields, and known information spliced with search information.

[0032] Further, in step S105, the deployment model is deployed using vLLM and Ollama tools, and API services are provided for RAG module calling; the specific process includes the following steps:

[0033] Step S105-1: Download and install vLLM using pip tool;

[0034] Step S105-2: Start the model service for the first time, specify the model address as the storage address of the general LLM model, and update the storage address of the VertiLLM model after subsequent training;

[0035] Step S105-3: Start the model service, and start the online service of vLLM using the python command line;

[0036] Step S105-4: Call the model to generate answers, and the model answers user questions based on the fusion of search results.

[0037] Further, in step 3, the retraining module generates training data using the cleaned data, uses a training method to adjust the parameters of LLM to generate a new model VertiLLM, and updates the new model VertiLLM to the RAG module after testing and verification, and deletes the trained data in the RAG knowledge base:

[0038] The retraining module generates training data using the cleaned data, which is to extract the cleaned text data from the RAG module to assemble into json format text data, and extract the json key-value pairs to generate training data, wherein the json format text data contains three keys: "instruction", "input", and "ouput", representing questions or instructions, inputs and outputs, respectively;

[0039] The new model VertiLLM is generated by using a training method to adjust the parameters of LLM, which is to use the fine-tuning training method to adjust the parameters of LLM based on the generated training data, so that the original pre-training model LLM is aligned with the data distribution of the target task, and the new model VertiLLM is obtained;

[0040] The new model VertiLLM is updated into the RAG module after being tested and verified, specifically, M pieces of data are randomly extracted from the training data, K pieces of test data with consistent topics are generated by calling RAG, and the fine-tuned model is verified and evaluated.

[0041] Further, in step 1, the modal conversion mechanism unit controls the LLM to be in the wake mode, the RAG module collects and cleans new data, constructs a knowledge base, retrieves the knowledge base based on the user question, assembles the Prompt, and calls the VertiLLM model to generate the answer to the user question, and further includes:

[0042] When retrieving the knowledge base based on the user question, it is necessary to determine whether the model in the RAG module is known for the user question, specifically, by setting a prompt word, the model in the RAG module is called to identify the current question, that is, whether the VertiLLM model after the continuation training module is identified in the RAG module; if it is the VertiLLM model after the continuation training module, it indicates that the answer of the model to the current user question is known, and has the ability to answer, and can directly assemble the Prompt and call the VertiLLM model to generate the answer to the user question; otherwise, fusion retrieval is performed in the constructed knowledge base according to the user question.

[0043] Compared with the prior art, the present application has the following advantages:

[0044] 1. Optimizing knowledge updating and response efficiency: through the dual-modal mechanism, the LLM quickly responds to user demand in the wake mode, and consolidates professional knowledge in the sleep mode, realizing continuous updating and efficient response of knowledge.

[0045] 2. Reduce dependence on external knowledge base: continuation training reduces the dependence of LLM on external knowledge base, reduces the capacity demand of knowledge base, and improves the retrieval efficiency.

[0046] 3. Overcome the limitations of single method: combining the advantages of RAG and continuation training, the shortcomings of each single use are made up, so that the LLM can maintain efficient learning ability while quickly responding.

[0047] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and obtained by the structure indicated in the specification, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0049] Figure 1 is a flow chart of a large language model knowledge ability continuous expansion method based on a dual-mode dynamic cycle;

[0050] Figure 2 is a RAG module flow chart of a large language model knowledge ability continuous expansion method based on a dual-mode dynamic cycle;

[0051] Figure 3 is a retraining module flow chart of a large language model knowledge ability continuous expansion method based on a dual-mode dynamic cycle;

[0052] Figure 4 is a test flow chart of a large language model knowledge ability continuous expansion method based on a dual-mode dynamic cycle. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0054] Embodiment 1:

[0055] In this embodiment, please refer to Figure 1 , a large language model knowledge ability continuous expansion method based on a dual-mode dynamic cycle is proposed, and the method steps include:

[0056] Step S1: control the LLM to switch between the wake and sleep modes through the modal conversion mechanism unit, the RAG module collects and cleanses new data, constructs a knowledge base, retrieves the knowledge base based on user questions, assembles a Prompt, and calls the VertiLLM model to generate an answer to the user question;

[0057] Step S2: when the new data collected by the RAG module reaches a preset threshold and the computing resources are idle, the modal conversion mechanism unit switches the LLM to the sleep mode, the RAG module stops collecting new data, and the retraining module is started;

[0058] Step S3: The retraining module generates training data using the cleaned data, adjusts the parameters of the LLM using a training method to generate a new model VertiLLM, and updates the new model VertiLLM to the RAG module after testing and verification, while deleting the trained data in the RAG knowledge base;

[0059] Step S4: After the RAG module data is updated, the modal conversion mechanism unit switches the LLM to the wakeful mode, and repeats the above steps S1-S3 to realize the continuous expansion of the knowledge and ability of the large language model.

[0060] Further, in the large language model knowledge and ability continuous expansion method based on the dual-mode dynamic cycle, the modal conversion mechanism unit, the RAG module, and the retraining module jointly support the iterative cycle memory and optimization of the LLM in the wakeful and sleep modes, promote the LLM to be more powerful with use, and ultimately break through the limitations of memory, wherein:

[0061] The modal conversion mechanism unit is configured to control the cycle conversion and continuous iteration between the wakeful mode and the sleep mode of the LLM, realize the continuous updating and optimization of the knowledge of the LLM, and achieve the wakeful mode and the sleep mode of the LLM.

[0062] The RAG module is configured to run in the wakeful mode of the LLM, improve the instant knowledge of the LLM through the RAG technology, and enhance the short-term memory.

[0063] The retraining module is configured to run in the sleep mode of the LLM, consolidate the professional field knowledge of the LLM through continuous training, and deepen the long-term memory.

[0064] Embodiment 2:

[0065] Based on the above embodiment 1, please refer to Figure 2 The modal conversion mechanism unit controls the LLM to be in the wakeful mode, the RAG module collects and cleanses new data, constructs a knowledge base, retrieves the knowledge base based on a user question, assembles a Prompt, and calls the VertiLLM model to generate an answer to the user question, and specifically includes the following steps:

[0066] Step S101: Collect and clean data: collect new data, and use the automated tool pandas to clean the collected new data;

[0067] Further, the new data includes word, excel, and pdf file data; and the use of the automated tool pandas to clean the collected new data includes identifying and removing duplicate data entries, identifying and filling missing values, identifying and modifying incorrect and inconsistent data entries.

[0068] Step S102: Construct a knowledge base: organize the cleaned data into a structured form for subsequent retrieval;

[0069] Further, the cleaning of the data into a structured form specifically includes:

[0070] Step S102-1: Extract text data from the cleaned data using the langchain tool, store it as a markdown format document, and then split it according to the markdown title to obtain the split document fragments;

[0071] Step S102-2: Vectorize the split document fragments using a vector model and store them in a Faiss vector database to form a knowledge base for subsequent retrieval processes to improve retrieval efficiency.

[0072] Further, the vector model includes a BGE-M3 model and a Text2vec model.

[0073] Further, in this embodiment, the BGE-M3 vector model is used as an example. The BGE-M3 vector model supports more than 100 languages and provides excellent multilingual and cross-language retrieval capabilities. This model can comprehensively and high-quality process different granularity text inputs from sentences, paragraphs, chapters to documents, with a maximum input length of 8192 tokens. It integrates dense retrieval, sparse retrieval, and multi-vector retrieval functions, and meets the diverse retrieval needs in one station.

[0074] Step S103: Perform fusion retrieval on the knowledge base based on the user question and return the retrieval result;

[0075] Further, in step S103, the fusion retrieval on the knowledge base based on the user question and returning the retrieval result is specifically: vectorizing the user question using the vector model, and the RAG module performs fusion retrieval in the knowledge base based on the vectorized user question to obtain the most relevant document fragments and provide them to the large model.

[0076] Further, in this embodiment, the vector model includes a BGE-M3 model and a Text2vec model. The fusion retrieval uses a BM25 model and a semantic retrieval based on a vector model, which is a fusion retrieval strategy combining the advantages of traditional information retrieval technology and modern deep learning technology.

[0077] The BM25 model is a widely used probability ranking model based on Term Frequency-Inverse Document Frequency (TF-IDF) improvement. It calculates the relevance score between the query and the document by considering the frequency of words in the entire document and their distribution in the entire document set. The expression of the BM25 model is:

[0078]

[0079] Where q i represents the i-th query word, where i∈[1,n], n is a natural number, representing the number of query words; f(q i ,D) represents the i-th query word q i The word frequency in document D, i.e. the i-th query word q i The number of times a word appears in document D, where the higher the word frequency, the higher the relevance score of the document to the query; |D| represents the length of document D, measured in words. The document length is used to adjust the score balance between short and long documents; avgdl represents the average document length of the document collection, which is used to standardize the document length. By comparing the length of a single document with the average document length, the impact on the document length can be adjusted; k1 represents the adjustment parameter for controlling the degree of word frequency saturation, with a value range of 1.2 <k1<2,k1值越高,词频对得分的影响越弱;b表示控制文档长度的归一化程度的调节参数,取值范围为0<b<1,b越接近1,文档长度对得分的影响越大;IDF(q i ) represents the query word q i Inverse Document Frequency, used to measure the query term q i The rarity in the document collection; N represents the total number of documents in the document collection; m(q i ) indicates that the query word q is included i The number of documents; inverse document frequency IDF(q i ) is higher, the rarer the word is, and the higher the score is;

[0080] The vector-based model maps document fragments in the knowledge base to a high-dimensional vector space. In the high-dimensional vector space, semantically similar texts are close to each other. This means that even if two texts use different words but have similar meanings, such as "woman" and "queen", they are very close in the vector space and can be identified as related in the vector space. This is particularly useful for understanding complex or ambiguous user questions because it goes beyond simple word matching and enters into semantic-level understanding.

[0081] Furthermore, the fusion search adopts the BM25 model and the semantic search based on the vector model, which specifically includes the following steps:

[0082] Step S103-1: Preprocessing stage: First, preprocess the user question, i.e., perform word segmentation and remove stop words;

[0083] Step S103-2: BM25 scoring, using the BM25 algorithm to score each document based on keyword matching;

[0084] Step S103-3: semantic encoding, converting the user question and the document into vector representations generated by a vector-based model;

[0085] Step S103-4: semantic scoring, calculating the similarity score between the user query vector and each document vector as a measure of semantic relevance;

[0086] Step S103-5: integrated scoring, combining the BM25 score and the semantic similarity score in a pre-set ratio of 6:4 to obtain the final integrated score;

[0087] Step S103-6: ranking and returning results, ranking all documents according to the integrated score, and selecting the top 3 documents with the highest scores as the search results.

[0088] Step S104: assembling Prompt based on the returned search results;

[0089] Further, the Prompt is a designed input sequence, containing three main contents, one is the role positioning of the professional field business assistant, which can answer user questions according to known information; the second is the known information spliced with search information; and the last is the user question.

[0090] Step S105: deploying the model, starting and calling the model to generate the answer to the user question according to the assembled Prompt.

[0091] Further, the model is deployed, specifically using vLLM (Virtual Large Language Model), Ollama, etc. for deployment, and providing API services for RAG module calling; wherein:

[0092] The vLLM is a high-efficiency system for LLM inference, aiming to accelerate the performance of LLM in the inference stage by reducing computational overhead and improving throughput; the key features of vLLM include dynamic tensor parallelism, asynchronous inference and efficient memory management, which enable vLLM to deploy more complex language models under limited hardware resources and provide more intelligent dialogue generation experience.

[0093] Further, in step S105, the calling model includes the following steps:

[0094] Step S105-1: install vLLM, download and install vLLM using pip tool;

[0095] Step S105-2: specify the model address, initially start the model service, which can be specified as the storage address of the general LLM model, and subsequently updated to the storage address of the VertiLLM model after retraining;

[0096] In this embodiment, the base model is ChatGLM-9B-Chat;

[0097] Step S105-3: Start the model service, start the online service of vLLM using the python command line;

[0098] Step S105-4: Call the model to generate an answer, the model answers the user's question based on the fusion search results.

[0099] Further, in step S1, the modal conversion mechanism unit controls the LLM to be in the sober mode, the RAG module collects and cleans new data, constructs a knowledge base, searches the knowledge base based on the user's question, assembles a Prompt, and calls the VertiLLM model to generate an answer to the user's question, and further includes:

[0100] When searching the knowledge base based on the user's question, it is necessary to determine whether the model in the RAG module is known for the user's question. Specifically, by setting a prompt word, the model in the RAG module is called to identify the current question, i.e., whether the model in the RAG module is a VertiLLM model after the continuation training module. If it is a VertiLLM model after the continuation training module, it indicates that the model has known the answer to the current user's question and has the ability to answer, and can directly assemble a Prompt and call the VertiLLM model to generate an answer to the user's question. Otherwise, fusion search is performed in the constructed knowledge base according to the user's question.

[0101] Embodiment 3:

[0102] In this embodiment, please refer to Figure 3 On the basis of the above-mentioned embodiment 1, the step S3: the continuation training module generates training data using the cleaned data, adjusts the parameters of the LLM using a training method to generate a new model VertiLLM, and updates the new model VertiLLM to the RAG module after testing and verification, and deletes the trained data in the RAG knowledge base, and further includes:

[0103] The continuation training module generates training data using the cleaned data, specifically extracts the cleaned text data from the RAG module to assemble into json format text data, and extracts the json key-value pairs to generate training data;

[0104] Further, the json format text data includes three keys of "instruction", "input", and "ouput" for each data, respectively representing the question or instruction, the input, and the output.

[0105] The use of the training method adjusts the parameters of the LLM to generate a new model VertiLLM, specifically: using the continued pre-training method or the fine-tuning method to adjust the parameters of the LLM to the generated training data, so that the original pre-trained model LLM aligns with the data distribution of the target task, and obtains a new model VertiLLM, wherein:

[0106] The continued pre-training method is a process of re-pre-training a specific domain data set based on an already pre-trained model, which is suitable for scenarios where there is a large difference between the pre-trained model and the data distribution of the target task, and the data volume is sufficient.

[0107] The fine-tuning usually uses a small-scale data set with clear task labels to adjust its parameters to optimize the performance of a specific task; the advantage of fine-tuning is that it can quickly adjust model parameters using a relatively small amount of labeled data to achieve the purpose of optimizing the performance of a specific task.

[0108] Further, the fine-tuning method includes Fine-Tuning, P-tuning, and LoRA (Low-Rank Adaptation); wherein the Fine-Tuning is the most direct method, which adjusts all parameters of the entire model, this method can maximize the use of data in the target field, and usually can obtain the best performance improvement, the disadvantage is that the computing resource consumption is large, and a large amount of GPU memory and training time is required, especially in the case of limited data, which is prone to overfitting; the P-tuning is a lighter fine-tuning method, which adjusts the behavior of the model by introducing a learnable prompt embedding, rather than directly modifying the model's parameters, this method can achieve effective fine-tuning without changing the original model weights, thereby reducing the demand for computing resources, but on complex tasks, the effect of P-tuning may not be as significant as Fine-Tuning.

[0109] In this embodiment, the LoRA (Low-Rank Adaptation) method in the fine-tuning method is taken as an example for illustration, the LoRA (Low-Rank Adaptation) method is the most commonly used method for efficient parameter fine-tuning, its core idea is to inject a trainable low-rank decomposition matrix into each layer of the Transformer architecture after freezing the pre-trained model weights, thereby greatly reducing the number of trainable parameters on the downstream task.

[0110] Further, in step S3, please refer to Figure 4The new model VertiLLM is updated into the RAG module after being tested and verified, specifically: after the model fine-tuning ends and enters the testing stage, M pieces of data are randomly extracted from the training data, K pieces of test data with consistent topics are generated by calling the RAG, and the fine-tuned model is verified and evaluated;

[0111] Further, in order to evaluate the consistency of the model output with the true facts, to identify and reduce errors, common evaluation indicators include accuracy AC, BLEU and ROUGE-L, wherein:

[0112] The AC is an indicator for measuring the proportion of correct predictions or generated results of the model, and its expression is:

[0113]

[0114] In the formula, TP is the number of correctly predicted samples, that is, the number of responses accurately predicted by the model;

[0115] The BLEU is an indicator for evaluating the quality of generated text by comparing the n-gram overlap degree of the model output and the reference text, and its expression is:

[0116]

[0117] In the formula, BP is a short sentence penalty factor, c is the length of the model answer, a is the length of the shortest reference answer, w n is an n-gram weight, and p n is an n-gram matching degree;

[0118] The ROUGE-L is an n-gram optimized to a common subsequence and allows discontinuous matching to occur, and its expression is:

[0119]

[0120] In the formula, LCS represents the longest common subsequence, m is the length of the reference answer, n is the length of the model answer, and β is a hyperparameter.

[0121] In this embodiment, taking AC as an example, when the AC accuracy is greater than a threshold T, it is considered that the model training is completed, otherwise, it is judged whether the model reaches the maximum training step N, if yes, the training strategy is adjusted, such as reducing or expanding the r of LoRA to enhance the performance of the model, and the retraining process is entered again, otherwise, the retraining is continued from the breakpoint. Wherein, T is a configurable hyperparameter. After the retraining is completed, the retrained model is saved in time.

[0122] Embodiment 4:

[0123] In this embodiment, based on the same inventive concept as in the above embodiment, a large language model knowledge capability continuous expansion system based on a dual-mode dynamic cycle is provided, the system comprises:

[0124] A modal conversion mechanism unit is configured to control switching of the large language model between the wake-up mode and the sleep mode.

[0125] Further, the modal conversion mechanism unit further comprises a state detector configured to monitor data accumulation amount and CPU / GPU utilization rate; a switching controller configured to trigger modal conversion according to a preset threshold; and a log recording device configured to save all state change records.

[0126] An RAG processing module is configured to work in the wake-up mode of the large language model, and comprises a data collection sub-module, a data cleaning engine, a vector knowledge base, a Prompt assembler, and a VertiLLM calling interface.

[0127] Further, the data collection sub-module is configured to collect external new data in real time; the data cleaning engine is configured to perform standardization processing on original data; the vector knowledge base is configured to store structured knowledge data; and the Prompt assembler is configured to dynamically generate query instructions.

[0128] Further, the RAG processing module adopts a hierarchical architecture design, and comprises a data access layer (supporting multiple access methods such as API / crawler), a data processing layer (ETL pipeline), and a service layer (query response engine).

[0129] A retraining module is configured to work in the sleep mode of the large language model, and comprises a training data generator, a parameter optimizer, a model verification platform, and a version management system.

[0130] Further, the retraining module comprises an incremental learning algorithm library, a distributed training framework, a model evaluation index system, and an automatic deployment tool chain.

[0131] A resource monitoring unit is configured to monitor the usage state of computing resources in real time.

[0132] Embodiment 5:

[0133] In this embodiment, a computer electronic device is provided, which comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement a large language model knowledge capability continuous expansion method based on a dual-mode dynamic cycle as described above.

[0134] Embodiment 6:

[0135] In the embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement a method for continuously expanding a large language model knowledge capability based on a dual-mode dynamic cycle.

[0136] In the present application, a plurality of professional terms are involved, and the specific term meanings are shown in the following Table 1: Term Explanation.

[0137] Table 1: Term Explanation

[0138]

[0139]

[0140] Although the present application is described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for continuously expanding the knowledge capabilities of a large language model based on a dual-mode dynamic cycle, characterized in that: The method comprises: The modal conversion mechanism unit controls the LLM to be in the awake mode. The RAG module collects and cleans new data, builds a knowledge base, searches the knowledge base based on user questions, assembles prompts, and calls the VertiLLM model to generate answers to user questions. When the new data collected by the RAG module reaches the preset threshold and the computing resources are idle, the mode conversion mechanism unit controls the LLM to switch to the sleep mode, the RAG module stops collecting new data, and starts the continuous training module; The continuous training module generates training data using the cleaned data, and uses the training method to adjust the parameters of LLM to generate a new model VertiLLM. After testing and verification, the new model VertiLLM is updated to the RAG module, and the trained data in the RAG knowledge base is deleted. After the RAG module data is updated, the modality conversion mechanism unit switches the LLM to the awake mode and repeats the above operations to achieve continuous expansion of the knowledge capabilities of the large language model; in: The RAG module operates in the LLM awake mode and improves the LLM's immediate knowledge and enhances short-term memory through RAG technology; The continuous training module runs in the LLM sleep mode, consolidating the LLM's professional knowledge and deepening long-term memory through continued training; The mode conversion mechanism unit is used to control the cyclic conversion between the LLM awake mode and the sleep mode.

2. The method for continuously expanding the knowledge capability of a large language model based on a dual-mode dynamic cycle as claimed in claim 1, characterized in that: The LLM is in awake mode. The RAG module collects and cleans new data, builds a knowledge base, searches the knowledge base based on user questions, assembles prompts, and calls the VertiLLM model to generate answers to user questions. The specific process includes: Collect new data and use the automated tool pandas to clean the collected new data; Organize the cleaned data into a structured form and build a knowledge base for subsequent retrieval; Perform fusion search on the knowledge base based on user questions and return the search results; Assemble prompts based on the returned search results; Deploy the model, start and call the model to generate answers to user questions based on the assembled prompt.

3. The method for continuously expanding the knowledge capability of a large language model based on a dual-mode dynamic cycle as described in claim 2, characterized in that: In the process of collecting new data, we use the automated tool pandas to clean the collected new data: The new data includes word, excel, and pdf file data; The automated tool pandas is used to clean the collected new data, including identifying and removing duplicate data entries, identifying and filling missing values, and identifying and correcting erroneous and inconsistent data entries.

4. The method for continuously expanding the knowledge capability of a large language model based on a dual-mode dynamic cycle as described in claim 2, characterized in that: The cleaned data is organized into a structured form and built into a knowledge base for subsequent retrieval. The specific process includes: Use the langchain tool to extract text data from the cleaned data, store it in markdown format, and then segment it according to the markdown title to obtain the segmented document fragments; Use one of the BGE-M3 vector models or the Text2vec vector model to vectorize the segmented document fragments and store them in the Faiss vector database to form a knowledge base.

5. The method for continuously expanding the knowledge capability of a large language model based on a dual-mode dynamic cycle as claimed in claim 2, characterized in that: The fusion search of the knowledge base based on the user question and the return of the search results are specifically as follows: the user question is vectorized using one of the BGE-M3 vector model or the Text2vec vector model, and the RAG module performs a fusion search in the knowledge base based on the vectorized user question, obtains the most relevant document fragments and provides them to the large model, wherein the fusion search adopts the BM25 model and the vector model-based semantic search, specifically including: In the preprocessing stage, user questions are first preprocessed, that is, word segmentation and stop word removal; BM25 scoring, using the BM25 algorithm to score each document based on keyword matching; Semantic encoding, which converts user questions and documents into vector representations generated by a vector-based model; Semantic scoring, which calculates the similarity score between the user query vector and each document vector as a measure of semantic relevance; Comprehensive score: The BM25 score and semantic similarity score are combined in a pre-set ratio of 6:4 to obtain the final comprehensive score; Sort and return the results. Sort all documents according to the comprehensive score and select the three documents with the highest scores as the search results.

6. The method for continuously expanding the knowledge capabilities of a large language model based on a dual-mode dynamic cycle as described in claim 2, characterized in that: The prompt is a designed input sequence, which includes three main contents: user questions, role positioning in professional fields, and known information spliced ​​with retrieval information.

7. The method for continuously expanding the knowledge capability of a large language model based on a dual-mode dynamic cycle as described in claim 2, characterized in that: The deployment model is specifically deployed using vLLM and Ollama tools, while providing API services for RAG modules to call; The specific process of deploying the model includes: Use pip to download and install vLLM; When you start the model service for the first time, specify the model address as the storage address of the general LLM model. It will be updated to the storage address of the VertiLLM model after continued training. Start the model service and use the Python command line to start the vLLM online service; The model is called to generate answers. Based on the fusion search results, the LLM intelligently answers user questions.

8. The method for continuously expanding the knowledge capability of a large language model based on a dual-mode dynamic cycle as claimed in claim 1, characterized in that: The continuous training module generates training data using the cleaned data, uses the training method to adjust the parameters of the LLM to generate a new model VertiLLM, updates the new model VertiLLM to the RAG module after testing and verification, and deletes the trained data in the RAG knowledge base at the same time, wherein: The continuous training module generates training data using the cleaned data, specifically extracting the cleaned text data from the RAG module to assemble it into text data in json format, and extracting json key-value pairs to generate training data; The method uses the training method to adjust the parameters of the LLM to generate a new model VertiLLM, specifically, the fine-tuning training method is used to adjust the parameters of the LLM on the generated training data so that the original pre-trained model LLM is aligned with the data distribution of the target task to obtain the new model VertiLLM; The new model VertiLLM is updated to the RAG module after testing and verification. Specifically, M data are randomly extracted from the training data, and RAG is called to generate K test data with the same theme, and the fine-tuned model is verified and evaluated.

9. The method for continuously expanding the knowledge capabilities of a large language model based on a dual-mode dynamic cycle as claimed in claim 8, characterized in that: Each of the JSON formatted text data contains three keys: "instruction", "input" and "ouput", representing questions or instructions, input and output respectively.

10. A method for continuously expanding the knowledge capabilities of a large language model based on a dual-mode dynamic cycle according to any one of claims 1 to 9, characterized in that: When searching the knowledge base based on user questions, the method needs to determine whether the model in the RAG module is known to the user question, specifically: By setting the prompt word, the model in the RAG module is called to identify the current problem, that is, to identify whether the model in the RAG module is the VertiLLM model after the continuous training module; If the VertiLLM model has undergone the retraining module, it means that the model knows the answer to the current user question and is capable of answering it. You can directly assemble Prompt and call the VertiLLM model to generate the answer to the user question. Otherwise, a fusion search is performed in the constructed knowledge base based on the user question.

Images