Large language model training method and device, equipment and storage medium
By screening and constructing Q&A samples and rejection samples in the training stage of the large language model, the low accuracy problem caused by information redundancy and uncorrelation in the training stage of the large language model is solved, and the accuracy and rejection ability of the model are improved.
Patent Information
- Application Number
- CN202411570038.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-06
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
During the training stage, large language models have low accuracy due to the information that is not related to the problem, redundant or wrong in the retrieved information.
By obtaining the original training dataset during the training stage of the large language model, searching based on the knowledge base, filtering out relevant and unrelated documents, building Q&A samples and rejection samples, and using these samples to train the model to improve the model's Accuracy and rejection ability.
By denoising related documents and constructing rejection samples, the accuracy and rejection ability of large language models are improved, the risk of generating irrelevant content is reduced, and the overall performance of the model is improved.
Smart Images

Figure CN119066155A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of natural language processing, and specifically relates to a training method, device, equipment and storage medium for a large language model. Background Art
[0002] Pre-trained large language models have shown amazing capabilities in natural language processing tasks. Despite this, large language models still have some limitations. When dealing with problems in specific fields or immediate questions, since the large model has not seen relevant knowledge during training, it may generate responses containing incorrect information. This phenomenon is also called the hallucination of the large language model. Using retrieved documents to assist in answer generation (retrieval enhancement) is an effective way to alleviate hallucinations. The specific approach is to use retrieval technology to obtain some knowledge documents related to the question, and then input them into the large language model together with the question.
[0003] Generally, retrieval enhancement is only used in the testing phase of the large language model. In order to avoid inconsistencies between the model training and testing phases and limit the effect of knowledge introduction and the final performance, retrieval enhancement methods are also currently applied in the training phase of the large language model. However, the current retrieval enhancement in the training phase directly inputs the retrieved information into the large language model. The retrieved information may contain information that is irrelevant to the problem, redundant, or even erroneous, resulting in a low accuracy rate for the trained large language model. Summary of the invention
[0004] In order to solve the problem of low accuracy of large language models, the present invention provides a training method, apparatus, device and storage medium for a large language model.
[0005] In order to achieve the above object, the present invention provides the following technical solutions: A large language model training method, comprising: Obtain the original training data set of the large language model, which includes questions and answers corresponding to the questions; search the knowledge base based on the questions in the original training data set to obtain multiple relevant documents and irrelevant documents of the questions; Predict the answers and corresponding probabilities of the questions in multiple related documents, compare the predicted probabilities with the preset threshold, and if the predicted probability corresponding to the target answer is greater than or equal to the preset threshold, take the document containing the target answer as the target document; Constructing question-answer samples based on target documents and constructing refusal-to-answer samples based on irrelevant documents; wherein the question-answer samples include questions, target documents and answers, and the refusal-to-answer samples include questions, irrelevant documents and preset refusal-to-answer templates; The large language model is trained based on question-answer samples and refusal samples.
[0006] Optionally, before searching the knowledge base based on the questions in the original training dataset, the following is further included: The documents in the knowledge base are segmented according to a fixed length to obtain multiple document blocks, and two adjacent document blocks include overlapping areas; Based on the questions in the original training data set, we search the knowledge base and get multiple relevant documents for the questions, including: Embed each document block to obtain multiple vectors, perform clustering operations on the multiple vectors, and determine the cluster center; Embed the question to get the question vector, and find w cluster centers that are close to the question vector among the cluster centers; Traverse the postings chains of w cluster centers, calculate the distance between the problem vector and the vector on the postings chain, and put the vector with the closest distance The vector corresponding to Document blocks as related documents.
[0007] Optionally, in the case of a The vector corresponding to After the document block as related documents, it also includes: Through the BGE-Reranker model The document blocks are reordered and the top similarity rankings are selected. The document block of the location is used as related documents. .
[0008] Optionally, predicting answers to the question in multiple related documents and corresponding probabilities, comparing the predicted probabilities with a preset threshold, and if the predicted probability corresponding to the target answer is greater than or equal to the preset threshold, taking the document where the target answer is located as the target document, includes: Reading comprehension through Encoder small model prediction problem The answers and corresponding probabilities in the document blocks are collected. If the predicted probability corresponding to the target answer is greater than or equal to the preset threshold, the sentence where the target answer is located and the three sentences above and below are selected to filter the document block, and the union of the sentences in the filtered document block is taken as the target document.
[0009] Optionally, the method further includes: during the training process of the large language model, fine-tuning the large language model by using the LoRA method.
[0010] The present invention also provides a large language model training device, comprising: A retrieval module is used to obtain the original training data set of the large language model, which includes questions and answers corresponding to the questions; based on the questions in the original training data set, a search is performed in the knowledge base to obtain multiple relevant documents and irrelevant documents of the questions; A processing module is used to predict the answers and corresponding probabilities of the questions in multiple related documents, compare the predicted probabilities with a preset threshold, and if the predicted probability corresponding to the target answer is greater than or equal to the preset threshold, the document containing the target answer is used as the target document; A sample construction module is used to construct a question-answering sample based on a target document and to construct a refusal-to-answer sample based on an irrelevant document; wherein the question-answering sample includes a question, a target document and an answer, and the refusal-to-answer sample includes a question, an irrelevant document and a preset refusal-to-answer template; The training module is used to train the large language model based on question-answering samples and rejection samples.
[0011] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned large language model training method when executing the program.
[0012] The present invention also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the training method of the large language model is implemented.
[0013] The large language model training method provided by the present invention has the following beneficial effects: In the training stage of the large language model, the present invention retrieves relevant documents of the question, predicts the answer of the question in the relevant documents and the corresponding probability, compares the predicted probability with the preset probability, and if the predicted probability corresponding to the target answer is greater than or equal to the preset threshold, the document where the target answer is located is used as the target document, and the retrieved relevant documents can be further denoised to make the relevant documents more matched with the question; secondly, question and answer samples are constructed based on the target documents, and refusal samples are constructed based on irrelevant documents, and the large language model is trained based on the question and answer samples and the refusal samples; since the target document is more accurate, the quality of the question and answer samples constructed based on the target document is higher; by constructing refusal samples for training, the large language model can have the ability to refuse to answer, and when faced with relevant information that does not contain the answer question, it can refuse to answer instead of outputting an incorrect answer, thereby reducing the risk of the large language model generating irrelevant content; therefore, the present invention improves the accuracy of the large language model by denoising the retrieved relevant documents and constructing refusal samples based on irrelevant documents in the training stage of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0015] Figure 1 A schematic flow chart of a large language model training method provided by an embodiment of the present invention; Figure 2 A schematic diagram of a construction process of a simple refusal sample provided by an embodiment of the present invention; Figure 3 A schematic diagram of a construction process of a difficult refusal sample provided by an embodiment of the present invention; Figure 4 An example diagram of a large language model training process provided by an embodiment of the present invention; Figure 5 A schematic diagram of the structure of a large language model training device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0016] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.
[0017] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the technical solutions of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0018] In addition, the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless otherwise clearly specified or limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. In the description of the present invention, unless otherwise specified, "plurality" means two or more, which will not be described in detail here.
[0019] Based on the problem of low accuracy of large language models in the prior art, the present invention proposes a training method for large language models, such as Figure 1 As shown, the method comprises the following steps: S1: Obtain the original training data set of the large language model, search the knowledge base based on the questions in the original training data set, and obtain multiple relevant and irrelevant documents of the questions.
[0020] It should be understood that the original training dataset may be a commonly used question-answering dataset, such as the SQuAD dataset, the TriviaQA dataset, etc., and the original training dataset includes questions and answers corresponding to the questions.
[0021] The knowledge base can be Wikipedia, academic paper library or other databases. In the knowledge base, each corpus consists of two parts: title and content. The title part is the question, and the content part contains documents that answer the question and documents that are not related to the question (redundant data).
[0022] Furthermore, documents can be retrieved from the knowledge base based on questions through a retriever, and relevant documents and irrelevant documents can be determined based on the similarity between the retrieved documents and the questions. For example, documents and questions are converted into vectors (such as through TF-IDF, Word2Vec, BERT and other models), and retrieved through vector similarity metrics (such as cosine similarity). Documents with vector similarity greater than or equal to the target threshold are relevant documents, and documents with vector similarity less than the target threshold are irrelevant documents.
[0023] It should be understood that the relevant documents of a question can be understood as documents that are closely related to the content of the question and can provide useful information or answers. These documents include but are not limited to: Direct matching documents: those that contain the key words or phrases in the question.
[0024] Semantically relevant documents: Even if the question keywords do not appear directly in the documents, their topic, content, or context is closely related to the question.
[0025] Documents that provide answers or explanations: These documents may directly answer the question or provide enough information to answer the question indirectly.
[0026] It should be understood that since the content part in the knowledge base is relatively long and is not conducive to retrieval, the present invention divides the documents in the knowledge base into blocks before retrieval, and uses fixed-length segmentation to divide the documents into multiple shorter document blocks, thereby reducing the complexity of each document block and improving processing speed and retrieval efficiency.
[0027] It should be understood that by segmenting the original lengthy content into multiple document blocks, the retrieval of specific information becomes easier and more efficient. When searching, these document blocks can be indexed and searched separately, which shortens the query time.
[0028] However, simply segmenting by fixed length may cut off in the middle of a sentence or a semantically important part, resulting in information loss or difficulty in understanding. Therefore, in order to prevent cutting off in the middle of a sentence or a semantically important part, the embodiment of the present invention introduces an overlapping area, that is, between two adjacent document blocks, there will be a part of the content overlapping.
[0029] The size of the document block and the size of the overlapping area can be set by yourself. For example, the size of the document block is set to 512 characters and the size of the overlapping area is set to 64 characters.
[0030] Furthermore, based on the block-based retrieval, it should be understood that since the number of document blocks obtained after content segmentation is large, matching one by one consumes a lot of resources and time, and cannot meet the real-time requirements. Therefore, the present invention adopts an inverted index method for retrieval, and the steps are as follows: A1: Embed each document block to obtain multiple vectors, perform clustering operations on the multiple vectors, and determine the cluster center.
[0031] For example, the open source Chinese-English semantic vector model BGE (BAAI General Embedding) can be used to embed document blocks to obtain vectors, and then the k-means method is used to cluster the vectors to find the cluster centers. It should be understood that each cluster center corresponds to all vectors belonging to this category.
[0032] A2: Embed the question to get the question vector, and find w cluster centers that are close to the question vector among the cluster centers.
[0033] Exemplarily, BGE is first used to obtain a vector representation of the problem, and then w cluster centers closest to the problem are searched in the clustering results, where w is a positive integer greater than or equal to 1, for example, w is 3.
[0034] A3: Traverse the postings chains of w cluster centers, calculate the distance between the problem vector and the vectors on the postings chains, and return the vectors with the closest distances to the cluster centers. The vector corresponding to Document blocks as related documents.
[0035] Specifically, we construct an inverted index, obtain all vectors belonging to w cluster center categories according to the inverted index, calculate the similarity between the problem vector and these vectors one by one, and take the vector with the highest similarity ranking. (For example, 5) document blocks are used as the relevant documents for retrieval. This embodiment uses the IndexIVFFlat index of the Faiss vector database as the inverted index.
[0036] Optionally, in order to improve the accuracy of relevant documents, after the above preliminary search, the above retrieved documents can be The document blocks are re-ranked using the bge-reranker model, and the top similarity rankings are selected (e.g., 3) chunks of documents as related documents, .
[0037] It should be understood that since the user's expression may be inaccurate, there are some differences between the semantic space of the question and the semantic space of the document. Therefore, the question can be expanded before retrieval. The embodiment of the present invention completes the expansion through the large language model ChatGLM2-6B. Specifically, the question is input into the large language model ChatGLM2-6B, some thoughts and answers about the question are generated, and then the original question is spliced with the generated content to obtain the expanded question.
[0038] The expansion of the problem can be expressed by the following formula: ; in, Indicates the expanded problem, Indicates the original problem, Represents relevant content generated by a large language model based on a prompt template. For the splicing method.
[0039] S2: Predict the answers and corresponding probabilities of the questions in multiple related documents, compare the predicted probabilities with the preset thresholds, and if the predicted probability corresponding to the target answer is greater than or equal to the preset threshold, the document containing the target answer is taken as the target document.
[0040] For example, an Encoder reading comprehension model (such as deepset / roberta-base-squad2) can be used to predict the answer and corresponding probability of the question in each relevant document for each retrieved relevant document. If the predicted probability corresponding to the target answer is less than a preset threshold (such as 0.7), it can be considered that the relevant document does not contain valid information and the document is filtered; if the predicted probability corresponding to the target answer is greater than or equal to the preset threshold, it is considered that the document where the target answer is located contains valid information and the document is retained as the target document.
[0041] In the above embodiment, by screening the retrieved relevant documents through the small model, further denoising of the relevant documents can be achieved, making the relevant documents more accurate.
[0042] Optionally, the present invention can also use the above method to retrieve The document blocks are screened to determine the target document. Specifically: the problem is predicted by the Encoder reading comprehension model. The answers and corresponding probabilities in the document blocks are collected. If the predicted probability corresponding to the target answer is greater than or equal to the preset threshold, the sentence where the target answer is located and the three sentences above and below are selected to filter the document block, and the union of the sentences in the filtered document block is taken as the target document.
[0043] S3: Construct question-answer samples based on target documents and construct refusal samples based on irrelevant documents.
[0044] Among them, the question and answer samples include questions, target documents and answers; the refusal to answer samples include questions, irrelevant documents and preset refusal to answer templates.
[0045] The present invention can construct instruction data through an instruction construction module to fine-tune the large language model, the instruction data includes two categories: question and answer samples and refusal to answer samples, and the instruction data is in the form of a triple of {question, document, answer}.
[0046] Optionally, question and answer samples can be constructed based on the target document retrieved above, i.e., {question, target document, answer}; there are no refusal to answer samples in general knowledge bases. In order to improve the refusal to answer ability of large language models, the present invention designs two types of refusal to answer sample construction methods, namely simple refusal to answer samples and difficult refusal to answer samples.
[0047] For simple refusal samples, we first retrieve several relevant documents from the knowledge base based on the question, sort them by similarity, select documents with lower similarity rankings as irrelevant documents, set a refusal template, such as "Sorry, I cannot answer this question based on the reference materials", and then construct a simple refusal sample of {question, irrelevant document, refusal template}.
[0048] Next, combine Figure 2 This paper introduces the construction process of simple refusal samples. Figure 2 A schematic diagram of a construction process of a simple refusal sample provided in an embodiment of the present invention.
[0049] For example, Figure 2 As shown in the figure, based on the question, the top 10 document blocks with the highest similarity are first retrieved from the knowledge base, and then the document blocks ranked 7th, 8th, 9th, and 10th in similarity are selected as irrelevant documents (which can be regarded as relevant but invalid reference materials), and the answer is set to "Sorry, it cannot be answered based on the reference materials" to construct negative samples, that is, samples that refuse to answer.
[0050] For difficult refusal samples, the construction process can refer to Figure 3 , Figure 3 A schematic diagram of a construction process of a difficult refusal sample provided in an embodiment of the present invention.
[0051] For example, Figure 3As shown, relevant documents are retrieved from the knowledge base based on the question, and valid information that answers the question is deleted from the relevant documents through a large language model to obtain reference materials with deleted answers, i.e., irrelevant documents. The answer is set to "Sorry, it cannot be answered based on the reference materials" to construct negative samples, i.e., samples that refuse to answer.
[0052] In the above embodiment, the present invention uses a combination of retrieval enhancement and fine-tuning to introduce knowledge. The fine-tuning part is to use the instruction data constructed above to fine-tune the large language model, thereby introducing knowledge in the knowledge base, which is an internalized knowledge. Retrieval enhancement is to use retrieval technology to obtain some documents related to the query, which is an external knowledge.
[0053] S4: Train a large language model based on question-answer samples and refusal samples.
[0054] Specifically, question-answer samples and refusal-to-answer samples are input into a large language model as training sets to train the large language model.
[0055] It should be understood that during the training phase of the large language model, in addition to inputting questions, documents, and answers, prompt templates can also be input to guide the model to process inputs and generate special formats or instructions for outputs.
[0056] In order to improve the generalization and response quality of the large language model, the present invention introduces some uniformly distributed noise into the embedding vector during the fine-tuning of the large language model. Specifically, these noises The shape of is exactly the same as the shape of the embedding vector. The noise value is sampled from a uniform distribution of [-1,1]. The noise is changed to , added to the embedding vector to obtain the final embedding vector. In this embodiment, is an adjustable hyperparameter that can be set to 0.8. is the dimension of the large language model embedding layer, which can be set to 4096, and L represents the input length.
[0057] Optionally, during the training process of the large language model, the present invention fine-tunes the large language model through the LoRA method. This method aims to improve the performance of the pre-trained large language model on new tasks by minimizing the number of fine-tuning parameters and computational complexity, thereby alleviating the training cost of the pre-trained large language model.
[0058] Specifically, two low-rank matrices A and B are initialized on specific layers of the pre-trained large language model (such as the attention layer or feedforward network layer of the Transformer). During training, they serve as bypasses of the large language model, freeze the parameters of the large language model itself, and only update the two low-rank matrices A and B. In the process of training the large language model using question-and-answer samples and rejection samples, A and B are updated through back-propagation to minimize the loss function.
[0059] Furthermore, when the large language model is inferring or generating text, the original weights The weight update calculated by the low-rank matrix Combined with, in the process of forward propagation, and will multiply the same input , and then add them together to get the final output , the process is expressed as follows: ; ; in, represents the dimension of the original weight matrix, represents the rank of the matrix, , Represents the set of real numbers.
[0060] This merging approach allows large language models to be quickly adapted to specific tasks or datasets while maintaining their generality.
[0061] Optionally, the embodiment of the present invention places the low-rank matrix in the attention mechanism and The mapping matrix and Next, the rank of the LoRA matrix .
[0062] In the above embodiment, the LoRA method is used to efficiently fine-tune the large language model. During training, only the parameters of the LoRA are updated and the original parameters of the large language model are frozen, thereby reducing resource consumption and adapting to the new data distribution during training.
[0063] In an embodiment of the present invention, the large language model is a decoder based on the Transformer structure. The large language model can adopt the ChatGLM2-6B large language model, which is an open source text generation dialogue model based on the General Language Model (GLM) framework and has 6.2 billion parameters. This model uses similar technology to ChatGPT and is optimized for Chinese question-answering and dialogue. After bilingual training in Chinese and English with about 1T identifiers, supplemented by technologies such as supervised fine-tuning, feedback self-help, and human feedback reinforcement learning, the 6.2 billion parameter ChatGLM2-6B can already generate answers that are quite in line with human preferences. Large language models can choose such open source models.
[0064] Figure 4 An example diagram of a large language model training process provided by an embodiment of the present invention.
[0065] For example, Figure 4 As shown in the figure, a vector search is performed in the knowledge base for the question, and the search results are filtered and denoised through a small model to obtain relevant documents. The relevant documents are screened through a preprocessing model, and instruction data (including question and answer samples and refusal samples) and a Prompt template are constructed. The Prompt template is used as a prompt project to train and fine-tune the large language model.
[0066] Based on the same inventive concept, the embodiment of the present invention also provides a training device for a large language model, such as Figure 5 As shown, the device comprises: The retrieval module 51 is used to obtain the original training data set of the large language model, which includes questions and answers corresponding to the questions; based on the questions in the original training data set, a search is performed in the knowledge base to obtain multiple relevant documents and irrelevant documents of the questions.
[0067] The processing module 52 is used to predict the answers to the question and the corresponding probabilities in multiple related documents, and compare the predicted probabilities with a preset threshold. If the predicted probability corresponding to the target answer is greater than or equal to the preset threshold, the document where the target answer is located is used as the target document.
[0068] The sample construction module 53 is used to construct question-answer samples based on the target document and to construct refusal-to-answer samples based on irrelevant documents; wherein the question-answer samples include questions, target documents and answers, and the refusal-to-answer samples include questions, irrelevant documents and preset refusal-to-answer templates.
[0069] The training module 54 is used to train the large language model based on the question and answer samples and the rejection samples.
[0070] Each module in the training device of the large language model can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.
[0071] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in the training method embodiment of the large language model. The specific implementation method can be found in the method embodiment, which will not be repeated here.
[0072] Furthermore, the present invention also provides a non-temporary computer-readable storage medium containing instructions, and a computer program is stored on the storage medium. For example, a memory containing instructions, the above instructions can be executed by a processor of a computer device to complete the above method. For example, a non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, and an optical data storage device. When the computer program is executed by the processor, the steps in the embodiment of the training method of the large language model can be implemented. The specific implementation method can be found in the method embodiment, which will not be repeated here.
[0073] It should be understood by those skilled in the art that embodiments of the present invention may provide methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0074] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0075] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0077] It should be pointed out that the specific implementation methods described above can enable those skilled in the art to understand the invention more comprehensively, but do not limit the invention in any way. Therefore, although the invention has been described in detail in this specification and embodiments, those skilled in the art should understand that the invention can still be modified or replaced by equivalents; and all technical solutions and improvements that do not deviate from the spirit and scope of the invention are included in the protection scope of the invention. Any figure mark in the claims should not be regarded as limiting the claims involved. Any simple change or equivalent replacement of the technical solution that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention belongs to the protection scope of the present invention.
Claims
1. A training method for a large language model, It is characterized in that include: Obtain the original training data set of the large language model, which includes questions and answers corresponding to the questions; Search the knowledge base based on the questions in the original training data set to obtain multiple relevant and irrelevant documents of the questions; Predict the answers and corresponding probabilities of the questions in multiple related documents, compare the predicted probabilities with the preset threshold, and if the predicted probability corresponding to the target answer is greater than or equal to the preset threshold, take the document containing the target answer as the target document; Constructing question-answer samples based on target documents and constructing refusal-to-answer samples based on irrelevant documents; wherein the question-answer samples include questions, target documents and answers, and the refusal-to-answer samples include questions, irrelevant documents and preset refusal-to-answer templates; The large language model is trained based on question-answer samples and refusal samples.
2. The method for training a large language model according to claim 1, It is characterized in that Before searching the knowledge base based on the questions in the original training dataset, it also includes: The documents in the knowledge base are segmented according to a fixed length to obtain multiple document blocks, and two adjacent document blocks include overlapping areas; Based on the questions in the original training data set, we search the knowledge base and get multiple relevant documents for the questions, including: Embed each document block to obtain multiple vectors, perform clustering operations on the multiple vectors, and determine the cluster center; Embed the question to get the question vector, and find w cluster centers that are close to the question vector among the cluster centers; Traverse the postings chains of w cluster centers, calculate the distance between the problem vector and the vector on the postings chain, and put the vector with the closest distance The vector corresponding to Document blocks as related documents.
3. The method for training a large language model according to claim 2, It is characterized in that In the distance close The vector corresponding to After the document block as related documents, it also includes: Through the BGE-Reranker model The document blocks are reordered and the top similarity rankings are selected. The document block of the location is used as related documents. .
4. The method for training a large language model according to claim 2 or 3, It is characterized in that Predict the answers and corresponding probabilities of the questions in multiple related documents, compare the predicted probabilities with the preset thresholds, and if the predicted probability corresponding to the target answer is greater than or equal to the preset threshold, take the document where the target answer is located as the target document, including: Reading comprehension through Encoder small model prediction problem The answers and corresponding probabilities in the document blocks are collected. If the predicted probability corresponding to the target answer is greater than or equal to the preset threshold, the sentence where the target answer is located and the three sentences above and below are selected to filter the document block, and the union of the sentences in the filtered document block is taken as the target document.
5. The method for training a large language model according to any one of claims 1 to 3, It is characterized in that Also includes: During the training process of the large language model, the large language model is fine-tuned using the LoRA method.
6. A training device for a large language model, It is characterized in that include: A retrieval module is used to obtain the original training data set of the large language model, where the original training data set includes questions and answers corresponding to the questions; Search the knowledge base based on the questions in the original training data set to obtain multiple relevant and irrelevant documents of the questions; A processing module is used to predict the answers and corresponding probabilities of the questions in multiple related documents, compare the predicted probabilities with a preset threshold, and if the predicted probability corresponding to the target answer is greater than or equal to the preset threshold, the document containing the target answer is used as the target document; A sample construction module is used to construct a question-answering sample based on a target document and to construct a refusal-to-answer sample based on an irrelevant document; wherein the question-answering sample includes a question, a target document and an answer, and the refusal-to-answer sample includes a question, an irrelevant document and a preset refusal-to-answer template; The training module is used to train the large language model based on question-answering samples and rejection samples.
7. A computer device, It is characterized in that The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the training method of the large language model according to any one of claims 1 to 5 is implemented.
8. A computer-readable storage medium, It is characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the training method of the large language model according to any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Intelligent question and answer system and method
CN109766421A
Question answer selection method and device, computer equipment and storage medium
CN110287296A
Data interaction method, device and storage medium
CN112905768A
Document-based intelligent question and answer model training method and device and application thereof
CN116910210A
Document-based question answering method and device, storage medium and computer equipment
CN117453887A
Cited By
Question and answer pair generation method and device, storage medium and electronic equipment
CN119293193A
Question-answer pair generation method, device, storage medium and electronic device
CN119293193B
Application level retrieval enhancement generation strategy method and device
CN119336873A
Data retrieval method, device and equipment, medium and product
CN119474161A
Answer prediction model training method and answer checking method
CN119783750A
