A processing method and device for self-correcting fact enhancement of a large language model
By constructing a pre-trained corpus of question-answering models with positive and negative labels and modifying the models, and using a self-correction module to perform fact-enhanced correction on large language models, the illusion problem of large language models in intelligent question-answering tasks is solved, improving processing efficiency and user experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING DP TECH CO LTD
- Filing Date
- 2024-04-09
- Publication Date
- 2026-07-07
AI Technical Summary
Large language models suffer from the illusion problem in intelligent question answering tasks, and existing fact verification systems have low processing efficiency, resulting in reduced overall processing efficiency and poor user experience.
We construct a pre-training corpus for a question-answering model with positive and negative labels, pre-train and fine-tune it based on a reference model with a Transformer model structure, and modify the model to include a self-correction module. The self-correction module performs fact-enhancing correction on the decoder output, reducing the probability of incorrect answer candidate texts and increasing the probability of correct answer candidate texts.
It improves the accuracy of fact-correct answer matching in intelligent question-answering tasks using large language models, reduces the probability of illusions, shortens processing time, and enhances user experience.
Smart Images
Figure CN118377869B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a processing method and apparatus for self-correcting fact enhancement of a large language model. Background Technology
[0002] In the field of Natural Language Processing (NLP), Large Language Models (LLMs) are maturing and have broad application prospects, with intelligent question answering being one such application scenario. Before being applied to intelligent question answering, LLMs are pre-trained on massive datasets. These pre-trained LLMs then serve as the text generation engine for intelligent question answering tasks. Given the powerful language processing capabilities of LLMs, applying them to intelligent question answering undoubtedly greatly improves the processing efficiency of various intelligent question answering tasks. However, the hallucination problem of LLMs can also pose challenges to intelligent question answering tasks.
[0003] To reduce the likelihood of illusions in large language models (LLMs), researchers have proposed several methods. One such method involves using a fact-verification system to validate the output of the LLM. This system primarily relies on external databases or knowledge graphs to carefully examine the text generated by the model. The process generally consists of four steps: first, extracting key information from the model-generated text; second, matching the extracted information with external knowledge sources; third, verifying the authenticity of the information through matching to determine its veracity and identify any inaccurate or fabricated content; and fourth, feeding the verification results back to the LLM to help the model adjust its output text. However, this approach also suffers from low processing efficiency: each model output requires additional processing time to query external databases or knowledge graphs, which undoubtedly reduces the overall processing efficiency of the LLM and negatively impacts the user experience. Summary of the Invention
[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a method, apparatus, electronic device, and computer-readable storage medium for self-correcting fact enhancement processing of a large language model. This invention selects a large language model based on the Transformer model structure as a reference model (comprising a first encoder, a first decoder, and a first generator), and constructs a pre-training corpus for a question-answering model with both positive and negative labels. Based on the positive labels of the corpus, the reference model is pre-trained to obtain the corresponding base model parameter set. Based on the negative labels of the corpus, the decoder parameters of the reference model are fine-tuned to obtain fine-tuned decoder parameters. The reference model is then modified to obtain a modified model (comprising a first encoder, a first decoder, a second decoder, a first self-correcting module, and a first generator). Based on the base encoder / decoder / generator parameters, the parameters of the first encoder / decoder / generator are set. Based on the fine-tuned decoder parameters, the parameters of the second decoder, which has the same structure as the first decoder, are set. The first self-correcting module performs fact enhancement self-correction processing on the output of the first decoder based on the output of the second decoder. Finally, the modified model is used to generate corresponding answer text for any question text input by the user. The key feature of this invention's modified model is that it uses a first self-correcting module to further enhance the factual accuracy of the correct answer candidate text decoded by the first decoder using the incorrect answer candidate text decoded by the second decoder. This correction further reduces the probability of correct answer candidate texts that are highly similar to the incorrect answer candidate texts, thereby achieving a relative enhancement of the probability of correct answer candidate texts that are more consistent with the facts. The first and second decoders of this invention operate in parallel, without increasing sequential processing time. The first self-correcting module also eliminates the need for significant additional processing time to query external databases or knowledge graphs. Therefore, using this modified model to handle question-and-answer text generation tasks not only reduces the likelihood of illusions in large language models but also shortens model processing time, improves model efficiency, and enhances the user experience.
[0005] To achieve the above objectives, a first aspect of the present invention provides a method for self-correcting fact enhancement of a large language model, the method comprising:
[0006] Construct a question-answering model pre-training corpus with positive and negative labels, denoted as the corresponding first corpus;
[0007] Choose a type of large language model based on the Transformer model structure as the corresponding first reference model;
[0008] The first reference model is pre-trained based on the positive labels of the first corpus to obtain the corresponding base model parameter set; the base model parameter set includes base encoder parameters, base decoder parameters and base generator parameters;
[0009] Based on the reverse labels of the first corpus, the base decoder parameters of the first reference model are fine-tuned to obtain the corresponding fine-tuned decoder parameters.
[0010] The first reference model is modified to obtain a corresponding first modified model; and the model parameters of the first modified model are set based on the base encoder parameters, the base decoder parameters, the base generator parameters, and the fine-tuning decoder parameters.
[0011] Based on the first modification model, the corresponding answer text is generated for any question text input by the user, and the corresponding current answer text is obtained and displayed.
[0012] Preferably, the first corpus includes multiple first corpus records; each first corpus record includes a first input text, a first positive label output text, and multiple first negative label output texts; the first input text is a question text; the first positive label output text is the correct answer text corresponding to the first input text; each first negative label output text is an incorrect answer text corresponding to the first input text.
[0013] Preferably, the large language model implemented based on the Transformer model structure includes at least the BART model, T5 model, Marian model, and mBART model;
[0014] The first reference model consists of a first encoder, a first decoder, and a first generator; the output of the first encoder is connected to the input of the first decoder; the output of the first decoder is connected to the input of the first generator; the first encoder and the first decoder are implemented based on the encoder-decoder structure of the Transformer model; the first generator is implemented based on a linear network.
[0015] The first encoder is used to perform feature encoding on the model input text to obtain a corresponding first feature encoding tensor; the first decoder is used to decode candidate text based on the first feature encoding tensor to obtain a corresponding first candidate text vector sequence; the first candidate text vector sequence includes multiple first candidate text vectors; the first generator is used to perform output text generation processing based on the model input text and the first candidate text vector sequence to obtain the corresponding model output text; the first candidate text vector consists of a first candidate text and a corresponding first candidate probability.
[0016] Preferably, the step of pre-training the first reference model based on the positive labels of the first corpus to obtain the corresponding base model parameter set specifically includes:
[0017] The first input text of each first corpus record in the first corpus is used as the corresponding first training text, and the first positive label output text of each first corpus record is used as the corresponding first label text; and each first training text and the corresponding first label text are combined to form a corresponding first training data; and all the obtained first training data are combined to form a corresponding first training dataset.
[0018] The model parameters of the first reference model are initialized based on a preset initial model parameter set; the first reference model is trained based on the first training dataset according to the model pre-training method of large language models; the model parameters of the first encoder, the first decoder and the first generator obtained at the end of pre-training are used as the corresponding base encoder parameters, base decoder parameters and base generator parameters; and the base encoder parameters, base decoder parameters and base generator parameters are used to form the corresponding base model parameter set and saved.
[0019] Preferably, the step of fine-tuning the base decoder parameters of the first reference model based on the inverse labels of the first corpus to obtain the corresponding fine-tuned decoder parameters specifically includes:
[0020] The first input text of each first corpus record in the first corpus is used as the corresponding second training text, and the first reverse label output text of each first corpus record is used as the corresponding second label text; and each second label text and the corresponding second training text are combined to form a corresponding second training data; and all the obtained second training data are combined to form a corresponding second training dataset.
[0021] Based on the base encoder parameters, base decoder parameters, and base generator parameters of the base model parameter set, the model parameters of the first encoder, first decoder, and first generator in the first reference model are initialized; and the parameters of the first encoder and first generator are fixed after the initialization is completed.
[0022] The first decoder is trained by fine-tuning its parameters based on the second training dataset according to the parameter fine-tuning algorithm of the large language model; and the model parameters of the first decoder obtained at the end of the pre-training are used as the corresponding fine-tuned decoder parameters and saved.
[0023] Preferably, the first modification model comprises a first encoder, a first decoder, a second decoder, a first self-calibration module, and a first generator; the first and second decoders have identical decoder structures; the output of the first encoder is connected to the inputs of the first and second decoders respectively, and the encoder parameters of the first encoder are the corresponding base encoder parameters; the outputs of the first and second decoders are connected to the two inputs of the first self-calibration module respectively, and the decoder parameters of the first decoder are the corresponding base decoder parameters, while the decoder parameters of the second decoder are the corresponding fine-tuning decoder parameters; the output of the first self-calibration module is connected to the input of the first generator, and the generator parameters of the first generator are the base generator parameters;
[0024] The first encoder is used to perform feature encoding on the model input text to obtain a corresponding second feature encoding tensor; the first and second decoders are respectively used to decode candidate text based on the second feature encoding tensor to obtain a corresponding second candidate text vector sequence and a third candidate text vector sequence; the first self-correction module is used to perform fact-enhanced self-correction processing on the second candidate text vector sequence based on the third candidate text vector sequence to obtain a corresponding fourth candidate text vector sequence; the first generator is used to perform output text generation processing based on the model input text and the fourth candidate text vector sequence to obtain the corresponding model output text; the second, third, or fourth candidate text vector sequences are respectively composed of multiple corresponding second, third, or fourth candidate text vectors; the second, third, or fourth candidate text vector consists of a second, third, or fourth candidate text and a corresponding second, third, and fourth candidate probability.
[0025] Furthermore, the first self-correction module is used to perform fact-enhancing self-correction processing on the second candidate text vector sequence based on the third candidate text vector sequence to obtain the corresponding fourth candidate text vector sequence, specifically including:
[0026] The first self-correction module counts the total number of the second candidate text vectors in the second candidate text vector sequence to obtain a corresponding first total number M; and counts the total number of the third candidate text vectors in the third candidate text vector sequence to obtain a corresponding second total number N; the first and second total numbers M and N are all integers greater than 0;
[0027] And each of the second candidate text vectors in the second candidate text vector sequence is denoted as the corresponding H. i (t i ,p i );t i For the corresponding second candidate text, p iLet i be the index of the second candidate text, and 1 ≤ i ≤ M.
[0028] Each of the third candidate text vectors in the third candidate text vector sequence is denoted as the corresponding H. j (t j ,p j );t j For the corresponding third candidate text, p j Let j be the corresponding third candidate probability, and j be the third candidate text index, where 1 ≤ j ≤ N;
[0029] And for each of the second candidate texts t i Text vector encoding is performed to obtain the corresponding encoded vector v. i ; and for each of the aforementioned third candidate texts t j Text vector encoding is performed to obtain the corresponding encoded vector v. j ; and for each of the encoded vectors v i With each of the encoded vectors v j The vector similarity is calculated to obtain the corresponding similarity s. i,j ;
[0030] And based on all the second candidate probabilities p i All of the third candidate probabilities p j and all the aforementioned similarities s i,j For each of the second candidate texts t i The candidate probabilities are adjusted to obtain the corresponding fourth candidate probability pa. i ,
[0031] Each of the second candidate texts t i Each fourth candidate text is represented as a corresponding fourth candidate text; and each fourth candidate text and its corresponding fourth candidate probability form a corresponding fourth candidate text vector; and all the obtained fourth candidate text vectors form a corresponding fourth candidate text vector sequence.
[0032] A second aspect of the present invention provides an apparatus for implementing the self-correcting fact enhancement processing method for a large language model as described in the first aspect above. The apparatus includes: a corpus processing module, a reference model processing module, a base model pre-training module, a model fine-tuning module, a modified model construction module, and a modified model application module.
[0033] The corpus processing module is used to construct a question-answering model pre-training corpus with positive and negative labels, denoted as the corresponding first corpus;
[0034] The reference model processing module is used to select a type of large language model based on the Transformer model structure as the corresponding first reference model.
[0035] The base model pre-training module is used to pre-train the first reference model based on the positive labels of the first corpus to obtain the corresponding base model parameter set; the base model parameter set includes base encoder parameters, base decoder parameters and base generator parameters;
[0036] The model fine-tuning module is used to fine-tune the base decoder parameters of the first reference model based on the reverse labels of the first corpus to obtain the corresponding fine-tuned decoder parameters.
[0037] The modified model construction module is used to modify the first reference model to obtain a corresponding first modified model; and to set the model parameters of the first modified model based on the base encoder parameters, the base decoder parameters, the base generator parameters, and the fine-tuning decoder parameters;
[0038] The modified model application module is used to generate corresponding answer text for any question text input by the user based on the first modified model, and then display the corresponding current answer text.
[0039] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;
[0040] The processor is used to couple with the memory, read and execute instructions in the memory to implement the steps of the method described in the first aspect above;
[0041] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.
[0042] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the instructions described in the first aspect.
[0043] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for self-correcting fact enhancement processing of a large language model. As described above, this invention selects a large language model based on a Transformer model structure as a reference model (composed of a first encoder, a first decoder, and a first generator), constructs a question-answering model pre-training corpus with both positive and negative labels, and pre-trains the reference model based on the positive labels of the corpus to obtain the corresponding base model parameter set. The decoder parameters of the reference model are then fine-tuned based on the negative labels of the corpus to obtain fine-tuned decoder parameters. The reference model is then modified to obtain a modified model (composed of a first encoder, a first decoder, a second decoder, a first self-correcting module, and a first generator). The parameters of the first encoder / decoder / generator are set based on the base encoder / decoder / generator parameters, and the parameters of the second decoder, which has the same structure as the first decoder, are set based on the fine-tuned decoder parameters. The first self-correcting module performs fact enhancement self-correction processing on the output of the first decoder based on the output of the second decoder. Finally, the modified model is used to generate corresponding answer text for any question text input by the user. The characteristic of the modified model in this embodiment of the invention is that the first self-correction module uses the incorrect answer candidate text decoded by the second decoder to further perform fact-enhancing correction on the correct answer candidate text decoded by the first decoder. This correction further reduces the candidate probability of correct answer candidate texts that are highly similar to the incorrect answer candidate texts, thereby achieving a relative enhancement of the candidate probability of correct answer candidate texts that are more consistent with the facts. The first and second decoders in this embodiment of the invention are parallel processing mechanisms and do not increase sequential processing time. The first self-correction module in this embodiment of the invention also does not need to consume a large amount of processing time to query external databases or knowledge graphs. Therefore, using the modified model of this embodiment of the invention to process question-and-answer text generation tasks not only improves the matching degree between the generated text and the factual correct answers and reduces the illusion probability of large language models, but also shortens the model processing time, improves model processing efficiency, and enhances the user experience. Attached Figure Description
[0044] Figure 1 This is a schematic diagram of a self-correcting fact enhancement processing method for a large language model provided in Embodiment 1 of the present invention;
[0045] Figure 2 A module structure diagram of the first reference model and the first modified model provided in Embodiment 1 of the present invention;
[0046] Figure 3 This is a module structure diagram of a processing device for self-correcting fact enhancement of a large language model provided in Embodiment 2 of the present invention;
[0047] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation
[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0049] Embodiment 1 of the present invention provides a method for self-correcting fact enhancement of a large language model, such as... Figure 1 The diagram illustrates a method for self-correcting fact enhancement of a large language model according to Embodiment 1 of the present invention. This method mainly includes the following steps:
[0050] Step 1: Construct a pre-training corpus for a question-answering model with positive and negative labels, denoted as the corresponding first corpus.
[0051] Here, the first corpus of this embodiment of the invention includes multiple first corpus records; each first corpus record includes a first input text, a first positive label output text, and multiple first negative label output texts; the first input text is a question text; the first positive label output text is the correct answer text corresponding to the first input text; each first negative label output text is an incorrect answer text corresponding to the first input text.
[0052] Step 2: Select a large language model based on the Transformer model structure as the corresponding first reference model.
[0053] Here, the large language models implemented based on the Transformer model structure mentioned in the embodiments of the present invention include at least the BART model, T5 model, Marine model, and mBART model.
[0054] like Figure 2 As shown in the module structure diagram of the first reference model and the first modified model provided in Embodiment 1 of the present invention, the first reference model of the present invention consists of a first encoder, a first decoder, and a first generator; the output of the first encoder is connected to the input of the first decoder; the output of the first decoder is connected to the input of the first generator; the first encoder and the first decoder are implemented based on the encoder-decoder structure of the Transformer model; the first generator is implemented based on a linear network; wherein:
[0055] 1) The first encoder is used to encode the features of the input text of the model to obtain the corresponding first feature encoding tensor; here, the encoding principle of the encoder of various large language models can be obtained by consulting the publicly available technical literature corresponding to each model, and will not be repeated here;
[0056] 2) The first decoder is used to decode the candidate texts based on the first feature encoding tensor to obtain the corresponding first candidate text vector sequence; the first candidate text vector sequence includes multiple first candidate text vectors; the first candidate text vector consists of a first candidate text and a corresponding first candidate probability; here, the processing mechanism of the decoder of various large language models to obtain multiple candidate texts by decoding and assigning a prediction probability (i.e., candidate probability) to each candidate text can also be found by consulting the publicly available technical literature corresponding to each model, and will not be repeated here;
[0057] 3) The first generator is used to generate the corresponding model output text based on the model input text and the first candidate text vector sequence. Here, the processing mechanism of various large language models for text generation, such as how to concatenate the maximum possible answer text based on the decoder output and how to format and concatenate the model input text and the maximum possible answer text to obtain the final model input text, can also be found by consulting the publicly available technical literature corresponding to each model. It will not be repeated here.
[0058] Step 3: Pre-train the first reference model based on the positive labels of the first corpus to obtain the corresponding base model parameter set;
[0059] The base model parameter set includes base encoder parameters, base decoder parameters, and base generator parameters;
[0060] Specifically, it includes: Step 31, taking the first input text of each first corpus record in the first corpus as the corresponding first training text, and taking the first positive label output text of each first corpus record as the corresponding first label text; and forming a corresponding first training data with each first training text and the corresponding first label text; and forming a corresponding first training dataset with all the obtained first training data;
[0061] Step 32, and perform model parameter initialization settings on the first reference model based on a preset initialization model parameter set;
[0062] Step 33, and train the first reference model based on the first training dataset according to the model pre-training method of the large language model;
[0063] Here, various large language models can be trained based on multiple large model pre-training methods when obtaining labeled training datasets, such as supervised large model pre-training, semi-supervised large model pre-training, self-supervised large model pre-training, and unsupervised large model pre-training, etc. The specific choice of one or more training methods can be determined according to the specific application. The large model pre-training methods for each type of large language model can also be found by consulting the publicly available technical literature corresponding to each model, and will not be repeated here.
[0064] Step 34: The model parameters of the first encoder, first decoder and first generator obtained at the end of pre-training are used as the corresponding base encoder parameters, base decoder parameters and base generator parameters; and the base encoder parameters, base decoder parameters and base generator parameters are used to form the corresponding base model parameter set and saved.
[0065] Step 4: Fine-tune the base decoder parameters of the first reference model based on the inverse labels of the first corpus to obtain the corresponding fine-tuned decoder parameters.
[0066] Specifically, it includes: step 41, taking the first input text of each first corpus record in the first corpus as the corresponding second training text, and taking the first reverse label output text of each first corpus record as the corresponding second label text; and forming a corresponding second training data by each second label text and the corresponding second training text; and forming a corresponding second training dataset by all the obtained second training data.
[0067] Step 42: Initialize the model parameters of the first encoder, first decoder and first generator in the first reference model based on the base encoder parameters, base decoder parameters and base generator parameters of the base model parameter set; and solidify the parameters of the first encoder and first generator after the setting is completed.
[0068] Step 43, and train the first decoder to fine-tune the decoder parameters based on the second training dataset according to the parameter fine-tuning algorithm of the large language model;
[0069] Here, various large language models can fine-tune the local parameters of the model based on multiple large model parameter fine-tuning algorithms when obtaining labeled training datasets. The training methods can include supervised, semi-supervised, and self-supervised methods. Common large model parameter fine-tuning algorithms include LoRA fine-tuning algorithm and AdaLoRA fine-tuning algorithm. The specific choice of which fine-tuning algorithm and which one or more training methods to use can be determined according to the specific application.
[0070] Step 44, and save the model parameters of the first decoder obtained at the end of pre-training as the corresponding fine-tuned decoder parameters.
[0071] Step 5: Modify the first reference model to obtain the corresponding first modified model; and set the model parameters of the first modified model based on the base encoder parameters, base decoder parameters, base generator parameters, and fine-tuning decoder parameters.
[0072] Here is as Figure 2 As shown, the first modification model of this embodiment of the invention consists of a first encoder, a first decoder, a second decoder, a first self-calibration module, and a first generator; wherein:
[0073] 1) The output of the first encoder is connected to the inputs of the first and second decoders respectively; after the parameter settings are completed, the encoder parameters of the first encoder are the corresponding base encoder parameters; the first encoder is used to perform feature encoding on the input text of the model to obtain the corresponding second feature encoding tensor;
[0074] 2) The outputs of the first and second decoders are connected to the two inputs of the first self-calibration module, respectively. The decoder structures of the first and second decoders are identical, that is, the second decoder is actually obtained by copying the decoder structure of the first decoder. After the parameter settings are completed, the decoder parameters of the first decoder are the corresponding base decoder parameters, and the decoder parameters of the second decoder are the corresponding fine-tuning decoder parameters. The first and second decoders are used to decode candidate texts according to the second feature encoding tensor to obtain the corresponding second candidate text vector sequence and third candidate text vector sequence, respectively. The second or third candidate text vector sequence is composed of multiple corresponding second or third candidate text vectors. The second or third candidate text vector is composed of a second or third candidate text and a corresponding second or third candidate probability. Here, as can be seen from the aforementioned model training method, each pair of second candidate texts and second candidate probabilities output by the first decoder is actually a correct answer candidate text and its corresponding prediction probability, while each pair of third candidate texts and third candidate probabilities output by the second decoder is actually an incorrect answer candidate text and its corresponding prediction probability.
[0075] 3) The output of the first self-correction module is connected to the input of the first generator; the first self-correction module is used to perform fact-enhancing self-correction processing on the second candidate text vector sequence based on the third candidate text vector sequence to obtain the corresponding fourth candidate text vector sequence; the fourth candidate text vector sequence consists of multiple fourth candidate text vectors; the fourth candidate text vector consists of a fourth candidate text and a corresponding fourth candidate probability; here, the first self-correction module actually uses the incorrect answer candidate text decoded by the second decoder to perform further fact-enhancing correction on the correct answer candidate text decoded by the first decoder. Through correction, the candidate probability of the correct answer candidate text that is highly similar to the incorrect answer candidate text can be further reduced, thereby achieving the effect of relatively enhancing the candidate probability of the correct answer candidate text that is more consistent with the facts;
[0076] The first self-correction module is used to perform fact-enhancing self-correction processing on the second candidate text vector sequence based on the third candidate text vector sequence to obtain the corresponding fourth candidate text vector sequence, specifically including:
[0077] Step A1: The first self-correction module counts the total number of second candidate text vectors in the second candidate text vector sequence to obtain the corresponding first total number M; and counts the total number of third candidate text vectors in the third candidate text vector sequence to obtain the corresponding second total number N;
[0078] Where the first and second totals M and N are both integers greater than 0;
[0079] Step A2, and denote each second candidate text vector in the second candidate text vector sequence as its corresponding H. i (t i ,p i ); and denote each third candidate text vector in the third candidate text vector sequence as its corresponding H. j (t j ,p j );
[0080] Among them, t i For the corresponding second candidate text, p i Let t be the probability of the second candidate, i be the index of the second candidate text, 1≤i≤M; j For the corresponding third candidate text, p j Let j be the probability of the third candidate, and j be the index of the third candidate text, where 1 ≤ j ≤ N;
[0081] Step A3, and for each second candidate text t i Text vector encoding is performed to obtain the corresponding encoded vector v. i ; and for each third candidate text t jText vector encoding is performed to obtain the corresponding encoded vector v. j ; and for each encoded vector v i With each encoded vector v j The vector similarity is calculated to obtain the corresponding similarity s. i,j ;
[0082] Step A4, and based on all second candidate probabilities p i The probability p of all third candidates j and all similarity s i,j For each second candidate text t i The candidate probabilities are adjusted to obtain the corresponding fourth candidate probability pa. i ,
[0083] in,
[0084] Step A5, select each second candidate text t i Each fourth candidate text is considered as a corresponding fourth candidate text; and each fourth candidate text and its corresponding fourth candidate probability form a corresponding fourth candidate text vector; and all the obtained fourth candidate text vectors form a corresponding fourth candidate text vector sequence.
[0085] 4) After completing the parameter settings, the generator parameters of the first generator are the base generator parameters; the first generator is used to generate the corresponding model output text by performing output text generation processing based on the model input text and the fourth candidate text vector sequence.
[0086] The second, third, or fourth candidate text vector sequences are each composed of multiple corresponding second, third, or fourth candidate text vectors; each second, third, or fourth candidate text vector consists of a second, third, or fourth candidate text and a corresponding second, third, and fourth candidate probability.
[0087] Step 6: Based on the first transformation model, process the user-inputted question text to generate the corresponding answer text and display the current answer text;
[0088] Specifically, the process is as follows: The user-inputted question text is used as the first question text and input into the first modification model; the first encoder performs feature encoding on the first question text to obtain the corresponding third feature encoding tensor; the first and second decoders respectively decode candidate texts based on the third feature encoding tensor to obtain the corresponding fifth and sixth candidate text vector sequences; the first self-correction module performs fact-enhancing self-correction processing on the fifth candidate text vector sequence based on the sixth candidate text vector sequence to obtain the corresponding seventh candidate text vector sequence; the first generator performs output text generation processing based on the first question text and the seventh candidate text vector sequence to obtain the corresponding current answer text; and the current answer text is then displayed.
[0089] Figure 3 This is a module structure diagram of a processing device for self-correcting fact enhancement of a large language model provided in Embodiment 2 of the present invention. This device can be a terminal device or server implementing the aforementioned method embodiments, or it can be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiments. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 3 As shown, the device includes: a corpus processing module 201, a reference model processing module 202, a base model pre-training module 203, a model fine-tuning module 204, a modified model construction module 205, and a modified model application module 206.
[0090] The corpus processing module 201 is used to construct a pre-training corpus for a question-answering model with positive and negative labels, denoted as the corresponding first corpus.
[0091] The reference model processing module 202 is used to select a large language model based on the Transformer model structure as the corresponding first reference model.
[0092] The base model pre-training module 203 is used to pre-train the first reference model based on the positive labels of the first corpus to obtain the corresponding base model parameter set; the base model parameter set includes base encoder parameters, base decoder parameters and base generator parameters.
[0093] The model fine-tuning module 204 is used to fine-tune the base decoder parameters of the first reference model based on the reverse labels of the first corpus to obtain the corresponding fine-tuned decoder parameters.
[0094] The modified model construction module 205 is used to modify the first reference model to obtain the corresponding first modified model; and to set the model parameters of the first modified model based on the base encoder parameters, base decoder parameters, base generator parameters and fine-tuning decoder parameters.
[0095] The transformation model application module 206 is used to generate corresponding answer text for any question text input by the user based on the first transformation model, and then display the corresponding current answer text.
[0096] The processing apparatus for self-correcting fact enhancement of a large language model provided in this embodiment of the invention can execute the method steps in the above method embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.
[0097] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, the corpus processing module can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and called and executed by a processing element of the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.
[0098] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs). Furthermore, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling program code. Additionally, these modules can be integrated together as a system-on-a-chip (SoC).
[0099] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the foregoing method embodiments are generated. The computer described above can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).
[0100] Figure 4 This is a schematic diagram of an electronic device provided in Embodiment 3 of the present invention. This electronic device can be a terminal device or server implementing the methods of the aforementioned embodiments, or it can be a terminal device or server connected to the aforementioned terminal device or server implementing the methods of the aforementioned embodiments. Figure 4 As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the processing steps described in the foregoing embodiments. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.
[0101] exist Figure 4The system bus 305 mentioned can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). Memory may include Random Access Memory (RAM) and may also include Non-Volatile Memory, such as at least one disk storage device.
[0102] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0103] It should be noted that the embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to perform the methods and processes provided in the above embodiments.
[0104] This invention also provides a chip for executing instructions, which is used to perform the processing steps described in the foregoing method embodiments.
[0105] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for self-correcting fact enhancement processing of a large language model. As described above, this invention selects a large language model based on a Transformer model structure as a reference model (composed of a first encoder, a first decoder, and a first generator), constructs a question-answering model pre-training corpus with both positive and negative labels, and pre-trains the reference model based on the positive labels of the corpus to obtain the corresponding base model parameter set. The decoder parameters of the reference model are then fine-tuned based on the negative labels of the corpus to obtain fine-tuned decoder parameters. The reference model is then modified to obtain a modified model (composed of a first encoder, a first decoder, a second decoder, a first self-correcting module, and a first generator). The parameters of the first encoder / decoder / generator are set based on the base encoder / decoder / generator parameters, and the parameters of the second decoder, which has the same structure as the first decoder, are set based on the fine-tuned decoder parameters. The first self-correcting module performs fact enhancement self-correction processing on the output of the first decoder based on the output of the second decoder. Finally, the modified model is used to generate corresponding answer text for any question text input by the user. The characteristic of the modified model in this embodiment of the invention is that the first self-correction module uses the incorrect answer candidate text decoded by the second decoder to further perform fact-enhancing correction on the correct answer candidate text decoded by the first decoder. This correction further reduces the candidate probability of correct answer candidate texts that are highly similar to the incorrect answer candidate texts, thereby achieving a relative enhancement of the candidate probability of correct answer candidate texts that are more consistent with the facts. The first and second decoders in this embodiment of the invention are parallel processing mechanisms and do not increase sequential processing time. The first self-correction module in this embodiment of the invention also does not need to consume a large amount of processing time to query external databases or knowledge graphs. Therefore, using the modified model of this embodiment of the invention to process question-and-answer text generation tasks not only improves the matching degree between the generated text and the factual correct answers and reduces the illusion probability of large language models, but also shortens the model processing time, improves model processing efficiency, and enhances the user experience.
[0106] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0107] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0108] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for self-correcting fact enhancement in a large language model, characterized in that, The method includes: Construct a question-answering model pre-training corpus with positive and negative labels, denoted as the corresponding first corpus; Choose a type of large language model based on the Transformer model structure as the corresponding first reference model; The first reference model is pre-trained based on the positive labels of the first corpus to obtain the corresponding base model parameter set; the base model parameter set includes base encoder parameters, base decoder parameters and base generator parameters; Based on the reverse labels of the first corpus, the base decoder parameters of the first reference model are fine-tuned to obtain the corresponding fine-tuned decoder parameters. The first reference model is modified to obtain a corresponding first modified model; and the model parameters of the first modified model are set based on the base encoder parameters, the base decoder parameters, the base generator parameters, and the fine-tuning decoder parameters. Based on the first modification model, the corresponding answer text is generated for any question text input by the user to obtain the current answer text and then displayed. The first modified model comprises a first encoder, a first decoder, a second decoder, a first self-correction module, and a first generator. The first encoder is used to perform feature encoding on the input text of the model to obtain a corresponding second feature encoding tensor. The first and second decoders are respectively used to decode candidate texts based on the second feature encoding tensor to obtain corresponding second candidate text vector sequences and third candidate text vector sequences. The first self-correction module is used to perform fact-enhanced self-correction processing on the second candidate text vector sequence based on the third candidate text vector sequence to obtain a corresponding fourth candidate text vector sequence. The first generator is used to perform output text generation processing based on the input text of the model and the fourth candidate text vector sequence to obtain the corresponding model output text. The second, third, or fourth candidate text vector sequences are each composed of multiple corresponding second, third, or fourth candidate text vectors. The second, third, or fourth candidate text vector consists of a second, third, or fourth candidate text and a corresponding second, third, and fourth candidate probability.
2. The method for self-correcting fact enhancement using a large language model according to claim 1, characterized in that, The first corpus includes multiple first corpus records; each first corpus record includes a first input text, a first positive label output text, and multiple first negative label output texts; the first input text is a question text; the first positive label output text is the correct answer text corresponding to the first input text; each first negative label output text is an incorrect answer text corresponding to the first input text.
3. The method for self-correcting fact enhancement of a large language model according to claim 2, characterized in that, The large language model implemented based on the Transformer model structure includes at least the BART model, T5 model, Marian model, and mBART model; The first reference model consists of a first encoder, a first decoder, and a first generator; the output of the first encoder is connected to the input of the first decoder; the output of the first decoder is connected to the input of the first generator; the first encoder and the first decoder are implemented based on the encoder-decoder structure of the Transformer model; the first generator is implemented based on a linear network. The first encoder is used to perform feature encoding on the model input text to obtain a corresponding first feature encoding tensor; the first decoder is used to decode candidate text based on the first feature encoding tensor to obtain a corresponding first candidate text vector sequence; the first candidate text vector sequence includes multiple first candidate text vectors; the first generator is used to perform output text generation processing based on the model input text and the first candidate text vector sequence to obtain the corresponding model output text; the first candidate text vector consists of a first candidate text and a corresponding first candidate probability.
4. The method for self-correcting fact enhancement using a large language model according to claim 3, characterized in that, The step of pre-training the first reference model based on the positive labels of the first corpus to obtain the corresponding base model parameter set specifically includes: The first input text of each first corpus record in the first corpus is used as the corresponding first training text, and the first positive label output text of each first corpus record is used as the corresponding first label text; and each first training text and the corresponding first label text are combined to form a corresponding first training data; and all the obtained first training data are combined to form a corresponding first training dataset. The model parameters of the first reference model are initialized based on a preset initial model parameter set; the first reference model is trained based on the first training dataset according to the model pre-training method of large language models; the model parameters of the first encoder, the first decoder and the first generator obtained at the end of pre-training are used as the corresponding base encoder parameters, base decoder parameters and base generator parameters; and the base encoder parameters, base decoder parameters and base generator parameters are used to form the corresponding base model parameter set and saved.
5. The method for self-correcting fact enhancement of a large language model according to claim 3, characterized in that, The step of fine-tuning the base decoder parameters of the first reference model based on the inverse tags of the first corpus to obtain the corresponding fine-tuned decoder parameters specifically includes: The first input text of each first corpus record in the first corpus is used as the corresponding second training text, and the first reverse label output text of each first corpus record is used as the corresponding second label text; and each second label text and the corresponding second training text are combined to form a corresponding second training data; and all the obtained second training data are combined to form a corresponding second training dataset. Based on the base encoder parameters, base decoder parameters, and base generator parameters of the base model parameter set, the model parameters of the first encoder, first decoder, and first generator in the first reference model are initialized; and the parameters of the first encoder and first generator are fixed after the initialization is completed. The first decoder is trained by fine-tuning its parameters based on the second training dataset according to the parameter fine-tuning algorithm of the large language model; and the model parameters of the first decoder obtained at the end of the pre-training are used as the corresponding fine-tuned decoder parameters and saved.
6. The method for self-correcting fact enhancement of a large language model according to claim 3, characterized in that, The first and second decoders have the same decoder structure; the output of the first encoder is connected to the input of the first and second decoders respectively, and the encoder parameters of the first encoder are the corresponding base encoder parameters; the outputs of the first and second decoders are connected to the two input terminals of the first self-calibration module respectively, and the decoder parameters of the first decoder are the corresponding base decoder parameters, and the decoder parameters of the second decoder are the corresponding fine-tuning decoder parameters. The output of the first self-calibration module is connected to the input of the first generator, and the generator parameters of the first generator are the base plate generator parameters.
7. The method for self-correcting fact enhancement of a large language model according to claim 6, characterized in that, The first self-correction module is used to perform fact-enhanced self-correction processing on the second candidate text vector sequence based on the third candidate text vector sequence to obtain the corresponding fourth candidate text vector sequence, specifically including: The first self-correction module counts the total number of the second candidate text vectors in the second candidate text vector sequence to obtain a corresponding first total number M; and counts the total number of the third candidate text vectors in the third candidate text vector sequence to obtain a corresponding second total number N; the first and second total numbers M and N are all integers greater than 0; And each of the second candidate text vectors in the second candidate text vector sequence is denoted as the corresponding H. i (t i ,p i );t i For the corresponding second candidate text, p i Let i be the probability of the second candidate, and i be the index of the second candidate text, where 1 ≤ i ≤ M; Each of the third candidate text vectors in the third candidate text vector sequence is denoted as the corresponding H. j (t j ,p j );t j For the corresponding third candidate text, p j Let j be the probability of the third candidate, and j be the index of the third candidate text, where 1 ≤ j ≤ N; And for each of the second candidate texts t i Text vector encoding is performed to obtain the corresponding encoded vector v. i ; and for each of the aforementioned third candidate texts t j Text vector encoding is performed to obtain the corresponding encoded vector v. j ; and for each of the encoded vectors v i With each of the encoded vectors v j The vector similarity is calculated to obtain the corresponding similarity s. i,j ; And based on all the second candidate probabilities p i All of the third candidate probabilities p j and all the aforementioned similarities s i,j For each of the second candidate texts t i The candidate probabilities are adjusted to obtain the corresponding fourth candidate probability pa. i , ; Each of the second candidate texts t i Each fourth candidate text is represented as a corresponding fourth candidate text; and each fourth candidate text and its corresponding fourth candidate probability form a corresponding fourth candidate text vector; and all the obtained fourth candidate text vectors form a corresponding fourth candidate text vector sequence.
8. An apparatus for performing the processing method of self-correcting fact enhancement for a large language model as described in any one of claims 1-7, characterized in that, The device includes: a corpus processing module, a reference model processing module, a base model pre-training module, a model fine-tuning module, a modified model construction module, and a modified model application module; The corpus processing module is used to construct a question-answering model pre-training corpus with positive and negative labels, denoted as the corresponding first corpus; The reference model processing module is used to select a type of large language model based on the Transformer model structure as the corresponding first reference model. The base model pre-training module is used to pre-train the first reference model based on the positive labels of the first corpus to obtain the corresponding base model parameter set; the base model parameter set includes base encoder parameters, base decoder parameters and base generator parameters; The model fine-tuning module is used to fine-tune the base decoder parameters of the first reference model based on the reverse labels of the first corpus to obtain the corresponding fine-tuned decoder parameters. The modified model construction module is used to modify the first reference model to obtain a corresponding first modified model; and to set the model parameters of the first modified model based on the base encoder parameters, the base decoder parameters, the base generator parameters, and the fine-tuning decoder parameters; The modified model application module is used to generate corresponding answer text for any question text input by the user based on the first modified model, and then display the corresponding current answer text.
9. An electronic device, characterized in that, include: Memory, processor, and transceiver; The processor is configured to be coupled to the memory, read and execute instructions in the memory to implement the method according to any one of claims 1-7; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1-7.