Verifiable text generation method and device

Abstract

The invention discloses a verifiable text generation method and a verifiable text generation device, relates to the field of natural language processing, and aims to improve the illusion and incredibility problems of retrieval enhancement generation task execution. The method comprises: receiving a text generation task; analyzing the task type of the text generation task; extracting key information of the text generation task; when each lexical element is generated, the uncertainty of the text generation model is evaluated; when the condition is triggered, a retrieval word is generated, and evidence retrieval is triggered; retrieving the evidence according to the retrieval word; the retrieved evidence enhanced historical session is submitted to a text generation model to generate a corresponding lexical element, and a reply text is generated based on the lexical element; associating the reply text with the evidence; verifying the consistency of the reply text and the associated evidence; and if the verification is not passed, triggering to rewrite the reply text. According to the method, the problem of static disjunction between retrieval and generation is solved, and the fact accuracy and traceability of the text are improved.

Description

Technical Field

[0001] This invention relates to the field of Natural Language Processing (NLP) technology, and in particular to a verifiable text generation method and apparatus. Background Technology

[0002] Text generation technology has a wide range of applications in natural language question answering, creation and other scenarios. It is usually achieved by using text generation models (generative artificial intelligence tools) represented by large language models (LLM).

[0003] Common text generation models employ Retrieval-Augmented Generation (RAG) technology. Its generation principle is to perform external knowledge retrieval based on the input text generation task (such as a problem description or task description), and the retrieved knowledge is called evidence. Then, referring to the retrieved evidence, it combines LLM to generate response text.

[0004] However, practical experience has shown that while the aforementioned text generation models are acceptable for short texts, they often produce fabricated texts when generating longer texts, even when the context seems reasonable. This phenomenon is referred to in the field as the "illusion" phenomenon. The existence of this phenomenon poses a serious risk when applying known text generation models to serious scenarios such as finance, healthcare, law, and technology.

[0005] Furthermore, the response texts generated by the aforementioned text generation models lack traceability, making it difficult to verify their authenticity and ensuring their credibility.

[0006] Most known optimization methods focus on the fluency of the response text and the relevance of the context, but have not effectively suppressed the "illusion" problem and credibility problem of text generation models. Summary of the Invention

[0007] The purpose of this invention is to provide a verifiable text generation method and apparatus to address all or part of the problems mentioned above, so as to effectively suppress the "illusion" problem in the execution of retrieval enhancement generation tasks and improve the credibility of response text.

[0008] The technical solution adopted in this invention is as follows: In a first aspect, the present invention provides a verifiable text generation method, comprising: S1. Receive a text generation task; parse the task type of the text generation task; extract the key information of the text generation task; S2. When the text generation model generates each word for the text generation task, evaluate the uncertainty of the text generation model; when the uncertainty reaches the triggering condition, generate search terms based on the key information and trigger evidence retrieval; the triggering condition is associated with the task type. S3. Retrieve evidence based on the search terms; enhance historical conversations using the retrieved evidence; generate corresponding word units by a text generation model based on the enhanced historical conversations, and generate response text based on the word units; associate the response text with the evidence corresponding to the word units; S4. Verify the consistency between the response text and the associated evidence; if the verification fails, trigger the rewriting of the response text.

[0009] Optionally, evaluate the uncertainty of the text generation model, including: Calculate the entropy value of the predicted probability distribution of the current word element, and determine whether the entropy value reaches a first preset threshold. If so, determine that the triggering condition has been met. The first preset threshold is positively correlated with the factual level coefficient corresponding to the task type. Alternatively, determine whether a predetermined entity exists in the response text to which the current word belongs; if so, determine that the triggering condition has been met; the predetermined entity is determined by the task type. Alternatively, determine whether the semantic relevance between the response text to which the current word belongs and the historical conversation reaches a second predetermined threshold. If not, determine that the triggering condition has been met. The second predetermined threshold is determined by the task type.

[0010] Optionally, evidence can be retrieved based on the search terms, including: Perform multi-source parallel evidence retrieval based on the search terms; The retrieved evidence is preprocessed, and each piece of preprocessed evidence is uniquely identified.

[0011] Optionally, the retrieved evidence is preprocessed, and each piece of preprocessed evidence is uniquely identified, including: The retrieved evidence is deduplicated, and then a predetermined number of pieces of evidence with the highest relevance to the search terms are selected. Each piece of evidence is segmented into sentences, and a unique evidence identifier is assigned to each segmented sentence.

[0012] Optionally, historical conversations can be enhanced using evidence retrieved, including: The retrieved evidence is then concatenated with the historical sessions and encoded. Alternatively, the retrieved evidence and the historical conversation can be encoded separately and then concatenated. Alternatively, the retrieved evidence can be encoded by embedding it into the historical session using a preset identifier.

[0013] Optionally, associating the response text with evidence corresponding to the lexical units includes: The cross-attention mechanism is used to evaluate the relevance of the lexical to be generated with all evidence, and the evidence with the highest relevance is implicitly associated with the lexical. Furthermore, when generating a response text based on the lexical, an explicit traceability marker for evidence associated with the lexical contained in the response text is generated based on the implicit association.

[0014] Optionally, verify the consistency between the response text and the associated evidence, including: Assess the risk of contradiction between the response text and the associated evidence. If the risk of contradiction reaches a third predetermined threshold, the verification fails.

[0015] Optionally, the method for assessing the conflict risk includes: Examine the probability that the entities or relationships in the response text contradict the evidence; Alternatively, examine the proportion of inconsistencies in the evidence associated with the response text across multiple searches based on the same search term.

[0016] Optionally, the training method for the text generation model includes: Define the reference response text for the sample; The sample is input into the text generation model, and the language modeling loss and fact consistency loss of the text generation model are calculated respectively; the language modeling loss is the cross-entropy loss between the reference response text and the response text generated by the text generation model; the fact consistency loss is the degree to which the response text generated by the text generation model is not implied by the evidence. The total loss is obtained by weighted summation of the language modeling loss and the fact consistency loss. The model parameters of the text generation model are optimized based on the total loss.

[0017] In a second aspect, the present invention also provides a verifiable text generation apparatus, comprising a processor and a storage medium; the storage medium stores computer instructions, and the processor executes the computer instructions to perform the verifiable text generation method described above.

[0018] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: Compared with known search enhancement generation techniques, the present invention has the following advantages: 1. Significantly Improves the Factual Accuracy of Generated Content: Due to the adoption of an uncertainty-driven dynamic retrieval mechanism, the system can promptly introduce external knowledge when model uncertainty is high, supplementing information gaps at the source and effectively avoiding the illusion problem caused by insufficient knowledge. By combining factual consistency loss with model parameter optimization, the model is explicitly guided to pursue factual correctness from the training stage. Experiments show that this invention significantly improves the factual accuracy index on knowledge-intensive tasks compared to the traditional static RAG method.

[0019] 2. Achieving fine-grained verifiability and traceability of generated results: Through evidence tagging and traceability-based fusion generation mechanisms, the system strongly correlates key assertions in the generated text with specific evidence fragments and outputs intuitive traceability tags. This breaks the "black box" operation of text generation models, making the source of each important fact clearly traceable, greatly enhancing the credibility of the results and their usability in serious scenarios.

[0020] 3. Adaptable to complex, open-domain long text generation tasks: The dynamic, iterative retrieval-generation process frees the system from the initial retrieval context window. When generating long texts (such as technical reports, academic reviews, etc.), the system can continuously introduce new and relevant knowledge as the discussion progresses, ensuring consistency of information throughout the text and overall factual accuracy.

[0021] 4. Provides flexible deployment and optimization paths: Most implementation stages of this invention (such as dynamic retrieval and closed-loop supervision) can be used as plug-ins with different LLMs, eliminating the need for training from scratch and reducing the cost of technology implementation. Simultaneously, the closed-loop supervision and feedback mechanism provides a feasible technical path for the system's continuous self-optimization in specific domains. Attached Figure Description

[0022] The present invention will be described by way of example and with reference to the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating the implementation of a verifiable text generation method.

[0023] Figure 2 This is a flowchart of the response text generation process that introduces a dynamic retrieval mechanism.

[0024] Figure 3 This is a diagram illustrating the connection between the response text and the evidence.

[0025] Figure 4 This is a flowchart of the training process for the text generation model. Detailed Implementation

[0026] All features disclosed in this specification, or steps in all methods or processes disclosed herein, may be combined in any way, except for mutually exclusive features and / or steps.

[0027] Any feature disclosed in this specification (including any appended claims and abstract) may be replaced by other equivalent or similar features, unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is merely one example of a series of equivalent or similar features.

[0028] Known text generation models employing retrieval-enhanced generation techniques have the following limitations: (1) Disconnect between retrieval and generation: Known methods usually perform a knowledge retrieval before generating the response text, using the retrieval results as a fixed context. This approach cannot dynamically introduce new knowledge during the generation of long texts, which can easily lead to the generated content deviating from the facts or becoming outdated.

[0029] (2) Factual verification is difficult: The generated final response text lacks clear and fine-grained factual support, and users cannot quickly verify the knowledge source of each key statement, which reduces the reliability and credibility of the result.

[0030] (3) Single optimization objective: The known methods mainly optimize the fluency and relevance of the generated response text, lack an explicit optimization mechanism for the overall factual consistency and traceability of the generated content, and do not effectively suppress the "illusion" problem and credibility problem of text generation.

[0031] In response to this situation, this application proposes a verifiable text generation method, which aims to effectively solve the "illusion" problem and credibility problem of text generation models using retrieval-enhanced generation technology.

[0032] Specifically, the problems addressed or to be solved by the embodiments of this application include: (1) Solve the problem of static disconnect between retrieval and generation processes: The existing RAG paradigm usually performs a one-time retrieval before generation, which makes it impossible to dynamically introduce the latest or more relevant knowledge based on the generated content and the current context when generating long texts, causing the subsequent generated content to easily deviate from the facts or become outdated.

[0033] (2) Solve the problem that the source of the generated results is untraceable and difficult to verify (i.e., credibility): The known RAG method uses the retrieved knowledge / evidence as the overall context input to the LLM model. When the model generates the response text, its internal decision-making process is a "black box". Users cannot know which piece of evidence each key fact in the final output response text comes from, which seriously damages the credibility and auditability of the results.

[0034] (3) Solving the problem of mismatch between the optimization goal of LLM model and the requirement of factual accuracy: The known text generation model mainly optimizes the fluency (confusion) and contextual relevance of the response text, and lacks explicit supervision and optimization mechanism for the factual consistency between the generated content and external evidence, which makes it difficult to effectively suppress the inherent "illusion" tendency of the model.

[0035] To effectively address one or more of the aforementioned problems, the verifiable text generation method disclosed in this application, based on an intelligent generation framework of task awareness, dynamic retrieval, traceability fusion, and closed-loop supervision, dynamically assesses information needs, actively retrieves evidence, and deeply integrates evidence into the generation process in a traceable manner at each key stage of generating the response text, while simultaneously performing closed-loop optimization through a factual supervision mechanism.

[0036] like Figure 1 As shown, the verifiable text generation method disclosed herein includes the following steps: S1, Task Awareness Phase.

[0037] The task awareness phase is responsible for receiving text generation tasks; parsing the task type of the text generation task; and extracting key information of the text generation task.

[0038] The so-called text generation task refers to the text generation requirements put forward by the user, such as query tasks, instruction tasks, etc.

[0039] Using pre-trained models (such as natural language understanding models) or rules (such as keyword recognition), the task type of the text generation task is identified. For example, the identified task types include three categories: "creative" (such as creative writing), "question-answering" (such as technical Q&A), and "report" (such as academic reports). The task type is determined through task type parsing. Different task types have different retrieval trigger conditions, which directly affect the trigger frequency of subsequent dynamic evidence retrieval and the strictness of consistency verification. The pre-trained model or rules can directly utilize existing technologies or be learned using sample data. For example, using historical text generation tasks labeled with task types as sample data, a three-output-channel classifier model is trained, with each output channel corresponding to the classification probability of "creative," "question-answering," and "report," respectively. The task type corresponding to the highest classification probability is output.

[0040] In one alternative implementation, when identifying task types using a pre-trained model or rules, a quantified factual requirement level coefficient (a normalized value between 0 and 1) is also output. This coefficient can be obtained by attaching it to sample data during classifier model training and then training the classifier model across multiple tasks. This factual requirement level coefficient can then be directly used as reference information for quantifying trigger condition criteria and the strictness of factual supervision.

[0041] In addition, key information from the text generation task is extracted during this stage to assist in generating search terms during evidence retrieval.

[0042] Specifically, the key information extracted from text generation tasks can be entities, relationships, or lists of factual judgments regarding when to retrieve them. This key information serves as an important component of the search terms used in the initial retrieval.

[0043] For example, suppose the user's task is to generate a technical report on the "Latest Advances in Neural Network Transformer Architecture." The user's input text generation task is: "Write a technical briefing on the latest advancements in Transformer architecture, ensuring accuracy and citing sources."

[0044] S101: The model parses the text generation task as belonging to the "report" task type. Furthermore, the model calculates a factual requirement level coefficient of 0.9, assuming that the uncertainty threshold (corresponding to the first predetermined threshold) for the "report" task type with a factual requirement level coefficient of 0.9 is 0.5. The extracted key information includes entities such as "Transformer," "improvement," "attention mechanism," and "sparseness."

[0045] S2, Dynamic Retrieval Stage.

[0046] The dynamic retrieval phase is responsible for retrieving supporting evidence from external knowledge sources to generate the response text, ensuring the factual accuracy of the response text.

[0047] Specifically, when the text generation model generates each token for the text generation task, the uncertainty of the text generation model is evaluated. When this uncertainty reaches a trigger condition, a search term is generated based on key information and evidence retrieval is triggered; this trigger condition is associated with the task type.

[0048] Different task types have different triggering conditions for evidence retrieval. The uncertainty of the text generation model changes dynamically when generating each word. Therefore, whether or not retrieval is triggered at different word generation stages for different task types is a dynamically changing branch task, hence the term "dynamic retrieval stage." Different methods can be used to evaluate the uncertainty of the text generation model.

[0049] In one alternative implementation, the uncertainty of the text generation model can be evaluated using the following methods: The entropy value (prediction entropy) of the predicted probability distribution of the current word generated by the text generation model is calculated. It is then determined whether this entropy value reaches a first preset threshold. If so, the model is considered to be in a state of "high uncertainty" when generating the current word, and the triggering condition is met. This first preset threshold is positively correlated with the factual level coefficient corresponding to the task type introduced above. That is, the higher the factual level coefficient, the larger the first preset threshold, and correspondingly, the lower the triggering frequency of evidence retrieval and the lower the strictness of factual supervision.

[0050] In another alternative implementation, the uncertainty of the text generation model can also be evaluated using the following method: Determine if a predetermined entity exists in the response text to which the current word belongs. If so, the trigger condition is met. The predetermined entity is determined by the task type of the text generation task.

[0051] For example, different task types can be assigned names, organizations, dates, etc., as predefined entities. Different task types can use the same or different predefined entities, depending on actual needs.

[0052] Another alternative implementation, in which the method for evaluating the uncertainty of the text generation model includes: The system determines whether the semantic relevance between the current word's response text and the historical conversation reaches a second predetermined threshold. If not, the trigger condition is met. This second predetermined threshold is also determined by the task type. Similarly, different task types can have the same second predetermined threshold or different second predetermined thresholds.

[0053] In other words, after generating one or more lexical units and generating a complete response text based on these lexical units, the semantic encoding of the response text is judged to be relevant to the existing context, and the uncertainty of the lexical units is characterized by this relevantness.

[0054] When the uncertainty of the text generation model is assessed using any of the above methods and the triggering condition for evidence retrieval is met, search terms are generated and evidence retrieval is triggered.

[0055] As an optional implementation method, the method for generating search terms includes: The currently generated context, key information extracted from the text generation task (such as entities), and evidence retrieved in the previous round (if any) are integrated and input into the search term generation model to produce refined and specific search terms. This search term generation model can directly use existing technologies, and this model is not an improvement of this application.

[0056] Once the search terms are generated, a search command (carrying the search terms) can be sent to the relevant search module to trigger evidence retrieval.

[0057] For example, such as Figure 2 As shown in S201: The text generation model generates the first sentence of the response text: "The core of the Transformer architecture is the self-attention mechanism". The average entropy value of this generation process is calculated to be 0.1, which is lower than the current first predetermined threshold of 0.5. Therefore, the retrieval is not triggered and the output is directly output.

[0058] S202: The text generation model begins generating the next response text: "In recent years, its improvement directions include." This is a transitional statement. After generating "including," the text generation model needs to predict the next key term. At this point, the entropy value of the text generation model rises sharply to 0.8, exceeding the first predetermined threshold of 0.5, thus triggering an interruption and immediately initiating a retrieval.

[0059] S203: Generate Search Terms: Combine the current context "In recent years, its improvement directions include" with the extracted key information "Transformer" and "improvement" as input into the search term generation model to generate refined search terms: "Transformer, efficient, improved, sparse, attention, latest research". Perform multi-source parallel retrieval based on these search terms and return the Top-3 relevant evidence. Examples of the retrieved evidence include: [Evidence 1] Efficient Transformers: reducing computational complexity through sparse attention; [Evidence 2]...; [Evidence 3]...

[0060] S3, the tracing and integration stage.

[0061] The traceability fusion stage is responsible for fusing the retrieved evidence with the lexical units generated based on that evidence. This ensures the traceability of the final response text while also facilitating the verification of the factual accuracy of the response text. It avoids the disconnect between the long text generation process and the evidence, thereby improving the credibility and auditability of the response text.

[0062] Specifically, in the retrospective fusion stage, evidence is retrieved based on the search terms; the retrieved evidence is used to enhance historical conversations; a text generation model generates corresponding lexical units based on the enhanced historical conversations, and a response text is generated based on these lexical units; the response text is then associated with the evidence corresponding to the lexical units (i.e., the most relevant evidence retrieved when generating the lexical units).

[0063] In one alternative implementation, multiple external knowledge sources are employed. Methods for retrieving evidence based on search terms include: Perform multi-source parallel evidence retrieval based on search terms; The retrieved evidence is preprocessed, and each piece of preprocessed evidence is uniquely identified.

[0064] After receiving search terms, the retrieval module performs evidence retrieval in parallel across multiple knowledge sources. These sources may include vector databases, structured knowledge bases, and trusted network sources. The multi-source parallel retrieval includes dense retrieval based on vector similarity, sparse retrieval based on keywords, and structured queries on domain-specific knowledge graphs. Clearly, this is only one example of multi-source parallel retrieval, and adjustments can be made for practical applications.

[0065] Because it involves multi-source information retrieval, this disclosure also preprocesses the retrieved evidence to ensure that the evidence is concise and reliable.

[0066] In one alternative implementation, the method for preprocessing the retrieved evidence includes: The retrieved evidence is deduplicated, and then a predetermined number (Top-K, where K is the set predetermined number) of evidence is selected based on its relevance to the search terms (relevance score during retrieval, such as the relevance score between the generated term and the retrieved evidence term calculated using cross-attention heads). Deduplication and selection effectively ensure the uniqueness and reliability of the evidence.

[0067] In addition, the pre-processed evidence needs to be traced and integrated with the response text. Therefore, the pre-processed evidence is also uniquely identified to ensure the traceability of each piece of evidence.

[0068] Specifically, in one optional implementation, each piece of preprocessed evidence is segmented into sentences, and a unique evidence identifier (such as an ID number) is assigned to each segmented sentence. These evidence identifiers are evidence supported by lexical tags, thereby granting the response text traceability and auditability.

[0069] After retrieving relevant evidence based on search terms, the retrieved evidence is used to enhance the historical conversation, that is, to enhance the generated context. This allows the text generation model to learn more factual knowledge from the context when generating lexical units and response text, thereby improving the factual credibility of the response text.

[0070] There are also many ways to enhance historical conversations.

[0071] In one alternative implementation, the method of enhancing historical conversations using retrieved evidence includes: The retrieved evidence (preprocessed evidence) is concatenated with historical sessions and then encoded.

[0072] In another alternative implementation, the retrieved evidence and historical conversations can be encoded separately and then concatenated.

[0073] Alternatively, another alternative implementation is to encode the retrieved evidence by embedding it into historical sessions using a preset identifier (e.g., "|| ||").

[0074] Regardless of the method used, after enhancing the historical conversation with retrieved evidence, the data is input into the text generation model. The model then generates corresponding lexical units based on the enhanced historical conversation, and subsequently generates sentence-based response text from these lexical units. This process of generating each sentence of response text is repeated to obtain the final complete response result.

[0075] To ensure the traceability and verifiability of the response text, the response text is associated with evidence corresponding to the word units.

[0076] Specifically, in one alternative implementation, the method of associating the response text with evidence corresponding to lexical units includes: A cross-attention mechanism is used to evaluate the relevance of the lexical to be generated to all evidence, and the evidence with the highest relevance is implicitly associated with the lexical. Furthermore, when generating the response text based on the lexical, explicit traceability tags are generated for the evidence associated with the lexical contained in the response text based on the implicit association.

[0077] Implicit associations, invisible to the user, are relationships built within the model for validation during the closed-loop supervision phase. These associations can take the form of tagged evidence identifiers or additional relevance scores, such as [Evidence ID: Relevance Score]. Explicit traceability markers, on the other hand, are visible to the user. Users can view the original evidence through these markers, such as superscripts, highlighted links, emphasis marks, or sidebar references at the end of the response text. The marker content can also be evidence identifiers.

[0078] In the previous evidence retrieval process, the relevance between the retrieved evidence and the terms to be generated was calculated (represented by association weights). This calculated data can be directly used when associating evidence here. When generating a sentence-based response text, a single response text may contain multiple terms, meaning it may be associated with multiple different pieces of evidence. In this case, a traceability tag can be generated for each term based on its associated evidence, and the corresponding traceability tag can be added after each term. Alternatively, multiple different pieces of evidence can be written into the traceability tag and uniformly labeled in the response text (e.g., at the end of the sentence).

[0079] For example, suppose the user's text generation task is: "What are the bottlenecks of the Transformer model in processing long sequences? What are some ways to improve it?". After performing stages S1 and S2, stage S3 includes: S301: Conduct the first round of retrieval to obtain general evidence regarding "double growth in computational complexity" and "memory consumption".

[0080] S302: Evidence Preprocessing and Identification: Segment all retrieved evidence and assign a unique evidence ID to each fact sentence, for example: [E1] The self-attention mechanism of the standard Transformer has O(n) time complexity. 2 The computational complexity is ).

[0081] [E2] Linear attention and sparse attention are two mainstream improvement paradigms.

[0082] like Figure 3 The diagram illustrates how the text generation model associates generated words with evidence words when generating response text. Figure 3 In the diagram, the right side shows the evidence text block with evidence labels, and the left side shows the generated response text. The connecting line between the two represents the attention weight. The term "sparse attention" is mainly associated with "sparse attention" in evidence E21, while the term "computational complexity" is mainly associated with "computational complexity" in evidence E1.

[0083] S303: Tracing Fusion. The text generation model generates text based on the task and evidence. Inside the decoder, a dedicated tracing cross-attention head comes into play.

[0084] When the model generates the word "computational complexity", the cross-attention head calculates that the word has the highest weight relative to the "computational complexity" in the evidence [E1].

[0085] When the model generates the word “improvement”, the cross-attention head calculates that the word has the highest weight relative to the “improvement paradigm” in the evidence [E2].

[0086] When the model subsequently generates the term “sparse attention”, the cross-attention head calculates that the term has the highest weight relative to “sparse attention” in the evidence [E2].

[0087] Based on this weighted record, the model performs retrospective fusion of evidence E1 with the context "The main bottleneck of Transformer in processing long sequences is" and retrospective fusion of evidence E2 with the context "The main improvement methods include". After the text generation model generates the complete sentence-style response text "The main bottleneck of Transformer in processing long sequences is that the computational complexity of its self-attention mechanism increases quadratically with the sequence length" based on the enhanced historical conversation, it automatically adds a retrospective marker [E1] to the end of the sentence and automatically adds a retrospective marker [E2] to the end of the sentence after generating "The main improvement methods include sparse attention".

[0088] S304: Output verifiable text. The final response text generated by the text generation model is as follows: "The main bottleneck of Transformer in handling long sequences is that the computational complexity of its self-attention mechanism increases quadratically with the sequence length [E1]. The main improvement methods include sparse attention [E2], linear attention [E2], etc." When the user clicks [E1] or [E2], the system can display the original document of evidence [E1] or [E2] in a floating manner.

[0089] Iterate through stages S2 and S3 until the report is completed, resulting in a verifiable report where all key assertions are accompanied by evidence source markers.

[0090] S4, Closed-loop monitoring stage.

[0091] The closed-loop phase is responsible for verifying the factual accuracy of the response text to effectively suppress the "illusion" problem caused by the generation of long texts.

[0092] Because the generated lexical units are associated with supporting evidence during the traceability fusion phase, the response text is traceable. Therefore, in the closed-loop supervision phase, the consistency between the response text and the associated evidence can be verified. This response text is generated by the text generation model based on lexical units (generating fluent sentence-like response text based on lexical units is a basic function of LLM). If the verification fails, the response text is rewritten. The rewritten response text only contains the sentence that failed the consistency verification, not the entire generated result of the text generation task.

[0093] In the aforementioned stages, the evidence supporting the generation of lexical units was associated during the generation of the response text. Therefore, based on this association, the final generated response text can be subjected to consistency verification (i.e., post-verification) with the associated evidence, thereby verifying both the authenticity and factual accuracy of the response text. Furthermore, the text generation model can be optimized based on the verification results. This optimization operation is significantly different from traditional optimization of text fluency; it focuses on optimizing the consistency of external evidence (i.e., factual accuracy).

[0094] In one alternative implementation, the method for verifying the consistency between the response text and the associated evidence includes: The system assesses the risk of contradiction between the response text and the associated evidence (the evidence with the highest relevance to the contained terms). If this risk of contradiction reaches a third predetermined threshold, the verification fails. This third predetermined threshold can also be determined by the task type, or it can be set based on empirical values.

[0095] The so-called contradiction risk refers to the probability that the response text contradicts the evidence. In one alternative implementation, the contradiction risk can be assessed by the following method: Examine the probability that the entities or relationships in the response text contradict the evidence.

[0096] This approach utilizes a powerful fact checker (e.g., directly verifying entities or relationships in the generated response text through evidence) to scan the response text and identify potential contradictions in the response text (in sentence form).

[0097] In another alternative implementation, the risk of conflict can also be assessed using the following method: The test examines the proportion of inconsistencies in the evidence linked to multiple searches based on the same search term in the response text. If the inconsistency proportion reaches a predetermined third threshold, the sentence is considered to be of high risk of contradiction.

[0098] If the consistency check fails, a rewrite operation is triggered on the response text. The rewritten response text only includes sentences with a high risk of contradiction (i.e., those that failed the check), not all responses. The rewrite operation involves returning to the S2 phase, retrieving the latest evidence, and re-executing subsequent operations.

[0099] To further improve the ability of the text generation model to generate response text that meets the requirements of factual accuracy in one go, in some optional embodiments of this disclosure, the supervision capability of the S4 stage is integrated into the retraining stage of the text generation model, and the loss of consistency verification is integrated into the total training loss of the text generation model to help optimize the model parameters.

[0100] Specifically, such as Figure 4 As shown, the training methods for text generation models include: Define the reference response text for the sample, which is the standard answer to the text generation task that serves as the sample data.

[0101] The sample is input into the text generation model, and the language modeling loss and fact consistency loss of the text generation model are calculated respectively. Among them, the language modeling loss is the cross-entropy loss between the reference response text and the response text generated by the text generation model; the fact consistency loss is the degree to which the response text generated by the text generation model is not implied by evidence.

[0102] The total loss is obtained by weighted summation of the language modeling loss and the fact consistency loss.

[0103] The model parameters of the text generation model are optimized based on this total loss.

[0104] For example, step S401: For each training sample, construct a quadruple (text generation task Q, retrieved evidence E, the reference response text A_std as defined, and the response text A_gen generated by the model).

[0105] Step S402: Calculate the language modeling loss.

[0106] Using the reference response text A_std as the supervision target, the traditional language modeling loss generated by the model is calculated and denoted as L_lm, which is the cross-entropy loss.

[0107] Step S403: Calculate the fact consistency loss.

[0108] The response text A_gen generated by the model and the evidence E are input into a pre-trained natural language inference model NLI.

[0109] The NLI model outputs three classification probabilities: P_entail, P_neutral, and P_contradict.

[0110] The fact consistency loss (L_fact) is defined as 1 - P_entail. That is, the less the content generated by the model is "implied" by the evidence, the greater the fact consistency loss.

[0111] Step S404: Joint optimization.

[0112] The total loss function is defined as: ; in, and Here are the hyperparameters, representing the weights of the language modeling loss L_lm and the fact consistency loss L_fact, respectively. ,For example , By optimizing the total loss L_total through backpropagation, the text generation model learns to generate content consistent with the evidence while simultaneously learning fluent generation capabilities.

[0113] Step S405: Rewriting during inference (optional). During inference, if the sentence generated by the trained text generation model is judged by the built-in lightweight NLI model to be seriously contradictory to the evidence (P_contradict>0.8, where 0.8 is the set third predetermined threshold, which is adjustable), the "local rewriting" sub-process can be triggered, which only re-retrieves and generates that sentence, i.e., stages S2 and S3.

[0114] Based on the ideas of this application, this disclosure also proposes a verifiable text generation apparatus, which includes a processor and a storage medium. The storage medium stores computer instructions, and the processor executes these computer instructions to perform the verifiable text generation method of the above embodiments or optional implementations.

[0115] This invention is not limited to the specific embodiments described above. The invention extends to any new feature or combination disclosed in this specification, as well as any new method or process step or combination disclosed herein.

Claims

1. A verifiable text generation method, characterized in that, include: S1. Receive the text generation task; parse the task type of the text generation task; Extract key information from the text generation task; S2. When the text generation model generates each word for the text generation task, evaluate the uncertainty of the text generation model; when the uncertainty reaches the triggering condition, generate search terms based on the key information and trigger evidence retrieval; the triggering condition is associated with the task type. S3. Retrieve evidence based on the search terms; Enhance historical conversations with evidence retrieved; The text generation model generates corresponding lexical units based on the enhanced historical conversation, and generates response text based on the lexical units; the response text is then associated with evidence corresponding to the lexical units. S4. Verify the consistency between the response text and the associated evidence; If the verification fails, the response text will be rewritten.

2. The verifiable text generation method as described in claim 1, characterized in that, Evaluate the uncertainty of the text generation model, including: Calculate the entropy value of the predicted probability distribution of the current word element, and determine whether the entropy value reaches a first preset threshold. If so, determine that the triggering condition has been met. The first preset threshold is positively correlated with the factual level coefficient corresponding to the task type. Alternatively, determine whether a predetermined entity exists in the response text to which the current word belongs; if so, determine that the triggering condition has been met; the predetermined entity is determined by the task type. Alternatively, determine whether the semantic relevance between the response text to which the current word belongs and the historical conversation reaches a second predetermined threshold. If not, determine that the triggering condition has been met. The second predetermined threshold is determined by the task type.

3. The verifiable text generation method as described in claim 1, characterized in that, Evidence retrieved based on the search terms includes: Perform multi-source parallel evidence retrieval based on the search terms; The retrieved evidence is preprocessed, and each piece of preprocessed evidence is uniquely identified.

4. The verifiable text generation method as described in claim 3, characterized in that, The retrieved evidence is preprocessed, and each piece of preprocessed evidence is uniquely identified, including: The retrieved evidence is deduplicated, and then a predetermined number of pieces of evidence with the highest relevance to the search terms are selected. Each piece of evidence is segmented into sentences, and a unique evidence identifier is assigned to each segmented sentence.

5. The verifiable text generation method as described in claim 3 or 4, characterized in that, Enhance historical conversations with evidence retrieved, including: The retrieved evidence is then concatenated with the historical sessions and encoded. Alternatively, the retrieved evidence and the historical conversation can be encoded separately and then concatenated. Alternatively, the retrieved evidence can be encoded by embedding it into the historical session using a preset identifier.

6. The verifiable text generation method as described in claim 3 or 4, characterized in that, Associating the response text with evidence corresponding to the lexical units includes: The cross-attention mechanism is used to evaluate the relevance of the lexical to be generated with all evidence, and the evidence with the highest relevance is implicitly associated with the lexical. Furthermore, when generating a response text based on the lexical, an explicit traceability marker for evidence associated with the lexical contained in the response text is generated based on the implicit association.

7. The verifiable text generation method as described in claim 1, characterized in that, Verify the consistency between the response text and the associated evidence, including: Assess the risk of contradiction between the response text and the associated evidence. If the risk of contradiction reaches a third predetermined threshold, the verification fails.

8. The verifiable text generation method as described in claim 7, characterized in that, The methods for assessing the risks of conflict include: Examine the probability that the entities or relationships in the response text contradict the evidence; Alternatively, examine the proportion of inconsistencies in the evidence associated with the response text across multiple searches based on the same search term.

9. The verifiable text generation method as described in claim 7 or 8, characterized in that, The training method for the text generation model includes: Define the reference response text for the sample; The sample is input into the text generation model, and the language modeling loss and fact consistency loss of the text generation model are calculated respectively; the language modeling loss is the cross-entropy loss between the reference response text and the response text generated by the text generation model; the fact consistency loss is the degree to which the response text generated by the text generation model is not implied by the evidence. The total loss is obtained by weighted summation of the language modeling loss and the fact consistency loss. The model parameters of the text generation model are optimized based on the total loss.

10. A verifiable text generation device, characterized in that, It includes a processor and a storage medium; the storage medium stores computer instructions, and the processor executes the computer instructions to perform the verifiable text generation method as described in any one of claims 1-9.

Images