Knowledge constrained tree search decoding and token level illusion detection

Abstract

Provided herein is a novel method referred to as knowledge constrained tree search decoding (KCTS), which is a first discriminator guided constrained decoding method for knowledge-based generation. The KCTS method also uses Monte Carlo Tree Search Decoding (MCTS) rather than a conventional LM decoding method that reorders the importance of tokens starting from each token using multiple simulations based on the estimated future loyalty. In combination with a novel token hierarchy hallusion detection method which is trained to predict the starting point of a hallusion token and enables a knowledge classifier defined on a sequence hierarchy to effectively adapt to the token hierarchy: Reward Inflection Point Approximation (RIPA), the novel method can greatly relieve the hallusion problem of LLM without fine tuning, and can improve the hallusion detection efficiency. Thus, high computational costs are prevented from being caused.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 588,297, filed on October 6, 2023, the disclosure of which is incorporated by reference in its entirety. Technical Field

[0002] The present invention relates to large language models. More specifically, the present invention proposes a method for decoding by a heuristic tree search process, combined with an algorithm for detecting and reducing hallucination detections. Background Art

[0003] Large Language Models (LLMs) are a type of artificial intelligence (AI) designed to understand and generate human-level natural language. The LLMs are built using machine learning techniques, especially deep learning, and are trained based on large amounts of text data.

[0004] However, LLM can generate false information, specifically text that appears grammatically and semantically sound but is actually incorrect or nonsensical, often referred to as "hallucinations".

[0005] To mitigate the hallucination problem, previous studies have attempted to augment the input of language models with contextual knowledge, using reader language models that take context and reference knowledge as input to generate responses. While this approach has shown promise, it involves fine-tuning the reader language model, which poses considerable challenges. First, the increasing size of language models makes training language models computationally expensive, which is becoming increasingly expensive, not to mention that some API-based LLMs (e.g., OpenAI API) are not trainable by end users. Second, many state-of-the-art language models are designed as multi-task zero-shot models through instruction tuning, aiming to perform well across a variety of tasks.

[0006] Another common approach to mitigate the hallucination problem is to fine-tune the LLM using knowledge from its input. However, extensively fine-tuning a language model on a specific task may lead to catastrophic forgetting or catastrophic interference, which is the phenomenon that the model suddenly forgets previously learned information after learning new information, which in turn greatly impairs its generalizability across different tasks and hurts its overall performance.

[0007] Additionally, although designing decoding methods for LLMs is a natural way to mitigate hallucinations without fine-tuning, current plug-and-play guided decoding works, such as weighted decoding (WD), are still not directly adaptable to knowledge-based scenarios due to their inability to identify the necessary knowledge required for generation, which can lead to hallucinations.

[0008] Therefore, there is a need to develop a novel model for LLM decoding that reduces hallucinations without fine-tuning. The present invention addresses this need. Summary of the Invention

[0009] In one aspect, the present invention provides a computer-implemented model-agnostic natural language generation method.

[0010] The computer-implemented model-agnostic natural language generation method includes: providing a plain text prompt containing information; automatically generating tokens based on the plain text prompt containing information; reordering the importance of each generated token and selecting, through a heuristic search process, the token with the highest knowledge fundamentality in each scenario; using a specially trained knowledge classifier to determine token-level hallucinations by identifying the inflection point of hallucinations and replacing hallucination tokens with additionally generated tokens; and using the tokens with the least hallucinations and the highest knowledge fundamentality to generate a plain text response.

[0011] In one embodiment, the heuristic search process of the computer-implemented model-agnostic natural language generation method is Monte Carlo tree search, which includes: selecting to traverse the tree from the root node until reaching the child node of the leaf node; expanding the leaf node until reaching the final state; evaluating an approximate token-level knowledge fundamentality score of the expansion result; recursively backpropagating the approximate token-level knowledge fundamentality score from the leaf node to the root node; and selecting the child node with the highest approximate token-level knowledge fundamentality score for further traversing the tree to reach subsequent leaf nodes.

[0012] In another embodiment, training the knowledge classifier of the computer-implemented model-agnostic natural language generation method includes: providing training instances training instances (y, x, k) ∼ D; randomly swapping knowledge k from a knowledge source with another knowledge k' to form a negative example; sampling for positions 1 < i < T, where T is the length of y; truncating the response y to the i'-th token and obtaining y i ; and completing the sequence y by sampling from P "M (y|x, y i , k) with the knowledge text k included in the instruction i .

[0013] In another embodiment, during the training process of the knowledge classifier, the tokens are marked by starting with marking the tokens as positive, marking the first hallucination token as negative; and predictively marking all tokens after the first hallucination as negative. Brief Description of the Drawings

[0014] The accompanying drawings, in which the same reference numerals refer to the same or functionally similar elements, contain diagrams of certain embodiments to further illustrate and clarify the above and other aspects, advantages and features of the present invention. It should be understood that these drawings depict embodiments of the present invention and are not intended to limit the scope thereof. The present invention will be described and explained with additional particularity and detail through the use of the accompanying drawings, in which:

[0015] Figure 1 A schematic diagram of the Monte Carlo Tree Search (MCTS) decoding algorithm is provided, which is the first key component of the Knowledge Constraint Tree Search (KCTS) decoding framework.

[0016] Figure 2 A schematic diagram of another key component of the KCTS decoding framework provided in the present invention is provided, which is called Reward Inflection Point Approximation (RIPA) training of the discriminator. DETAILED DESCRIPTION

[0017] As used in the disclosure herein, "decoding" refers to the process of generating text from an LLM when given a prompt or input. Decoding involves selecting words or tokens in sequence in response to a prompt or input for forming a coherent sentence. Common decoding strategies include: greedy search, in which the LLM selects the token with the highest probability at each step; beam search, in which multiple candidate sequences, called beams, are maintained and expanded in parallel, retaining only the top k beams based on cumulative probabilities; top-k sampling, in which the top k tokens are sampled by the LLM at each step with the highest probability; top-p sampling, in which at each step, the LLM samples from the smallest set of tokens that exceed any cumulative probability; and temperature sampling, in which the probability of a token is adjusted by applying a temperature parameter T, under which lower temperatures produce steeper distributions and higher temperatures flatten the distribution.

[0018] As used in the disclosure herein, "token" refers to the smallest unit of text that is processed individually by LLM. A token can be a word, a subword (i.e., each word can be further split into smaller units under different tokenization methods), a character, or a special token for a specific purpose in LLM.

[0019] As used in the disclosure herein, "knowledge-based" refers to the degree to which the decoding result (ie, the generated text) is consistent with the facts and context-appropriate information. Therefore, knowledge-based also reflects the degree of illusion of the output of the LLM.

[0020] As used in the disclosure herein, "fidelity" refers to the extent to which tokens or generated text from an LLM accurately reflect the information or data input to the LLM such that the generated output is contextually consistent with the information received by the LLM.

[0021] Hallucinations, or the generation of non-factual or unsupported content by evidence, by language models have been studied and reported in various fields such as machine translation, abstractive summarization, or image captioning. Recently developed LLMs (e.g., Bing Chat or perplexity.ai) are used as generative search engines, although the responses are not always verifiable. Several detection methods and benchmarks have been designed to automatically detect and quantify hallucinations in model-generated text.

[0022] Knowledge-based generation is mainly driven by retrieving relevant knowledge and training the generator to produce responses enhanced on the retrieved knowledge; or by learning and storing entity or fact representations provided as input to the generator. While these methods all address the problem of knowledge-based generation, all of them require comprehensive fine-tuning of the generator, which may degrade the zero-shot capability of the base model (i.e., the ability of the model to perform unseen tasks without explicit training on relevant examples based solely on training from which the LLM acquires prior knowledge) due to catastrophic forgetting and thus incurring significant computational costs.

[0023] Guided decoding includes supervised controllable text generation, discriminator-guided decoding, and constrained decoding, where the user controls the sentiment or style of the generated text, or constrains the generation to vocabulary constraints. Different LMs adopt different guided decoding frameworks. For example, Plug and Play ML (PPLM) introduces the key concept of Bayesian decomposition P(y|x, c) ∝ P(y|x) P(c|y), where c is the control attribute. PPLM trains a small discriminator on the frozen LM and performs gradient ascent from the discriminator to maximize P(c|y). FUDGE achieves this by applying the auxiliary classifier probability P(c|y ＜t ) directly to the token probability P(y t |y ＜t , x) are reweighted to perform weighted decoding (WD). To perform reordering for each step, P(y|x)P(c|y) is decomposed into a token level, and a token level attribute classifier P(c|y) is used. ＜t ). NADO is also derived from P(c|y ＜t ) is sampled from a similar token-level distribution weighted by P(c| y＜t ) is defined as an approximation of the sequence-level oracle P(c|y). GeDi and DExperts also adopt a weighted decoding approach, but avoid enumerating the vocabulary P(c|y by training a generative classifier. ＜t，i ).

[0024] Constrained decoding methods focus on constraint satisfaction, such as lexical constraints or right-hand coherence. Since constraint satisfaction can be measured after the sequence is fully generated, search-based methods have been proposed that estimate future scores (rewards) at each decoding step. Unlike weighted decoding, these methods commit tokens based not only on the score of the current token but also on an estimate of the future reward.

[0025] In this invention, a novel method called Knowledge Constrained Tree Search Decoding (KCTS) is provided, which is a first discriminator guided constrained decoding method for knowledge-based generation. KCTS applies an auxiliary knowledge classifier on a frozen LM to detect hallucinations, and uses its knowledge-based score to guide the decoding process.

[0026] The KCTS method also uses Monte Carlo Tree Search Decoding (MCTS) instead of the conventional LM decoding method, which uses multiple simulations to re-rank the importance of tokens starting from each token based on the estimated future fidelity. This can better estimate the value of tokens compared to the existing techniques that only re-rank tokens based on the current fidelity score. By combining a knowledge classifier and a classifier-guided decoding algorithm, KCTS constrains the generated text to ensure its fidelity to the reference knowledge.

[0027] In addition, a novel token-level hallucination detection method, Reward Inflection Point Approximation (RIPA), is trained to predict the starting point of hallucinated tokens, so that the knowledge classifier defined at the sequence level can be effectively adapted to the token level. The RIPA knowledge classifier solves two problems in the prior art: the RIPA knowledge classifier alleviates the expensive retraining of LM weights by keeping the LM frozen and only training the adapter weights, and the RIPA knowledge classifier accurately determines whether the generated text contains hallucinations. This provides a better signal to the MCTS decoding algorithm, resulting in more faithful text generation.

[0028] In addition, the KCTS method of the present invention is model-agnostic, so that it can be applied to previously trained models. Moreover, the KCTS model does not involve fine-tuning of the LLM, which would incur a large amount of computational cost.

[0029] Examples

[0030] Example 1 - Problem Statement

[0031] The fact generation capability of the instruction-tuned LM will be improved under the constrained decoding setting. The problem can be formulated as y~P LM (y|x,k,α k ), (1), (1) Where y is the generated text, x is the input text with the task description, k is the knowledge fragment to which y must be constrained, and ak is an attribute representing the basicity of y to k.

[0032] Let f(y, k) = P(α k =1|y, k) is a function that defines the fidelity of generating y and k. After Bayesian decomposition, Bayes' rule is applied to the above equation (1) to obtain the following equation (2): P LM (y|x,k,α k )∝P LM (y|x)f(y, k). (2)

[0033] From an optimization perspective, obtaining the most knowledge-based generation while remaining faithful to the task instructions can be written as follows: y * = argmaxP LM (y|x)f(y, k). (3)

[0034] Next, taking into account the autoregressive nature of language models, this equation can be decomposed into a token hierarchy as found in FUDGE:

[0035] 1.1 Token-level loyalty

[0036] The knowledge fidelity f (or hallucination in the opposite perspective) is explicitly defined at the sequence level, which can be modeled as an entailment or fact verification problem. However, in order to guide generation at each step, it is necessary to ＜t Define f(y ＜t , k). Following NADO (Meng et al., 2022), f(y ＜t , k) is defined as an approximation of future fidelity as shown in the following equation: f(y ＜t , k)≈f(P(y|y ＜t , x), k). (5)

[0037] Example 2 - KCTS Method

[0038] 2.1 Monte Carlo Tree Search Decoding

[0039] While weighted decoding (WD) reweights the token distribution by the knowledge basicity score at each step, it selects the most basic tokens in a greedy manner, which can lead to suboptimal solutions. This is particularly problematic considering that basicity is only explicitly defined after the sequence is fully generated, and the guidance signal from the classifier at each step is an approximation. For this purpose, it is proposed to use the Monte Carlo Tree Search algorithm (MCTS), which can better estimate future knowledge basicity through multiple simulations, which has been shown to be effective in other scenarios such as sentiment polarity control.

[0040] MCTS consists of four main steps: selection, expansion, rollout, and backpropagation.

[0041] Selection: Start from the root (the currently generated sequence), traverse the tree until you reach a leaf node, using the PUCT algorithm to select a child node: Where V(s * ) is node s * The estimated value of * For node s * The visit count (i.e., the number of simulations after the node), and N * For * The number of access counts of the parent node. +,-. is a hyperparameter that controls the trade-off between exploration and exploitation, where higher c +,-. Promote exploration. P represents the language model probability.

[0042] Extension: If the selected leaf node is not EOS (final state), then the node is deeply expanded by k children by decoding using LM for one step and selecting the first k tokens as children.

[0043] Preview (Evaluation): From the selected leaf node s, use the language model to generate until EOS, then use the knowledge classifier denoted as f to evaluate the faithfulness of the generated sequence f(y,k). Let this be the value of s, V(s) = f(y,k). However, this full preview can be expensive and lead to large discrepancies. Therefore, an approximate token-level fundamentality score is used instead: V(s)←f(y )',( ,k), where y )',( refers to the currently generated sequence y with token s appended )' .

[0044] Backpropagation: This score is then backpropagated recursively from the previously evaluated nodes back to the root. After PPL-MCTS, the average aggregate of all simulations played after this node is used. This results in This is for all s on the path from leaf node s to the root * , where ρ(s i ) is s i These values ​​will be used in the PUCT algorithm to select the node in step 1 in the next simulation.

[0045] Steps 1 to 4 are repeated for a predefined number of simulations. For the present invention, the number of simulations is set to 50 in order to have a robust and reliable performance in knowledge-based generation.

[0046] 2.2 Token-level hallucination detection

[0047] f is first modeled as a fact verification problem, and the binary classifier f(y, k) = P f (α k = 1|y, k) is based on sequence-level training. In order to adapt f to the token-level fidelity f(y ＜t ), previous methods employ either random input sequence truncation or token-level labeling to train classifiers. The random truncation method may be sample-inefficient because only a portion of the input is used during training, and the random truncation method may add noise to the training because the input sequence may no longer contain hallucinated content after truncation while still receiving negative labels. Although the token-level labeling method may be more sample-efficient, it may induce even more noise during training and hinder convergence.

[0048] To alleviate these shortcomings, a novel method called Reward Inflection Point Approximation (RIPA) is developed to approximate the future f for an unfinished token sequence by explicitly providing token-level labels for the underlying. A schematic diagram of RIPA’s comparison with previous methods can be found in Figure 1 Inspired by the “hallucination snowball” effect where an initial hallucination of a language model can completely lead to further unsupported claims, identifying the inflection point of this fundamentality is a more effective approximation of future scores. Therefore, RIPA trains a classifier to identify the starting point or inflection point of the reward (fundamentality) with a token-level label that starts at 1 (positive) and becomes 0 (negative) after the first hallucination token.

[0049] RIPA does not associate benign tokens with hallucination labels, which may lead to more stable training. In addition, RIPA is trained to predict 0 for all tokens after hallucination is detected, which will further reduce future exploration under the node in MCTS, thereby preventing the selection of the token. RIPA and MCTS (i.e., KCTS) together provide better estimates of equations (4) and (5).

[0050] Training RIPA requires fine-grained token-level annotation of hallucinations, which is difficult to obtain through human annotation. Alternatively, two simple methods for generating synthetic data are listed below.

[0051] Knowledge shuffling: Given a training instance (y, x, k) ∼ D, randomly swap k from a knowledge source with another knowledge k′ to form a negative example. Then, although the correlation between y and x remains unchanged, the grounding of y in k becomes negative because y is no longer based on k. All tokens in y are labeled 0.

[0052] Partial hallucination: Similar to above, given a training instance (y, x, k) ∼ D, first randomly swap k with another knowledge k′. Then, randomly sample a position 1 < i < T, where T is the length of y, and truncate the response y to the i'th token and obtain y i . Then require the LM to sample from P "M (y|x, y i , k) in a zero-shot manner by including the knowledge text k in the instruction to complete the sequence y i . Note that the goal here is to utilize the hallucination of the LM: thus, sample the completion at a temperature greater than 1. In this method, only the completion tokens (y / i ) are labeled 0.

[0053] Use a balanced mixture of both to obtain the training set; however, for tasks where x and k are indistinguishable (e.g., summarization), the problem becomes P(y|k). Thus, only the partial hallucination method is employed. Further details of the hyperparameters used in each task are presented below.

[0054] Example 3 - Experimental Setup

[0055] 3.1 Dataset

[0056] To demonstrate the advantages of the guided decoding method in knowledge-based generation, two well-studied tasks are selected: knowledge-based dialogue and abstract summarization. In both tasks, the language model is given a reference knowledge snippet in the input and is required to generate a response using the knowledge.

[0057] Knowledge-based dialogue: Knowledge-based dialogue (KGD) can be formulated as P LM (y|x, k), where y is the generated response, x is the dialogue history, and k is the relevant knowledge. Experiments were conducted on the gold knowledge because the focus of this study is to demonstrate the potential of constrained decoding in the knowledge base. The unseen topic portion of the test set of the Wikipedia Wizard (WoW) dataset was used as the benchmark dataset for this task.

[0058] Summarization: Abstractive summarization can be naturally considered as a knowledge-based generation task, since the generated summary should only contain information from the reference document. Improving the factual fidelity in abstractive summarization is a challenging task. The CNN / DM dataset is used as a benchmark dataset for the study.

[0059] 3.2 Evaluation Metrics

[0060] Various evaluation metrics applied to knowledge base and natural language generation are used. The metrics are classified into three categories: token-based, knowledge-based, and multifaceted. For token-based automatic metrics, BLEU-4, Rouge-L, ChrF, and METEOR 5 are used. For knowledge-based metrics, first Knowledge-F1 (KF1) is used, which measures the unary overlap between generated tokens and knowledge tokens and K-replicas, as defined in equation (8): Where LD represents the Levenshtein distance between the generated response and the reference knowledge string. This metric captures the amount of verbatim copying of knowledge in the generation. The purpose of this metric is to monitor whether the model simply copies knowledge as a response, which defeats the purpose of using a generative LM. Therefore, a too high copy rate (e.g., ≥70%) may indicate a reduced utility of the response.

[0061] Finally, UniEval, a multifaceted model-based evaluator trained using the Boolean QA format, is also utilized. For the dialogue task, naturalness, coherence (with dialogue context), and groundedness (to knowledge) are utilized; and for summarization, coherence (within the summary), consistency (with the article), fluency, and relevance (with the golden answer) are considered as fine-grained evaluation dimensions. For summarization, the MFMA (Lee et al., 2022) pre-trained metric is also adopted, which shows SOTA level correlation with human annotations on CNN / DM (data partitioning of FRANK and QAGS benchmarks).

[0062] 3.3 Baseline

[0063] Popular API-based LLMs and publicly available instruction-tuned language models of various sizes were used as initial baselines. LLMs provided through the OpenAI API were studied; namely, ChatGPT (gpt-3.5-turbo-0301) and GPT 3.5 (text-davinci-003). For instruction-tuned models, the Flan-T5 (FT5) model family and two different sizes of T0++ (XL and XXL) were studied. It should be noted that due to significant differences in model size and training cost, they are not directly compared with the described methods.

[0064] It should be noted that although the FT5 and T0++ models have been fine-tuned based on some dialogue data, knowledge-based dialogue is still an unseen task for these models. Therefore, we first collect zero-shot results from models tuned with various instructions, and experiment with guided decoding on the zero-shot results. On the other hand, the CNN / DM summarization task is included in the T0 dataset mixture and the natural instruction dataset, which is part of the Flan fine-tuning. Therefore, we can consider performing knowledge-constrained decoding (KCD) on the CNN / DM test set to guide the already fine-tuned model to further improve the fact dimension.

[0065] Next, weighted decoding (WD) is applied and it constrains the decoding baselines directly on the KCD setting, namely FUDGE, NADO and MCTS, which serve as strong baselines directly comparable to the inventive method.

[0066] 3.4 Implementation Details

[0067] To train the classifier, the lightweight adapter is applied only to the decoder layer of the language model via LoRA, and a single linear layer is added in addition to the last hidden state. This adds only 0.21% of additional trained weights that can be disabled or enabled at test time, which does not compromise the rich multi-task capabilities of the basic instruction following model. More details on model training are available further below.

[0068] Example 4 - Main Assessment

[0069] 4.1 Results Analysis

[0070] The performance of the zero-shot LLM and various instruction-fine-tuned models is reported in the upper half of Table 1, and the performance of the directly comparable decoding-based baseline and the method of the present invention is reported in the lower half. The performance of the supervised fine-tuning (SFT) version of FT5-XL on the KGD task is also studied. It should be noted that the performance in the upper half (LLM and SFT) is only used to provide an overview of the power of each language model when tested on WoW, rather than a direct comparison with the method of the present invention. The instructions used for each model are further listed below.

[0071] Table 1: Results on the WoW test set (unseen topics) SFT stands for supervised fine-tuning, and FT5 is an abbreviation for Flan-T5. Under the UniEval metric, each letter represents the following: N for naturalness, C for coherence, and G for basicity. For all metrics, except K-copy, larger numbers are preferred. It should be noted that the performance of LLM in the upper half is for reference only. For each column, bold represents the best score in the KCD method under the FT5-XL backbone, and the underline indicates the second best.

[0072] According to the results in the upper part of Table 1, ChatGPT demonstrates a strong ability to generate responses that highly overlap with knowledge. On the other hand, FT5-XL, although the smallest in size (3B) among all the models studied, demonstrates the best performance in generating responses that are most based on reference knowledge, as indicated by the KF1 and Basicity columns of Table 1. Therefore, FT5-XL was selected as the base model in additional studies of guided decoding.

[0073] Comparisons of baseline decoding methods with KCTS of the present invention can also be found in the lower half of Table 1. Knowledge Weighted Decoding (KWD) - a WD variant of RIPA guidance, also experimented for ablation purposes. The penultimate group contains results for baseline-guided decoding methods, used together with each of their own proposed approximations of token-level classifiers. Both FUDGE and MCTS use random truncation approximations, and NADO uses a token-level labeling approach. All decoding methods show improvements over the kernel sampling baseline on the basis of knowledge, as indicated by the higher KF1 scores and the basis column of UniEval. The results also clearly show that RIPA provides a better token-level guidance signal for KCD, as both KWD and KCTS show improvements on dimensions other than naturalness. KCTS also results in the highest f activation, confirming the hypothesis that MCTS, which also estimates the future rewards of token selections through simulation, can produce higher rewards.

[0074] To show that KCTS ultimately guides future generation trajectories towards more grounded responses, token generation is constrained for the initial T tokens, and then the original language model is used to complete the sentence using kernel sampling. The results in Table 2 indicate that the initial grounded tokens provide good context for the LM to achieve future fidelity f(y ＜t , k)≈f(P(y|y ＜t ), k) to generate a more basic response. In addition, this provides a performance / speed trade-off parameter that can be adjusted by the user.

[0075] Table 2: Ablation study of the number of initial tokens to be constrained in knowledge using KCTS

[0076] Human evaluation of the generated responses was also conducted to assess their quality. 100 instances were randomly sampled and three different evaluators were asked to measure their fluency, relevance to the conversation context, grounding on the reference knowledge, and whether the response was an unnatural copy of the knowledge. The human evaluation results in Table 3 further confirm the advantage of KCTS, which scored better than the baseline on all dimensions. In addition, compared to FUDGE, KCTS produced higher relevance and grounding while copying less knowledge, indicating that KCTS responses have higher perceived utility. The results in the non-copy group also indicate that KCTS outperforms the baseline even if knowledge copy responses are excluded.

[0077] Table 3: Human evaluation on a 3-point Likert scale on the WoW test set Fl., Rev., and Gr. represent fluency, relevance, and foundation, respectively. The intra-rater consistency of Krippendorff alpha was 0.57, 0.46, 0.77, and 0.31. Non-replication refers to the average score of instances where annotators agreed to generate knowledge that was not replicated.

[0078] 4.2 Summarizing the results

[0079] The same model as in 4.1 is used. Based on the results found in Table 4, it can be observed that ChatGPT again outperforms the other models in most dimensions except for the BLEU and Rouge metrics. On the other hand, the instruction-tuned models show a different trend from the KGD setting; the T0++ model outperforms the FT5 model, presumably because Flan fine-tuning introduces a large number of additional tasks compared to the T0 fine-tuning dataset hybrid, which leads to a drop in performance on some tasks.

[0080] Table 4: Results on CNN / DM test set Bootstrap decoding was performed using the FT5-XL model as the base model. Coh., Cons., and Relv. represent coherence, consistency, and relevance, respectively. Since the performance of LM is for reference only, the best scores for the last two groups are highlighted in bold, and the second best are highlighted in underline.

[0081] For efficiency, the smallest Flan-T5-XL model is used throughout the guided decoding experiments on the summarization task. KCTS again shows superior performance over the baselines, with token overlap and MFMA showing significant improvements. RIPA-guided decoding also outperforms all baselines on UniEval, with KWD showing slightly better performance than KCTS.

[0082] 50 samples were randomly selected for human evaluation, with three human raters identical to the KGD task as described above. The raters were asked to evaluate the summary on three dimensions: fluency, basicity, and completeness. Basicity is analogous to knowledge precision, while completeness is analogous to recall. It can be observed from Table 5 that the raters prefer KCD over the baseline on all dimensions (including basicity and completeness).

[0083] Table 5: Human evaluation on CNN / DM This followed a 3-point Likert scale with consistency alphas of 0.35, 0.44, and 0.19.

[0084] Example 5 - Instruction Template

[0085] The instructions for different models and tasks are listed in Table 6. For ChatGPT with KGD task, the chat completion API is used, where each dialogue turn is separated and formatted as user / assistant message, and the instructions are given at the end as system messages.

[0086] Table 6: Instruction templates for different models for different tasks

[0087] Example 6 - Application to LLM

[0088] As a preliminary study, knowledge-constrained decoding is applied to LLM, specifically GPT-3.5 (text-davinci-003). A limitation of the OpenAI API is that it does not return token probability distributions over the entire vocabulary; at most the top 5 log probabilities are returned. This significantly limits the search space for all WD methods, which may reduce the ability to direct generation towards the goal. Therefore, a new method called pre-WD is proposed, in which a proxy model is used to first propose the top k tokens, RIPA is applied to re-rank the tokens, and then included in the logit bias field in the API request, which is added to the logit of the LLM before sampling. This can be expressed as: Where Z i is the logit of token i, Z LLM is the logit of the basic LLM, and is the logit of the smaller proxy model. α is another hyperparameter that controls the strength of the logit bias. Since GPT2 and GPT3 series share their vocabulary, GPT2-XL is used to calculate

[0089] 100 instances are randomly sampled from the WoW test set for this experiment. The results in Table 7 show that applying post-bootstrapping does not improve much because the search space is finite: without sufficient width, the generation is constrained to what the base model thinks. On the other hand, although the overlap between the tokens proposed by the proxy model and the actual token distribution is unknown, the empirical results show that this method can successfully increase the bias towards tokens based on reference knowledge. Finally, using both pre-bootstrapping and periodic post-reweighting together can produce the most faithful generation.

[0090] Table 7: GPT3.5+RIPA(WD) on 100 random instances from the WoW test set (unseen topics)

[0091] Example 7 - Implementation Details

[0092] The response length is set to 64 tokens for summarization and to 32 tokens for KGD. For kernel sampling, the top-p terms are 0.95 with temperature = 1, which also applies to the OpenAI model. For all decoding methods studied, top-k filtering is applied with k = 50. In addition, the constraint factor α is set to 0.25 in NADO and the constant c is set to 0.25 in MCTS. puct = 3. According to the original implementation, a repetition of 1.2 was also applied to MCT.

[0093] The generated synthetic data has the following statistics: For WoW, 8,832 partially hallucinated instances were generated in a zero-shot manner using FT5-XL with temperature T=1.4 to promote hallucination. 10,000 knowledge shuffled instances were also sampled, along with 20,000 original instances, resulting in a balanced mixture of 20,000 positive and 18,800 negative examples. For CNN / DM, 12,811 partially hallucinated negative examples were generated using the same procedure. The final dataset also contains 13,180 positive examples. A random 9:1 split was then applied to obtain training and validation sets.

[0094] All experiments were performed using NVIDIA GPUs with CUDA 11.7 and CuDNN 7.5 or higher enabled. RTX A6000 48GB, RTX 3090Ti 24GB, or RTX 3090 24GB GPUs were used. All classifier training was performed for 2000 steps with an effective batch size of 64 for KGD and 32 for generalization. For efficiency, models were loaded with 8-bit quantization in both bits and bytes during both training and inference.

[0095] All implementations are based on the huggingface transformer and peft libraries. The evaluation library is used to benchmark the implementations. All pre-trained model weights are downloaded from the huggingface hub. Human evaluation is performed by three NLP graduate researchers with at least one year of experience in the field.

[0096] Example 8 - Generated Example

[0097] In the case of the study, each method includes some exemplary outputs for KGD in Table 8 and summarization in Table 9. Examples 1 and 2 of Table 8 demonstrate the advantages of our method. In Example 1 of Table 8, although FT5-XL failed to fully address the knowledge and FUDGE outputted "sweater" that was not mentioned, KCTS used the knowledge in a natural way to generate a response. ChatGPT also generated a well-grounded response. In Example 2 of Table 8, FT5-XL was not very informative, FUDGE made an unsupported statement about the popularity of golden retrievers, and KCTS generated the most knowledgeable response. On the other hand, Example 3 of Table 8 demonstrates the failure of all models. Although the knowledge provided was about whether "fish protein powder" was hygienic, ChatGPT mentioned the high protein content and the other models mentioned the taste.

[0098] Table 9 contains two successful examples of KCD. ChatGPT responses tend to be more detailed, which may contribute to higher completeness in human evaluation, but may not be concise enough to serve as a good summary. FT5-XL can generate seemingly irrelevant comments, and FUDGE may degrade in fluency during token reordering. KCTS generates the most concise and faithful summary, which captures the core of the article.

[0099] Table 8: Example generation from the WoW test set

[0100] Table 9: Example generation from the CNN / DM test set

[0101] Throughout this specification, unless the context requires otherwise, the word "comprise" or variations such as "comprises" or "comprising" should be understood to imply the inclusion of the stated integer or groups of integers, but not the exclusion of any other integer or groups of integers. It should also be noted that in this disclosure and particularly in the claims and / or paragraphs, terms such as "comprises", "comprised", "comprising", etc. may have the meanings ascribed to them in U.S. patent law; for example, they allow for elements not expressly stated, but exclude elements that are found in the prior art or that affect the basic or novel characteristics of the invention.

[0102] Furthermore, throughout the present specification and claims, unless the context requires otherwise, the word "include" or variations such as "includes" or "including", will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers.

[0103] References in this specification to "one embodiment", "an embodiment", "an exemplary embodiment", etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but each embodiment may not necessarily include the particular feature, structure, or characteristic. In addition, such phrases do not necessarily refer to the same embodiment. In addition, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it should be understood that it is within the knowledge of a person skilled in the art to affect this feature, structure, or characteristic in conjunction with other embodiments, whether or not explicitly described.

[0104] Other definitions of selected terms used herein can be found within the detailed description of the invention and are applicable to the entire text. Unless otherwise defined, all other technical terms used herein have the same meaning as those generally understood by those skilled in the art to which the invention belongs.

[0105] Those skilled in the art will appreciate that, in light of these teachings, alternative embodiments may be implemented without undue experimentation or without departing from the spirit or scope of the invention as set forth in the appended claims. The present invention shall be limited solely by the appended claims, which encompass all such embodiments and modifications when viewed in conjunction with the above description and drawings.

Claims

1. A computer-implemented model-agnostic method for natural language generation, characterized in that Comprising: Providing a plain text prompt containing information; Automatically generating tokens based on the plain text prompt containing information; Re - ordering the importance of each generated token and selecting, through a heuristic search process, the token with the highest knowledge - fundamentality in each scenario; Using a specially trained knowledge classifier to determine token - level hallucinations by identifying the inflection point of hallucinations and replacing hallucination tokens with additionally generated tokens; And Using the tokens with the least hallucinations and the highest knowledge - fundamentality to generate a plain text response.

2. The computer-implemented model-agnostic natural language generation method of claim 1, wherein: The heuristic search process is a tree search, which includes: Selecting to traverse the tree from the root node until reaching the child node of the leaf node; Expanding the leaf node until reaching the final state; Evaluating an approximate token - level knowledge - fundamentality score of the expansion result; Back - propagating the approximate token - level knowledge - fundamentality score recursively from the leaf node to the root node; and Selecting the child node with the highest approximate token - level knowledge - fundamentality score for further traversing the tree to reach subsequent leaf nodes.

3. The computer-implemented model-agnostic natural language generation method of claim 1, wherein: Training the knowledge classifier includes: Providing training instances (y, x, k) ∼ D; Randomly swapping knowledge k from a knowledge source with another knowledge k′ to form a negative example; Sampling for 1 < i < T, where T is the length of y; Truncate the response y to the i'th token and obtain y i ;as well as By including the knowledge text k in the instruction, from P "# (y|x,y i ,k) sampling to complete the sequence y i .

4. The computer - implemented model - agnostic natural language generation method according to claim 3, wherein: Marking the token by starting with marking the token as positive; Marking the first hallucination token as negative; And Predictively marking all tokens after the first hallucination token as negative.

Images