Explainer, output verification, and hallucination correction for output of large language models

Abstract

A computer-implemented, machine learning method for explaining and verifying correctness of an output of a large language model (LLM) includes splitting two sets of documents into a plurality of text spans by inputting the two sets of documents into a coarse chunking algorithm. Attribution links are generated using the text spans as input to a specialized LLM that has been trained to output a likelihood that one of the text spans in the first portion relates to one of the text spans in the second portion. Hallucination candidate text spans are identified in the text spans based on the attribution links. The hallucination candidate text spans and the attribution links are presented via a user interface. The method has applications including, but not limited to, use cases in computational biology and medical Al and healthcare for disease classification or supporting decision making in diagnosis and treatment of patients.

Description

Atorney Docket No. 820805 (Client Ref. NLE-1414-24-WO)EXPLAINER, OUTPUT VERIFICATION, AND HALLUCINATION CORRECTION FOR OUTPUT OF LARGE LANGUAGE MODELSCROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims benefit to European Patent Application No. EP 24198219.8, filed on September 3, 2024, which is hereby incorporated by reference herein.FIELD

[0002] The present disclosure relates to Artificial Intelligence (Al) and machine learning (ML), and in particular to a method, system, data structure, computer program product and computer-readable medium for explaining and verifying correctness of an output of a large language model (LLM).BACKGROUND

[0003] Large Language Models (LLMs) are conventionally known to hallucinate, that is, they generate information that is factually incorrect. Therefore, a human user may be required to verify the output of an LLM before using it, especially in high risk situations such as diagnosing diseases or identifying cybersecurity threats. Currently there exists no way to prevent such hallucinations being generated by LLMs. LLMs are more likely to hallucinate on lesser-known topics. This can create dangerous scenarios because a user might start trusting the LLM when validating known knowledge (e.g. about a well-known disease), and then not realize that the LLM very likely hallucinates on uncommon knowledge (e.g. a rare disease).

[0004] Retrieval augmented generation (RAG) has been introduced as a tool to help combat hallucinations. An RAG system retrieves relevant information from a database for a given user query and integrates this to the input prompt that is given to the LLM as additional context. The expectation is that the LLM will utilize this contextual information that was retrieved to generate its output. This can reduce hallucinations but it does not eliminate them. Conventional methods do not provide an efficient and accurate explainer for output generated by an LLM that can be used to verify whether the output is correct.SUMMARY

[0005] In an embodiment, the present disclosure provides a computer-implemented method for explaining and verifying correctness of an output of a large language model (LLM). Two sets of documents are split into a plurality of text spans by inputting the two sets of documents into a coarse chunking algorithm. A first portion of the text spans corresponds to the output of the LLM and a second portion of the text spans corresponds to a context. Attribution links are generated using the text spans as input to a specialized LLM that has been trained to output a likelihood that one of the text spans in the first portion relates to one of the text spans in theAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) second portion. Hallucination candidate text spans are identified in the text spans based on the attribution links. The hallucination candidate text spans and the attribution links are presented via a user interface. The method has applications including, but not limited to, use cases in computational biology and medical Al and healthcare for disease classification or supporting decision making in diagnosis and treatment of patients. As used herein, the term “document” includes strings or sequences of text or words, such as the output from a RAG system, as well as documents containing more than just text.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Embodiments of the present disclosure will be described in even greater detail below based on the exemplary figures. The present disclosure is not limited to the exemplary embodiments. All features described and / or illustrated herein can be used alone or combined in different combinations in embodiments of the present disclosure. The features and advantages of various embodiments of the present disclosure will become apparent by reading the following detailed description with reference to the attached drawings which illustrate the following:

[0007] FIG. 1 is an exemplary illustration of links from text of interest (e.g. output of an LLM) to a provided context (e.g. text provided to the LLM as input) for providing attribution information as an explainer to the output of the LLM according to embodiments of the present disclosure;

[0008] FIG. 2 illustrates an example of a standard LLM architecture;

[0009] FIG. 3 illustrates an exemplary architecture that adds an LLM Explainer to the LLM architecture according to embodiments of the present disclosure;

[0010] FIG. 4 illustrates an exemplary architecture of the LLM Explainer according to embodiments of the present disclosure;

[0011] FIG. 5 illustrates an exemplary architecture of the LLM Attributor according to embodiments of the present disclosure;

[0012] FIG. 6A illustrates an exemplary architecture of a fine-tuned attributor (fine-tuned attribution module) that includes data augmentation and fine-tuning a smaller model (3 and 4 from FIG. 4) according to embodiments of the present disclosure;

[0013] FIG. 6B illustrates an exemplary architecture of a filtering step for a Hallucination Module of the LLM Explainer according to embodiments of the present disclosure;

[0014] FIG. 7 illustrates an exemplary architecture of a Hallucination Module of the LLM Explainer and its constituent modules for detecting, labeling, and correcting hallucinations in the output of an LLM, a correction module is further comprised of verifier and rewriting modules that enable context-aware corrections to the erroneous outputs according to embodiments of the present disclosure;Atorney Docket No. 820805 (Client Ref. NLE-1414-24-WO)

[0015] FIG. 8 illustrates an exemplary architecture of a task improvement module of the LLM Explainer according to embodiments of the present disclosure; and

[0016] FIG. 9 is a block diagram of an exemplary processing system, which can be configured to perform any and all operations disclosed herein.DETAILED DESCRIPTION

[0017] This present disclosure provides users with an LLM explainer that enables fast verification of output generated by the LLM that is free of hallucinations and can present attribution links between the output generated by the LLM to the input provided to the LLM that was used to generate the output. Final control over the output is given to the user by providing them with a tool that explains the LLM output and that can be used to efficiently verify whether or not the LLM output is correct. The features of the present disclosure may be used as a tool that enables the safe and efficient usage of LLMs in high risk scenarios (medical diagnosis). In addition, the present disclosure improves the performance of downstream tasks at hand (e.g., diagnosing a patient), thereby making it useful tool for both human inspection and for improving the performance of the task at hand. Embodiments of the present disclosure provide solutions to the technical challenges in identifying and correcting hallucinations generated by LLMs while also providing an explanation to users to verify the correctness of an output generated by the LLM or identified hallucinations.

[0018] The features of the present disclosure can provide links (alignments) from a document of interest to a provided context, thus providing attribution information from the input text to the context text as depicted in FIG. 1. For example, the LLM Explainer of the present disclosure may take a piece of text as input (e.g., this could be the output of a generative Al model, such as an LLM) and then it links pieces of this text to a previously provided context information (e.g., LLM input 100). These links 102 suggest that the linked text from the context (e.g. text chunk 104 in the LLM input 100) is the source of information for the generated (linked) text (e.g. text chunk 106 of the LLM output 108). As described above, the features described herein can be used to identify and correct hallucinations generated by LLMs while also providing an explanation to users to verify the correctness of an output generated by the LLM or identified hallucinations. This can, for example, be used to generate more accurate and safe medical diagnosis of patients, determine treatment plans for patients, facilitate vaccine discovery, generate health plans associated with behavior changes for disease prevention, classify diseases, automated report generation, identify contradictions between sets of documents, generate medical reports, automated responses to queries such as in a ticketing system, and / or generate cybersecurity reports or identify cyber security threats. The present disclosure also saves time and resources in the design of vaccines, diagnosing diseases, andAtorney Docket No. 820805(Client Ref. NLE-1414-24-WO) developing treatments or treatment plans as the predictions or output generated by LLMs can be verified more efficiently and correct any identified hallucinations faster than conventional methods. The ability to quickly and accurately identify and correct hallucinations in output generated by an LLM represents an improvement in the technical field of LLMs, where acting quickly to design vaccines, diagnose, or provide particular treatments using the output of an LLM is time -critical. The more accurate output generated by LLMs of the present disclosure can also be used to provide valuable information to doctors or clinicians to determine possible treatment strategies, as well as for operating wet lab equipment or medical testing equipment, provide automated suggestions or interactive guidance for vaccines or to design vaccines in an automated or semi -automated manner, provide for automated diagnoses or treatments, or to design molecular diagnostic tools or molecular diagnostic tests. Machine learning systems, and in particular, LLMs generate and output incorrect information which is a technical problem in the field of machine learning and for computers that implement the LLMs. The functioning of computers that implement LLMs are improved by the embodiments of the present disclosure by enabling the identification of hallucinations generated by LLMs and correcting the hallucinations in output generated by the LLMs.

[0019] The machine-learning method according to an embodiment of the present disclosure includes the steps of: splitting two sets of documents into a plurality of text spans by inputting the two sets of documents into a coarse chunking algorithm, wherein a first portion of the text spans corresponds to the output of the LLM and a second portion of the text spans corresponds to a context provided to the LLM for a particular task used to generate the output of the LLM; generating attribution links using the text spans as input to a specialized LLM that has been trained to output a likelihood that one of the text spans in the first portion relates to one of the text spans in the second portion; identifying hallucination candidate text spans in the text spans based on the attribution links; and presenting the hallucination candidate text spans and the attribution links via a user interface. The above-described features are complex in nature and correctly identifying hallucinations in output generated by an LLM is computationally complex and cannot be performed in the human mind. Moreover, LLMs can be refined to be more accurate or generate less hallucinations for a particular task by contrasting an output with and an output without hallucinations with each other using the features described herein. The abovedescribed features identify hallucinations in a more accurate and efficient manner than conventional methods which rely on multiple re-query iterations thereby saving computer resources and time.

[0020] In a first aspect, the present disclosure provides a computer-implemented machinelearning method for explaining and verifying correctness of an output of a large language modelAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO)(LLM), the method including: splitting two sets of documents into a plurality of text spans by inputting the two sets of documents into a coarse chunking algorithm, wherein a first portion of the text spans corresponds to the output of the LLM and a second portion of the text spans corresponds to a context; generating attribution links using the text spans as input to a specialized LLM that has been trained to output a likelihood that one of the text spans in the first portion relates to one of the text spans in the second portion; identifying hallucination candidate text spans in the text spans based on the attribution links; and presenting the hallucination candidate text spans and the attribution links via a user interface.

[0021] In a second aspect, the present disclosure provides the method according to the first aspect wherein the specialized LLM is trained by: generating a silver labeled training data set by providing as input a golden data set that does not include hallucinations, an unlabeled training data set, and prompts to a plurality of LLMs, wherein the silver labeled training data set includes hallucination free data points and hallucinated data points; implementing a weight learning phase for learning a weight for each LLM of the plurality of LLMs based on the golden data set and the silver labeled training data set; implementing an online filtering phase that provides, as input, a data point from the unlabeled training data set to each of the plurality of LLMs that provide as output the weight of the LLM and a predicted label for the data point, wherein the online filtering phase further includes computing a soft label for the predicted label and comparing the soft label to a threshold to determine a final label; generating a trusted training data set based on the final labels of each data point from the unlabeled training data set; and training the specialized LLM based on a light-weight context attribution LLM and the trusted training data set, wherein the weight learning phase includes determining a first weight for each LLM for attribution links of hallucination free text spans and a second weight for each LLM for attribution links of text spans with hallucinations.

[0022] In a third aspect, the present disclosure provides the method according to the first aspect or the second aspect, wherein the context was provided to the LLM for a particular task used to generate the output of the LLM, and wherein the method further includes: determining that a stopping condition has not been met; generating an updated prompt that incorporates the hallucination candidate text spans and the attribution links; providing the updated prompt as input to the LLM for solving the particular task, wherein the determining, generating, and providing steps are executed iteratively until the stopping condition is met, wherein the stopping condition includes a maximum number of iteration loops or until no hallucination candidate text spans are identified in the text spans, wherein the LLM generates an updated output by inputting the updated prompt for solving the particular task; and updating the LLM by implementing a contrastive loss function based on the updated output representing a corrected version of theAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) output generated by the LLM and the output that represents a hallucinated version of the output generated by the LLM.

[0023] In a fourth aspect, the present disclosure provides the method according to any of the first to third aspects, wherein at least one of the two sets of documents is an output of a retrieval augmented generation (RAG) system.

[0024] In a fifth aspect, the present disclosure provides the method according to any of the first to fourth aspects, wherein identifying the hallucination candidate text spans from the text spans includes identifying entities, words, or concepts that are only identified in the first portion, identifying the text spans of the first portion that have no relevancy to the text spans of the second portion, implementing another LLM configured to detect hallucinations that are provided the first portion, the second portion, and the attribution links as input, or by comparing the text spans of the first portion to a set of text spans that correspond to known hallucinations that are stored in a database.

[0025] In a sixth aspect, the present disclosure provides the method according to any of the first to fifth aspects, wherein the method further includes, subsequent to generating the attribution links but prior to identifying the hallucination candidate text spans: splitting the first portion into a sub-set first portion and the second portion into a sub-set second portion by inputting the first portion, the second portion, and the attribution links into a sub-chunking algorithm; and generating sub-attribution links using the sub-set first portion and the sub-set second portion as input to the specialized LLM that has been trained to output a likelihood that one of the text spans in the sub-set first portion relates to one of the text spans in the sub-set second portion, wherein the sub-attribution links indicate which of the sub-set first portion are linked to the sub-set second portion.

[0026] In a seventh aspect, the present disclosure provides the method according to any of the first to sixth aspects, wherein the method further includes assigning an intrinsic hallucination label or an extrinsic hallucination label to the first portion or the second portion based on the attribution links, the extrinsic hallucination label and the intrinsic hallucination label for the first portion of the second portion also associated with a corresponding confidence score representing a confidence of the first portion or the second portion as being an intrinsic hallucination or an extrinsic hallucination.

[0027] In an eighth aspect, the present disclosure provides the method according to any of the first to seventh aspects, wherein the method further includes: verifying extrinsic hallucination labeled text spans from the first portion and the second portion by: generating an information-seeking query based on the extrinsic hallucination labeled text spans of the first portion or the second portion; obtaining a set of documents from a database based on theAtorney Docket No. 820805(Client Ref. NLE-1414-24-WO) information-seeking query; reranking the set of documents based on the attribution links to serve as relevance feedback; and generating an editing context for removing hallucinations of the extrinsic hallucination labeled text spans, the editing context comprising portions of the reranked set of documents that are attributed to the two sets of documents based on the attribution links, the extrinsic hallucination labeled text spans, and an instruction to the LLM for text reformulation based on the attribution links and the extrinsic hallucination labeled text spans.

[0028] In a ninth aspect, the present disclosure provides the method according to any of the first to eighth aspects, wherein the method further includes: rewriting either the first portion or the second portion to remove the extrinsic hallucination labeled text spans based on the editing context; rewriting either the first portion or the second portion to remove intrinsic hallucinations based on intrinsic hallucination labeled text spans and the attribution links; updating the attribution links based on the rewritten first portion or second portion; and identifying updated hallucination candidate text spans from the plurality of text spans based on the rewritten first portion or second portion and the updated attribution links.

[0029] In a tenth aspect, the present disclosure provides the method according to any of the first to ninth aspects, wherein one of the two sets of documents includes text output generated by the LLM in response to being provided a set of requirements as input to the LLM, wherein another of the two sets of documents includes a report, and wherein the hallucination candidate text spans and the attribution links that are presented via the user interface represent potential hallucinations and the attribution links between the text output generated by the LLM and the set of requirements provided as input to the LLM.

[0030] In an eleventh aspect, the present disclosure provides the method according to any of the first to tenth aspects, wherein one of the two sets of documents includes text output generated by the LLM in response to being provided cybersecurity reports as input to the LLM, wherein another of the two sets of documents includes the cybersecurity reports, and wherein the hallucination candidate text spans and the attribution links that are presented via the user interface represent potential hallucinations and the attribution links between the text output generated by the LLM and the cybersecurity reports provided as the input to the LLM.

[0031] In a twelfth aspect, the present disclosure provides the method according to any of the first to eleventh aspects, wherein one of the two sets of documents includes medical reports of a patient generated by the LLM in response to being provided historic medical information for the patient, wherein another of the two sets of documents includes the historic medical information for the patient and medical resources, and wherein the hallucination candidate text spans and the attribution links that are presented via the user interface represent potentialAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) hallucinations and the attribution links between the medical reports of the patient generated by the LLM and the historic medical information for the patient provided as the input to the LLM.

[0032] In a thirteenth aspect, the present disclosure provides a machine learning model stored on a tangible, non-transitory computer readable medium for explaining and verifying correctness of an output of a large language model (LLM), the machine learning model including: a chunker that generates text spans by implementing a coarse chunking algorithm on two sets of input documents, wherein a first portion of the text spans corresponds to the output of the LLM and a second portion of the text spans corresponds to a context; an attribution layer that takes the text spans as input and is trained to output a likelihood that one of the text spans in the first portion relates to one of the text spans in the second portion in order to generate attribution links; and a hallucination detection layer trained to identify hallucination candidate text spans in the text spans based on the attribution links.

[0033] In a fourteenth aspect, the present disclosure provides the machine learning model according to the thirteenth aspect, wherein the attribution layer includes a granularity adjustment layer and a large language model (LLM) attributor that includes an embedding layer for embedding the text spans into a latent space, and wherein the hallucination detection layer includes a hallucination detection layer that identifies the hallucination candidate text spans, a hallucination classification layer for classifying the hallucination candidate text spans as intrinsic hallucinations or extrinsic hallucinations, and a hallucination correction layer for rewriting the hallucination candidate text spans to generate a corrected output.

[0034] In a fifteenth aspect, the present disclosure provides a computer system for using a machine learning method for explaining and verifying correctness of an output of a large language model (LLM), the computer system including one or more hardware processors which, alone or in combination, are configured to provide for execution of steps including: splitting two sets of documents into a plurality of text spans by inputting the two sets of documents into a coarse chunking algorithm, wherein a first portion of the text spans corresponds to the output of the LLM and a second portion of the text spans corresponds to a context; generating attribution links using the text spans as input to a specialized LLM that has been trained to output a likelihood that one of the text spans in the first portion relates to one of the text spans in the second portion; identifying hallucination candidate text spans in the text spans based on the attribution links; and presenting the hallucination candidate text spans and the attribution links via a user interface.

[0035] Without loss of generality, the present disclosure uses the term “LLM input” for the source text and the term “LLM output” for the text to be verified. Note that the input chunks of the LLM input can be a prompt and / or context given to an LLM, and / or it could also be anyAtorney Docket No. 820805(Client Ref. NLE-1414-24-WO) other reference text that is relevant. Likewise, output chunks of the LLM output can be the output of an LLM, and / or it could also be any other text to be verified. These pieces of text are then provided to a chunker (2 of FIG. 5), which splits the text into adequate text spans (e.g., sentences or sub-sentences, which are also referred to herein as “chunks”).

[0036] The overall architecture of current use of LLMs is shown on FIG. 2. For example, providing a context 200 and a prompt 202 as input 204, an LLM 206 (e.g. model 208) generates a response 210 as output 212.

[0037] In an embodiment, the present disclosure adds an explainer (LLM Explainer) on top of any LLM, which will generate links between the response text (output text) and the context text (input text), thus exposing the source of information for the generated text to the user. FIG. 3 depicts this architecture. For example, providing a context 300 and a prompt 302 as input 304, an LLM 306 (e.g. model 308) generates a response 310 as output 312. As described herein, the response 310 and the context 300 can be provided to the LLM Explainer 314 for generating attribution links 316 which can be used to identify and correct hallucinations in the response 310. Although some examples described herein include a context 300 that is provided to the LLM for generating the response 310 that corresponds to the output associated with a particular task, embodiments described herein are not limited to such scenarios. For example, the context 300 can include any type of context, not just context provided to the LLM 306 to generate the response 310.

[0038] To inform users about potentially hallucinated content generated by an LLM, as well as to improve the task at hand executed by the LLM (e.g., patient diagnosis), the present disclosure describes three major modules of the LLM Explainer: an attribution module 400 that provides attribution links between the generated text (e.g., by LLM) and the context information (e.g., provided by user or RAG-based systems); a hallucination module 402 that provides information about potential hallucinations in the generated text, as well as a functionality to automatically correct them; and a task module 406 for taking the attribution links and the hallucinations detected as input information and then uses them to improve the task itself (e.g., automatic diagnosis of a patient). An exemplary architecture including the modules 400-406 is depicted in FIG. 4. As is depicted in FIG. 4, the attribution module (attributor) 400 includes a granularity adjustment method or module 408 (1 of FIG. 4), and an LLM attributor 410 (2 of FIG. 4). In embodiments, the hallucination module 402 includes modules or processes for hallucination detection 412 (5 of FIG. 4), hallucination classification 414 (6 of FIG. 4), and hallucination correction 416 (7 of FIG. 4). The task module 406 includes task improvement module 418 (8 of FIG. 4). In embodiments, the exemplary architecture depicted in FIG. 4 includes a fine-tuned attribution model module or process 420. The fine-tuned attribution modelAtorney Docket No. 820805(Client Ref. NLE-1414-24-WO) module or process 420 includes a data augmentation module 422 (3 of FIG. 4) and a step or feature fortraining a smaller model 424 (4 of FIG. 4). The details of each of the above described modules or processes are described in more detail below with reference to FIGs. 5-9. The output of the attribution module 400 includes the attribution links which are provided 426 to the task module 406.

[0039] The following describes the granularity adjustment method or module 408 (1 of FIG. 4), and an LLM attributor 410 (2 of FIG. 4) of the attribution module 400. Some of the features described for these modules or methods are also depicted in more detail in FIG. 5.

[0040] As depicted in FIG. 5, and in accordance with embodiments of the present disclosure, a specialized LLM is used to perform attribution on two pieces of text, where the first piece is assumed to be the context provided to an LLM for a particular task (LLM input 00), while the other piece is assumed to be the output of the LLM given the context of the query (LLM output 502) . Given these two sets of text 500 and 502, the LLM attributor 410 of FIG. 4 chunks both and returns a set of links between the chunks from the LLM input 500 and the LLM output 502, whereas each link from the LLM output 502 to the LLM input 500 attributes the information in one LLM output chunk 504 to the list of LLM input chunks 506. In particular, the links to one or multiple LLM input chunks 506 explain and give evidence that either complements or contradicts the validity of the LLM output chunk 504.

[0041] The LLM Attributor 410 of FIG. 4 generates the attributions as follows. Note that the numbers in parenthesis in the following description refer to the circled numbers in FIG. 5. The LLM Attributor 410 of FIG. 4 takes the LLM input 500 and LLM output 502 as inputs (1). These pieces of text are then fed to a chunker 508 (2), which splits the text into adequate text spans (e.g., documents, paragraphs, sentences or sub-sentences). In embodiments, the chunker 508 (2) comes from the granularity adjustment 408 ((1) from FIG. 4). Then, the resulting chunks 504 and 506 (3) are fed to an LLM attributor 510 (4) which yields attribution links 512 (5) that indicate which output chunks 504 should be linked to which input chunks 506.

[0042] In embodiments, the LLM attributor 510 (4) can be implemented in the following ways:- Lexical: This aligner splits the text of the LLM output 502 and LLM input 500 into lexical units (i.e., words). Then, for each chunk from the LLM input 500 (e.g., a sentence) the aligner verifies how many words are overlapping with the LLM output chunk 504. Then, a similarity is computed between the text chunks 504 and 506 based on the lexical overlap (e.g., similarity metrics like the n-gram overlap).- Embedding model: this process includes leveraging the embeddings of the chunks 504 and 506. The embeddings are fed or provided as input to an embedding model. TheAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) embedding model can be any suitable sentence or text embedding model (e.g., EmbEddings from bidirEctional Encoder rEpresentations (E5)). The semantic similarity scores are obtained between the LLM input chunks 506 and LLM outputs chunks 504 by using a similarity or distance (inverse similarity) measure of the embeddings of the chunks 504 and 506; e.g., cosine similarity, 12-distance, etc. A threshold matcher is implemented that yields the complete alignments between the LLM input chunks 506 and the LLM output chunks 504. The threshold matcher can return all chunk pairs with a minimum similarity score or return the top-k most similar matches for an output chunk 504 or a combination of both.- Pre-rank / rerank: Similar to the embedding model, after the first embedding model, a second embedding model is employed to rerank. Given the preliminary alignments between the chunks 504 and506 (i.e., the pre-ranking) the top-k elements of the (preranked) similarities matrix are identified. These top-k elements are then provided to the re-ranker model. In embodiments, the second model has higher precision, which might come at a slower inference speed, which is why the top-k elements are preselected with an initial (faster) model(s).- Entity aligner: The entity aligner works similar to the lexical aligner described above. The difference is that, instead of aligning based on every lexical units, the entity aligner targets the alignment based on domain-specific recognized entities in the text (i.e., named entities). In particular, the chunks 504 and 506 are first run through a Named Entity Recognition (NER) model, which provides a list of entities. Then, a similarity between the text chunks is computed based on the lexical overlap only of the lexical units that belong to a recognized named entity.- LLM aligner model: Includes using a large generative model, such as Mistral, to directly generate output links.- Fine-tuned LLM aligner models: A fine-tuned LLM that works specifically well for the attribution task - described in more detail below with reference to the Fine-tuned Attribution Module 420.

[0043] Given the attributions 512, a pipeline depicted in FIG. 5 allows for applying another chunking step using a sub-chunker 514 (6), which results in a set of output sub-chunks and a set of input sub-chunks with corresponding sub-chunked attributions 516 (7). In embodiments, the sub-chunker 514 (6) can be implemented in one of the following ways, including not chunking the output chunks 504 and input chunks 506 any further:Atorney Docket No. 820805 (Client Ref. NLE-1414-24-WO)- In embodiments, the sub-chunker 514 takes from a database which parts of a chunk should become sub-chunks (e.g. from a triple extraction algorithm that extracted triples from the original chunk and this results was saved in a database).- In some embodiments, an Al trained model that is configured or trained to know how human users want the sub-chunks to look like.- In embodiments, embedding vectors are obtained from an LLM to perform pooling (i.e., selection) of only the token embeddings relevant to the chunks of interest to the current iteration or relevant to the input chunks 506 and output chunks 504.

[0044] The chunker 508 may not chunk the output chunks 504 and input chunks 506 any further in scenarios where the chunks 504 and 506 are already small enough, which can be decided in a per use case basis or depending on the original chunker 508 selected. In embodiments, the attribution LLM 510 (4) should embed a coherent information unit in a vector space so that the full benefit of encoding each word correctly given the relevant context can be taken advantage of when generating attributions. Additionally, human users might require finegrained links, so narrowing down to the desired sub-chunks in the vector space enables the system to generate or consider sub-sentence embeddings or specific phrase embeddings. These can be subsequently used to generate links that are more fine-grained compared to the original ones (e.g. absent a sub-chunk process). This also provides the advantage that the attributions generated by the system leverage the complete context. This process is unique in achieving attribution between texts (e.g. obtaining the fine-grained links desired), as in any other use case chunking would occur beforehand.

[0045] The output of both the attributions 512 (5) and the sub-chunked attributions 516 (6) have the following format: For each output chunk 504, a series of input chunks 506 is listed or provided that the output chunk 504 should have a link with. Having such a link indicates that a given input chunk 506 might be relevant to produce the given output chunk 504 and is depicted at 518 (8) of FIG. 5.

[0046] The following describes the Fine-tuned Attribution Model 420 that includes two parts: 422 (1) Data augmentation; and 424 (2) Train smaller model of FIG. 4. The following section describes more details of these modules or processes in further detail with reference to FIG. 6A. Note that the numbers in parenthesis in the following description refer to the circled numbers in FIG. 6A.

[0047] Assume data points d where d is a pair of chunks and the label serves as is the attribution links between the pair chunks. The exemplary architecture depicted in FIG. 6A includes starting with the context attribution golden examples 600 (1) where the chunk pairs of these examples do not contain any hallucinations. These are examples of context attribution thatAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) are trusted the most, e.g. because they were previously annotated by human experts, where theLLM input chunks are linked to the appropriate LLM output chunks. Then, the annotated examples are used in order to form a (few-shot in-context learning) prompt 602 (2), such that the prompt 602 (2) instructs an LLM to label previously unlabeled data points 604 (3). The exemplary architecture of FIG. 6A includes using one or more LLMs 606 in order to generate silver labeled training data (4). As used herein, silver labeled training data (4) may refer to cleaned data generated by the LLMs 606 and includes pairs of text chunks with attributions between them. One or more LLMs 606 may be used as they are likely to produce better output for use as training data, and several LLMs 606 are used as if several LLMs agree how to label a training data point - then the likelihood is higher that the labeling was correct therefore leading to a better quality dataset.

[0048] In a second step, data points d are also added, where the output chunk of a pair specifically contains hallucinations (3). This can be generated by instructing an LLM 606 to add hallucinations. The one or more LLMs 606 can also be used in order to generate silver labeled training data (4).

[0049] The following filtering step is implemented for both hallucination free and hallucinated data points included in the silver labeled training data (4).

[0050] Since multiple LLMs 606 may be used to perform the context attribution, various filtering and aggregation methods may be applied to fuse the predictions of these diverse LLM models 606 at filtering 608 (5). The filtering methods 608 can take various forms, for example, but not limited to: (a) majority voting of the LLMs 606; (b) weighing the outputs of the LLMs 606 based on the corresponding LLM confidence; and / or (c) weighing the outputs of the individual LLMs 606 based on their performance (accuracy, precision, etc.) on the labeled golden data samples 600 (1). The filtering step (5) is described in more detail below with reference to FIG. 6B. Upon the filtering step (5), the entire pipeline of FIG. 6A yields a high- quality silver training dataset 610 (6), which can be used to train a context attribution method. Subsequently, in step (7), a light-weight context attribution LLM 612 is chosen or selected, which is fine-tuned 614 (7) on the silver training dataset 610. The light-weight context attribution LLM 612 does not have as many parameters or has less parameters than the LLMs 606. The resulting smaller LLM (e.g. smaller model 616) obtained in step (8), post-fine-tuning on the synthetically labeled dataset (e.g. high-quality silver training dataset 610 (6)) is significantly more accurate than the individual LLMs 606 constituent in step (4). Once the smaller model 616 is trained, it can be provided to the LLM Attributor 410 of FIG. 4, represented as the final attribution model ((4) from FIG. 4). The overall architecture of the LLM attributor is illustrated on FIG. 5.Atorney Docket No. 820805(Client Ref. NLE-1414-24-WO)

[0051] The following describes the filtering 608 (5) of FIG. 6A in more detail. The following section describes more details of these modules or processes in further detail with reference to FIG. 6B. Note that the numbers in parenthesis in the following description refer to the circled numbers in FIG. 6B. In embodiments. The filtering depicted in FIG. 6B includes a method that contains two major phases: a (1) weight learning phase and an (2) online filtering phase as depicted in FIG. 6B.

[0052] In the weight learning phase, a weight wt is learned for every LLM i (such as LLMs 606 of FIG. 6A). This weight determines how much importance each LLM 606 should carry for classifying a data point for attribution. In embodiments, such a weight is learned for each LLM 606 twice, one weight for how well a LLM 606 works to create attribution links of hallucination free chunk pairs, and another weight for chunk pairs where the output side contains a hallucination. This is important so that the later attribution model also works well when a hallucination is present - which is exactly what the present disclosure is directed to - discovering and exposing hallucinations.

[0053] To learn the weights for the hallucination free instance, the architecture of FIG. 6B first starts with a database of gold data (G) 618. For each data point in G 618, the system first checks what the attribution golden label lgis and the predicted label by the LLM: lp- depicted at 620 (label correctness (1)). Then, the system checks if lgand lpare equivalent - also depicted at 620 (label correctness (1)). Note that the present disclosure is flexible as to what “equivalence” might mean. For example, this could also be viewed as a sequence labeling problem, in which case “equivalence” might have a strict meaning (e.g., the chunks must exactly match the alignments) or, in other scenarios, “equivalence’ could have a more relaxed meaning (e.g., significant portion of the chunks are overlapping). If the golden annotations match the predicted labels, then a label correctness score of 1 is given, otherwise the data point is given a label correctness score of 0. Next, a scoring module 622 takes each data point in the database of gold data (G) 618 as input. Then, each golden data point is passed to an LLM 624 for attribution. The LLM 624 may be the same LLM 612 in that it is a light-weight LLM with less parameters than LLMs 606 so that it can generate the attributions in an efficient time-period (e.g. a few seconds). The LLM 624 does not have access to the actual label. Then, (1) the golden label is compared with the predicted label against an evaluation framework 626, which, when the data points from G 618 are aggregated, gives a final evaluation score 628 for the LLM 606 (e,); (2) a confidence score 630 (ci) is obtained about the prediction from the LLM 606 (e.g., this can be direct feedback from the LLM 606, or a confidence score 630 based on the logits of the final layer of the LLM 606 if available) from a confidence framework 632. Then, for each LLM 606 i, the aggregated information about the evaluation score e 628 and confidence score c 630 are used,Atorney Docket No. 820805(Client Ref. NLE-1414-24-WO) together with the label correctness (1) to compute the weighted score per LLM 606 (3). Note that the present disclosure is not limited to deriving the LLM 606 weight from these two scores alone; and instead the embodiments described herein can use other scores derived from the context or the LLM 606 itself to derive the weight.

[0054] The above described weight learning process is repeated for the hallucination chunk pairs. For this an LLM, such as LLM 606, corrupts the gold data 600 by adding hallucination to the output side.

[0055] Once the weighted score for each LLM 606 is computed, the system implements a phase of online filtering (Phase 2 of FIG. 6B). The online filtering phase starts by assuming a data point d that has no label, where is a pair of chunks and the label is the link between the chunks. Then, this data point is passed to several LLMs 634 as input. As output, the LLMs 634 provide two coefficients: wt (the LLM 634 weight) and if (the predicted label by the LLM 634) (depicted at (4) of FIG. 6B). A soft label 5 is then computed, which will give a probabilistic label about the attribution alignments (5). Then, through a previously defined threshold (6), it is decided whether the label is 0 or 1 (i.e., aligned or not aligned). The previously defined threshold may be selected from a previously tested set of thresholds that are used with a small dataset to determine an optimal threshold. The threshold can either be manually defined by an expert, or learned offline by a machine learning model. Based on this, the final label (7) is derived. As depicted in FIG. 6B it is then decided whether to filter (8) out this data point or not based on the information from the LLMs 634, threshold (6), LLM weights, etc. For example, if there is too much variance between the soft scores of each LLM 634, it could be an indication that the label is not easy to derive, which means that it should be filtered out.

[0056] Similar to the weight learning phase, the online filtering phase is repeated for both hallucination free and hallucination containing chunk pairs. Crucially the decision thresholds for the data point with and without hallucination can be set differently in order to make sure to keep tricky (true but difficult to identify) hallucination cases where one would still want attribution to work. For the hallucination cases the system may specifically keep the data points where only one LLM (e.g. particularly good / high weight for getting attributions correct with hallucinations) generated a high confidence attribution score.

[0057] The final output is a trusted dataset of data points d labelled with attribution links and where d can be both hallucination free or containing a hallucination.

[0058] The following describes the Hallucination Model 402 of FIG. 4 in more detail with reference to FIG. 7. The hallucination module 700 of FIG. 7 is responsible for (1) automatically detecting candidate chunks in the output that are possible hallucinations via a detection module 702, (2) automatically labeling the type of hallucination for a given candidate via a labelingAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) module 704, and (3) correcting the hallucinated candidates by rewriting them in a context-aware manner via the correction module 706 which includes a verification module 708 and rewriting module 710. The corrected output will be refined by realigning only the modified parts from the correction module 706. This improves efficiency and consolidates the entire pipeline of the present disclosure as a self-contained evaluation of output quality of LLMs.

[0059] In embodiments, the hallucination module 700 takes or obtains as input the attributions, represented in FIG. 7 as [SI] returned by the attribution module 400 of FIG. 4.

[0060] Next, given the input and output chunks, the detection module 702 [S2] enumerates chunks that are only in the output but not in the input as “candidates” for hallucinations. As used herein and as depicted in FIG. 7, “candidates” are chunks or text spans that likely contain a hallucination. Such a detection module 702 can for example be implemented by:- Using existing state-of-the art models for identifying entities, words, or concepts that can only be found in the output.- Identifying output chunks where S 1 (the attributions) did not find or include a relevant input chunk.- An LLM that specializes in detecting hallucinations. An example of this type of LLM is described by Jiaxin Zhang, Zhuohang Li, Kamalika Das, Bradley Malin, Sricharan Kumar, “SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic -aware Cross-check Consistency,” Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15445-15458, December 6-10, 2023, which is hereby incorporated by reference herein.- A set of chunks stored as hallucinations in a database (as for example labelled by a human who used SI (attributions) to investigate).- A combination of any of the above.

[0061] After obtaining the candidate chunks from the detection module 702, the labeling module 704 [S3] assigns one of the labels (with a certain confidence) to each chunk for its hallucination type: intrinsic, extrinsic. As used herein, an intrinsic hallucination is an output chunk that wrongly modifies information from the set of input documents and can be attributed to the input chunks. As used herein, an extrinsic hallucination is an output chunk that cannot be fully attributed to any of the input chunks and demands additional efforts from users to verify its correctness. The labeling module 704 could for example be an existing LLM or an LLM trained on data for hallucination types and leverages the attributions [SI] as additional context.

[0062] The correction module 706 [S4] accepts as input the labeled candidate chunks from the labeling module 704 and the attributions [SI], and rewrites the output chunks to remove hallucinated content. As described above, the correction module 706 includes two submodulesAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) for verification (verification module 708 [S4.1]) and rewriting (rewriting module 710 [S4.2]).The verifier module 708 is especially relevant for extrinsic hallucinations which require additional information beyond the set of input documents.

[0063] The verification of a hallucinated chunk that cannot be attributed to any of the input chunks (e.g., an extrinsic hallucination) requires the look up for relevant information pertaining to the hallucination. In embodiments, the verification module 708 does this by performing the constituent subtasks of: (1) formulating an information-seeking query 712 based on the labeled hallucination chunk, (2) finding evidence 714 (e.g. a set of relevant documents) from the web or a database, (3) re-ranking 716 the retrieved evidence by computing the attributions with the source (SI) and use this as pseudo relevance feedback, and (4) formulating an “editing context” 718 with the top ranked evidence for the rewriting module 710. An example of editing context 718 would be a set of (a) relevant parts of the evidence 714 that are (partially) attributed to the source, (b) the extrinsic hallucination chunk, and (c) an instruction to the LLM for text reformulation composed from the two former elements.

[0064] In embodiments, the rewriting module 710 accepts as input the labeled hallucination chunks from the labeling module 704 and the editing context 718 from the verification module 708, and rewrites the output chunks to remove the hallucinations. For intrinsic hallucinations, the rewriting module 710 may skip the verification by the verification module 708 and directly use the attributions [SI] from the attribution module 400 to rewrite new output chunks with no hallucinations.

[0065] After rewriting the hallucinated output chunks via the rewriting module 710, the refinement module (represented by the pipeline of FIG. 5) re-computes attributions with the set of input documents for only the modified output to ensure that the hallucinations have been removed. For example, the modified pairs are sent again through the pipeline of FIG. 5 and the re-computed attributions are from 512 of FIG. 5. This step involves redetecting hallucinated chunks via the detection module 702.

[0066] The following describes the task module 406 (task improvement (8)) from FIG. 4 in more detail with reference to FIG. 8. Note that the numbers in parenthesis in the following description refer to the circled numbers in 8. In embodiments, the goal of the task module 406 is to improve the performance of an LLM for a task at hand (e.g., patient diagnosis) by reducing the chance of hallucinations. First, as depicted in FIG. 8, a regular prompt 800 is provided to solve a task (1). This prompt 800 is fed to the LLM 802 (2), which then solves the task 804 (3). The output 806 (4) is then fed to the previous modules for hallucinations and attributions (5) such as by identifying hallucinations 808, correcting hallucinations 810, determining attribution alignments 812, and determining hallucination types 814 (e.g. intrinsic or extrinsic). ThisAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) provides information about hallucinations and attributions. Then, the process determines if the stopping conditions 816 are met (6). In embodiments, the stopping conditions include determining that running a rewritten section does not lead to the detection of any further hallucinations or determining that a maximum number of iteration loops has been reached.

[0067] While the stopping condition 816 is not met, the loop is rerun by giving the prompt 818 to the LLM again, where the prompt 818 incorporates the information about the hallucinations and the alignments (7). Finally, when the stopping criterion 816 is met, the final output 820 (8) is presented, such as via a user interface to a user.

[0068] In embodiments, the loop depicted in FIG. 8 may be run or implemented prior to deployment in order to fine-tune a LLM to reduce the chance of hallucinations. For example, in a scenario where there exists an instance with a hallucination and a corrected version, an LLM can be updated using this as a positive and negative example in a contrastive loss function, e.g. a ramp loss, rather than standard maximum likelihood estimation.

[0069] One embodiment of the present disclosure can be practically applied in the field of automated report generation. Here, a use case is that a person who works in a knowledge- intensive field can increase their efficiency by letting LLMs draft reports / summaries or answer questions. Absent the features of the present disclosure, a person would still have to read all the source material and the LLM generated output in order to verify the correctness of the output generated by the LLM. Even still, the person might get tired or forget elements and fail to catch mistakes in the output generated by the LLM. With the LLM Explainer of the present disclosure, the person can verify the output more efficiently, leading to a great increase in efficiency and ensuring that it is safe to deploy a LLM to complete certain tasks associated with drafting reports / summaries, or answering questions. The data source and input for this use case includes an input text given to an LLM. The output can be the text produced by an LLM. Further, the present disclosure generates links between chunks in the output and chunks in the input which can be used to warn of hallucinated information in the output. In this way, the present disclosure enables the generation of attribution links which can warn of hallucinated output generated by an LLM thereby ensuring safe implementation of LLMs for generating summaries, reports, or answering questions in an efficient and accurate manner. A user interface can provide a user with relevant knowledge, such as highlighting where potential hallucinations are and generated links between the input and output which explain why the features of the present disclosure identified certain portions as including potential hallucinations. This directly allows the user to verify that the output is correct or modify it if needed.

[0070] Another embodiment of the present disclosure can be practically applied in the field of document generation, verification, and / or identifying contradictions between documents.Atorney Docket No. 820805(Client Ref. NLE-1414-24-WO)Here, a use case is that a person has two sets of documents and would like to find out if one document contradicts the other document. Absent the features of the present disclosure, a person would still have to read both sets of documents and the LLM generated output in order to verify any identified contradictions in the output generated by the LLM. Even still, the person might get tired or forget elements and fail to catch mistakes, such as incorrectly identified contradictions, in the output generated by the LLM. With the LLM Explainer of the present disclosure, the person can easily verify the output generated by the LLM, leading to an increase in efficiency and ensuring that it is safe to deploy a LLM to complete certain tasks associated with identifying contradictions between documents. The data source and input for this use case includes the two sets of documents of interest to the person. The output can be text produced by the LLM which can be presented via a user interface that provides the person with relevant knowledge such as highlighting where potential hallucinations are and the generated attribution links which explain why the LLM Explainer has identified certain portions of the LLM output as being hallucinations. This directly allows the user to verify that the output is correct or modify it if needed.

[0071] One embodiment of the present disclosure can be practically applied in the field of medical report generation. Here, a use case is that a medical doctor needs to read previous reports about a patient before writing their current report. This process can typically take a lot of time and effort that would be better spent with interacting with the patient or another patient. A doctor cannot directly ask a LLM to take over the report writing because it could hallucinate which would be harmful if it then leads to the wrong treatment for the patient. This leads to the scenario where even if an LLM writes the text of the medical report, the doctor would still have to read all previous reports and verify the correctness of the text generated by the LLM. With the LLM Explainer of the present disclosure, the doctor can quickly and efficiently verify the correctness of the report, therefore saving time and ensuring that it is safe to deploy a LLM to complete certain tasks such as generating medical reports. The data source and input for this use case includes the input text given to an LLM and the output text produced by a LLM. The features of the present disclosure can use the input to generate attribution links between chunks in the output and chunks in the input which can be used to identify potential hallucination candidates and warn of hallucinated information in the medical report generated by the LLM. The output can be the medical report produced by the LLM which can be presented via a user interface that also provides the medical doctor with relevant knowledge such as highlighting where potential hallucinations are, and the generated attribution links which explain why the LLM Explainer has identified certain portions of the LLM output as being hallucinations. This directly allows the medical doctor to verify that the output is correct or modify it if needed. InAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) embodiments, an interactive and iterative process may be implemented by embodiments of the present disclosure by guiding users, such as the medical doctor, in verifying that the output is correct or modifying the output of the LLM by receiving input, prompts, or information from the medical doctor to correct any identified hallucinations generated by the LLM.

[0072] Another embodiment of the present disclosure can be practically applied in the field of automated responses to queries such as in a ticketing system. For example, many companies receive queries from users, e.g. via a ticketing system. Typically, a large amount of resources are required to answer such queries. Here, a use case is that the features of the present disclosure can support the support people answering the queries. For example, an LLM produces a draft answer, and the LLM Explainer highlights why the different sentences in the draft answer were written, allowing the support person to quickly verify or adapt the response. This can greatly increase the support person’s work efficiency and generate more accurate answers. The data source and input for this use case includes the input text given to an LLM and the output text produced by a LLM. The features of the present disclosure can use the input to generate attribution links between chunks in the output and chunks in the input which can be used to identify potential hallucination candidates and warn of hallucinated information in the draft answers generated by the LLM. The output can be the draft answer produced by the LLM which can be presented via a user interface that also provides the user with relevant knowledge such as highlighting where potential hallucinations are, and the generated attribution links which explain why the LLM Explainer has identified certain portions of the LLM output as being hallucinations. This directly allows the users to verify that the output is correct or modify it if needed. In embodiments, the rewriting module (e.g. 710 of FIG. 7) can ensure that the final output of the LLM that is given to solve the task at hand, e.g., generating draft answers to queries, has fixed any potential hallucination issues as much as possible.

[0073] One embodiment of the present disclosure can be practically applied in the field of cyberthreat security and intelligence. Here, a use case is that, a Cybersecurity operations may require an analyst to parse a large amount of information. An LLM can support the analyst by providing summaries of reports and answering questions about the information included in the reports. However, the LLM might produce hallucinated output. If the analyst has to verify a summary generated by the LLM they will have to read and analyze a large amount of information from original documents thereby spending their time in an inefficient manner. In this scenario, the LLM Explainer can speed up an analyst’s work and ensure that it is safe to use LLM output by generating links from the LLM output to the input. The data source and input for this cause can include the original documents that were summaries or it could be snippets retrieved by a RAG, such as input text given to an LLM and the output text produced by theAtorney Docket No. 820805(Client Ref. NLE-1414-24-WO)LLM. The features of the present disclosure can use the input to generate attribution links between chunks in the output and chunks in the input which can be used to identify potential hallucination candidates and warn of hallucinated information in the summaries of the reports generated by the LLM. The output can be the summaries of the reports produced by the LLM which can be presented via a user interface that also provides the analyst with relevant knowledge such as highlighting where potential hallucinations are, and the generated attribution links which explain why the LLM Explainer has identified certain portions of the LLM output as being hallucinations. This directly allows the analyst to verify that the output is correct or modify it if needed. In embodiments, an interactive and iterative process may be implemented by embodiments of the present disclosure by guiding users, such as the analyst, in verifying that the output is correct or modifying the output of the LLM by receiving input, prompts, or information from the analyst to correct any identified hallucinations generated by the LLM.

[0074] Another embodiment of the present disclosure can be practically applied in the field of generating health plans associated with behavior change applications for disease prevention. Here, a use case is that a phone app is given to a user who wants to improve their health via behavior changes to prevent diseases. In this scenario, an LLM can personalize the answer (e.g., health plan, advice, behavior change recommendations, etc.,) to the user, therefore providing the user with a better support ‘buddy’ to keep them on track. However, the LLM might produce incorrect information, advice, or recommendations. The features of the present disclosure enable the user to verify the correctness of the output generated by the LLM before following the advice. The data source and input for this use case includes the input text given to an LLM enriched by a RAG that retrieves information from a trusted source and the output text produced by a LLM. The features of the present disclosure generates links between chunks in the output and chunks in the input which can be used to warn of hallucinated information in the output. In this way, the present disclosure enables the generation of attribution links which can warn of hallucinated output generated by an LLM (the advice or recommendations generated by the LLM) thereby ensuring safe implementation of LLMs for generating recommendations or advice associated with behavior changes for preventing disease in an efficient and accurate manner. A user interface can provide a user with relevant knowledge, such as highlighting where potential hallucinations are and generated links between the input and output which explain why the features of the present disclosure identified certain portions as including potential hallucinations. This directly allows the user to verify that the output is correct or modify it if needed. In embodiments, the rewriting module (e .g . 710 of FIG. 7) can ensure that the final output of the LLM that is given to solve the task at hand, e.g., generating advice or recommendations associated with behavior changes for preventing disease, has fixed any potential hallucinationAtorney Docket No. 820805(Client Ref. NLE-1414-24-WO) issues as much as possible. In embodiments, an interactive and iterative process may be implemented by embodiments of the present disclosure by guiding users, such as the users, in verifying that the output is correct or modifying the output of the LLM by receiving input, prompts, or information from the user to correct any identified hallucinations generated by the LLM.

[0075] One embodiment of the present disclosure can be practically applied in the field of disease classification. Here, a use case is that, given past information about a patient and, optionally, also relevant medical resources, an LLM diagnoses the patient and gives a reasoning for the diagnosis (for transparency and safety). With the LLM Explainer of the present disclosure, a doctor or medical personnel can safely and accurately rely on the disease classifications generated by an LLM by verifying the output generated by the LLM in an efficient manner. For example, a doctor may be presented with potential hallucinations along with relevant information including attributions explaining why the LLM Explainer believes certain portions of the output to be hallucinations generated by the LLM. The data source and input for this use case includes past information (historic health information, previously generated medical reports, etc.,) about a patient and optionally also relevant medical resources. Further, the present disclosure generates links between chunks in the output and chunks in the input which can be used to warn of hallucinated information in the output. In this way, the present disclosure enables the generation of attribution links which can warn of hallucinated output generated by an LLM thereby ensuring safe implementation of LLMs for disease classification or diagnosis in an efficient and accurate manner. A user interface can provide a user with relevant knowledge, such as highlighting where potential hallucinations are and generated links between the input and output which explain why the features of the present disclosure identified certain portions as including potential hallucinations. This directly allows the user to verify that the output is correct or modify it if needed. The output generated by the LLM Explainer can include the diagnosis generated by the LLM along with attribution links and an explanation for potentially identified hallucinations. The user interface may enable a doctor to view trace attribution which the doctor can use before accepting an Al-based diagnosis generated by the LLM. In embodiments, an interactive and iterative process may be implemented by embodiments of the present disclosure by guiding users, such as the doctor, in verifying that the output is correct or modifying the output of the LLM by receiving input, prompts, or information from the doctor to correct any identified hallucinations generated by the LLM. In embodiments, the doctor or users can provide input or correct identified hallucinations by interacting with the user interface.Atorney Docket No. 820805 (Client Ref. NLE-1414-24-WO)

[0076] In an embodiment, the present disclosure provides a method for explaining LLM output in order to verify its correctness and fixing hallucinations by determining hallucination candidates for an output of an LLM, the method comprising the steps of:1) Selecting / training the following:1. an LLM for generation.2. an LLM for generating attributions.In some embodiments, this can include the trainings described herein with reference to the fine-tuned attribution model and task module.3. a chunking method that includes the coarse chunking and, in some embodiments, the sub-chunking. A simpler chunker or chunking method could include a program that creates a new chunk each time a is encountered which would result in chunks which are sentences in most cases, however, other chunkers or sub-chunkers can be used which create chunks that are single words or noun phrases.4. a hallucination detection module which includes: i. a detection module; ii. a labeling module; iii. a correction module; and iv. a rewriting module.2) Receiving two sets of documents A and B, where one or both could be the output of an LLM selected in step 1.1 above, and chunking both A and B according to the coarse chunking method selected in step 1.3 above.3) Generate LLM attributions (attribution links) using the LLM from step 1.2 above. The attribution links are an output of the system of the present disclosure. In some embodiment, the sub-chunking module is applied to further refine the granularity level of the attribution between the input and output of the LLM.4) Use the attributions for detecting candidate chunks for hallucinations. The hallucination candidates are an output of the system of the present disclosure. In some embodiments, the method includes labelling, correcting, and rewriting the labeled hallucination candidate chunks using the modules from 4.i.-4.iv above. In some embodiments, the new output generated by the system of the present disclosure may be refined by realigning only the rewritten chunks with the source documents. In embodiments, the hallucinations and attributions are used to inform an LLM about solving a given task (e.g., automatic disease diagnosis).Atorney Docket No. 820805 (Client Ref. NLE-1414-24-WO)5) Present the final output with attribution links, hallucination candidates, and in some embodiments, correct hallucination chunks in a user interface. Users may provide input via the user interface to correct hallucination chunks by editing text to rewrite a hallucinated section. In such scenarios, the system can save the rewritten text and use this information (feedback information) for further updates.

[0077] Embodiments of the present disclosure provide for the following improvements and technical advantages over existing technology:1) Using a specialized EEM which takes pairs of text chunks and outputs the likeliness that one chunk relates to the other chunk in order to do source attribution ((4) of FIG. 5).2) Chunk text before and after creating attribution links. This produces more fine-grained links while making sure the initial attribution takes into account the entire meaning of the sentence.3) Generate synthetic data that specifically contains hallucinations and re-rank the data points by the difficulty of detecting the hallucinations according to the participating EEMs in order to train an attribution model that is more accurate and efficient at attributing despite tricky hallucinations.4) Use attribution links (and lack thereof) in order to identify hallucinations, finding more relevant information, re -ranking with the attribution, and rewriting the hallucination, whereas the rewritten context is verified to be hallucination free by rerunning the attribution module. In particular: a. For finding supporting evidence to confirm the factuality of extrinsic hallucinations, supporting evidence is re-ranked for relevance by leveraging the computed attributions as pseudo relevance feedback, b. Create detailed context comprising of attributions and relevant evidence for the rewriting module, and c. Verify the automatically corrected output by re-computing attributions.5) Using a contrastive loss function to fine-tune an EEM to have less hallucinations for a particular task by contrasting an output with hallucinations to an output without hallucinations.

[0078] Currently, both LLMs and RAGs can lead to hallucinations or generate hallucinations. In contrast, embodiments of the present disclosure can prevent this while achieving state of the art performance, and offer features and enhanced computer functionality that do not exist elsewhere by generating attribution links for identifying hallucination candidates.Atorney Docket No. 820805(Client Ref. NLE-1414-24-WO)

[0079] There are several solutions according to existing technology that try to address enhancing writing efficiency while ensuring the accuracy and reliability of Al-generated content. Generally, one can categorize these solutions into several groups: explainability / attribution methods that clarify why models make predictions and trace information origins; interpretability methods that reveal how models make decisions; domainspecific LLMs designed to reduce inaccuracies with specialized knowledge; govemance / auditing software ensuring compliance and risk management; and content safety evaluation tools that detect harmful inputs and outputs and methods that fact-check outputs.

[0080] The sections below describe comparisons between different solutions currently being provided according to existing technology to features of the present disclosure. Different solutions according to existing technology are compared based on the desired capabilities of a system according to an embodiment of the present disclosure that both enhances writing and ensures correctness of the writing. Such a system according to an embodiment of the present disclosure provides the following improvements:- Enhances writing efficiency by automating the writing process using LLMs, thereby significantly reducing the time and effort required to generate comprehensive reports or summaries.- Provides efficient verification by incorporating real-time source attribution and error detection features, enabling users to quickly and confidently verify the accuracy of AI- generated content.- Maintains high processing speed by delivering results faster than a human can read, ensuring that the integration does not disrupt the workflow and is suitable for real-time applications.- Offers fine-granular and actionable insights by presenting detailed, sentence-level explanations and visualizations that allow users to understand and act upon the AI- generated outputs and explanations effectively.

[0081] Embodiments of the present disclosure provide an LLM Explainer that is both efficient and safe for collaboration on tasks such as disease classification. In comparison, conventional LLMs or RAGs generate more hallucinations as they are tasked with providing more assistance. Humans might write without the assistance of an LLM but might still make mistakes and complete tasks in an inefficient manner. To further illustrate, as described below, different market solutions are analyzed based on two main comparisons.

[0082] Lor the first comparison — Efficiency vs. Verification — a range of products exist. Traditional human writing without Al assistance, while reliable in terms of understanding context, proves to be low in both efficiency and verification due to its time-consuming natureAtorney Docket No. 820805(Client Ref. NLE-1414-24-WO) and susceptibility to human errors. On the other end of the spectrum, general LLMs significantly enhance writing efficiency by automating the process, but they typically require extensive manual verification because of their tendency to produce inaccurate or hallucinated content. Fact-checking engines provide a middle ground with improved verification capabilities through fact checking via web-search. However, they are limited in their scope, essentially as they are not suitable for confidential data and on-premise use-cases. In contrast, the LLM explainer of the present disclosure improves on existing technology by combining high writing efficiency with robust, real-time verification tools, ensuring that users can generate and validate content quickly and reliably. The LLM explainer of the present disclosure enables to detect hallucinations and offers source attributions, resulting in more efficient and safe collaboration than conventionally available LLMs or RAGs.

[0083] For the second comparison — Speed vs. Granularity — solutions also vary widely. Existing tools are slow and provide low granularity, offering only summary-level insights that may not be actionable. Other existing tools operate at mid speed but still fall short in delivering detailed granularity, limiting their insights to paragraph-level explanations. Some other existing systems excel in speed by quickly blocking harmful content, but they do not support collaborative, detailed insights, leaving the user without the option to take action. Another existing system can provide highly granular, sentence-level insights, but at the cost of processing speed, making it less practical for real-time applications. In contrast, the LLM Explainer of the present disclosure distinguishes itself and improves on existing technology by integrating highspeed processing with detailed, sentence-level explanations. This combination ensures that users receive instant, precise and actionable insights, making it an unmatched tool in the realm of AI- assisted writing and validation that seamlessly integrates into the users' workflow with any type of LLM. The LLM Explainer of the present disclosure stands out from the above-described approaches by offering real-time processing and fine insights as well as a side-by-side editor.

[0084] The LLM Explainer of the present disclosure distinguishes itself from other solutions on the market or conventionally used and presents itself as a highly efficient, effective, and safe solution for automating writing and documentation in high-risk applications. Differentiation features include compatibility with any LLM and any domain without the need for prior finetuning, real-time source attributions and hallucination warnings, easy visual displays of references, links, and warnings, and faster-than-human or ChatGPT processing speeds. The all- in-one editor allows users to run the LLM, explain outputs, visualize attributions, and edit the generated text seamlessly. By offering unparalleled writing efficiency, robust verification, rapid processing, and precise insights, the LLM Explainer ensures that Al -generated content is bothAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) accurate and reliable, making it an indispensable tool for enhancing productivity and safety in critical contexts.

[0085] In contrast to Benjamin Cohen-Wang, Harshay Shah, Kristian Georgiev, Aleksander Madry, “ContextCite,” blog post, May 6 2024, which is hereby incorporated by reference herein, and which approximates the attribution by randomly deleting information as re-querying the LLM that produced the text several times (e.g. 32 times in their experiments) - embodiments of the present disclosure are different from the approach described in “ContextCite” as it is implemented using an incredibly costly approach that requires multiple re-queries. Rerunning a large LLM that produces the output even a second time would consume significant computational resources and time. In contrast, a prototype implementing an embodiment of the present disclosure avoided such consumption of computational resources and time.

[0086] Yifei Li, Xiang Yue, Zeyi Liao, Huan Sun, “AttributionBench: How Hard is Automatic Attribution Evaluation?,” Findings of the Association for Computational Linguistics ACL, pages 14919-14935, 2024, which is hereby incorporated by reference herein, provides a benchmark that measures the success of attributions. However, the approach described in the above work does not propose a novel method for solving the problem of identifying hallucination candidates in the output of LLM compared to the embodiments of the present disclose. Instead, the approach described above simply prompts an LLM directly and fine-tunes an LLM on a subset of self-provided data.

[0087] In contrast to Tobias Schimanski, Jingwei Ni, Mathias Kraus, Elliott Ash, Markus Leippold, “Towards Faithful and Robust LLM Specialists for Evidence-Based Question- Answering,” arXiv:2402.08277, 2024, which is hereby incorporated by reference herein, which proposes a data generation strategy for evidence-based question answering which is different from the features of the present disclosure as it describes focusing exclusively on the task of question answering, and is not applicable to other tasks. Instead, the embodiments of the present disclosure are generic across many natural language processing (NLP) tasks, and leverages the ensemble approach of multiple LLMs, which is different from the data generation approach of described above.

[0088] The approach detailed in Satyapriya Krishna, Jiaqi Ma, Dylan Slack, Asma Ghandeharioun, Sameer Singh, and Himabindu Lakkaraju, “Post Hoc Explanations of Language Models Can Improve Language Models,” NeurlPS, (2024), which is hereby incorporated by reference herein, proposes a method for improving a task with the use of LLMs. This approach is restricted to certain types of classification explanations (i.e., feature importance), which is used as a signal for improving the task at hand, requires annotated data in order to work, and does not use signals from alignments or hallucinations to improve the task. In contrast,Atorney Docket No. 820805(Client Ref. NLE-1414-24-WO) embodiments of the present disclosure advantageously use signals from alignments or hallucinations to improve the task and do not require annotated data.

[0089] In the approach described by Abhika Mishra, Akari Asai, Widhisha Balachandran, Yizhong Wang, Graham Neubig, Yuha Tsvetkov, Hannaneh Hajishirzi, “Fine-grained Hallucination Detection and Editing for Language Models, ”arXiv:2401.06855, 2024, which is hereby incorporated by reference herein, the authors propose a new benchmark of hallucination in LLM responses to diverse queries, each annotated for different hallucination types. However, the work of the authors mentioned above focuses on open-ended text generation and very specific error types that are subjectively annotated. In contrast, the embodiments of the present disclosure have a more objective and minimal classification of hallucination types fully based on the attributions with a clear task in mind. The approach of the above mentioned authors to editing the hallucinations relies solely on synthetic data without any scope for further improvement. In contrast, the embodiments of the present disclosure focus on obtaining feedback from the downstream task performance to further iterate and mitigate any remaining hallucinations.

[0090] In the approach described by Postawee Manakul, Adian Liusie, Mark J. F. Gales, “SELFCHECKGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models,” arXiv:2303.08896, 2023, which is hereby incorporated by reference herein, the authors propose a sampling -based approach for fact-checking responses of black-box LLMs without relying on additional resources. However, the assumptions of the authors mentioned above that if an LLM has knowledge of a concept, sampled responses are likely to be consistent and hold similar facts, is insufficient for enumerating the type of hallucinations generated in the output. In contrast, the approach of the present disclosure leverages the source document attributions which excludes reliance on the internal knowledge of the LLM and can thus be applied to evaluating text generation in any domain. The embodiments of the present disclosure additionally rely on external sources such as document databases which makes the fact-checking more reliable and customizable to the domain at hand which is absent from the above mentioned approach.

[0091] U.S. Patent App. Pub. No. 2024 / 0289395 Al, which is hereby incorporated by reference herein, is related to the reduction of hallucinations when using an LLM. However, this patent application is focused on finding the relevant context, in contrast to the embodiments of the present disclosure which assumes it is already there. Moreover, the above referenced patent application focuses on fine-tuning LLMs so that they don’t hallucinate as much, while the embodiments of the present disclosure provide post-hoc explanations of already trained LLMs. The features of the above referenced patent application may provide attributions to support factsAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) in the generated responses, however, these are obtained by a search engine which is different from the embodiments of the present disclosure.

[0092] U.S. Patent App. Pub. No. 2024 / 0289395 Al, which is hereby incorporated by reference herein, proposes a method which is related to the reduction of hallucinations for LLMs. The approach described in the above referenced patent application also provides attributions that link back the LLM output to source documents. However, this approach is focused on text summarization and question answering only, while the embodiments of the present disclosure can be applied to any task with LLM input & output. The approach of the above referenced patent application describes how the LLM is encouraged to generate quotes from the source document, which means that the output of the LLM is dependent on the LLM. In contrast, the embodiments of the present disclosure execute everything post-hoc, and the LLM is not instructed to input quotes or anything similar. The approach of the above referenced patent application is related to prompt engineering. In contrast, the embodiments of the present disclosure use synthetic data generation, fine-tuning smaller models, comparing embedding vectors, etc.

[0093] U.S. Patent App. Pub No. 2024 / 0320445 Al, which is hereby incorporated by reference herein, proposes a method for abstractive summarization with LLMs by using attribution information. The approach described in the above referenced patent application is different from the embodiments of the present disclosure as it focuses on text summarization only. In contrast, the embodiments of the present disclosure are applicable to any NLP task that is tackled with an LLM. While the above referenced patent application does describe attribution the exact steps of their “attribution engine” are lacking and instead seems to attribute back to sentences used for training / fme-tuning the LLM, i.e. the "internal knowledge" of the LLM, with a target to then force the summary to be also based on the input document - which is different from the attributions of the present disclosure.

[0094] Referring to FIG. 9, a processing system 900 can include one or more processors 902, memory 904, one or more input / output devices 906, one or more sensors 908, one or more user interfaces 910, and one or more actuators 912. Processing system 900 can be representative of each computing system disclosed herein.

[0095] Processors 902 can include one or more distinct processors, each having one or more cores. Each of the distinct processors can have the same or different structure. Processors 902 can include one or more central processing units (CPUs), one or more graphics processing units (GPUs), circuitry (e.g., application specific integrated circuits (ASICs)), digital signal processors (DSPs), and the like. Processors 902 can be mounted to a common substrate or to multiple different substrates.Atorney Docket No. 820805 (Client Ref. NLE-1414-24-WO)

[0096] Processors 902 are configured to perform a certain function, method, or operation (e.g., are configured to provide for performance of a function, method, or operation) at least when one of the one or more of the distinct processors is capable of performing operations embodying the function, method, or operation. Processors 902 can perform operations embodying the function, method, or operation by, for example, executing code (e.g., interpreting scripts) stored on memory 904 and / or trafficking data through one or more ASICs. Processors 902, and thus processing system 900, can be configured to perform, automatically, any and all functions, methods, and operations disclosed herein. Therefore, processing system 900 can be configured to implement any of (e.g., all of) the protocols, devices, mechanisms, systems, and methods described herein.

[0097] For example, when the present disclosure states that a method or device performs task “X” (or that task “X” is performed), such a statement should be understood to disclose that processing system 900 can be configured to perform task “X”. Processing system 900 is configured to perform a function, method, or operation at least when processors 902 are configured to do the same.

[0098] Memory 904 can include volatile memory, non-volatile memory, and any other medium capable of storing data. Each of the volatile memory, non-volatile memory, and any other type of memory can include multiple different memory devices, located at multiple distinct locations and each having a different structure. Memory 904 can include remotely hosted (e.g., cloud) storage.

[0099] Examples of memory 904 include a non-transitory computer-readable media such as RAM, ROM, flash memory, EEPROM, any kind of optical storage disk such as a DVD, a Blu- Ray® disc, magnetic storage, holographic storage, a HDD, a SSD, any medium that can be used to store program code in the form of instructions or data structures, and the like. Any and all of the methods, functions, and operations described herein can be fully embodied in the form of tangible and / or non-transitory machine-readable code (e.g., interpretable scripts) saved in memory 904.

[0100] Input-output devices 906 can include any component for trafficking data such as ports, antennas (i.e., transceivers), printed conductive paths, and the like. Input-output devices 906 can enable wired communication via USB®, DisplayPort®, HDMI®, Ethernet, and the like. Input-output devices 906 can enable electronic, optical, magnetic, and holographic, communication with suitable memory 906. Input-output devices 906 can enable wireless communication via WiFi®, Bluetooth®, cellular (e.g., LTE®, CDMA®, GSM®, WiMax®, NFC®), GPS, and the like. Input-output devices 906 can include wired and / or wireless communication pathways.Atorney Docket No. 820805 (Client Ref. NLE-1414-24-WO)

[0101] Sensors 908 can capture physical measurements of environment and report the same to processors 902. User interface 910 can include displays, physical buttons, speakers, microphones, keyboards, and the like. Actuators 912 can enable processors 902 to control mechanical forces.

[0102] Processing system 900 can be distributed. For example, some components of processing system 900 can reside in a remote hosted network service (e.g., a cloud computing environment) while other components of processing system 900 can reside in a local computing system. Processing system 900 can have a modular design where certain modules include a plurality of the features / functions shown in FIG. 9. For example, I / O modules can include volatile memory and one or more processors. As another example, individual processor modules can include read-only-memory and / or local caches.

[0103] While embodiments of the disclosure have been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. It will be understood that changes and modifications may be made by those of ordinary skill within the scope of embodiments of the present disclosure. In particular, the present disclosure covers further embodiments with any combination of features from different embodiments described above and below. Additionally, statements made herein characterizing the invention or disclosure refer to an embodiment of the invention or disclosure and not necessarily all embodiments.

[0104] The terms used in the claims should be construed to have the broadest reasonable interpretation consistent with the foregoing description. For example, the use of the article “a” or “the” in introducing an element should not be interpreted as being exclusive of a plurality of elements. Likewise, the recitation of “or” should be interpreted as being inclusive, such that the recitation of “A or B” is not exclusive of “A and B,” unless it is clear from the context or the foregoing description that only one of A and B is intended. Further, the recitation of “at least one of A, B and C” should be interpreted as one or more of a group of elements consisting of A, B and C, and should not be interpreted as requiring at least one of each of the listed elements A, B and C, regardless of whether A, B and C are related as categories or otherwise. Moreover, the recitation of “A, B and / or C” or “at least one of A, B or C” should be interpreted as including any singular entity from the listed elements, e.g., A, any subset from the listed elements, e.g., A and B, or the entire list of elements A, B and C.

Claims

Atorney Docket No. 820805 (Client Ref. NLE-1414-24-WO)CLAIMSWhat is claimed is:

1. A computer-implemented, machine learning method for explaining and verifying correctness of an output of a large language model (LLM), the method comprising: splitting two sets of documents into a plurality of text spans by inputting the two sets of documents into a coarse chunking algorithm, wherein a first portion of the text spans corresponds to the output of the LLM and a second portion of the text spans corresponds to a context; generating attribution links using the text spans as input to a specialized LLM that has been trained to output a likelihood that one of the text spans in the first portion relates to one of the text spans in the second portion; identifying hallucination candidate text spans in the text spans based on the attribution links; and presenting the hallucination candidate text spans and the attribution links via a user interface.

2. The method according to claim 1, wherein the specialized LLM is trained by: generating a silver labeled training data set by providing as input a golden data set that does not include hallucinations, an unlabeled training data set, and prompts to a plurality of LLMs, wherein the silver labeled training data set includes hallucination free data points and hallucinated data points; implementing a weight learning phase for learning a weight for each LLM of the plurality of LLMs based on the golden data set and the silver labeled training data set; implementing an online filtering phase that provides, as input, a data point from the unlabeled training data set to each of the plurality of LLMs that provide as output the weight of the LLM and a predicted label for the data point, wherein the online filtering phase further includes computing a soft label for the predicted label and comparing the soft label to a threshold to determine a final label; generating a trusted training data set based on the final labels of each data point from the unlabeled training data set; and training the specialized LLM based on a light-weight context attribution LLM and the trusted training data set, wherein the weight learning phase includes determining a first weight for each LLM for attribution links of hallucination free text spans and a second weight for each LLM for attribution links of text spans with hallucinations.Atorney Docket No. 820805 (Client Ref. NLE-1414-24-WO)3. The method according to any of the preceding claims, wherein the context was provided to the LLM for a particular task used to generate the output of the LLM, and wherein the method further comprises: determining that a stopping condition has not been met; generating an updated prompt that incorporates the hallucination candidate text spans and the attribution links; providing the updated prompt as input to the LLM for solving the particular task, wherein the determining, generating, and providing steps are executed iteratively until the stopping condition is met, wherein the stopping condition includes a maximum number of iteration loops or until no hallucination candidate text spans are identified in the text spans, wherein the LLM generates an updated output by inputting the updated prompt for solving the particular task; and updating the LLM by implementing a contrastive loss function based on the updated output representing a corrected version of the output generated by the LLM and the output that represents a hallucinated version of the output generated by the LLM.

4. The method according to any of the preceding claims, wherein at least one of the two sets of documents is an output of a retrieval augmented generation (RAG) system.

5. The method according to any of the preceding claims, wherein identifying the hallucination candidate text spans from the text spans includes identifying entities, words, or concepts that are only identified in the first portion, identifying the text spans of the first portion that have no relevancy to the text spans of the second portion, implementing another LLM configured to detect hallucinations that are provided the first portion, the second portion, and the attribution links as input, or by comparing the text spans of the first portion to a set of text spans that correspond to known hallucinations that are stored in a database.

6. The method according to any of the preceding claims, wherein the method further comprises, subsequent to generating the attribution links but prior to identifying the hallucination candidate text spans: splitting the first portion into a sub-set first portion and the second portion into a sub-set second portion by inputting the first portion, the second portion, and the attribution links into a sub-chunking algorithm; and generating sub-attribution links using the sub-set first portion and the sub-set second portion as input to the specialized LLM that has been trained to output a likelihood that one of the text spans in the sub-set first portion relates to one of the text spans in the sub-set secondAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) portion, wherein the sub-attribution links indicate which of the sub-set first portion are linked to the sub-set second portion.

7. The method according to any of the preceding claims, wherein the method further comprises assigning an intrinsic hallucination label or an extrinsic hallucination label to the first portion or the second portion based on the attribution links, the extrinsic hallucination label and the intrinsic hallucination label for the first portion of the second portion also associated with a corresponding confidence score representing a confidence of the first portion or the second portion as being an intrinsic hallucination or an extrinsic hallucination.

8. The method according to claim 7, wherein the method further comprises: verifying extrinsic hallucination labeled text spans from the first portion and the second portion by: generating an information-seeking query based on the extrinsic hallucination labeled text spans of the first portion or the second portion; obtaining a set of documents from a database based on the information-seeking query; reranking the set of documents based on the attribution links to serve as relevance feedback; and generating an editing context for removing hallucinations of the extrinsic hallucination labeled text spans, the editing context comprising portions of the reranked set of documents that are attributed to the two sets of documents based on the attribution links, the extrinsic hallucination labeled text spans, and an instruction to the LLM for text reformulation based on the attribution links and the extrinsic hallucination labeled text spans.

9. The method according to claim 8, wherein the method further comprises: rewriting either the first portion or the second portion to remove the extrinsic hallucination labeled text spans based on the editing context; rewriting either the first portion or the second portion to remove intrinsic hallucinations based on intrinsic hallucination labeled text spans and the attribution links; updating the attribution links based on the rewritten first portion or second portion; and identifying updated hallucination candidate text spans from the plurality of text spans based on the rewritten first portion or second portion and the updated attribution links.

10. The method according to any of the preceding claims, wherein one of the two sets of documents includes text output generated by the LLM in response to being provided a set ofAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) requirements as input to the LLM, wherein another of the two sets of documents includes a report, and wherein the hallucination candidate text spans and the attribution links that are presented via the user interface represent potential hallucinations and the attribution links between the text output generated by the LLM and the set of requirements provided as input to the LLM.

11. The method according to any of the preceding claims, wherein one of the two sets of documents includes text output generated by the LLM in response to being provided cybersecurity reports as input to the LLM, wherein another of the two sets of documents includes the cybersecurity reports, and wherein the hallucination candidate text spans and the attribution links that are presented via the user interface represent potential hallucinations and the attribution links between the text output generated by the LLM and the cybersecurity reports provided as the input to the LLM.

12. The method according to any of the preceding claims, wherein one of the two sets of documents includes medical reports of a patient generated by the LLM in response to being provided historic medical information for the patient, wherein another of the two sets of documents includes the historic medical information for the patient and medical resources, and wherein the hallucination candidate text spans and the attribution links that are presented via the user interface represent potential hallucinations and the attribution links between the medical reports of the patient generated by the LLM and the historic medical information for the patient provided as the input to the LLM.

13. A machine learning model stored on a tangible, non-transitory computer readable medium for explaining and verifying correctness of an output of a large language model (LLM), the machine learning model comprising: a chunker that generates text spans by implementing a coarse chunking algorithm on two sets of input documents, wherein a first portion of the text spans corresponds to the output of the LLM and a second portion of the text spans corresponds to a context; an attribution layer that takes the text spans as input and is trained to output a likelihood that one of the text spans in the first portion relates to one of the text spans in the second portion in order to generate attribution links; and a hallucination detection layer trained to identify hallucination candidate text spans in the text spans based on the attribution links.

14. The machine learning model according to claim 13, wherein the attribution layer includes a granularity adjustment layer and a large language model (LLM) attributor thatAtorney Docket No. 820805 (Client Ref. NLE-1414-24-WO) includes an embedding layer for embedding the text spans into a latent space, and wherein the hallucination detection layer includes a hallucination detection layer that identifies the hallucination candidate text spans, a hallucination classification layer for classifying the hallucination candidate text spans as intrinsic hallucinations or extrinsic hallucinations, and a hallucination correction layer for rewriting the hallucination candidate text spans to generate a corrected output.

15. A computer system for using a machine learning method for explaining and verifying correctness of an output of a large language model (LLM), the computer system comprising one or more hardware processors which, alone or in combination, are configured to provide for execution of the following steps: splitting two sets of documents into a plurality of text spans by inputting the two sets of documents into a coarse chunking algorithm, wherein a first portion of the text spans corresponds to the output of the LLM and a second portion of the text spans corresponds to a context; generating attribution links using the text spans as input to a specialized LLM that has been trained to output a likelihood that one of the text spans in the first portion relates to one of the text spans in the second portion; identifying hallucination candidate text spans in the text spans based on the attribution links; and presenting the hallucination candidate text spans and the attribution links via a user interface.

Images