System and method for proactively reducing hallucinations in generative artificial intelligence (AI) model responses

Abstract

A method, computer program product, and computing system for processing a prompt for a target generative AI model and a corresponding response generated by the target generative AI model for the prompt. The prompt and the corresponding response from the generative AI model are compared to a plurality of predefined verified prompt-response pairs. In response to determining at least a threshold similarity between the prompt and the corresponding response and a predefined verified prompt-response pair, the corresponding response from the target generative AI model is provided to a source of the prompt.

Description

SYSTEM AND METHOD FOR PROACTIVELY REDUCING HALLUCINATIONS IN GENERATIVE ARTIFICIAL INTELLIGENCE (Al) MODEL RESPONSESBackground

[0001] Generative Al models are being used in customer support service (CSS) and other domains to solve a variety of problems and to ease data processing from various sources to improve the efficiency and accuracy of customer support by automating repetitive and simple tasks and providing consistent responses. However, hallucination in a generative Al model response results in factually incorrect or nonsensical information being presented to a user. Despite advancements in natural language processing, this challenge remains due to the complexity of language understanding and generation. Additionally, current approaches rely on reactive processes to correct hallucinations that have already been provided in a response to a user.Brief Description of the Drawings

[0002] FIGS. 1 A-l B are a flow chart of one implementation of a response verification process;

[0003] FIG. 2 is a diagrammatic view of the response verification process of FIGS. 1A-1B generating predefined verified prompt-response pairs;

[0004] FIG. 3 is a diagrammatic view of the response verification process of FIGS. 1A-1B verifying generative Al model responses; and

[0005] FIG. 4 is a diagrammatic view of the response verification process of FIGS. 1A-1B verifying user feedback concerning generative Al model responses; and

[0006] FIG. 5 is a diagrammatic view of computer system and the response verification process coupled to a distributed computing network.

[0007] Like reference symbols in the various drawings indicate like elements.Detailed Description of the Embodiments

[0008] Implementations of the present disclosure provide a two-point comparator model that improves the accuracy of generative Al model responses by proactively reducing hallucinations. For example, generative Al models that include natural language processing (NLP) have advanced in understanding and generating human-like text. However, these generative Al models are not immune to certain challenges, and one such issue is ’‘hallucination.” Hallucination refers to thephenomenon where the generative Al model generates responses that contain fictional or incorrect information, without any factual basis in the input data or context. The manifestations of hallucination may vary in severity, ranging from minor factual errors to generating entire passages of imaginative content. In some cases, the generative Al model may confidently present hallucinated information, leading users to believe in the validity of the generated text. This phenomenon can be problematic, especially in critical domains where misinformation may have serious consequences.

[0009] To combat hallucination issues, the two-point comparator model of the present disclosure involves a multi-stage process through predefined, verified prompt-response pairs generated by generative Al model(s). As will be discussed in greater detail below, a first stage verifies responses by processing generative Al model responses through predefined verified prompt-responses pairs prior to displaying the response to the user. In this manner, generative Al model responses are verified before being presented to a prompting user and hallucinations are filtered from the response. In a second stage, user feedback is verified by processing through predefined verified prompt-response pairs prior to invoking subject matter experts (SMEs) to verily / correct biased or incorrect user feedback responses. In this manner, user feedback is filtered through predefined verified prompt-response pairs before applying the user feedback to generative Al model training or tuning.

[0010] As will be descnbed in greater detail below, implementations of the present disclosure processes a prompt for a target generative Al model and a corresponding response generated by the target generative Al model for the prompt. The prompt and the corresponding response from the generative Al model are compared to a plurality of predefined verified prompt-response pairs. In response to determining at least a threshold similarity between the prompt and the corresponding response and a predefined verified prompt-response pair, the corresponding response from the target generative Al model is provided to a source of the prompt.

[0011] As will be described in greater detail below, implementations of the present disclosure provide a process for automatically and proactively identifying and removing hallucinations generated by generative Al models and for improving the accuracy of the responses provided by removing incorrect or factually false information by converting each prompt into a vector ofembeddings and each corresponding generative Al model response into a vector embeddings and performing vector similarity between the embedding representations of the prompt and the corresponding response to predefined verified prompt-response pairs. When the vector similarity between the embedding representations of the prompt and the corresponding response and the predefined verified prompt-response pairs is above a threshold, the corresponding response is verified and provided to a source of the prompt. In some implementations, this approach improves the accuracy of generative Al model responses and prevent hallucinations in a manner that is scalable to monitor any number (i.e., millions) of incorrect responses in real-time.

[0012] The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features and advantages will become apparent from the description, the drawings, and the claims.The Response Verification Process:

[0013] Referring to FIGS. 1 A-4, response verification process 10 processes 100 a prompt for a target generative Al model and a corresponding response generated by the target generative Al model for the prompt. The prompt and the corresponding response from the generative Al model are compared 102 to a plurality of predefined verified prompt-response pairs. In response to determining at least a threshold similarity between the prompt and the corresponding response and a predefined verified prompt-response pair, the corresponding response from the target generative Al model is provided 104 to a source of the prompt.

[0014] In some implementations, response verification process 10 proactively reduces hallucinations found in responses generated by generative Al models when processing prompts. For example. Al models use neural networks to identify patterns and structures within a data set to perform a particular task (e.g., convert speech to text, generate new data (e.g., generative Al model), identify a biometric profile within a plurality of biometric profiles, solve complex mathematical problems, etc.). In some implementations, a generative Al model (e.g., generative Al model 200) is configured to receive natural language prompts and / or example entries and / or contextual information concerning a request to generate a response. In some implementations, the Al model includes a Large Language Model (LLM). A LLM (e.g., Bing® Chat from Microsoft®, GPT-4 from OpenAI®, and Bard from Google®) is a language model consisting of a neuralnetwork with many parameters (typically billions of weights or more), trained on large quantities of unlabeled text using self-supervised learning or semi-supervised learning. Though trained on simple tasks along the lines of predicting the next word in a sentence, LLMs with sufficient training and parameter counts capture the syntax and semantics of human language. In addition, LLMs demonstrate considerable general knowledge and are able to ‘'memorize” large quantities of facts during training.

[0015] However, because LLMs and generative Al models learn patterns from diverse sources that may not include training data that addresses a prompt and / or because a prompt may be incomplete or ambiguous, these models can “hallucinate” responses that may have no direct source. Rather, these hallucinated responses are generated using content from sources that are most likely (e.g., based on probabilities and / or weighting within the generative Al model) to address the prompt. Accordingly, hallucinated responses present significant issues for users who rely on generative Al models to provide factual information (e.g., for safety instructions, for instructions on how to assemble a product, for medical care directions, for information on how to resolve emergency issues, etc.). With response verification process 10, hallucinated responses are identified before, and filtered from, being presented to a source of a prompt. In this manner, response verification process 10 automatically (i.e., without human or manual intervention) and proactively (i.e., in real-time and prior to providing responses to users) prevents hallucinated responses from being provided to a user.Generating Predefined Verified Prompt-Response Pairs:

[0016] In some implementations, response verification process 10 generates 106 the plurality of predefined verified prompt-response pairs by: extracting 108 each paragraph of content from a verified document; generating 110 a prompt for each extracted paragraph using a generative Al model; and generating 112 a corresponding response to each prompt by processing the prompt and a corresponding extracted paragraph using the generative Al model. For example and in some implementations, response verification process 10 uses a plurality of predefined verified promptresponse pairs as a '‘ground truth” against which prompts and corresponding responses generated by generative Al models are compared to determine whether the response generated by the target generative Al model is a hallucination. Conventional approaches to defining ground truth for agenerative Al model include relying on subject matter experts (SMEs) to define or identify facts or other content that describes accurate information. However, this approach cannot scale to rate by which new content is provided to generative Al models. For example, as more content becomes available to generative Al models, the ability to identify7ground truth becomes limited by SME processing of that content. As such, response verification process 10 resolves this limitation by using generative Al model(s) to convert content into prompts and corresponding responses that define the ground truth. In this manner, response verification process 10 automates the generation of predefined verified prompt-response pairs using generative Al model(s).

[0017] In some implementations, response verification process 10 generates 106 the plurality of predefined verified prompt-response pairs by extracting 108 each paragraph of content from a verified document. Referring also to FIG. 2. response verification process 10 obtains a document (e.g., document 200). In one example, document 200 is a portion of pre- verified content. Preverified content generally includes information that has been verified by a subject matter expert (SME) for its accuracy or factual basis. In some implementations, document 200 is stored in a verified content database (e.g., verified content database 202). Response verification process 10 extracts 108 each paragraph of content (e.g., paragraphs 204, 206, 208) and generates unique identifiers for each document and each paragraph (e.g., document and paragraph identifiers 210, 212, 214). In some implementations, extracting 108 each paragraph includes performing data cleaning. In one example, data cleaning includes using natural language processing (NLP) techniques to clean irrelevant punctuation marks, characters, and to fix formatting issues. In another example, data cleaning does not include performing lemmatization or stemming on documents to avoid contextual information loss. In some implementations, paragraph-level extraction is particularly useful for generative Al models and information retrieval systems when the answers require more extensive context, and single-sentence answers may be insufficient. By selecting entire paragraphs, response verification process 10 can provide more comprehensive and coherent responses to user prompts, leading to a better user experience and more informative interaction. In one example, paragraph level extraction is performed by parsing breaks (e.g., An”) or specific tags (e.g., <\p>) in HTML documents. Each extracted paragraph is denoted with a paragraph identifier (e.g., “Para-Id”) and it is saved along with an associated a document identifier(e.g., “Doc-Id’'). In some implementations, content of each paragraph is vectorized through a word embedding model. Accordingly, each extracted paragraph may be defined as a vector of Para-Id, Paragraph Content, Doc-Id, Para-Embeddings.

[0018] In some implementations, response verification process 10 generates 110 a prompt for each extracted paragraph using a generative Al model. For example, response verification process 10 provides each paragraph (e.g., paragraphs 204, 206. 208) to a generative Al model (e.g., generative Al model 216) such as T5 (text-to-text transfer), BART (bi-directional auto transformer model) or a Davinci model to generate questions I prompts (e.g., prompts 218, 220, 222). In some implementations, each generated question / prompt is stored along with their document identifier and their paragraph identifier. In one example, response verification process 10 generates a vector or other identifier formed from the combination of a prompt identifier, a document identifier, and a paragraph identifier (e.g., Ques ID, Doc ID, Para ID). In some implementations and as will be described below, each prompt (e.g., prompts 218, 220, 222) is vectorized as embeddings that are used to search answers / responses from paragraphs through vector similarity metrics.

[0019] In some implementations, response verification process 10 generates 112 a corresponding response to each prompt by processing the prompt and a corresponding extracted paragraph. For example, response verification process 10 provides the prompts (e.g., prompts 218, 220, 222) along with their content (e.g., by paragraph identifier and document identifier) to a generative Al model (e.g., generative Al model 224) to generate prompts. In one example, generative Al model 224 is the same as generative Al model 216 used to generate prompts 218, 220, 222. In another example, generative Al model 224 is a different generative Al model than generative Al model 216. Using generative Al model 224, response verification process 10 generates a corresponding response (e.g., corresponding responses 226, 228, 230) for prompts 218, 220, 222.

[0020] In some implementations and for each prompt, response verification process 10 performs a vector si mi I an ty search on each prompt embedding and paragraph embedding. In some implementations, this results in fetching a “top N” answers / responses signifying ranking of the paragraphs based on relevance to the question I prompt. The fetched top paragraph embeddings indicate the lines in the paragraph which are relevant to construct the answers / responses. In oneexample, response verification process 10 performs vector similarity metrics to fetch top answers / responses 226, 228, 230 for prompts 218, 220, 222. Accordingly, response verification process 10 controls the quality of responses 226, 228, 230 by adding a threshold on the similarity score such that if similarity score is greater than threshold value, then response verification process 10 only saves that answer in the plurality of predefined verified prompt-response pairs. In one example, the threshold for the similarity score is high enough to achieve high accuracy for generated answers / responses. In this example, other parameters such as temperature and top probability are set to “0” and "1”, respectively. In this manner, response verification process 10 generates a plurality of predefined verified prompt-response pairs (e.g., predefined verified prompt-response pairs 232, 234, 236). In one example, predefined verified prompt-response pairs 232, 234, 236 are stored in a database of predefined verified prompt-response pairs (e.g., predefined verified prompt-response pair database 238). In some implementations, predefined verified prompt-response pair database 238 includes a plurality of predefined verified promptresponse pairs in the following format: (Ques-Id, Question. Question Embeddings, Ans-Id, Answers, Answer Embeddings, Para-Id, Doc-Id). As will be discussed in greater detail below, the plurality of predefined verified prompt-response pairs are generated automatically without human intervention and can be used to proactively identify and remove hallucinated responses from being provided to users.Generative Al Model Response Verification:

[0021] In some implementations, response verification process 10 processes 100 a prompt for a target generative Al model and a corresponding response generated by the target generative Al model for the prompt. For example, response verification process 10 provides a proactive approach for verifying generative Al model responses for hallucinations in real-time by comparing a response generated by a target generative Al model against a plurality of predefined verified prompt-response pairs as described above. Referring also to FIG. 3, a target generative Al model (e.g., target generative Al model 300) is the generative Al model that a user (or other source) provides with a prompt in order to obtain a response. In this manner, target generative Al model 300 is the generative Al model that response verification process 10 interacts with to identify and restrict hallucinated responses from. In some implementations, target generative Al model 300processes a prompt (e.g., prompt 302) to generate a corresponding response (e.g., corresponding response 304).

[0022] In some implementations, response verification process 10 compares 102 the prompt and the corresponding response from the generative Al model to a plurality of predefined verified prompt-response pairs. As discussed above, a predefined verified prompt-response pair is a combination of prompts / questions and generative Al model responses / answers that are verified by a SME and / or by response verification process 10 as described above. In this manner, predefined verified prompt-response pairs represent ground truth that can validate or verify responses generated by the target generative Al model. Referring again to FIG. 3 and in some implementations, response verification process 10 provides prompt 302 and corresponding response 304 to a comparator system (e.g., comparator system 306). Comparator system 306 is a hardware and / or software component that processes prompt 302 and corresponding response 304 and compares 402 the combination of prompt 300 and corresponding response 304 to a plurality of predefined verified prompt-response pairs (e.g.. predefined verified prompt-response pairs 232, 234, 236). In some implementations, comparator system 306 is also referred to as a “content moderator" or “data tuner" as response verification process 10 is able to provide data enrichment

[0023] In some implementations, comparing 102 the prompt and the corresponding response from the generative Al model to a plurality of predefined verified prompt-response pairs includes generating 114 embeddings representative of the prompt. For example, an embedding or a vector embedding is a way to convert words and sentences and other data into numbers that capture their meaning and relationships. They represent different data types as points in a multidimensional space, where similar data points are clustered closer together. These numerical representations help machines understand and process this data more effectively. In some implementations, response verification process 10 generates 114 a vector embedding representative of prompt 302. In one example, response verification process 10 generates 114 the vector embedding for prompt 302 using comparator system 306. In another example, response verification process 10 generates 114 the vector embedding for prompt 302 using a separate system / embedding system.

[0024] In some implementations, response verification process 10 performs 116 a vector similarity search for the prompt from the plurality of predefined verified prompt-response pairsusing the embeddings representative of the prompt. For example, a vector similarity search includes identifying similar data using approximate nearing neighbor (ANN) algorithms. Compared to a traditional keyword search, vector search yields more relevant results and executes faster. In some implementations, response verification process 10 identifies 118 a threshold number of most similar prompts from the plurality of predefined verified prompt-response pairs. Referring again to FIG. 3, when performing 116 a vector similarity search for prompt 302, response verification process 10 identifies 118 a threshold number of most similar prompts (i.e., the top “N” most similar prompts) from a plurality of predefined verified prompt-response pairs (e.g., predefined verified prompt-response pairs 232, 234, 236). In one example, the threshold number is a default number (e.g., three). In another example, the threshold number is a user- defined value. In some implementations, “most similar’7prompt is defined as a threshold level of similarity when performing vector similarity' searching. This threshold may be a default value or a user-defined value.

[0025] As shown in FIG. 3, in response to performing 116 a vector similarity search for prompt 302 from predefined verified prompt-response pairs 232, 234, 236, response verification process 10 identifies 118 a threshold number of most similar prompts (e.g., predefined verified prompt-response pair 308 with prompt 310). In this example, prompt 310 is the most similar prompt of predefined verified prompt-response pairs 232, 234, 236 when compared with prompt 302. In some implementations, if there is not a sufficiently similar prompt in the plurality of predefined verified prompt-response pairs, response verification process 10 may alert (e.g., by providing a pop up window or other electronic message) a source of prompt 302 that response verification process 10 is unable to perform hallucination prevention based on prompt 302. In some implementations, a source of prompt 302 can selectively enable (i.e., choose when to enable and when to disable) response verification process 10 from performing verification of responses from a target generative Al model. In this manner, users are able to determine when they need hallucination-free responses and when they are searching for creative content (i.e., content not based on verified information).

[0026] In some implementations, response verification process 10 obtains 120 the corresponding responses for the threshold number of most similar prompts from the plurality ofpredefined verified prompt-response pairs. For example, with the most similar prompts (e.g., prompt 310), response verification process 10 obtains 120 the corresponding responses (e.g., corresponding response 312). In some implementations, response verification process 10 generates 122 embeddings representative of each corresponding response for the threshold number of most similar prompts. As with prompt 302, response verification process 10 generates 122 embeddings representative of corresponding response 304 by converting the words and sentences and other data of corresponding response 304 into numbers that capture their meaning and relationships.

[0027] In some implementations, response verification process 10 performs 124 a vector similarity search for the corresponding responses from the plurality of predefined verified promptresponse pairs using the embeddings representative of the prompt. For example, response verification process 10 compares the embedding representation of response 304 generated by target generative Al model 300 and the corresponding responses from the plurality of predefined verified prompt-response pairs (e.g., corresponding response 312). In some implementations, response verification process 10 uses a threshold level of similarity (e.g., threshold 314) to determine whether response 304 is sufficiently similar to a corresponding response from the plurality of predefined verified prompt-response pairs (e.g., corresponding response 312). When response 304 is not sufficiently similar (i.e., the level of similarity is below threshold 314), response 304 is indicative of a hallucination because it cannot be verified with a corresponding response from a sufficiently similar prompt of the plurality of predefined verified prompt-response pairs. When response 304 is sufficiently similar (i.e., the level of similarity is at or above threshold 314). response 304 is indicative of verified response because it can be verified with a corresponding response from a sufficiently similar prompt of the plurality of predefined verified prompt-response pairs.

[0028] In some implementations and in response to determining at least a threshold similarity between the prompt and the corresponding response and a predefined verified prompt-response pair, response verification process 10 provides 104 the corresponding response from the target generative Al model to a source of the prompt. For example, suppose that response verification process 10 determines that there is at least a threshold similarity (i.e., the comparison of response304 to a corresponding response from the plurality of predefined verified prompt-response pairs (e.g., corresponding response 312)). In this example, response verification process 10 indicates that response 304 is not a hallucination because response 304 is verified relative to corresponding response 312 from the plurality of predefined verified prompt-response pairs. Accordingly, response verification process 10 provides 104 response 304 generated by target generative Al model 300 to a source of the prompt (e.g., user 316). In one example, providing 104 response 304 to the source of prompt 302 includes directing or permitting generative Al model 300 to reply to prompt 302 with response 304. In another example, providing 104 response 304 to the source of prompt 302 includes providing response 304 using comparator sy stem 306 or another hardware / software component that interacts with generative Al model 300. In some implementations when providing 104 response 304 to the source of prompt 302. response verification process 10 provides a notice that response 304 has been processed for possible hallucinations. In this manner, a source of prompt 302 may have greater confidence that response 304 is based on verified information.

[0029] In some implementations and in response to determining less than at least the threshold similarity between the prompt and the corresponding response and any of the plurality of predefined verified prompt-response pairs, response verification process 10 provides 126 a default response from the target generative Al model. For example, suppose that response verification process 10 determines that there is not at least a threshold similarity (i.e., the comparison of response 304 to a corresponding response from the plurality’ of predefined verified promptresponse pairs (e.g., corresponding response 312)). In this example, response verification process 10 indicates that response 304 may be a hallucination because response 304 is not verified relative to the plurality of predefined verified prompt-response pairs. In some instances, response verification process 10 provides a default response (e.g., default response 318) to a source of the prompt (e g., user 316). For example, default response 318 includes a notice that response 304 has been processed for possible hallucinations and that response 304 cannot be verified relative to a ground truth database. In this manner, a source of prompt 302 may have an understanding that response 304 is not based on verified information. In another example, default response 318 includes a request for the source of prompt 302 to provide a new prompt or to access particular sources of information to assist the source in obtaining verified information.

[0030] In some implementations and in response to determining less than at least the threshold similarity between the prompt and the corresponding response and any of the plurality of predefined verified prompt-response pairs, response verification process 10 provides 128 a most similar predefined verified response from the plurality of predefined verified prompt-response pairs. For example, suppose that response verification process 10 determines that there is not at least a threshold similarity (i. e. , the comparison of response 304 to a corresponding response from the plurality of predefined verified prompt-response pairs (e.g., corresponding response 312)). In this example, response verification process 10 indicates that response 304 may be a hallucination because response 304 is not verified relative to the plurality of predefined verified promptresponse pairs. In some instances, response verification process 10 provides a most similar predefined verified response (e.g.. corresponding response 312) from the plurality of predefined verified prompt-response pairs to a source of the prompt (e.g., user 316). For example, response verification process 10 provides 128 most similar predefined verified response 312 and a notice that the initially-generated response 304 has been processed for possible hallucinations and that response 304 cannot be verified relative to a ground truth database but that response 312 represents the most similar response that is based on verified information. In this manner, a source of prompt 302 be provided with verified information even if it is not as relevant to prompt 302.User Feedback Response Verification:

[0031] In some implementations and in response to providing the corresponding response from the target generative Al model to the source of the prompt, response verification process 10 processes 130 feedback concerning the corresponding response. For example and referring also to FIG. 4, suppose a source of prompt 302 (e.g., user 316) provides prompt 302 to a generative Al model (e.g., generative Al model 300) and response verification process 10 provides either a verified response (e.g., response 304), a default response (e.g., default response 318), or a most similar verified response (e.g., response 312). When the source of prompt 302 receives the response, response verification process 10 solicits and / or accepts feedback (e.g., feedback 400) concerning the response (e.g., response 304). In one example, feedback 400 is sentiment feedback (i.e., a selection of a positive response or a negative response; or a selection of a “like” or a “dislike”). In another example, feedback 400 is a user assertion of the accuracy of response 304(i.e., a selection of a user-based verification as “verified” / “accurate” / “true”, or “unverified” / “inaccurate” / “false”). However, as user-feedback may be inaccurate or vulnerable to biases, response verification process 10 processes 130 feedback 400 concerning a generative Al model response (e.g., response 304 from generative Al model 300) to determine whether feedback 400 should be applied to subsequent training of generative Al model 300.

[0032] In some implementations, processing 130 the feedback includes determining that the feedback is one of positive feedback and negative feedback. For example, positive feedback is feedback that supports the accuracy or truthfulness of response 304 generated by generative Al model 300 and negative feedback is feedback that opposes the accuracy or truthfulness of response 304 generated by generative Al model 300. In some implementations, feedback 400 that is a “like” is positive feedback while a “dislike” is negative feedback.

[0033] In some implementations, response verification process 10 compares 132 the prompt and the corresponding response from the generative Al model to the plurality of predefined verified prompt-response pairs. For example and as discussed above, a predefined verified prompt-response pair is a combination of prompts / questions and generative Al model responses I answers that are verified by a SME and / or by response verification process 10 as described above. In this manner, predefined verified prompt-response pairs represent ground truth that can validate or verify responses generated by the target generative Al model. In some implementations, response verification process 10 provides prompt 302. corresponding response 304, and feedback 400 to a comparator system (e.g., comparator system 306). Comparator system 306 is a hardware and / or software component that processes prompt 302, corresponding response 304, and feedback 400 and compares 132 the combination of prompt 302, corresponding response 304, and feedback 400 to a plurality of predefined verified prompt-response pairs (e.g., predefined verified promptresponse pairs 232, 234, 236) to determine whether feedback 400 is applied to subsequent training of generative Al model 300 and / or to flag inaccurate user feedback.

[0034] In some implementations, comparing 132 the prompt and the response to the plurality of predefined verified prompt-response pairs includes generating 134 embeddings representative of the prompt. For example and as described above, an embedding or a vector embedding is a way to convert words and sentences and other data into numbers that capture their meaning andrelationships. In some implementations, response verification process 10 generates 134 a vector embedding representative of prompt 302. In one example, response verification process 10 generates 134 the vector embedding for prompt 302 using comparator system 306. In another example, response verification process 10 generates 134 the vector embedding for prompt 302 using a separate system / embedding system.

[0035] In some implementations, response verification process 10 performs 136 a vector similarity search for the prompt from the plurality of predefined verified prompt-response pairs using the embeddings representative of the prompt. For example and as described above, a vector similarity search includes identify ing similar data using approximate nearing neighbor (ANN) algorithms. In some implementations, response verification process 10 identifies 138 a threshold number of most similar prompts from the plurality of predefined verified prompt-response pairs. Referring again to FIG. 4, when performing 136 a vector similarity search for prompt 302, response verification process 10 identifies 138 a threshold number of most similar prompts (i.e., the top “N” most similar prompts) from a plurality of predefined verified prompt-response pairs (e.g., predefined verified prompt-response pairs 232, 234, 236). In one example, the threshold number is a default number (e.g., three). In another example, the threshold number is a user- defined value. In some implementations, “most similar"’ prompt is defined as a threshold level of similarity’ when performing vector similarity searching. This threshold may be a default value or a user-defined value.

[0036] As shown in FIG. 4, in response to performing 136 a vector similarity search for prompt 302 from predefined verified prompt-response pairs 232, 234, 236, response verification process 10 identifies 138 a threshold number of most similar prompts (e.g.. predefined verified prompt-response pair 308 with prompt 310). In this example, prompt 310 is the most similar prompt of predefined verified prompt-response pairs 232, 234, 236 when compared with prompt 302. In some implementations, if there is not a sufficiently similar prompt in the plurality of predefined verified prompt-response pairs, response verification process 10 may alert (e.g., by providing a pop up window or other electronic message) a source of prompt 302 that response verification process 10 is unable to perform hallucination prevention based on prompt 302.

[0037] In some implementations, response verification process 10 identifies 138 a thresholdnumber of most similar prompts from the plurality of predefined verified prompt-response pairs. In this example, prompt 310 is the most similar prompt of predefined verified prompt-response pairs 232, 234. 236 when compared with prompt 302. In some implementations, if there is not a sufficiently similar prompt in the plurality of predefined verified prompt-response pairs, response verification process 10 may alert (e.g., by providing a pop up window or other electronic message) a source of prompt 302 that response verification process 10 is unable to perform feedback verification based on prompt 302 and response 304.

[0038] In some implementations, response verification process 10 obtains 140 the corresponding responses for the threshold number of most similar prompts from the plurality of predefined verified prompt-response pairs. For example, with the most similar prompts (e.g., prompt 310), response verification process 10 obtains the corresponding responses (e.g., corresponding response 312). In some implementations, response verification process 10 generates 142 embeddings representative of each corresponding response for the threshold number of most similar prompts. As with prompt 302, response verification process 10 generates 142 embeddings representative of each corresponding response for the threshold number of most similar prompts by converting the words and sentences and other data of corresponding response 304 into numbers that capture their meaning and relationships.

[0039] In some implementations, response verification process 10 performs 144 a vector similarity search for the corresponding responses from the plurality of predefined verified promptresponse pairs using the embeddings representative of the prompt. For example, response verification process 10 compares the embedding representation of response 304 generated by target generative Al model 300 and the corresponding responses from the plurality of predefined verified prompt-response pairs (e.g., corresponding response 312). In some implementations, response verification process 10 uses a threshold level of similarity’ (e.g., threshold 314) to determine whether response 304 is sufficiently similar to a corresponding response from the plurality of predefined verified prompt-response pairs (e.g., corresponding response 312). When response 304 is not sufficiently similar (i.e., the level of similarity is below threshold 312), response 304 is indicative of a hallucination because it cannot be verified with a corresponding response from a sufficiently similar prompt of the plurality of predefined verified prompt-responsepairs. When response 304 is sufficiently similar (i.e.. the level of similarity is at or above threshold 312), response 304 is indicative of verified response because it can be verified with a corresponding response from a sufficiently similar prompt of the plurality of predefined verified prompt-response pairs.

[0040] In some implementations and in response to determining at least a threshold similarity between the prompt and the corresponding response and a predefined verified prompt-response pair, response verification process 10 applies 146 positive feedback to the target generative Al model. For example, suppose that response verification process 10 determines that there is at least a threshold similarity' (i.e., the comparison of response 304 to a corresponding response from the plurality of predefined verified prompt-response pairs (e.g., corresponding response 312)). In this example, response verification process 10 indicates that response 304 is not a hallucination because response 304 is verified relative to corresponding response 312 from the plurality of predefined verified prompt-response pairs. Accordingly, the content of response 304 is accurate or truthful as determined by the ground truth database formed from the plurality of predefined verified prompt-response pairs. In some implementations, with response 304 being verified by response verification process 10, feedback 400 should ideally be positive (i.e., indicating that the user confirms the information to be accurate or truthful). In one example, suppose that a user (e.g., user 316) provides positive feedback 400 concerning response 304. Further suppose that response verification process 10 determines that response 304 is verified by prompt-response pair 308. Accordingly, because feedback 400 is positive and response verification process 10 determines that response 304 is verified, response verification process 10 applies 146 feedback 400 to generative Al model 300. In some implementations, applying 146 feedback 400 to generative Al model 300 includes using feedback 400 in subsequent training or tuning of generative Al model 300. For example, during training or tuning of generative Al model 300, positive feedback 400 is used to improve the likelihood that generative Al model 300 generates response 304 for prompts similar to prompt 302.

[0041] In some implementations and in response to determining at least the threshold similarity between the prompt and the corresponding response and a predefined verified promptresponse pair, response verification process 10 prevents 148 negative feedback from being appliedto the generative Al model. For example, suppose that response verification process 10 determines that there is not at least a threshold similarity (i.e., the comparison of response 304 to a corresponding response from the plurality of predefined verified prompt-response pairs (e.g., corresponding response 312)). In this example, response verification process 10 indicates that response 304 is a hallucination because response 304 is not verified relative to corresponding response 312 from the plurality of predefined verified prompt-response pairs. Accordingly, the content of response 304 is accurate or truthful as determined by the ground truth database formed from the plurality of predefined verified prompt-response pairs. In some implementations, with response 304 being verified by response verification process 10, feedback 400 should ideally be positive (i.e., indicating that the user confirms the information to be accurate or truthful). In one example, suppose that a user (e.g.. user 316) provides negative feedback 400 concerning response 312. Further suppose that response verification process 10 determines that response 304 is verified by prompt-response pair 308. Accordingly, because feedback 400 is negative and response verification process 10 determines that response 304 is verified, response verification process 10 prevents 148 feedback 400 from being applied to generative Al model 300. In one example, preventing 148 feedback 400 from being applied to generative Al model 300 includes deleting feedback 400 from response verification process 10. In this manner, response verification process 10 identifies false negative feedback (e.g., false negative feedback 402) and prevents 148 false negative feedback 402 from being applied to generative Al model 300.

[0042] In some implementations and in response to determining less than at least the threshold similarity between the prompt and the corresponding response and any of the plurality of predefined verified prompt-response pairs, response verification process 10 prevents 150 positive feedback from being applied to the generative Al model. For example, suppose that response verification process 10 determines that there is not at least a threshold similarity (i.e., the comparison of response 304 to a corresponding response from the plurality of predefined verified prompt-response pairs (e.g., corresponding response 312)). In this example, response verification process 10 indicates that response 304 is not verified relative to any responses from the plurality of predefined verified prompt-response pairs. Accordingly, the content of response 304 is inaccurate or false as determined by the ground truth database formed from the plurality ofpredefined verified prompt-response pairs. In some implementations, with response 304 not being verified by response verification process 10, feedback 400 should ideally be negative (i.e., indicating that the user confirms the information to be inaccurate or false). In one example, suppose that a user (e.g., user 316) provides positive feedback 400 concerning response 304. Further suppose that response verification process 10 determines that response 304 is not verified by prompt-response pair 308. Accordingly, because feedback 400 is positive and response verification process 10 determines that response 304 is not verified, response verification process 10 prevents 150 feedback 400 from being applied to generative Al model 300. In this manner, response verification process 10 identifies false positive feedback (e.g., false positive feedback 404) and prevents 150 false positive feedback 404 from being applied to generative Al model 300.

[0043] In some implementations and in response to determining less than at least the threshold similarity between the prompt and the corresponding response and any of the plurality7of predefined verified prompt-response pairs, response verification process 10 applies 152 negative feedback to the target generative Al model. For example, suppose that response verification process 10 determines that there is not at least a threshold similarity7(i.e., the comparison of response 304 to a corresponding response from the plurality7of predefined verified promptresponse pairs (e.g., corresponding response 312)). In this example, response verification process 10 indicates that response 304 is not verified relative to any responses from the plurality of predefined verified prompt-response pairs. Accordingly, the content of response 304 is inaccurate or false as determined by the ground truth database formed from the plurality of predefined verified prompt-response pairs. In some implementations, with response 304 not being verified by response verification process 10, feedback 400 should ideally be negative (i.e., indicating that the user confirms the information to be inaccurate or false). In one example, suppose that a user (e.g., user 316) provides negative feedback 400 concerning response 304. Further suppose that response verification process 10 determines that response 304 is not verified by prompt-response pair 308. Accordingly, because feedback 400 is negative and response verification process 10 determines that response 304 is not verified, response verification process 10 applies 152 feedback 400 to generative Al model 300. In this manner, response verification process 10 applies 152 feedback 400 to generative Al model 300 to help train or tune generative Al model 300 to notgenerate response 304 for future prompts that are similar to prompt 302.System Overview:

[0044] Referring to FIG. 5, a response verification process 10 is shown to reside on and is executed by storage system 500, which is connected to network 502 (e.g., the Internet or a local area network). Examples of storage system 500 include: a Network Attached Storage (NAS) system, a Storage Area Network (SAN), a personal computer with a memory system, a server computer with a memory7system, and a cloud-based device with a memory system. A SAN includes one or more of a personal computer, a server computer, a series of server computers, a minicomputer, a mainframe computer, a RAID device, and a NAS system.

[0045] The various components of storage system 500 execute one or more operating systems, examples of which include: Microsoft® Windows®; Mac® OS X®; Red Hat® Linux®, Windows® Mobile, Chrome OS, Blackberry OS, Fire OS, or a custom operating system (Microsoft and Windows are registered trademarks of Microsoft Corporation in the United States, other countries or both: Mac and OS X are registered trademarks of Apple Inc. in the United States, other countries or both; Red Hat is a registered trademark of Red Hat Corporation in the United States, other countries or both; and Linux is a registered trademark of Linus Torvalds in the United States, other countries or both).

[0046] The instruction sets and subroutines of response verification process 10, which are stored on storage device 504 included within storage system 500. are executed by one or more processors (not shown) and one or more memory7architectures (not shown) included within storage system 500. Storage device 504 may include: a hard disk drive; an optical drive; a RAID device; a random-access memory (RAM); a read-only memory (ROM); and all forms of flash memory7storage devices. Additionally or alternatively, some portions of the instruction sets and subroutines of response verification process 10 are stored on storage devices (and / or executed by processors and memory architectures) that are external to storage system 500.

[0047] In some implementations, network 502 is connected to one or more secondary' networks (e.g., network 506), examples of which include: a local area network; a wide area network; or an intranet.

[0048] Various input / output (IO) requests (e.g., IO request 508) are sent from clientapplications 510. 512, 514, 516 to storage system 500. Examples of IO request 508 include data write requests (e.g., a request that content be written to storage system 500) and data read requests (e.g.. a request that content be read from storage system 500).

[0049] The instruction sets and subroutines of client applications 510, 512, 514, 516, which may be stored on storage devices 518, 520, 522, 524 (respectively) coupled to client electronic devices 526, 528, 530, 532 (respectively), may be executed by one or more processors (not shown) and one or more memory architectures (not shown) incorporated into client electronic devices 526, 528, 530, 532 (respectively). Storage devices 518, 520, 522, 524 may include: hard disk drives; tape drives; optical drives; RAID devices; random access memories (RAM); read-only memories (ROM), and all forms of flash memory storage devices. Examples of client electronic devices 526, 528, 530, 532 include personal computer 526, laptop computer 528, smartphone 530, laptop computer 532, a server (not shown), a data-enabled, and a dedicated network device (not shown). Client electronic devices 526, 528, 530, 532 each execute an operating system.

[0050] Users 534, 536. 538, 540 may access storage system 500 directly through network 502 or through secondary' network 506. Further, storage system 500 may be connected to netw ork 502 through secondary network 506, as illustrated with link line 542.

[0051] The various client electronic devices may be directly or indirectly coupled to network 502 (or network 506). For example, personal computer 526 is shown directly coupled to network 502 via a hardwired network connection. Further, laptop computer 532 is shown directly coupled to network 506 via a hardwired network connection. Laptop computer 528 is shown wirelessly coupled to network 502 via wireless communication channel 544 established between laptop computer 528 and wireless access point (e.g., WAP) 546. which is shown directly coupled to network 502. WAP 546 may be, for example, an IEEE 802.11a, 802.11b, 802.11g, 802. lln, WiFi®, and / or Bluetooth® device that is capable of establishing a wireless communication channel 544 between laptop computer 528 and WAP 546. Smartphone 530 is shown wirelessly coupled to network 502 via wireless communication channel 548 established between smartphone 530 and cellular network / bridge 550, which is shown directly coupled to network 502.General:

[0052] As will be appreciated by one skilled in the art, the present disclosure may be embodiedas a method, a system, or a computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a "circuit." “module” or “system.” Furthermore, the present disclosure may take the form of a computer program product on a computer-usable storage medium having computer-usable program code embodied in the medium.

[0053] Any suitable computer usable or computer readable medium may be used. The computer-usable or computer-readable medium may be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. More specific examples (a non-exhaustive list) of the computer- readable medium may include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a transmission media such as those supporting the Internet or an intranet, or a magnetic storage device. The computer-usable or computer-readable medium may also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory. In the context of this document, a computer-usable or computer-readable medium may be any medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-usable medium may include a propagated data signal with the computer-usable program code embodied therewith, either in baseband or as part of a carrier wave. The computer usable program code may be transmitted using any appropriate medium, including but not limited to the Internet, wireline, optical fiber cable, RF, etc.

[0054] Computer program code for carrying out operations of the present disclosure may be written in an object-oriented programming language. However, the computer program code for carrying out operations of the present disclosure may also be written in conventional proceduralprogramming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user’s computer, partly on the user’s computer, as a stand-alone software package, partly on the user’s computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user’s computer through a local area network / a wide area network / the Internet.

[0055] The present disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer / special purpose computer / other programmable data processing apparatus, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0056] These computer program instructions may also be stored in a computer-readable memory that may direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function / act specified in the flowchart and / or block diagram block or blocks.

[0057] The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0058] The flowcharts and block diagrams in the figures may illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computerprogram products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, not at all, or in any combination with any other flowcharts depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

[0059] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and / or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0060] The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present disclosure has been presented for purposes of illustration and description but is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. The embodiment was chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated.

[0061] A number of implementations have been described. Having thus described the disclosure of the present application in detail and by reference to embodiments thereof, it will be apparent that modifications and variations are possible without departing from the scope of the disclosure defined in the appended claims.

Claims

Claims1. A computer-implemented method, executed on a computing device, comprising: processing a prompt for a target generative Al model and a corresponding response generated by the target generative Al model for the prompt; comparing the prompt and the corresponding response from the generative Al model to a plurality’ of predefined verified prompt-response pairs; and in response to determining at least a threshold similarity between the prompt and the corresponding response and a predefined verified prompt-response pair, providing the corresponding response from the target generative Al model to a source of the prompt.

2. The computer-implemented method of claim 1, further comprising: in response to providing the corresponding response from the target generative Al model to the source of the prompt, processing feedback concerning the corresponding response; comparing the prompt and the corresponding response from the generative Al model to the plurality’ of predefined verified prompt-response pairs; and in response to determining at least a threshold similarity between the prompt and the corresponding response and a predefined verified prompt-response pair, applying the feedback to the target generative Al model.

3. The computer-implemented method of claim 2, in response to determining at least the threshold similarity between the correspond response and the prompt and a predefined verified prompt-response pair from the plurality of predefined verified prompt-response pairs, preventing negative feedback from being applied to the target generative Al model.

4. The computer-implemented method of claim 1, wherein comparing the prompt and the corresponding response to the plurality of predefined verified prompt-response pairs includes: generating embeddings representative of the prompt; performing a vector similarity search for the prompt from the plurality of predefined verified prompt-response pairs using the embeddings representative of the prompt; identify ing a threshold number of most similar prompts from the plurality ofpredefined verified prompt-response pairs; obtaining the corresponding responses for the threshold number of most similar prompts from the plurality of predefined verified prompt-response pairs; generating embeddings representative of each corresponding response for the threshold number of most similar prompts; and performing a vector similarity search for the corresponding responses from the plurality7of predefined verified prompt-response pairs using the embeddings representative of the prompt.

5. The computer-implemented method of claim 1, further comprising; in response to determining less than at least the threshold similarity between the prompt and the corresponding response and any of the plurality7of predefined verified prompt-response pairs, providing a default response from the target generative Al model.

6. The computer-implemented method of claim 1, further comprising: in response to determining less than at least the threshold similarity between the prompt and the corresponding response and any of the plurality7of predefined verified prompt-response pairs, providing a most similar predefined verified response from the plurality of predefined verified prompt-response pairs.

7. The computer-implemented method of claim 1, further comprising: generating the plurality of predefined verified prompt- response pairs by: extracting each paragraph of content from a verified document; generating a prompt for each extracted paragraph; and generating a corresponding response to each prompt by processing the prompt and a corresponding extracted paragraph.

8. A computing system comprising: a memory; and a processor configured to process feedback concerning a response generated by a target generative Al model for a prompt, to compare the response from the generative Al model and the prompt to a plurality of predefined verified prompt-response pairs, and, in response to determining at least a threshold similarity7between the response and the promptand a predefined verified prompt-response pair from a plurality of predefined verified prompt-response pairs, to apply positive feedback to the target generative Al model.

9. The computing system of claim 8, wherein the processor is further configured to: in response to determining at least the threshold similarity between the response and the prompt and a predefined verified prompt-response pair from a plurality of predefined verified prompt-response pairs, preventing negative feedback from being applied to the generative Al model.

10. The computing system of claim 8, wherein the processor is further configured to: process the prompt for the target generative Al model and the response generated by the target generative Al model for the prompt; compare the prompt and the response from the generative Al model to the plurality of predefined verified prompt-response pairs; and in response to determining at least a threshold similarity between the prompt and the response and a predefined verified prompt-response pair, provide the response from the target generative Al model to a source of the prompt.

11. The computing system of claim 9, in response to determining less than at least the threshold similarity between the prompt and the corresponding response and any of the plurality of predefined verified prompt-response pairs, providing a default response from the target generative Al model.

12. The computing system of claim 8, wherein comparing the prompt and the response to the plurality of predefined verified prompt-response pairs includes: generating embeddings representative of the prompt; performing a vector similarity' search for the prompt from the plurality of predefined verified prompt-response pairs using the embeddings representative of the prompt; identifying a threshold number of most similar prompts from the plurality of predefined verified prompt-response pairs; obtaining the corresponding responses for the threshold number of most similar prompts from the plurality of predefined verified prompt-response pairs;generating embeddings representative of each corresponding response for the threshold number of most similar prompts; and performing a vector similarity search for the corresponding responses from the plurality of predefined verified prompt-response pairs using the embeddings representative of the prompt.

13. The computing system of claim 8, wherein the processor is further configured to: in response to determining less than at least the threshold similarity between the prompt and the corresponding response and any of the plurality of predefined verified prompt-response pairs, preventing positive feedback from being applied to the generative Al model.

14. The computing system of claim 8, wherein the processor is further configured to: in response to determining less than at least the threshold similarity between the prompt and the corresponding response and any of the plurality of predefined verified prompt-response pairs, applying negative feedback to the target generative Al model.

15. A computer program product residing on a computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising: processing a prompt for a target generative Al model and a corresponding response generated by the target generative Al model for the prompt; comparing the prompt and the corresponding response from the generative Al model to a plurality of predefined verified prompt-response pairs; in response to determining at least a threshold similarity between the prompt and the corresponding response and a predefined verified prompt-response pair, providing the corresponding response from the target generative Al model to a source of the prompt; processing feedback concerning the corresponding response; comparing the prompt and the corresponding response from the generative Al model to the plurality of predefined verified prompt-response pairs; and in response to determining at least a threshold similarity between the prompt and the corresponding response and a predefined verified prompt-response pair, applying the feedback to the target generative Al model.

Images