System and method for proactively reducing hallusion in generative artificial intelligence (AI) model responses

Abstract

A method, computer program product, and computing system for processing a cue for a target generative AI model and a corresponding response generated by the target generative AI model for the cue. The cues and corresponding responses from the generative AI model are compared to a plurality of predefined validated cue-response pairs. In response to determining that there is at least a threshold similarity between the cue and the corresponding response and the predefined validated cue-response pair, a source of the cue is provided with the corresponding response from the target generative AI model.

Description

Background Technology

[0001] Generative AI models are being used in customer support services (CSS) and other fields to solve various problems and streamline data processing from diverse sources, improving the efficiency and accuracy of customer support by automating repetitive and simple tasks and providing consistent responses. However, illusions in the responses of generative AI models can lead to presenting users with information that is actually incorrect or meaningless. Despite advances in natural language processing, this challenge remains due to the complexity of language understanding and generation. Additionally, current methods rely on post-processing to correct for illusions that have already appeared in the responses provided to users. Attached Figure Description

[0002] Figures 1A-1B This is a flowchart illustrating one implementation method of the response verification process;

[0003] Figure 2 It generates predefined, validated prompt-response pairs. Figures 1A-1B A schematic view of the response verification process;

[0004] Figure 3 It is for validating the response of generative AI models. Figures 1A-1B A schematic view of the response verification process; and

[0005] Figure 4 It is to validate user feedback regarding the response of generative AI models. Figures 1A-1B A schematic view of the response verification process; and

[0006] Figure 5 It is a schematic view of a computer system coupled to a distributed computing network and the response verification process.

[0007] The same reference numerals in the various figures indicate the same elements. Detailed Implementation

[0008] The implementation disclosed herein provides a two-point comparator model that improves the accuracy of generative AI model responses by actively reducing illusions. For example, generative AI models, including those in Natural Language Processing (NLP), have made progress in understanding and generating human-like text. However, these generative AI models still face certain challenges, one of which is "illusion." Illusion refers to the phenomenon where a generative AI model generates responses containing fictitious or incorrect information without any real basis in the input data or context. The manifestation of illusion can vary in severity, ranging from minor factual errors to generating entire paragraphs of imagined content. In some cases, generative AI models may confidently present illusory information, leading users to believe in the validity of the generated text. This phenomenon can be problematic, especially in critical areas where misinformation can have serious consequences.

[0009] To combat the illusion problem, the two-point comparator model disclosed herein involves a multi-stage process using predefined, validated cue-response pairs generated by generative AI models(s). As will be discussed in more detail below, the first stage validates the response by processing the generative AI model response with predefined validated cue-response pairs before displaying the response to the user. In this way, the generated AI model response is validated before being presented to the user who made the cue, and illusions are filtered out from the response. In the second stage, user feedback is validated by processing with predefined validated cue-response pairs before invoking a subject matter expert (SME) to validate / correct biased or incorrect user feedback responses. In this way, user feedback is filtered out using predefined validated cue-response pairs before being applied to the training or tuning of the generative AI model.

[0010] As will be described in more detail below, the implementation of this disclosure processes a cue for a target generative AI model and a corresponding response generated by the target generative AI model for that cue. The cue and the corresponding response from the generative AI model are compared with a plurality of predefined, validated cue-response pairs. In response to determining that there is at least a threshold similarity between the cue and the corresponding response and the predefined validated cue-response pairs, the corresponding response from the target generative AI model is provided to the source of the cue.

[0011] As will be described in more detail below, implementations of this disclosure provide a process for automatically and proactively identifying and removing illusions generated by a generative AI model, and improving the accuracy of provided responses by removing incorrect or factually erroneous information. This process involves converting each cue into an embedded vector, converting each corresponding generative AI model response into a vector embedding, and performing vector similarity calculations between the embedded representations of the cue and the corresponding response and predefined verified cue-response pairs. When the vector similarity between the embedded representations of the cue and the corresponding response and the predefined verified cue-response pairs is greater than a threshold, the corresponding response is verified and provided to the source of the cue. In some implementations, this method improves the accuracy of generative AI model responses and prevents illusions in a scalable manner, enabling real-time monitoring of any number (i.e., millions) of incorrect responses.

[0012] Details of one or more implementations are set forth in the accompanying drawings and the description below. Other features and advantages will become apparent from the description, drawings, and claims. Response verification process:

[0013] Reference Figures 1A-4 The response verification process 10 performs processing 100, which includes a prompt for the target generative AI model and a corresponding response generated by the target generative AI model for the prompt. The prompt and the corresponding response from the generative AI model are compared with multiple predefined verified prompt-response pairs 102. In response to determining that there is at least a threshold similarity between the prompt and the corresponding response and the predefined verified prompt-response pairs, the corresponding response from the target generative AI model is provided to the source of the prompt.

[0014] In some implementations, the response verification process 10 proactively reduces illusions found in responses generated by the generative AI model when processing prompts. For example, the AI ​​model uses neural networks to identify patterns and structures within a dataset to perform specific tasks (e.g., converting speech to text, generating new data (e.g., generative AI models), identifying biometric profiles within multiple biometric profiles, solving complex mathematical problems, etc.). In some implementations, the generative AI model (e.g., generative AI model 200) is configured to receive natural language prompts and / or example entries and / or contextual information about the request used to generate the response. In some implementations, the AI ​​model includes a Large Language Model (LLM). An LLM (e.g., Bing Chat from Microsoft®, GPT-4 from OpenAI®, and Bard from Google®) is a language model consisting of neural networks with many parameters (typically billions or more weights), trained on large amounts of unlabeled text using self-supervised or semi-supervised learning. Although trained on simple tasks such as predicting the next word in a sentence, LLMs with sufficient training and parameter counts capture the syntax and semantics of human language. Furthermore, LLMs exhibit considerable common sense and are able to “memorize” a large number of facts during training.

[0015] However, because LLM and generative AI models learn patterns from different sources of training data that may not include the solution prompts and / or because the prompts may be incomplete or ambiguous, these models may produce “illusory” responses without a direct source. Instead, these illusory responses are generated using content from the source most likely (e.g., based on probabilities and / or weights within the generative AI model) to solve the prompt. Accordingly, illusory responses pose a serious problem for users who rely on generative AI models to provide factual information (e.g., for safety instructions, instructions on how to assemble a product, instructions on medical care, information on how to solve an emergency, etc.). Using response verification process 10, illusory responses are identified and filtered out before being presented to the source of the prompt. In this way, response verification process 10 automatically (i.e., without human intervention) and proactively (i.e., in real-time and before the response is provided to the user) prevents illusory responses from being provided to the user. Generate predefined, validated hint-response pairs:

[0016] In some implementations, the response verification process 10 generates 106 predefined verified cue-response pairs through the following steps: performing extraction 108 from the verified document to extract each paragraph of the content; generating 110 cue for each extracted paragraph using a generative AI model; and generating 112 corresponding responses for each cue by processing the cue and the corresponding extracted paragraph using the generative AI model. For example, in some implementations, the response verification process 10 uses multiple predefined verified cue-response pairs as “ground truths,” comparing the cue with the corresponding responses generated by the generative AI model to determine whether the responses generated by the target generative AI model are hallucinations. Conventional methods for defining ground truths for generative AI models include relying on subject matter experts (SMEs) to define or identify facts or other content that accurately describes the information. However, this method cannot scale to accommodate the rate at which new content is provided to generative AI models. For example, as more content becomes available to generative AI models, the ability to identify ground truths is limited by the SME’s capacity to process that content. Thus, the response verification process 10 addresses this limitation by using (multiple) generative AI models to transform content into hints and corresponding responses that define baseline truth values. In this way, the response verification process 10 uses (multiple) generative AI models to automatically generate predefined, verified hint-response pairs.

[0017] In some implementations, the response validation process 10 generates more than 106 predefined validated prompt-response pairs by extracting each paragraph of 108 content from the validated document. See also... Figure 2The response verification process 10 retrieves a document (e.g., document 200). In one example, document 200 is a portion of pre-verified content. Pre-verified content typically includes information that has been verified by subject matter experts (SMEs) for its accuracy or factual basis. In some implementations, document 200 is stored in a verified content database (e.g., verified content database 202). The response verification process 10 extracts each paragraph of the content 108 (e.g., paragraphs 204, 206, 208) and generates unique identifiers for each document and each paragraph (e.g., document and paragraph identifiers 210, 212, 214). In some implementations, extracting each paragraph 108 includes performing data cleaning. In one example, data cleaning includes using natural language processing (NLP) techniques to remove irrelevant punctuation, characters, and fix formatting issues. In another example, data cleaning does not include performing lemmatization or stemming on the document to avoid loss of contextual information. In some implementations, paragraph-level extraction is particularly useful for generative AI models and information retrieval systems when the response requires broader context and a single sentence may be insufficient. By selecting entire paragraphs, the response validation process 10 can provide users with more comprehensive and coherent responses, thereby improving the user experience and increasing the amount of information conveyed in the interaction. In one example, paragraph-level extraction is performed by parsing breaks (e.g., "\n") or specific tags (e.g., <\p>) in an HTML document. Each extracted paragraph is represented by a paragraph identifier (e.g., "Para-Id"), and it is stored together with its associated document identifier (e.g., "Doc-Id"). In some implementations, the content of each paragraph is vectorized using a word embedding model. Therefore, each extracted paragraph can be defined as a vector of Para-id, paragraph content, Doc-id, and paragraph embeddings.

[0018] In some implementations, the response verification process 10 uses a generative AI model to generate 112 hints for each extracted paragraph. For example, the response verification process 10 feeds each paragraph (e.g., paragraphs 204, 206, 208) to a generative AI model (e.g., generative AI model 216) to generate questions / hints (e.g., hints 218, 220, 222), such as a T5 (text-to-text), BART (bidirectional autoregressive transformer model), or Davinci model. In some implementations, each generated question / hint is stored along with its document identifier and its paragraph identifier. In one example, the response verification process 10 generates a vector or other identifier formed from a combination of hint identifiers, document identifiers, and paragraph identifiers (e.g., Ques ID, DocID, Para ID). In some implementations, and as described below, each hint (e.g., hints 218, 220, 222) is vectorized into an embedding used to search for answers / responses from the paragraphs via a vector similarity metric.

[0019] In some implementations, the response verification process 10 generates 112 a corresponding response for each prompt by processing the prompts and corresponding extracted paragraphs. For example, the response verification process 10 provides prompts (e.g., prompts 218, 220, 222) along with their content (e.g., by paragraph identifiers and document identifiers) to a generative AI model (e.g., generative AI model 224) to generate prompts. In one example, generative AI model 224 is the same as generative AI model 216 used to generate prompts 218, 220, 222. In another example, generative AI model 224 is a different generative AI model from generative AI model 216. Using generative AI model 224, the response verification process 10 generates corresponding responses (e.g., corresponding responses 226, 228, 230) for prompts 218, 220, 222.

[0020] In some implementations, and for each prompt, the response verification process 10 performs a vector similarity search on each prompt embedding and paragraph embedding. In some implementations, this results in obtaining the “top N” answers / responses representing paragraphs ranked based on their relevance to the question / prompt. The obtained head paragraph embeddings indicate the content within the paragraph relevant to constructing the answer / response. In one example, the response verification process 10 performs a vector similarity metric to obtain head answers / responses 226, 228, and 230 for prompts 218, 220, and 222. Accordingly, the response verification process 10 controls the quality of responses 226, 228, and 230 by adding a threshold to the similarity score, such that if the similarity score is greater than the threshold, the response verification process 10 only saves that answer in a plurality of predefined verified prompt-response pairs. In one example, the threshold for the similarity score is high enough to achieve high accuracy for the generated answers / responses. In this example, other parameters (such as temperature and head probability) are set to “0” and “1”, respectively. In this manner, the response verification process 10 generates multiple predefined verified prompt-response pairs (e.g., predefined verified prompt-response pairs 232, 234, and 236). In one example, the predefined verified prompt-response pairs 232, 234, and 236 are stored in a database of predefined verified prompt-response pairs (e.g., a predefined verified prompt-response pair database 238). In some implementations, the predefined verified prompt-response pair database 238 includes multiple predefined verified prompt-response pairs in the following formats: (Ques-Id, Question, Question Embedded, Ans-Id, Answer, Answer Embedded, Para-Id, Doc-Id). As will be discussed in more detail below, multiple predefined verified prompt-response pairs are automatically generated without human intervention and can be used to proactively identify and remove illusory responses presented to the user. Generative AI model response validation:

[0021] In some implementations, the response verification process 10 performs processing 100, including a cue for the target generative AI model and a corresponding response generated by the target generative AI model in response to the cue. For example, the response verification process 10 provides an active method for verifying the generative AI model's response to illusion in real time by comparing the response generated by the target generative AI model with multiple predefined verified cue-response pairs as described above. See also... Figure 3A target generative AI model (e.g., target generative AI model 300) is a generative AI model that receives a prompt from a user (or other source) to elicit a response. In this way, target generative AI model 300 is a generative AI model with which response verification process 10 interacts to identify and limit illusory responses from it. In some implementations, target generative AI model 300 processes prompts (e.g., prompt 302) to generate a corresponding response (e.g., corresponding response 304).

[0022] In some implementations, the response verification process 10 compares the prompt and the corresponding response from the generative AI model with multiple predefined verified prompt-response pairs 102. As described above, the predefined verified prompt-response pairs are combinations of prompts / questions and generative AI model responses / answers verified by the SME and / or response verification process 10 as described above. In this way, the predefined verified prompt-response pairs represent a baseline truth value that can confirm or verify the response generated by the target generative AI model. See again Figure 3 Furthermore, in some implementations, the response verification process 10 provides a prompt 302 and a corresponding response 304 to a comparator system (e.g., comparator system 306). The comparator system 306 is a hardware and / or software component that processes the prompt 302 and the corresponding response 304 and compares the combination of the prompt 302 and the corresponding response 304 with a plurality of predefined verified prompt-response pairs (e.g., predefined verified prompt-response pairs 232, 234, 236) 402. In some implementations, the comparator system 306 is also referred to as a "content conditioner" or "data tuner" because the response verification process 10 is capable of providing data enrichment.

[0023] In some implementations, comparing the cue and its corresponding response from the generative AI model with multiple predefined, validated cue-response pairs 102 includes generating 114 embeddings representing the cue. For example, embeddings, or vector embeddings, are ways to transform words and sentences, as well as other data, into numerical representations that characterize their meaning and relationships. They represent different data types as points in a multidimensional space, where similar data points are clustered together more closely. These numerical representations help machines understand and process the data more effectively. In some implementations, the response validation process 10 generates 114 vector embeddings representing the cue 302. In one example, the response validation process 10 uses a comparator system 306 to generate 114 vector embeddings for the cue 302. In another example, the response validation process 10 uses a separate system / embedding system to generate 114 vector embeddings for the cue 302.

[0024] In some implementations, the response verification process 10 uses embeddings representing the prompts to perform a 116-vector similarity search on prompts from multiple predefined verified prompt-response pairs. For example, the vector similarity search includes using an Approximate Nearest Neighbor (ANN) algorithm to identify similar data. Compared to traditional keyword search, vector search produces more relevant results and executes faster. In some implementations, the response verification process 10 identifies the most similar prompts for a threshold number of 118 from multiple predefined verified prompt-response pairs. See again... Figure 3 When performing a vector similarity search for cue 302, the response verification process 10 identifies the most similar cue (i.e., the top "N" most similar cue) from a plurality of predefined verified cue-response pairs (e.g., predefined verified cue-response pairs 232, 234, 236) with a threshold number of 118. In one example, the threshold number is a default number (e.g., 3). In another example, the threshold number is a user-defined value. In some implementations, when performing a vector similarity search, the "most similar" cue is defined as one whose similarity reaches a threshold level. This threshold can be a default value or a user-defined value.

[0025] like Figure 3 As shown, in response to performing a 116-vector similarity search on cue 302 from predefined verified cue-response pairs 232, 234, 236, response verification process 10 identifies the most similar cue (e.g., a predefined verified cue-response pair 308 with cue 310) by a threshold number of 118. In this example, cue 310 is the most similar cue among the predefined verified cue-response pairs 232, 234, 236 when compared to cue 302. In some implementations, if there are not sufficiently similar cue among the multiple predefined verified cue-response pairs, response verification process 10 may alert (e.g., by providing a pop-up window or other electronic message) the source of cue 302: response verification process 10 cannot perform hallucination prevention based on cue 302. In some implementations, the source of cue 302 can be selectively enabled (i.e., selected when to enable and when to disable) for response verification process 10 to perform verification of responses from the target generative AI model. In this way, users can determine when they need a non-illusion response and when they are searching for creative content (i.e., content not based on verified information).

[0026] In some implementations, the response verification process 10 obtains 120 corresponding responses to the most similar prompts for a threshold number from a plurality of predefined verified prompt-response pairs. For example, using the most similar prompt (e.g., prompt 310), the response verification process 10 obtains 120 corresponding responses (e.g., corresponding responses 312). In some implementations, the response verification process 10 generates an embedding of 122 representing each corresponding response for the most similar prompts for the threshold number. Similar to prompt 302, the response verification process 10 generates the embedding of the corresponding response 304 by converting the words and sentences of the corresponding response 304, along with other data, into numbers that capture its meaning and relationships.

[0027] In some implementations, the response verification process 10 uses embeddings representing the cue to perform a 124-vector similarity search against corresponding responses from multiple predefined verified cue-response pairs. For example, the response verification process 10 compares the embedding representation of response 304 generated by the target generative AI model 300 with corresponding responses from multiple predefined verified cue-response pairs (e.g., corresponding response 312). In some implementations, the response verification process 10 uses a similarity threshold level (e.g., threshold 314) to determine whether response 304 is sufficiently similar to corresponding responses from multiple predefined verified cue-response pairs (e.g., corresponding response 312). When response 304 is not sufficiently similar (i.e., the similarity level is below threshold 314), response 304 indicates a hallucination because it cannot be verified using corresponding responses to sufficiently similar cue responses from multiple predefined verified cue-response pairs. When responses 304 are sufficiently similar (i.e., the similarity level is at or above the threshold 314), response 304 indicates a validated response because it can be validated using the corresponding responses of sufficiently similar prompts from multiple predefined validated prompt-response pairs.

[0028] In some implementations, and in response to determining that there is at least a threshold similarity between the prompt and its corresponding response and predefined verified prompt-response pairs, the response verification process 10 provides the source of the prompt with 104 corresponding responses from the target generative AI model. For example, suppose the response verification process 10 determines that there is at least a threshold similarity (i.e., a comparison of response 304 with corresponding responses from multiple predefined verified prompt-response pairs (e.g., corresponding response 312)). In this example, the response verification process 10 indicates that response 304 is not a hallucination because response 304 is verified relative to corresponding responses 312 from multiple predefined verified prompt-response pairs. Accordingly, the response verification process 10 provides the source of the prompt (e.g., user 316) with 104 responses 304 generated by the target generative AI model 300. In one example, providing 104 responses 304 to the source of prompt 302 includes guiding or allowing the generative AI model 300 to respond to prompt 302 using response 304. In another example, providing response 304 to the source of cue 302 includes using comparator system 306 or another hardware / software component that interacts with generative AI model 300 to provide response 304. In some implementations, when response 304 is provided to the source of cue 302, response verification process 10 provides notification that response 304 has been processed for possible hallucinations. In this way, the source of cue 302 can have greater confidence in response 304 based on verified information.

[0029] In some implementations, and in response to determining a similarity of at least less than a threshold between the prompt and its corresponding response and any of a plurality of predefined verified prompt-response pairs, response verification process 10 provides 126 default responses from the target generative AI model. For example, suppose response verification process 10 determines that there is no similarity of at least the threshold (i.e., a comparison of response 304 with corresponding responses from a plurality of predefined verified prompt-response pairs (e.g., corresponding response 312)). In this example, response verification process 10 indicates that response 304 may be a hallucination because response 304 is not verified relative to the plurality of predefined verified prompt-response pairs. In some examples, response verification process 10 provides a default response (e.g., default response 318) to the source of the prompt (e.g., user 316). For example, default response 318 includes notification that response 304 has been processed for possible hallucinations and that response 304 cannot be verified relative to a benchmark truth database. In this way, the source of prompt 302 can understand that response 304 is not based on verified information. In another example, the default response 318 includes a request to the source of prompt 302 to provide new prompts or access specific information to help that source obtain verified information.

[0030] In some implementations, in response to determining a similarity of at least less than a threshold between the prompt and its corresponding response and any of a plurality of predefined verified prompt-response pairs, response verification process 10 provides 128 of the most similar predefined verified responses from the plurality of predefined verified prompt-response pairs. For example, suppose response verification process 10 determines that there is no similarity of at least the threshold (i.e., a comparison of response 304 with the corresponding response from the plurality of predefined verified prompt-response pairs (e.g., corresponding response 312)). In this example, response verification process 10 indicates that response 304 may be a hallucination because response 304 is not verified relative to the plurality of predefined verified prompt-response pairs. In some instances, response verification process 10 provides the source of the prompt (e.g., user 316) with the most similar predefined verified response from the plurality of predefined verified prompt-response pairs (e.g., corresponding response 312). For example, response verification process 10 provides 128 most similar predefined verification responses 312 and a notification that: the initially generated response 304 has been processed for possible illusions and that response 304 cannot be verified relative to the benchmark truth database, but response 312 represents the most similar response based on verified information. In this way, verified information is provided to the source of prompt 302 even if the verified information is not so relevant to prompt 302. User feedback response verification:

[0031] In some implementations, and in response to providing a corresponding response from the target generative AI model to the source of the prompt, the response verification process 10 processes 130 feedback regarding the corresponding response. For example, and also referring to... Figure 4Suppose that the source of cue 302 (e.g., user 316) provides cue 302 to a generative AI model (e.g., generative AI model 300), and the response verification process 10 provides a verified response (e.g., response 304), a default response (e.g., default response 318), or the most similar verified response (e.g., response 312). When the source of cue 302 receives a response, the response verification process 10 solicits and / or accepts feedback (e.g., feedback 400) regarding that response (e.g., response 304). In one example, feedback 400 is sentiment feedback (i.e., a choice between a positive or negative response; or a choice between "like" or "dislike"). In another example, feedback 400 is a user assertion of the accuracy of response 304 (i.e., based on the user's verification choice as "verified" / "accurate" / "true" or "unverified" / "inaccurate" / "false"). However, since user feedback may be inaccurate or susceptible to bias, the response verification process 10 processes 130 feedback 400 on the generative AI model response (e.g., response 304 from the generative AI model 300) to determine whether the feedback 400 should be applied to the subsequent training of the generative AI model 300.

[0032] In some implementations, handling feedback 130 includes determining whether the feedback is positive or negative. For example, positive feedback is feedback that supports the accuracy or realism of the response 304 generated by the generative AI model 300, while negative feedback is feedback that opposes the accuracy or realism of the response 304 generated by the generative AI model 300. In some implementations, "like" feedback 400 is positive feedback, while "dislike" is negative feedback.

[0033] In some implementations, the response verification process 10 compares the prompt and the corresponding response from the generative AI model with multiple predefined verified prompt-response pairs 132. For example, and as described above, the predefined verified prompt-response pairs are combinations of prompts / questions and generative AI model responses / answers verified by the SME and / or response verification process 10 as described above. In this way, the predefined verified prompt-response pairs represent a baseline truth value that can confirm or verify the response generated by the target generative AI model. In some implementations, the response verification process 10 provides a prompt 302, a corresponding response 304, and feedback 400 to a comparator system (e.g., comparator system 306). The comparator system 306 is a hardware and / or software component that processes the cue 302, the corresponding response 304, and the feedback 400 and compares the combination of the cue 302, the corresponding response 304, and the feedback 400 with a plurality of predefined, validated cue-response pairs (e.g., predefined validated cue-response pairs 232, 234, 236) to determine whether the feedback 400 is applied to the subsequent training of the generative AI model 300 and / or to label inaccurate user feedback.

[0034] In some implementations, comparing the cue and response with multiple predefined, validated cue-response pairs involves generating a 134-bit embedding representing the cue. For example, and as described above, an embedding, or vector embedding, is a way of transforming words and sentences, as well as other data, into numbers that capture their meaning and relationships. In some implementations, the response validation process 10 generates a 134-bit vector embedding representing the cue 302. In one example, the response validation process 10 uses a comparator system 306 to generate a 134-bit vector embedding for the cue 302. In another example, the response validation process 10 uses a separate system / embedding system to generate a 134-bit vector embedding for the cue 302.

[0035] In some implementations, the response verification process 10 performs a 136-vector similarity search on hints from multiple predefined verified hint-response pairs using embeddings representing the hints. For example, and as described above, the vector similarity search includes using an Approximate Nearest Neighbor (ANN) algorithm to identify similar data. In some implementations, the response verification process 10 identifies the most similar hint from multiple predefined verified hint-response pairs with a threshold number of 138. See again... Figure 4When a vector similarity search is performed for cue 302, the response verification process 10 identifies the most similar cue (i.e., the top "N" most similar cue) from a plurality of predefined verified cue-response pairs (e.g., predefined verified cue-response pairs 232, 234, 236) up to a threshold number of 138. In one example, the threshold number is a default number (e.g., 3). In another example, the threshold number is a user-defined value. In some implementations, when a vector similarity search is performed, the "most similar" cue is defined as a threshold level of similarity. This threshold can be a default value or a user-defined value.

[0036] like Figure 4 As shown, in response to performing a 136-vector similarity search on cue 302 from predefined verified cue-response pairs 232, 234, 236, response verification process 10 identifies the most similar cue (e.g., a predefined verified cue-response pair 308 with cue 310) by a threshold number of 138. In this example, cue 310 is the most similar cue among the predefined verified cue-response pairs 232, 234, 236 when compared to cue 302. In some implementations, if there are not sufficiently similar cue among the multiple predefined verified cue-response pairs, response verification process 10 may warn (e.g., by providing a pop-up window or other electronic message) the source of cue 302: response verification process 10 cannot perform hallucination prevention based on cue 302.

[0037] In some implementations, the response verification process 10 identifies the most similar prompt among a threshold number of predefined verified prompt-response pairs 138. In this example, prompt 310 is the most similar prompt among the predefined verified prompt-response pairs 232, 234, and 236 when compared to prompt 302. In some implementations, if there are not enough similar prompts among the predefined verified prompt-response pairs, the response verification process 10 may warn (e.g., by providing a pop-up window or other electronic message) the source of prompt 302: the response verification process 10 cannot perform feedback verification based on prompt 302 and response 304.

[0038] In some implementations, the response verification process 10 obtains 140 corresponding responses to the most similar prompts for a threshold number from a plurality of predefined verified prompt-response pairs. For example, using the most similar prompt (e.g., prompt 310), the response verification process 10 obtains corresponding responses (e.g., corresponding response 312). In some implementations, the response verification process 10 generates 142 embeddings representing each corresponding response to the most similar prompts for a threshold number. Similar to prompt 302, the response verification process 10 generates 142 embeddings representing each corresponding response to the most similar prompts for a threshold number by converting the words and sentences of the corresponding response 304, along with other data, into numbers that capture their meaning and relationships.

[0039] In some implementations, the response verification process 10 performs a 144-vector similarity search against corresponding responses from multiple predefined verified cue-response pairs using embeddings representing the cue. For example, the response verification process 10 compares the embedding representation of response 304 generated by the target generative AI model 300 with corresponding responses from multiple predefined verified cue-response pairs (e.g., corresponding response 312). In some implementations, the response verification process 10 uses a similarity threshold level (e.g., threshold 314) to determine whether response 304 is sufficiently similar to corresponding responses from multiple predefined verified cue-response pairs (e.g., corresponding response 312). When response 304 is not sufficiently similar (i.e., the similarity level is below threshold 312), response 304 indicates a hallucination because it cannot be verified using corresponding responses to cue that are sufficiently similar from multiple predefined verified cue-response pairs. When responses 304 are sufficiently similar (i.e., the similarity level is at or above the threshold 312), response 304 indicates a validated response because it can be validated by the corresponding response of a sufficiently similar cue from multiple predefined validated cue-response pairs.

[0040] In some implementations, and in response to determining at least a threshold similarity between the prompt and its corresponding response and predefined verified prompt-response pairs, the response verification process 10 applies positive feedback 146 to the target generative AI model. For example, suppose the response verification process 10 determines that at least a threshold similarity exists (i.e., a comparison of response 304 with corresponding responses from multiple predefined verified prompt-response pairs (e.g., corresponding response 312)). In this example, the response verification process 10 indicates that response 304 is not a hallucination because response 304 is verified relative to corresponding response 312 from multiple predefined verified prompt-response pairs. Accordingly, the content of response 304 is accurate or true, as determined by a benchmark truth database formed based on multiple predefined verified prompt-response pairs. In some implementations, when response 304 is verified by the response verification process 10, feedback 400 should ideally be positive feedback (i.e., indicating to the user that the information is accurate or true). In one example, suppose a user (e.g., user 316) provides positive feedback 400 regarding response 304. It is also assumed that response verification process 10 determines that response 304 is verified by cue-response pair 308. Accordingly, since feedback 400 is positive and response verification process 10 determines that response 304 is verified, response verification process 10 applies feedback 400 146 to generative AI model 300. In some implementations, applying feedback 400 146 to generative AI model 300 includes using feedback 400 in subsequent training or tuning of generative AI model 300. For example, during training or tuning of generative AI model 300, positive feedback 400 is used to increase the likelihood that generative AI model 300 generates response 304 for cue similar to cue 302.

[0041] In some implementations, and in response to determining that there is at least a threshold similarity between the prompt and its corresponding response and predefined verified prompt-response pairs, the response verification process 10 prevents negative feedback 148 from being applied to the generative AI model. For example, suppose the response verification process 10 determines that there is no at least a threshold similarity (i.e., a comparison of response 304 with corresponding responses from multiple predefined verified prompt-response pairs (e.g., corresponding response 312)). In this example, the response verification process 10 indicates that response 304 is an illusion because response 304 is not verified relative to corresponding response 312 from multiple predefined verified prompt-response pairs. Accordingly, the content of response 304 is accurate or true, as determined by a benchmark truth database formed based on multiple predefined verified prompt-response pairs. In some implementations, when response 304 is verified by the response verification process 10, feedback 400 should ideally be positive feedback (i.e., instructing the user to confirm that the information is accurate or true). In one example, suppose a user (e.g., user 316) provides negative feedback 400 regarding response 312. It is also assumed that response verification process 10 determines that response 304 is verified by prompt-response pair 308. Accordingly, since feedback 400 is negative and response verification process 10 determines that response 304 is verified, response verification process 10 prevents feedback 400 from being applied to generative AI model 300. In one example, preventing feedback 400 from being applied to generative AI model 300 includes removing feedback 400 from response verification process 10. In this way, response verification process 10 identifies false negative feedback (e.g., false negative feedback 402) and prevents false negative feedback 402 from being applied to generative AI model 300.

[0042] In some implementations, in response to determining that the prompt and corresponding response have at least a threshold similarity with any of a plurality of predefined verified prompt-response pairs, the response verification process 10 prevents 150 positive feedback from being applied to the generative AI model. For example, suppose the response verification process 10 determines that there is no at least threshold similarity (i.e., a comparison of response 304 with a corresponding response from a plurality of predefined verified prompt-response pairs (e.g., corresponding response 312)). In this example, the response verification process 10 indicates that response 304 has not been verified relative to any response from the plurality of predefined verified prompt-response pairs. Accordingly, the content of response 304 is inaccurate or erroneous, as determined by a benchmark truth database formed based on the plurality of predefined verified prompt-response pairs. In some implementations, in the case that response 304 has not been verified by the response verification process 10, feedback 400 should ideally be negative feedback (i.e., instructing the user to confirm that the information is inaccurate or erroneous). In one example, suppose a user (e.g., user 316) provides positive feedback 400 regarding response 304. It is also assumed that response verification process 10 determines that response 304 was not verified by prompt-response pair 308. Accordingly, since feedback 400 is positive and response verification process 10 determines that response 304 was not verified, response verification process 10 prevents feedback 400 from being applied to generative AI model 300. In this way, response verification process 10 identifies false positive feedback (e.g., false positive feedback 404) and prevents false positive feedback 404 from being applied to generative AI model 300.

[0043] In some implementations, in response to determining a similarity of at least less than a threshold between the prompt and its corresponding response and any of a plurality of predefined verified prompt-response pairs, the response verification process 10 applies negative feedback 152 to the target generative AI model. For example, suppose the response verification process 10 determines that there is no similarity of at least the threshold (i.e., a comparison of response 304 with corresponding responses from a plurality of predefined verified prompt-response pairs (e.g., corresponding response 312)). In this example, the response verification process 10 indicates that response 304 has not been verified relative to any response from the plurality of predefined verified prompt-response pairs. Accordingly, the content of response 304 is inaccurate or erroneous, as determined by a benchmark truth database formed based on the plurality of predefined verified prompt-response pairs. In some implementations, in the case where response 304 is not verified by the response verification process 10, feedback 400 should ideally be negative feedback (i.e., instructing the user to acknowledge that the information is inaccurate or erroneous). In one example, suppose a user (e.g., user 316) provides negative feedback 400 regarding response 304. It is also assumed that response verification process 10 determines that response 304 is not verified by prompt-response pair 308. Accordingly, since feedback 400 is negative and response verification process 10 determines that response 304 is not verified, response verification process 10 applies feedback 400 152 to the generative AI model. In this way, response verification process 10 applies feedback 400 152 to the generative AI model 300 to help train or tune the generative AI model 300 so as not to generate response 304 for future prompts similar to prompt 302. System Overview:

[0044] refer to Figure 5 The response verification process 10 is shown residing on and executed by storage system 500, which is connected to network 502 (e.g., the Internet or a local area network). Examples of storage system 500 include: network attached storage (NAS) systems, storage area networks (SANs), personal computers with storage systems, server computers with storage systems, and cloud-based devices with storage systems. SANs include personal computers, server computers, a series of server computers, minicomputers, mainframes, RAID devices, and one or more NAS systems.

[0045] Various components of the storage system 500 execute one or more operating systems, examples of which include: Microsoft® Windows®; Mac® OS X®; Red Hat® Linux®, Windows® Mobile, Chrome OS, Blackberry OS, Fire OS, or custom operating systems (Microsoft and Windows are registered trademarks of Microsoft Corporation in the U.S., other countries, or both; Mac and OS X are registered trademarks of Apple Inc. in the U.S., other countries, or both; Red Hat is a registered trademark of Red Hat Corporation in the U.S., other countries, or both; Linux is a registered trademark of Linus Torvalds in the U.S., other countries, or both).

[0046] The instruction set and subroutines of the response verification process 10, stored on storage device 504 included within storage system 500, are executed by one or more processors (not shown) and one or more memory architectures (not shown) included within storage system 500. Storage device 504 may include: hard disk drives; optical drives; RAID devices; random access memory (RAM); read-only memory (ROM); and all forms of flash memory. Additionally or alternatively, some portions of the instruction set and subroutines of the response verification process 10 may be stored on storage devices outside storage system 500 (and / or executed by processors and memory architectures).

[0047] In some implementations, network 502 is connected to one or more secondary networks (e.g., network 506), examples of which include: local area network; wide area network; or intranet.

[0048] Various input / output (IO) requests (e.g., IO request 508) are sent from client applications 510, 512, 514, and 516 to storage system 500. Examples of IO requests 508 include data write requests (e.g., requests to write content to storage system 500) and data read requests (e.g., requests to read content from storage system 500).

[0049] Instruction sets and subroutines of client applications 510, 512, 514, and 516, which may be stored on storage devices 518, 520, 522, and 524 coupled to client electronic devices 526, 528, 530, and 532, may be executed by one or more processors (not shown) and one or more memory architectures (not shown) coupled to client electronic devices 526, 528, 530, and 532, respectively. Storage devices 518, 520, 522, and 524 may include: hard disk drives; tape drives; optical drives; RAID devices; random access memory (RAM); read-only memory (ROM); and all forms of flash memory. Examples of client electronic devices 526, 528, 530, and 532 include personal computers 526, laptop computers 528, smartphones 530 and 532, servers (not shown), data-enabled devices, and dedicated network devices (not shown). Each client electronic device 526, 528, 530, and 532 executes an operating system.

[0050] Users 534, 536, 538, and 540 can access storage system 500 directly via network 502 or via secondary network 506. Additionally, storage system 500 can be connected to network 502 via secondary network 506, as illustrated by link 542.

[0051] Various client electronic devices can be directly or indirectly coupled to network 502 (or network 506). For example, personal computer 526 is shown as directly coupled to network 502 via a hardwired network connection. Furthermore, laptop computer 532 is shown as directly coupled to network 506 via a hardwired network connection. Laptop computer 528 is shown as wirelessly coupled to network 502 via a wireless communication channel 544 established between laptop computer 528 and a wireless access point (e.g., WAP) 546, which is shown as directly coupled to network 502. WAP 546 can be, for example, an IEEE 802.11a, 802.11b, 802.11g, 802.11n, Wi-Fi®, and / or Bluetooth® device capable of establishing a wireless communication channel 544 between laptop computer 528 and WAP 546. Smartphone 530 is shown wirelessly coupled to network 502 via a wireless communication channel 548 established between smartphone 530 and cellular network / bridge 550, which is shown directly coupled to network 502. General instructions:

[0052] As will be understood by those skilled in the art, this disclosure can be embodied as a method, system, or computer program product. Accordingly, this disclosure can take the form of an entire hardware embodiment, an entire software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which may be collectively referred to herein as a "circuit," "module," or "system." Furthermore, this disclosure can take the form of a computer program product on a computer-usable storage medium having computer-usable program code contained therein.

[0053] Any suitable computer-usable or computer-readable medium may be used. A computer-usable or computer-readable medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or transmission medium. More specific examples of computer-readable media (a non-exhaustive list) may include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage devices, transmission media (such as those supporting the Internet or intranet), or magnetic storage devices. A computer-usable or computer-readable medium can also be paper or another suitable medium on which a program is printed, because the program can be electronically captured, for example, by optical scanning of paper or other media, then compiled, interpreted, or processed in a suitable manner (if necessary), and then stored in computer memory. In the context of this document, a computer-usable or computer-readable medium can be any medium that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Computer-usable media may include propagated data signals having computer-usable program code embodied therein, either in baseband or as a portion of a carrier wave. The computer-usable program code may be transmitted using any suitable medium, including but not limited to the Internet, wired lines, fiber optic cables, RF, etc.

[0054] Computer program code for performing the operations of this disclosure can be written in an object-oriented programming language. However, it can also be written in a conventional programming language, such as the "C" programming language or a similar programming language. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer via a local area network (LAN) / wide area network (WAN) / the Internet.

[0055] This disclosure has been described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer / special-purpose computer / other programmable data processing apparatus, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams.

[0056] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of writing comprising instruction means that implement the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0057] Computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus, thereby producing a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0058] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code comprising one or more executable instructions for implementing a specified logical function(s). It should also be noted that in some alternative implementations, the functions indicated in the blocks may not occur in the order shown in the drawings. For example, depending on the function involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order, not at all, or in any combination with any other flowchart. It should also be noted that each block illustrated in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a system based on dedicated hardware, or a combination of dedicated hardware and computer instructions, that performs the specified function or action.

[0059] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0060] All means or steps plus functional elements in the following claims are intended to include any structure, material, action, and equivalent for performing a function in combination with other claimed elements as specifically claimed. The description of this disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the disclosure in its form. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of this disclosure. The embodiments were chosen and described in order to best explain the principles and practical application of this disclosure and to enable others skilled in the art to understand the disclosure of various embodiments with various modifications suitable for the particular intended use.

[0061] Several implementations have been described. The disclosure of this application has been described in such detail that it will be apparent from reference to its embodiments that modifications and variations are possible without departing from the scope of the disclosure as defined in the appended claims.

Claims

1. A computer-implemented method executed on a computing device, comprising: Processing prompts for a target generative AI model and corresponding responses generated by the target generative AI model in response to the prompts; The prompt and the corresponding response from the generative AI model are compared with a plurality of predefined, validated prompt-response pairs; as well as In response to determining that there is at least a threshold similarity between the prompt and the corresponding response and a predefined verified prompt-response pair, the corresponding response from the target generative AI model is provided to the source of the prompt.

2. The computer-implemented method according to claim 1 further includes: In response to providing the corresponding response from the target generative AI model to the source of the prompt, feedback on the corresponding response is processed; The prompt and the corresponding response from the generative AI model are compared with the plurality of predefined, validated prompt-response pairs; as well as In response to determining that there is at least a threshold similarity between the prompt and the corresponding response and a predefined verified prompt-response pair, the feedback is applied to the target generative AI model.

3. The computer-implemented method of claim 2, in response to determining that there is at least the threshold similarity between the corresponding response and the prompt and a predefined verified prompt-response pair from the plurality of predefined verified prompt-response pairs, negative feedback is prevented from being applied to the target generative AI model.

4. The computer-implemented method of claim 1, wherein comparing the prompt and the corresponding response with the plurality of predefined verified prompt-response pairs comprises: Generate an embedding representing the prompt; Using the embedding representing the prompt, perform a vector similarity search for the prompt from the plurality of predefined, verified prompt-response pairs; From the plurality of predefined, validated prompt-response pairs, select the prompt that is most similar to the number of identifier thresholds; Obtain the corresponding response of the most similar prompt for the threshold number from the plurality of predefined verified prompt-response pairs; Generate an embedding representing each corresponding response to the most similar prompt for the threshold number; as well as Using the embedding representing the prompt, a vector similarity search is performed from the plurality of predefined, verified prompt-response pairs for the corresponding responses.

5. The computer-implemented method according to claim 1, further comprising: In response to determining that the similarity between the prompt and the corresponding response and any of the plurality of predefined verified prompt-response pairs is at least less than the threshold, a default response is provided from the target generative AI model.

6. The computer-implemented method according to claim 1, further comprising: In response to determining that the similarity between the prompt and the corresponding response and any of the plurality of predefined verified prompt-response pairs is at least less than the threshold, the most similar predefined verified response from the plurality of predefined verified prompt-response pairs is provided.

7. The computer-implemented method according to claim 1, further comprising: The plurality of predefined, verified prompt-response pairs are generated in the following manner: Extract each paragraph of content from a verified document; Generate suggestions for each extracted paragraph; as well as A corresponding response is generated for each prompt by processing the prompts and the corresponding extracted paragraphs.

8. A computing system, comprising: Memory; as well as A processor configured to: process feedback regarding responses generated by a target generative AI model in response to prompts; The responses and prompts from the generative AI model are compared with a plurality of predefined, validated prompt-response pairs; Furthermore, in response to determining that there is at least a threshold similarity between the response and the prompt and a predefined verified prompt-response pair from a plurality of predefined verified prompt-response pairs, positive feedback is applied to the target generative AI model.

9. The computing system of claim 8, wherein the processor is further configured to: In response to determining that the response and the prompt have at least the threshold similarity with a predefined verified prompt-response pair from a plurality of predefined verified prompt-response pairs, negative feedback is prevented from being applied to the generative AI model.

10. The computing system of claim 8, wherein the processor is further configured to: Processing the prompts for the target generative AI model and the responses generated by the target generative AI model in response to the prompts; The prompts and responses from the generative AI model are compared with the plurality of predefined, validated prompt-response pairs; and In response to determining that there is at least a threshold similarity between the prompt and the response and a predefined verified prompt-response pair, the response from the target generative AI model is provided to the source of the prompt.

11. The computing system of claim 9, in response to determining that the similarity between the prompt and the corresponding response and any of the plurality of predefined verified prompt-response pairs is at least less than the threshold, a default response is provided from the target generative AI model.

12. The computing system of claim 8, wherein comparing the prompt and the corresponding response with the plurality of predefined verified prompt-response pairs comprises: Generate an embedding representing the prompt; Using the embedding representing the prompt, perform a vector similarity search for the prompt from the plurality of predefined, verified prompt-response pairs; From the plurality of predefined, validated prompt-response pairs, select the prompt that is most similar to the number of identifier thresholds; Obtain the corresponding response of the most similar prompt for the threshold number from the plurality of predefined verified prompt-response pairs; Generate an embedding representing each corresponding response to the most similar prompt for the threshold number; as well as Using the embedding representing the prompt, a vector similarity search is performed from the plurality of predefined, verified prompt-response pairs for the corresponding responses.

13. The computing system of claim 8, wherein the processor is further configured to: In response to determining that the similarity between the prompt and the corresponding response and any of the plurality of predefined verified prompt-response pairs is at least less than the threshold, positive feedback is prevented from being applied to the generative AI model.

14. The computing system of claim 8, wherein the processor is further configured to: In response to determining that the similarity between the prompt and the corresponding response and any of the plurality of predefined verified prompt-response pairs is at least less than the threshold, negative feedback is applied to the target generative AI model.

15. A computer program product residing on a computer-readable medium, the computer-readable medium storing a plurality of instructions, which, when executed by a processor, cause the processor to perform operations, the operations including: Processing prompts for a target generative AI model and corresponding responses generated by the target generative AI model in response to the prompts; The prompt and the corresponding response from the generative AI model are compared with a plurality of predefined, validated prompt-response pairs; In response to determining that there is at least a threshold similarity between the prompt and the corresponding response and a predefined verified prompt-response pair, the corresponding response from the target generative AI model is provided to the source of the prompt; Process feedback regarding the corresponding response; The prompt and the corresponding response from the generative AI model are compared with the plurality of predefined, validated prompt-response pairs; as well as In response to determining that there is at least a threshold similarity between the prompt and the corresponding response and a predefined verified prompt-response pair, the feedback is applied to the target generative AI model.

Images