Risk-based trust scoring and operationalization framework for generative artificial intelligence models

Abstract

A method for providing a Generative Artificial Intelligence (GenAI) system with trustworthiness evaluation, wherein the method comprises processing a training dataset that the GenAI was trained upon; constructing a plurality of latent knowledge anchors (LKAs) by applying topic modeling on the processed training dataset, wherein the plurality of LKAs comprises domain-specific knowledge representations within the processed training dataset; training a neural network classifier using the LKAs and the processed training data; evaluating, using the neural network classifier, a response generated by the GenAI to generate a trust score, wherein the trust score indicates the alignment level of the response to the training dataset and / or the comprehensiveness of the response compared to the training dataset; in response to the trust score exceeding a threshold, presenting the response to a user; and in response to the trust score not exceeding the threshold, issuing a command for the GenAI to conduct additional actions.

Description

Attorney Docket No.: 035006-872001 WO Customer Number: 76615 RISK-BASED TRUST SCORING AND OPERATIONALIZATION FRAMEWORK FOR GENERATIVE ARTIFICIAL INTELLIGENCE MODELS CROSS-REFERENCE

[0001] This application claims priority to U.S. Patent Applications 63 / 756,711, filed February 10, 2025, and U.S. Patent Application No. 19 / 078,999, filed March 13, 2025, the contents of which are fully incorporated by reference.TECHNICAL FIELD

[0002] The present disclosure relates to machine learning-based trust evaluation systems, and more specifically, to systems and methods for assessing the reliability of responses generated by Generative Artificial Intelligence (GenAI) models.BACKGROUND

[0003] Generative Artificial Intelligence (GenAI) models, such as large language models (LLMs), have gained significant adoption across various domains, including natural language processing, automated content generation, and complex reasoning tasks. These models are trained on vast datasets and utilize probabilistic techniques to generate human-like text based on input prompts. While their capabilities are impressive, concerns remain regarding the trustworthiness and reliability of their outputs.

[0004] A primary challenge associated with GenAI systems is their tendency to generate responses that may appear contextually relevant but lack factual accuracy. These inaccuracies, often referred to as hallucinations, can mislead users, particularly in applications requiring precise and verifiable information, such as legal, financial, or medical domains. Due to the non-deterministic nature of these models, two seemingly similar prompts can yield significantly different outputs, further complicating the reliability of their responses.

[0005] Additionally, GenAI models are inherently opaque, making it difficult for users to assess whether a given response aligns with the model’s training knowledge or if it is influenced by adversarial manipulations, or data sparsity issues. The inability to interpret or validate the reasoning behind generated responses limits the widespread adoption of these models in mission-critical applications.Attorney Docket No.: 035006-872001 WO Customer Number: 76615

[0006] Existing approaches attempt to mitigate these concerns through human-in-the-loop validation, heuristic-based filtering, or model fine-tuning. However, these methods are either labor-intensive, prone to inconsistencies, or fail to provide real-time assessments of response reliability. As a result, there is a growing need for systematic mechanisms that can evaluate the trustworthiness of Gen Al- generated responses, ensuring that only reliable and meaningful outputs are allowed to be presented to a user.SUMMARY

[0007] Methods, systems, and articles of manufacture, including computer program products, are provided for providing a Generative Artificial Intelligence (GenAI) system with trustworthiness evaluation, wherein the method comprises processing a training dataset that the GenAI was trained upon; constructing a plurality of latent knowledge anchors (LKAs) by applying topic modeling on the processed training dataset, wherein the plurality of LKAs comprises domain- specific knowledge representations within the processed training dataset; training a neural network classifier using the LKAs and the processed training data; evaluating, using the neural network classifier, a response generated by the GenAI to generate a trust score, wherein the trust score indicates the alignment level of the response to the training dataset and / or the comprehensiveness of the response compared to the training dataset; in response to the trust score exceeding a dynamically defined threshold, presenting the response to a user; and in response to the trust score not exceeding the threshold, issuing a command for the GenAI to conduct additional actions.

[0008] In some variations, the additional actions comprise generating an additional response, wherein generating an additional response comprises issuing a prompt to the GenAI to refine or supplement the response.

[0009] In some variations, training the neural network classifier comprises expanding the training dataset with out-of-domain samples to introduce negative example; generating LKA vectors for in-domain samples and out-of-domain training datasets; and training the neural network classifier using the in-domain and out-of-domain dataset, wherein the classifier is optimized to maximize a separation between in-domain and out-of-domain responses.Attorney Docket No.: 035006-872001 WO Customer Number: 76615

[0010] In some variations, the out-of-domain training samples are assigned to the neural network classifier’s training dataset based on their rank-ordered level of relevance to a pre-defined in-domain data definitions.

[0011] In some variations, the method further comprises: extracting N-gram sequences from the processed training dataset; calculating a frequency of occurrence of each N-gram within the training dataset to establish an N-gram statistical baseline; and generating an N-gram coverage statistics metric associated with the training dataset, wherein the N-gram coverage statistics metric is utilized in calculating the trust score.

[0012] In some variations, the method further comprises: converting the response to N-gram representation and LKA vector representations; generating, using the trained classifier, a first score indicating the alignment level of the response with the in-domain LKA space; calculating a second score based on the N-gram representation of the response and the N-gram coverage statistics metric associated with the training dataset; and generating the trust score based on the first score and the second score.

[0013] In some variations, the topic modeling for constructing the LKAs comprises applying latent Dirichlet allocation (LDA) topic modeling to the processed training dataset.

[0014] In some variations, the processing the training dataset comprises: performing text preprocessing on the training dataset, wherein the text preprocessing includes tokenization, lemmatization, and stopword removal.

[0015] In an aspect, provided herein is a computer program product for providing a Generative Artificial Intelligence (GenAI) system with trustworthiness evaluation, comprising a non-transient machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations including processing a training dataset that the GenAI was trained upon; constructing a plurality of latent knowledge anchors (LKAs) by applying topic modeling on the processed training dataset, wherein the plurality of LKAs comprises domain- specific knowledge representations within the processed training dataset; training a neural network classifier using the LKAs and the processed training data; evaluating, using the neural network classifier, a response generated by the GenAI to generate a trust score, wherein the trust score indicates the alignment level of the response to the training dataset and / or theAttorney Docket No.: 035006-872001 WO Customer Number: 76615 comprehensiveness of the response compared to the training dataset; in response to the trust score exceeding a dynamically defined threshold, presenting the response to a user; and in response to the trust score not exceeding the threshold, issuing a command for the GenAI to conduct additional actions.

[0016] In some variations, the additional actions comprise generating an additional response, wherein generating an additional response comprises issuing a prompt to the GenAI to refine or supplement the response.

[0017] In some variations, training the neural network classifier comprises expanding the training dataset with out-of-domain samples to introduce negative example; generating LKA vectors for in-domain samples and out-of-domain training datasets; and training the neural network classifier using the in-domain and out-of-domain dataset, wherein the classifier is optimized to maximize a separation between in-domain and out-of-domain responses.

[0018] In some variations, the out-of-domain training samples are assigned to the neural network classifier’s training dataset based on their rank-ordered level of relevance to a pre-defined in-domain data definitions.

[0019] In some variations, the operations further include extracting N-gram sequences from the processed training dataset; calculating a frequency of occurrence of each N-gram within the training dataset to establish an N-gram statistical baseline; and generating an N-gram coverage statistics metric associated with the training dataset, wherein the N-gram coverage statistics metric is utilized in calculating the trust score.

[0020] In some variations, the operations further includes converting the response to N-gram representation and LKA vector representations; generating, using the trained classifier, a first score indicating the alignment level of the response with the in-domain LKA space; calculating a second score based on the N-gram representation of the response and the N-gram coverage statistics metric associated with the training dataset; and generating the trust score based on the first score and the second score.

[0021] In some variations, the topic modeling for constructing the LKAs comprises applying latent Dirichlet allocation (LDA) topic modeling to the processed training dataset.Attorney Docket No.: 035006-872001 WO Customer Number: 76615

[0022] In some variations, the processing the training dataset comprises: performing text preprocessing on the training dataset, wherein the text preprocessing includes tokenization, lemmatization, and stopword removal.

[0023] In an aspect, provided herein is a system for providing a Generative Artificial Intelligence (GenAI) system with trustworthiness evaluation comprising at least one programmable processor; and a non-transient machine-readable medium storing instructions that, when executed by the processor, cause the at least one programmable processor to perform operations including processing a training dataset that the GenAI was trained upon; constructing a plurality of latent knowledge anchors (LKAs) by applying topic modeling on the processed training dataset, wherein the plurality of LKAs comprises domain-specific knowledge representations within the processed training dataset; training a neural network classifier using the LKAs and the processed training data; evaluating, using the neural network classifier, a response generated by the GenAI to generate a trust score, wherein the trust score indicates the alignment level of the response to the training dataset and / or the comprehensiveness of the response compared to the training dataset; in response to the trust score exceeding a dynamically defined threshold, presenting the response to a user; and in response to the trust score not exceeding the threshold, issuing a command for the GenAI to conduct additional actions.

[0024] In some variations, the additional actions comprise generating an additional response, wherein generating an additional response comprises issuing a prompt to the GenAI to refine or supplement the response.

[0025] In some variations, training the neural network classifier comprises expanding the training dataset with out-of-domain samples to introduce negative example; generating LKA vectors for in-domain samples and out-of-domain training datasets; and training the neural network classifier using the in-domain and out-of-domain dataset, wherein the classifier is optimized to maximize a separation between in-domain and out-of-domain responses. The in-domain samples may be part of the in-domain corpus.

[0026] In some variations, the out-of-domain training samples are assigned to the neural network classifier’s training dataset based on their rank-ordered level of relevance to a pre-defined in-domain data definitions.Attorney Docket No.: 035006-872001 WO Customer Number: 76615

[0027] In some variations, the operations further include extracting N-gram sequences from the processed training dataset; calculating a frequency of occurrence of each N-gram within the training dataset to establish an N-gram statistical baseline; and generating an N-gram coverage statistics metric associated with the training dataset, wherein the N-gram coverage statistics metric is utilized in calculating the trust score.

[0028] In some variations, the operations further includes converting the response to N-gram representation and LKA vector representations; generating, using the trained classifier, a first score indicating the alignment level of the response with the in-domain LKA space; calculating a second score based on the N-gram representation of the response and the N-gram coverage statistics metric associated with the training dataset; and generating the trust score based on the first score and the second score.

[0029] In some variations, the topic modeling for constructing the LKAs comprises applying latent Dirichlet allocation (LDA) topic modeling to the processed training dataset.

[0030] Implementations of the current subject matter can include, but are not limited to, methods consistent with the descriptions provided herein as well as articles that include a tangibly embodied machine-readable medium operable to cause one or more machines (e.g., computers, etc.) to result in operations implementing one or more of the described features. Similarly, computer systems are also described that may include one or more processors and one or more memories coupled to the one or more processors.

[0031] A memory, which can include a computer-readable storage medium, may include, encode, store, or the like one or more programs that cause one or more processors to perform one or more of the operations described herein. Computer implemented methods consistent with one or more implementations of the current subject matter can be implemented by one or more data processors residing in a single computing system or multiple computing systems. Such multiple computing systems can be connected and can exchange data and / or commands or other instructions or the like via one or more connections, including but not limited to a connection over a network (e.g. the Internet, a wireless wide area network, a local area network, a wide area network, a wired network, or the like), via a direct connection between one or more of the multiple computing systems, etc.Attorney Docket No.: 035006-872001 WO Customer Number: 76615

[0032] The details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features and advantages of the subject matter described herein will be apparent from the description and drawings, and from the claims. The claims that follow this disclosure are intended to define the scope of the protected subject matter.DESCRIPTION OF DRAWINGS

[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, show certain aspects of the subject matter disclosed herein and, together with the description, help explain some of the principles associated with the disclosed implementations. In the drawings,

[0034] FIG. 1 is a diagram illustrating an exemplary architecture of a GenAI system with a trustworthiness evaluation model, in accordance with one or more embodiments of the current subject matter referred to as the trust score risk model.

[0035] FIG. 2 illustrates the trust score risk model’s pipeline for independent verification of a generative artificial intelligence (GenAI) model’s responses, in accordance with one or more embodiments of the current subject matter.

[0036] FIG. 3 illustrates the probability density function of trust scores derived from documents categorized as category 1 and category 5, in accordance with one or more embodiments of the current subject matter.

[0037] FIG. 4 depicts a block diagram illustrating a computing system consistent with implementations of the current subject matter.

[0038] FIG. 5 is a diagram illustrating a flowchart of a process 500 for evaluating trustworthiness in a Generative Artificial Intelligence (GenAI) system, in accordance with one or more embodiments of the current subject matter.

[0039] When practical, like labels are used to refer to same or similar items in the drawings.Attorney Docket No.: 035006-872001 WO Customer Number: 76615 DETAILED DESCRIPTION

[0040] The details of one or more variations of the subject matter described herein are set forth in the accompanying drawings.

[0041] The disclosure relates to assessing the reliability of responses generated by a GenAI model. Aspects relate to optimizing an assessment (e.g., optimizing determination of reliability of an output generated by a GenAI system) by performing heavy computational tasks earlier (e.g., during a Modeling Stage). In this manner, during a later Use Stage, a trust score and the related operations and calculations can be performed, with minimal latency, relying on the earlier performed operations.

[0042] During the Modeling Stage, core modeling operations and statistical baselines (e.g., building a bounded vocabulary, generating Latent Knowledge Anchors (LKAs) via LDA topic modeling, and precomputing complex coverage and summarization statistics) are defined, built, calculated, and stored for future access.

[0043] During the Use Stage, the system avoids redoing time and resource intensive modeling operations and instead relies on lightweight operations, such as mapping the response to small LKA vectors and performing fast database lookups to access pre-stored values generated during the Modeling Stage. Trust assessment time is further minimized by calculating the LKA alignment score (via quick neural network inference) and the N-gram coverage score (via statistical retrieval) in parallel.

[0044] The modeling stage may be performed prior to receiving a response generated by a GenAI system for evaluation. The use stage may be performed after the modeling stage, in response to receiving a response generated by a GenAI system for evaluation.

[0045] Accordingly, the reliability of the GenAI system’s output (i.e., the steps needed for generating the final score) can be determined efficiently and quickly (i.e., substantially faster than the GenAI’ s text generation operations).

[0046] Modeling Stage:

[0047] The system first pre-processes an in-domain training corpus (training documents that are relevant to a specific topic) by:Attorney Docket No.: 035006-872001 WO Customer Number: 76615 • Extracting POS-filtered N-grams (e.g., bigrams) and converting them into a numeric TF-IDF representation, and building a bounded vocabulary (V) of size S. The point of bounding the vocabulary is to keep the linguistic feature set controlled and efficient so the model focuses on domain-relevant phrases.• Using that bounded vocabulary, the system trains a topic model (e.g., a latent Dirichlet allocation (EDA)) to discover T hidden topics / latent knowledge anchors (EKAs).• Converting each training document into a compact T-dimensional LKA vector (e.g., 10 dimensions).

[0048] Accordingly, the system does the heavier statistical work ahead of time (i.e., prior to generating the results or scoring the results) and stores the statistical analytic data needed for scoring, for fast access, once the generative results are generated (or while the results are being generated):• Each N-gram in the in-domain vocabulary is assigned a coverage statistic (how often that phrase appears across the training corpus).• Those coverage values are stored in a database.• For an LKA, the system identifies a predefined number of the most relevant N-grams (e.g., selected based on LDA weights), retrieves their coverage values, and computes summarization statistics (e.g., mean and standard deviation) for the LKA. These are stored in a database for later evaluation.

[0049] Use Stage:

[0050] At the time of use, when a GenAI model produces an output, the system does not need to build / rebuild the domain model. Instead the system uses what was already learned and stored. The GenAI generated response is converted into an N-gram representation and mapped to an LKA vector. The N-gram representation and LKA vector processing may take place in parallel to reduce latency:(1) an LKA alignment score may be produced by applying a trained classifier (e.g., a neural network) to the LKA vector in inference mode (a quick scoring pass), andAttorney Docket No.: 035006-872001 WO Customer Number: 76615 (2) an N-gram coverage score may be produced, using the previously calculated and stored statistical data, by finding the N-grams found in the generated GenAI output that already exist in the in-domain vocabulary, retrieving their stored coverage values, computing a mean coverage for the response, and retrieving the relevant LKA’s stored mean / standard deviation to standardize that coverage (e.g., z-score / capping / calibration).(3) The above component scores (LKA & N-gram) may be combined into a single trust score, which reflects the trustworthiness / reliability of the results generated.

[0051] This implementation optimizes trust scoring because expensive (time consuming, compute resource intensive) steps (e.g., LKAs via LDA, defining the bounded vocabulary, and computing coverage / summarization statistics) are performed earlier in time (e.g., offline). For example, LDA operations require iterative probabilistic inference across an extremely large textual database (corpus) to calculate topic-word distributions. The iterative complexity vastly increases with the number of documents, topics, and vocabulary size and becomes significantly more complex when dealing with the high topic density found in generative outputs.

[0052] As another example, generating a bounded vocabulary and computing real-time statistics add substantial processing overhead by requiring continuous, high-dimensional matrix operations and tracking as the data stream is generated by the GenAI model (i.e., token by token). Managing and comparing large vector representations for every token in a highdimensional semantic space requires significant memory and CPU / GPU resources that would substantially increase the total latency associated with generating an output and verifying GenAI output reliability.

[0053] If some or all of the above time and resource-intensive operations are performed during the modeling stage, at Use Stage (e.g., when the GenAI output is being generated) the evaluation of the output is quickly performed by mapping the generated GenAI output to small LKA vectors and using N-grams to retrieve pre-stored statistical data, based on which the two component scores can be quickly determined. In this manner, calculations and determinations are performed during the Modeling Stage and the results are stored and reused via fast database lookups and lightweight arithmetic during verification at the Use Stage.Attorney Docket No.: 035006-872001 WO Customer Number: 76615

[0054] The above approach involving performing the time consuming and resource intensive computation earlier allows for the GenAI output to be verified for reliability substantially faster and with lower utilization of resources. In certain embodiments, the two component scores may be computed in parallel. This additionally reduces compute time. In this way, the trust / reliability determination pipeline is optimized for real-time / large- scale deployments (i.e., the trust score inference latency is designed to be lower than response generation time).

[0055] As discussed above, there is a need for a systematic and automated approach to evaluate the trustworthiness of responses generated by Generative Artificial Intelligence (GenAI) models. Given the challenges of hallucinations, data biases, and non-deterministic outputs, existing solutions remain inadequate in providing real-time, scalable, and interpretable trust assessments. A robust framework is required to assess the reliability of GenAI responses dynamically, ensuring that only high-confidence outputs are presented to users while mitigating the risks associated with uncertain or misleading content.

[0056] The present disclosure addresses these challenges by introducing a trust evaluation mechanism that enables the automated assessment of GenAI-generated responses. The disclosed system provides a structured approach to determine the degree to which a response aligns with the model’s training knowledge and its statistical coverage of relevant domain- specific information. By implementing a trust scoring framework, referred to as the trust score risk model in FIG. 1, the system facilitates the filtering, refinement, and validation of generated responses before they are presented to end users. The following sections describe the technical components and methodologies underlying this approach.

[0057] FIG. 1 is a diagram illustrating an exemplary architecture of a GenAI system 100 with a trustworthiness evaluation model, in accordance with one or more embodiments of the current subject matter. As shown in FIG. 1, the GenAI system 100 includes a trust score risk model 110 that evaluates the reliability of responses generated by the GenAI model. The process starts with a prompt submission from a user, which is processed by the GenAI model to generate a response. The generated response is then evaluated by the trust score risk model 110 to compute a trust score. The computed trust score is passed to a comparison module 120,Attorney Docket No.: 035006-872001 WO Customer Number: 76615 which compares the score against a predefined threshold to categorize the response into three levels.

[0058] Low Trust Score: If the trust score does not meet the threshold (e.g., is below the threshold), the response is rejected. In some embodiments, the predefined threshold is dynamically defined based on response characteristics, including domain specificity, response length, and historical user feedback. The system may adjust the threshold dynamically in realtime to optimize accuracy and reliability in trust score assessments. The system then initiates a post-validation step to analyze the rejected response. User feedback may also be incorporated to improve future responses. Medium Trust Score: For responses with a medium trust score, the system invokes self-consistency checks and chain-of-thought reasoning within the GenAI model to refine or supplement the response. High Trust Score: If the trust score is above the threshold, the response is forwarded to a quality control mechanism 130 for final verification before being presented to the user.

[0059] The architecture incorporates a feedback loop that allows rejected or refined responses to be used as training inputs to improve the evaluation capabilities of the trust score risk model 110 and enhance the accuracy of the GenAI model over time. The diagram highlights a dynamic process that categorizes responses, applies refinement actions when needed, and ensures that only high-confidence outputs are presented to the user, improving reliability and user satisfaction.

[0060] In some embodiments, the trust score risk model 110 may generate the trust score by evaluating a response generated by the GenAI model using a combination of an LKA alignment score and an N-gram coverage score. The LKA alignment score is computed by determining the degree to which the response aligns with the latent knowledge anchors (LKAs) constructed from a processed training dataset. A Latent Knowledge Anchor may be a latent pattern (across activations, attention heads, or embedding subspaces) that consistently encodes data (e.g., a concept, fact, or rule) and allows the GenAI model to reliably retrieve or reason about the encoded data across contexts. The LKA may represent in-domain knowledge or samples. The N-gram coverage score is calculated based on the statistical coverage of N-gram sequences extracted from the training dataset, reflecting the comprehensiveness of the response in comparison to the training data. These two scores are then merged to generate the trust score, which serves as an indicator of both the alignment level of the response to the training dataset and its overall comprehensiveness.Attorney Docket No.: 035006-872001 WO Customer Number: 76615 LKA alignment score calculation

[0061] In some embodiments, the trust score risk model 110 calculates the LKA alignment score by evaluating the alignment of a response generated by the GenAI model with the latent knowledge anchors (LKAs) derived from the training dataset. This process involves several stages:

[0062] Raw Document Data Processing: The LLM's domain- or task-specific data is processed to convert raw text into an N-gram representation. In the preferred embodiment, all documents are converted to their bigram representation, which consists of sequences of two consecutive words. These bigrams are then filtered based on part-of- speech (POS) tags to capture meaningful co-occurrence relationships between words in the corpus. In some embodiments, the LKA construction process incorporates a specialized domain-specific vocabulary derived from the training dataset. This vocabulary is selected based on domainrelevant terminology and may be refined using linguistic techniques such as part-of-speech filtering, term frequency analysis, and external domain-specific lexicons to ensure the most relevant concepts are captured. The processing pipeline includes the following operations:• Tokenization: Text documents are tokenized into individual word strings (tokens), removing punctuation and stop words, and retaining only alphanumeric words no shorter than three characters. An N-gram is a contiguous sequence of N tokens, where N represents the number of tokens.• Lemmatization: Tokens are lemmatized to their base forms (e.g., "financed" becomes "finance").• Bigram / N-gram Generation: Bigrams or N-grams may be generated from the lemmatized tokens.• POS Tagging and Filtering: Bigrams are POS-tagged and filtered to ensure only certain parts of speech co-occur. In the preferred embodiment, the following POS tag combinations are allowed, according to the English Penn Treebank POS tag set:o JJ + NN (an adjective + a singular noun)o NN + NN (a singular noun + a singular noun)Attorney Docket No.: 035006-872001 WO Customer Number: 76615 o VB + NN (a verb + a singular noun)o NN + VB (a singular noun + a verb)

[0063] For example, in a financial context, the bigram "economic growth" (JJ + NN) may be considered, while in a medical context, "heart disease" (NN + NN) may be relevant. In alternative embodiments, different types of N-grams, such as trigrams, can be generated and filtered for different POS tag combinations, like an adjective + a singular noun + a singular noun. This processing pipeline produces arrays of bigrams, which are subsequently used for topic modeling.

[0064] Based on the POS-tag filtered N-grams in a numeric term frequency-inverse document frequency (TF-IDF) form, a machine learning model is trained to uncover underlying topics within the LLM's domain- or task-specific training data, the training data corpus may include the in-domain corpus. In the preferred embodiment, these underlying topics are defined as latent knowledge anchors (LKAs), representing distinct sub-domains within the larger domain of the LLM's training data corpus. LKAs are uncovered considering a specific and contained vocabulary (V) of size S, representing the POS-tag filtered N-grams. The creation of a bounded vocabulary is crucial, allowing granular control of linguistic patterns and efficient representations of a given domain's nomenclature as N-grams. Assuming an a priori dictionary is not provided, a vocabulary is constructed from the training data corpus by analyzing all documents to extract unique N-grams. The vocabulary size is bounded and controlled by an external parameter S. In the preferred embodiment, once the vocabulary is algorithmically established, an Al scientist audits all the N-grams within the vocabulary to maintain observability, traceability, and auditability. An Al scientist should always be part of the vocabulary creation stage to capture every aspect of data and LLM model development.

[0065] In the preferred embodiment, a latent Dirichlet allocation (LDA) algorithm is used for topic modeling. The LDA algorithm allows the LLM's training data to be explained by a T-number (e.g., 10) of hidden topics representing LKAs, describing why certain N-gram-transformed text documents are similar. The LDA model training iteratively updates the topic assignments for each N-gram in the vocabulary until convergence. In alternative embodiments, different generative probabilistic or non- probabilistic linear algebraic models, such as nonnegative matrix factorization (NMF), can be used to uncover the LKAs. Once completed, theAttorney Docket No.: 035006-872001 WO Customer Number: 76615 topic modeling stage converts the LLM's training data into T-dimensional (e.g., 10) LKA vectors.

[0066] Neural Network Training and Inference: In this stage, the LLM's domain- or task-specific dataset is expanded with additional exemplars outside of the domain to train a neural network classifier to discriminate between in-domain and out-of-domain data. Both the in-domain and out-of-domain data are processed using the aforementioned stages to generate T-dimensional LKA vectors. Following training, a raw neural network score is calibrated to a [1, 999] range.

[0067] In some embodiments, out-of-domain samples are specifically introduced as negative examples during classifier training. These samples are selected based on their deviation from the specialized domain-specific vocabulary and LKA structures. The classifier is trained to assign lower scores to responses resembling out-of-domain samples, ensuring that trust scores reflect domain alignment.

[0068] During the process of an independent LLM's response verification, the trained neural network model is used in inference mode to verify whether the domain-specific LKAs that the LLM was trained on are utilized by the LLM to provide a response to a specific prompt. The closer the score is to 999, the higher the probability that the LLM's response to a given prompt is within the latent knowledge anchor space of the in-domain data. High scores provide LLM users with greater confidence in the LLM and its responses.

[0069] In alternative embodiments, different machine learning classifiers, such as decision trees or support vector machines, can be used to learn to discriminate between indomain and out-of-domain data.Response coverage score calculation

[0070] In some embodiments, the trust score risk model calculates the N-gram coverage of a response generated by the GenAI model with respect to the latent knowledge anchors (LKAs) underlying the response as a second component of the trust score. To achieve this, the system performs multiple stages of processing and evaluation.

[0071] [Modeling Stage] Initially, each N-gram in the in-domain vocabulary, constructed during the LDA modeling process, is assigned a coverage statistic. This statisticAttorney Docket No.: 035006-872001 WO Customer Number: 76615 represents the frequency of an N-gram's occurrence across the training dataset and is calculated as the percentage of documents within the training corpus that include at least one instance of the n-gram. For example, the N-gram coverage percentage is defined as the number of documents containing a specific N-gram divided by the total number of documents in the indomain training set, multiplied by 100. These coverage statistics are stored in a database for independent verification of responses.

[0072] For each LKA, a summarization process is performed by identifying a predefined number (e.g., 50) of the most relevant N-grams associated with the LKA. In some embodiments, the system constructs coverage statistical measures specifically over a specialized domain-specific vocabulary. These measures quantify relationships of cooccurrence between tokens in the vocabulary, allowing the system to track term relevance and domain specificity. This statistical representation enhances the ability to assess trust scores by providing a structured measure of linguistic coherence and domain alignment. These N-grams are identified algorithmically, leveraging the LDA model’s weights, and optionally validated through human expertise to ensure domain relevance.

[0073] For a selected N-gram, the corresponding coverage statistic is retrieved, and both the mean and standard deviation of these values are calculated to represent the overall coverage characteristics of the LKA. These summarization statistics are stored in a database to facilitate the comparison and evaluation of responses. For instance, if an LKA represents personal finance, its relevant N-grams might include terms such as "credit score," "credit card," and "credit report," with individual coverage percentages. The mean coverage and standard deviation of a predefined number (e.g., 50) of the most relevant N-grams associated with the LKA provide a comprehensive overview of how these terms are distributed across the training data and in relation to each LKA.

[0074] In another example, if an LKA represents medical diagnostics, its relevant N-grams might include terms such as “blood pressure,” “heart rate,” and “MRI scan,” each with individual coverage percentages reflecting their prevalence in the training dataset. For instance, “blood pressure” may have a coverage percentage of 6.75%, “heart rate” 4.32%, and “MRI scan” 3.89%. The mean coverage and standard deviation for this LKA are calculated to provide a statistical overview of how these terms are distributed across the medical dataset. The statistical values are stored and later used for evaluating the comprehensiveness of GenAL generated responses in the medical domain.Attorney Docket No.: 035006-872001 WO Customer Number: 76615

[0075] In some embodiments, an in-domain LKA space is formed by consolidating the LKA vectors corresponding to in-domain samples. This consolidation process aggregates the learned LKA representations into a structured space that characterizes the key knowledge areas within the training dataset. By defining a well-bounded in-domain space, the system improves response evaluation accuracy. If a response includes medical terminology but deviates from the expected LKA-based coverage, its trust score may be adjusted accordingly to reflect potential gaps in domain knowledge representation.

[0076] [Use Stage] To evaluate a specific response generated by the GenAI model, the response is converted into an N-gram representation and mapped to an LKA vector. The N-grams within the response are compared against the in-domain vocabulary to determine the mutual elements. For instance, if a response contains 30 N-grams and 20 of them are found in the in-domain vocabulary, the system retrieves the coverage statistics for these 20 N-grams. Using an arithmetic mean formula, the system calculates the mean coverage value (MC) for the response. The LKA vector, which includes elements equal to the total number of LKAs (e.g., 10), is used to identify the LKA most strongly associated with the response. The LKA's mean and standard deviation coverage values are then retrieved from the summarization database for further calculations.

[0077] To standardize the MC, the system computes a z-score (referred to as the MC z-score) using the formula: (MC - LKA’s mean) / LKAs’ standard deviation. The z-score is capped to a range of [-3, 0], where values below the LKA's mean and within three standard deviations are retained. The MC z-score is subsequently calibrated to a range of [1, 999], with any z-score value greater than or equal to 0.0 automatically mapped to 999. The calibrated score is further weighted by the probability of the LKA most likely associated with the response. This weighting ensures that the system accounts for cases where responses with similar coverage metrics may differ semantically based on the LKA space they align with. In an alternative embodiment, for responses that cannot be distinctly assigned to a single LKA, the system may consider bigram information from two or three LKAs with the highest probabilities.

[0078] The resulting N-gram coverage score reflects how comprehensive and representative the response is in relation to the target domain and its specific LKA. A higher coverage score, approaching 999, indicates a higher level of comprehensiveness and alignment with the in-domain knowledge. In the preferred embodiment, this process is supplemented byAttorney Docket No.: 035006-872001 WO Customer Number: 76615 retrieving a specified number of relevant documents from the in-domain database, identified using a cosine similarity measure based on the LKA vectors. These documents provide additional context and validation for the response, further enhancing the system's trustworthiness evaluation capabilities.

[0079] FIG. 2 illustrates the trust score risk model’s pipeline for independent verification of a GenAI model’s responses. The diagram depicts the sequential process through which a trust score is generated based on two core components: the latent knowledge anchor (LKA) alignment score and the N-gram coverage score. As shown in FIG. 2, when a prompt is submitted to the GenAI model, the system generates a response. This response is simultaneously processed to produce both an N-gram representation and an LKA vector. These representations are then used as input for two parallel evaluation processes.

[0080] Component #1 calculates an LKA alignment score by assessing how well the response aligns with the in-domain knowledge represented by the LKAs derived during the training process. Component #2 computes the N-gram coverage score by comparing the response’s N-grams against the statistical distribution of N-grams from the in-domain training dataset. Both component scores are scaled within a range of [1, 999] and are then combined to produce the final trust score. The combination of these scores may involve arithmetic averaging, probabilistic merging, or machine learning-based mappings that account for nonlinear interactions between the two components.

[0081] In some embodiments, the neural network classifier integrates the LKA alignment score and the statistical coverage measures to compute the trust score. By combining these two components, the classifier accounts for both conceptual alignment with in-domain knowledge and statistical consistency with the training dataset, enhancing the accuracy of response evaluations. The resulting trust score represents the overall reliability of the response, with higher scores indicating a greater alignment with domain- specific knowledge and comprehensiveness.

[0082] The pipeline described in FIG. 2 is optimized for real-time applications and large-scale API deployments. Using latent Dirichlet allocation (LDA) technology, the system is capable of processing trust score evaluations efficiently. The computational latency of trust score inference is designed to be lower than the GenAI model’s response generation time, ensuring that the trust score can be computed rapidly and without affecting the timeliness ofAttorney Docket No.: 035006-872001 WO Customer Number: 76615 the overall system. This makes the approach suitable for scenarios requiring prompt evaluations, such as real-time user interactions or large-scale deployment environments.

[0083] In certain embodiments, an assessment model may be deployed in conjunction with a Generative Artificial Intelligence (GenAI) model utilizing the technical features provided herein. In one framework, while the GenAI model generates a response (e.g., an answer to a user query), the assessment model determines the reliability of the generated response in parallel and in real time.

[0084] To promote efficiency and lower latency, as the GenAI model begins streaming its response (e.g., token by token), the assessment model filters the stream and applies N-gram conversion and LKA vectorization. As a result, a trust score can be generated for the results streamed by the GenAI model.

[0085] In accordance with some aspects, a relatively small language model (e.g., a Focused Language Model) may be implemented for the GenAI to help reduce latency and increase topic accuracy. Further, the assessment model may run in parallel with the GenAI model during the generation process. That is, the assessment model need not wait for the GenAI model to complete the entire result and can continue the assessment as the GenAI model streams the results.

[0086] The features provided above considerably enhance resource and compute efficiency by limiting the volume of data analyzed by the assessment model. This reduces the latency associated with real-time processing of generative results and helps conserve storage resources, making the overall process faster and more efficient.Case Study

[0087] To demonstrate the novelty and value of this invention, a case study was conducted to evaluate the trust score risk model. The case study used a Focused Language Models (FLMs) trained on medical diagnostics domain data (further referred to as in-domain data). A FLM is a version of a large language model (LLM) specifically designed and trained to achieve reliable and trustworthy results within a narrow domain or task. In this case, the medical diagnostics domain was selected. FLMs are highly specialized because they are trained on carefully filtered and selected data sets relevant to the intended task. This reduces the likelihood of generating nonsensical information (known as hallucinations) and increasesAttorney Docket No.: 035006-872001 WO Customer Number: 76615 confidence in the generated outputs. After training the base (domain adaptation) FLM, additional task-specific FLMs were fine-tuned on specialized diagnostics tasks (e.g., identifying potential symptoms or analyzing lab results). FLMs allow granular control of language and linguistic patterns because they are built on bounded vocabularies created from high-quality and audited data sets.

[0088] The in-domain data included 56,390,304 documents from the medical diagnostics industry. These documents were categorized into five levels of relevance:Category 1: Document content has minimal relevant medical diagnostics information; mostly irrelevant or promotional.Category 2: Document mentions medical diagnostics concepts but lacks depth; contains significant irrelevant content.Category 3: Document clearly introduces key medical diagnostics concepts and terminology; focused on medical topics with minimal extraneous information.Category 4: Document is highly relevant and valuable; provides substantial information with examples or case studies; minimal irrelevant content.Category 5: Document is an exceptional educational resource; well-organized, highly informative, with deep insights into medical diagnostics; free from irrelevant content.

[0089] The training of the trust score risk model included the following stages:

[0090] LKAs Generation: To identify the latent knowledge anchors within the FLM’s data corpus, only documents of category 5 (837,016 in total) were used. These documents were converted into their bigram representations and transformed into numeric TF-IDF forms to train the LDA model. The final vocabulary size VW was 262,144. For each bigram, a coverage statistic was calculated as follows:A bigram coverage % = (# of documents including at least 1 occurrence of a given bigram / 56,390,304) x 100Attorney Docket No.: 035006-872001 WO Customer Number: 76615

[0091] Bigram-specific coverage statistics were stored in a database. Although the vocabulary was created only from category 5 documents, the denominator for the formula included the entirety of the FLM’s training data (categories 3, 4, and 5).

[0092] The LDA model identified TTT-number of latent knowledge anchors, where T=10T = 10T=10. An Al scientist conducted an analysis of the most relevant bigrams for each topic and assigned LKA-related categories such as:i. Diagnostic Imaging Techniquesii. Pathology and Lab Results Analysisiii. Disease Symptom Classificationiv. Preventive Healthcare Protocolsv. Treatment Outcome Predictionvi. Patient Risk Stratificationvii. Drug Efficacy Monitoringviii. Genetic and Biomarker Analysisix. Mental Health Assessmentx. Surgical Success Probability Models

[0093] For each LKA, summarization statistics were generated and stored in a database. All 837,016 documents were converted into LKA vectors following the LDA training process.

[0094] Neural Network Training: A dataset was created for training a neural network classifier. This included 837,016 documents from category 5 represented as LKA vectors and 771,270 documents from category 1 also represented as LKA vectors. The neural network, consisting of one hidden layer with six nodes, was trained for 200 epochs.

[0095] After training, the trust score risk model’s pipeline was tested on a holdout dataset of 250,000 documents sampled evenly across all five categories. The trust score for each document was computed as the arithmetic mean of the component #1 (LKA alignment) and component #2 (N-gram coverage) scores.trust score cumulative % of cumulative % of cumulative % of cumulative % of cumulative % of category 1 above category above category 3 above category 4 above category 5 above rangethe range the range the range the range the rangeAttorney Docket No.: 035006-872001 WO Customer Number: 76615 [1, 100] 99.09 100.00 100.00 100.00 100.00(100, 200](200, 300] 60.48 69.65 83.02 91.86 96.73(300, 400] 34.18 50.78 68.43 82.30 90.91(400, 500] 19.88 34.20 53.04 69.19 80.11(500, 600]* 10.87 19.35 36.57 53.57 67.97(600, 700]* 3.74 5.73 17.06 29.99 45.34Table 1: Trust Score Distribution Table for the Holdout Data

[0096] Table 1 demonstrates the trust score risk model’s ability to differentiate between in-domain and out-of-domain documents. For example, with a trust score threshold of >600, approximately 11% of category 1 documents are included, compared to 68% of category 5 documents. Similarly, a threshold of >700 retains only ~4% of category 1 documents, but 45% of category 5 documents.

[0097] The case study illustrates the trust score risk model’s efficacy in identifying reliable and relevant responses for highly specialized domains such as medical diagnostics, supporting its applicability across diverse real-world scenarios.

[0098] FIG. 3 illustrates the probability density function of trust scores derived from documents categorized as category 1 and category 5, according to one or more embodiments. As shown in FIG. 3, binning and observation counting are utilized to generate a kernel density estimate plot, smoothing the observed data with a Gaussian kernel to produce a continuous density distribution. This visualization demonstrates the effectiveness of the trust score risk model in distinguishing between in-domain and out-of-domain data. Specifically, documents from category 5, which are highly relevant and accurate, show a higher density of trust scores concentrated in the upper range, reflecting their strong alignment with domain- specificAttorney Docket No.: 035006-872001 WO Customer Number: 76615 knowledge. In contrast, documents from category 1, which are minimally relevant or irrelevant, exhibit trust scores predominantly in the lower range. This differentiation validates the robustness of the trust score risk model in accurately categorizing and evaluating GenAI responses.

[0099] The following example illustrates the inference process using the trust score risk model to independently verify a response generated by a focused language model (FLM). A prompt is submitted to the FLM: "What are common symptoms of diabetes?" The FLM generates the response: "Common symptoms of diabetes include increased thirst, frequent urination, extreme hunger, unintended weight loss, fatigue, blurred vision, slow-healing sores, and frequent infections. These symptoms are more noticeable in type 1 diabetes but can also occur in type 2 diabetes. It's essential to consult a healthcare provider for proper diagnosis and management." This response is processed into its bigram representation and latent knowledge anchor (LKA) vector.[000100] For example, the bigram array for the response includes phrases such as ['common symptom', 'symptom diabetes', 'include increase', 'increase thirst', 'frequent urination', 'extreme hunger', 'unintended weight', 'weight loss', 'loss fatigue', 'fatigue blurred', 'blurred vision', 'slow-healing sores', 'frequent infection', 'symptom noticeable', 'type diabetes', 'consult healthcare', 'healthcare provider', 'provider proper', 'proper diagnosis', 'diagnosis management'], and the corresponding LKA vector is [0.000123, 0.998652, 0.000245, 0.000342, 0.000198, 0.000276, 0.000321, 0.000145, 0.000267, 0.000132],[000101] Analysis of the LKA vector identifies LKA at index 1 (zero-based indexing) as the most probable match, representing the disease symptom classification subdomain in the medical diagnostics domain. The LKA vector is further processed by a neural network classifier to evaluate the response’s alignment with the latent knowledge anchors, resulting in an alignment score of 915. Simultaneously, the bigram array is analyzed to calculate the response’s n-gram coverage relative to the latent knowledge anchors, yielding an N-gram coverage score of 821. These two scores are combined as an arithmetic mean to derive the final trust score, which is 868 in this case. This high trust score indicates that the response is both well-aligned with the in-domain knowledge and comprehensive in its coverage.Attorney Docket No.: 035006-872001 WO Customer Number: 76615[000102] To further validate the response, five highly relevant documents from the indomain database are retrieved based on cosine similarity measures. These documents provide additional context for the response and are presented in Table 2 below.[000103] With a trust score of 868, the system confidently verifies that the response is accurate, comprehensive, and representative of the target domain. The retrieved documents further reinforce the reliability of the GenAI system's outputs, enhancing user trust in the response evaluation process.[000104] FIG. 5 is a diagram illustrating a flowchart of a process 500 for evaluating trustworthiness in a Generative Artificial Intelligence (GenAI) system, in accordance with one or more embodiments of the current subject matter.[000105] [Modeling Stage] As shown in FIG. 5, the process 500 may begin with operation 502, wherein the system processes a training dataset that the GenAI was trained upon. In some embodiments, this includes performing text preprocessing such as tokenization, lemmatization, and stopword removal to prepare the dataset for further analysis. The process may then proceed to operation 504, where the system constructs a plurality of latent knowledge anchors (LKAs) by applying a topic modeling mechanism to the processed training dataset.[000106] In some embodiments, the topic modeling mechanism involves latent Dirichlet allocation (LDA) or other probabilistic techniques to generate domain- specific knowledge representations within the training dataset. Each LKA represents a specific subdomain of the knowledge space. Next, in operation 506, the system trains a neural network classifier usingAttorney Docket No.: 035006-872001 WO Customer Number: 76615 the LKAs and the processed training dataset. The training process involves generating LKA vectors for in-domain data and augmenting the training dataset with out-of-domain data samples to introduce negative examples. The classifier is trained to maximize the separation between in-domain and out-of-domain responses by assigning higher LKA alignment scores to in-domain data and proportionally lower scores to out-of-domain data.[000107] [Use Stage] The process advances to operation 508, where the system evaluates a response generated by the GenAI system and generates a trust score. The evaluation includes generating two component scores: a first score indicating the response’s alignment level with the LKAs and a second score derived from an analysis of the response’s N-gram coverage with respect to the statistical baseline established from the training dataset. The trust score is calculated by combining these two component scores through methods such as arithmetic averaging, probabilistic merging, or other statistical techniques. At operation 510, the system compares the trust score against a dynamically defined threshold. If the trust score does not meet the threshold (e.g., exceeds the threshold, depending on implementation), the process proceeds to presenting the response to the user. If the trust score meets the threshold (e.g., does not exceed the threshold), the process advances to issuing a command to the GenAI to conduct additional actions. These additional actions may include generating an additional response by issuing a refined prompt to the GenAI to improve or supplement the response.[000108] FIG. 4 depicts a block diagram illustrating a computing system 400 consistent with implementations of the current subject matter. As shown in FIG. 4, the computing system 400 can include a processor 410, a memory 420, a storage device 430, and input / output devices 440. The processor 410, the memory 420, the storage device 430, and the input / output devices 440 can be interconnected via a system bus 450. The computing system 400 may additionally or alternatively include a graphic processing unit (GPU), such as for image processing, and / or an associated memory for the GPU. The GPU and / or the associated memory for the GPU may be interconnected via the system bus 450 with the processor 410, the memory 420, the storage device 430, and the input / output devices 440. The memory associated with the GPU may store one or more images described herein, and the GPU may process one or more of the images described herein. The GPU may be coupled to and / or form a part of the processor 410. The processor 410 is capable of processing instructions for execution within the computing system 400. Such executed instructions can implement one or more components. In some implementations of the current subject matter, the processor 410 can be a single-threadedAttorney Docket No.: 035006-872001 WO Customer Number: 76615 processor. Alternately, the processor 410 can be a multi-threaded processor. The processor 410 is capable of processing instructions stored in the memory 420 and / or on the storage device 430 to display graphical information for a user interface provided via the input / output device 440.[000109] The memory 420 is a computer readable medium such as volatile or non-volatile that stores information within the computing system 400. The memory 420 can store data structures representing configuration object databases, for example. The storage device 430 is capable of providing persistent storage for the computing system 400. The storage device 430 can be a floppy disk device, a hard disk device, an optical disk device, or a tape device, or other suitable persistent storage means. The input / output device 440 provides input / output operations for the computing system 400. In some implementations of the current subject matter, the input / output device 440 includes a keyboard and / or pointing device. In various implementations, the input / output device 440 includes a display unit for displaying graphical user interfaces.[000110] According to some implementations of the current subject matter, the input / output device 440 can provide input / output operations for a network device. For example, the input / output device 440 can include Ethernet ports or other networking ports to communicate with one or more wired and / or wireless networks (e.g., a local area network (LAN), a wide area network (WAN), the Internet).[000111] In some implementations of the current subject matter, the computing system 400 can be used to execute various interactive computer software applications that can be used for organization, analysis and / or storage of data in various (e.g., tabular) format (e.g., Microsoft Excel®, and / or any other type of software). Alternatively, the computing system 400 can be used to execute any type of software applications. These applications can be used to perform various functionalities, e.g., planning functionalities (e.g., generating, managing, editing of spreadsheet documents, word processing documents, and / or any other objects, etc.), computing functionalities, communications functionalities, etc. The applications can include various addin functionalities or can be standalone computing products and / or functionalities. Upon activation within the applications, the functionalities can be used to generate the user interface provided via the input / output device 440. The user interface can be generated and presented to a user by the computing system 400 (e.g., on a computer screen monitor, etc.).Attorney Docket No.: 035006-872001 WO Customer Number: 76615[000112] One or more aspects or features of the subject matter described herein can be realized in digital electronic circuitry, integrated circuitry, specially designed framework specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) computer hardware, firmware, software, and / or combinations thereof. These various aspects or features can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device. The programmable system or computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.[000113] These computer programs, which can also be referred to as programs, software, software frameworks, frameworks, components, or code, include machine instructions for a programmable processor, and can be implemented in a high-level procedural language, an object-oriented programming language, a functional programming language, a logical programming language, and / or in assembly / machine language. As used herein, the term “machine-readable medium” refers to any computer program product, apparatus and / or device, such as for example magnetic discs, optical disks, memory, and Programmable Logic Devices (PLDs), used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor. The machine-readable medium can store such machine instructions non-transitorily, such as for example as would a nontransient solid-state memory or a magnetic hard drive or any equivalent storage medium. The machine-readable medium can alternatively or additionally store such machine instructions in a transient manner, such as for example as would a processor cache or other random access memory associated with one or more physical processor cores.[000114] To provide for interaction with a user, one or more aspects or features of the subject matter described herein can be implemented on a computer having a display device, such as for example a cathode ray tube (CRT) or a liquid crystal display (LCD) or a lightAttorney Docket No.: 035006-872001 WO Customer Number: 76615 emitting diode (LED) monitor for displaying information to the user and a keyboard and a pointing device, such as for example a mouse or a trackball, by which the user may provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well. For example, feedback provided to the user can be any form of sensory feedback, such as for example visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including, but not limited to, acoustic, speech, or tactile input. Other possible input devices include, but are not limited to, touch screens or other touch-sensitive devices such as single or multi-point resistive or capacitive trackpads, voice recognition hardware and software, optical scanners, optical pointers, digital image capture devices and associated interpretation software, and the like.[000115] In the descriptions above and in the claims, phrases such as “at least one of’ or “one or more of’ may occur followed by a conjunctive list of elements or features. The term “and / or” may also occur in a list of two or more elements or features. Unless otherwise implicitly or explicitly contradicted by the context in which it used, such a phrase is intended to mean any of the listed elements or features individually or any of the recited elements or features in combination with any of the other recited elements or features. For example, the phrases “at least one of A and B;” “one or more of A and B;” and “A and / or B” are each intended to mean “A alone, B alone, or A and B together.” A similar interpretation is also intended for lists including three or more items. For example, the phrases “at least one of A, B, and C;” “one or more of A, B, and C;” and “A, B, and / or C” are each intended to mean “A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B and C together.” Use of the term “based on,” above and in the claims is intended to mean, “based at least in part on,” such that an unrecited feature or element is also permissible.[000116] The subject matter described herein can be embodied in systems, apparatus, methods, and / or articles depending on the desired configuration. The implementations set forth in the foregoing description do not represent all implementations consistent with the subject matter described herein. Instead, they are merely some examples consistent with aspects related to the described subject matter. Although a few variations have been described in detail above, other modifications or additions are possible. In particular, further features and / or variations can be provided in addition to those set forth herein. For example, the implementations described above can be directed to various combinations and subcombinations of the disclosed features and / or combinations and subcombinations of several further features disclosed above.Attorney Docket No.: 035006-872001 WO Customer Number: 76615 In addition, the logic flows depicted in the accompanying figures and / or described herein do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Other implementations may be within the scope of the following claims.[000115] Depending on implementation, certain aspects of the claimed subject matter are provided in itemized form below:Item 1. A method for providing a Generative Artificial Intelligence (GenAI) system with trustworthiness evaluation, comprising:processing a training dataset that the GenAI was trained upon;constructing a plurality of latent knowledge anchors (LKAs) by applying topic modeling on the processed training dataset, wherein the plurality of LKAs comprises domainspecific knowledge representations within the processed training dataset and specialized domain-specific vocabulary;constructing a plurality of coverage statistical measures over a specialized domain specific vocabulary where the statistical measures quantify a statistical relevance of relationships of co-occurrence between tokens on the vocabulary;training a neural network classifier using the LKAs and the processed training data;evaluating, using the neural network classifier and statistical coverage measures, a response generated by the GenAI to generate a trust score, wherein the trust score indicates an alignment level of the response to the training dataset given a specialized domain-specific vocabulary and / or a comprehensiveness of the response compared to the training dataset;in response to the trust score meeting a dynamically defined threshold, presenting the response to a user; andin response to the trust score not meeting the threshold, issuing a command for the GenAI to conduct additional actions.Attorney Docket No.: 035006-872001 WO Customer Number: 76615 Item 2. The method of item 1, wherein the additional actions comprise generating an additional response, wherein generating an additional response comprises issuing a prompt to the GenAI to refine or supplement the response.Item 3. The method of item 1, wherein training the neural network classifier comprises:expanding the training dataset with out-of-domain samples to introduce negative example;generating LKA vectors for in-domain samples and out-of-domain training datasets, wherein an in-domain LKA space is formed by consolidating the LKA vectors for the indomain samples;training the neural network classifier using the in-domain and out-of-domain dataset, wherein the classifier is optimized to maximize a separation between in-domain and out-of-domain responses.Item 4. The method of item 1, further comprising:extracting N-gram sequences from the processed training dataset;calculating a frequency of occurrence of each N-gram within the training dataset to establish an N-gram statistical baseline; andgenerating a N-gram coverage statistics metric associated with the training dataset, wherein the N-gram coverage statistics metric is utilized in calculating the trust score.Item 5. The method of item 4, further comprising:converting the response to N-gram representation and LKA vector representations;generating, using the trained classifier, a first score indicating the alignment level of the response with an in-domain LKA space;Attorney Docket No.: 035006-872001 WO Customer Number: 76615 calculating a second score based on the N-gram representation of the response and the N-gram coverage statistics metric associated with the training dataset; andgenerating the trust score based on the first score and the second score.Item 6. The method of item 1, wherein the topic modeling for constructing the LKAs comprises applying latent Dirichlet allocation (LDA) topic modeling to the processed training dataset.Item 7. The method of item 1, wherein the processing the training dataset comprises: performing text preprocessing on the training dataset, wherein the text preprocessing includes tokenization, lemmatization, and stopword removal.Item 8. A computer program product for providing a Generative Artificial Intelligence (GenAI) system with trustworthiness evaluation, comprising a non-transient machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:processing a training dataset that the GenAI was trained upon;constructing a plurality of latent knowledge anchors (LKAs) by applying topic modeling on the processed training dataset, wherein the plurality of LKAs comprises domainspecific knowledge representations within the processed training dataset and specialized domain-specific vocabulary;constructing a plurality of coverage statistical measures over a specialized domain specific vocabulary where the statistical measures quantify a statistical relevance of relationships of co-occurrence between tokens on the vocabulary;training a neural network classifier using the LKAs and the processed training data;Attorney Docket No.: 035006-872001 WO Customer Number: 76615 evaluating, using the neural network classifier and statistical coverage measures, a response generated by the GenAI to generate a trust score, wherein the trust score indicates an alignment level of the response to the training dataset given a specialized domain- specific vocabulary and / or a comprehensiveness of the response compared to the training dataset;in response to the trust score exceeding a dynamically defined threshold, presenting the response to a user; andin response to the trust score not exceeding the threshold, issuing a command for the GenAI to conduct additional actions.Item 9. The computer program product of item 8, wherein the additional actions comprise generating an additional response, wherein generating an additional response comprises issuing a prompt to the GenAI to refine or supplement the response.Item 10. The computer program product of item 8, wherein training the neural network classifier comprises:expanding the training dataset with out-of-domain samples to introduce negative example;generating LKA vectors for in-domain samples and out-of-domain training datasets, wherein an in-domain LKA space is formed by consolidating the LKA vectors for the indomain samples;training the neural network classifier using the in-domain and out-of-domain dataset, wherein the classifier is optimized to maximize a separation between in-domain and out-of-domain responses.Item 11. The computer program product of item 8, wherein the operations further comprise:Attorney Docket No.: 035006-872001 WO Customer Number: 76615 extracting N-gram sequences from the processed training dataset;calculating a frequency of occurrence of each N-gram within the training dataset to establish an N-gram statistical baseline; andgenerating a N-gram coverage statistics metric associated with the training dataset, wherein the N-gram coverage statistics metric is utilized in calculating the trust score.Item 12. The computer program product of item 11, wherein the operations further comprise:converting the response to N-gram representation and LKA vector representations;generating, using the trained classifier, a first score indicating the alignment level of the response with an in-domain LKA space;calculating a second score based on the N-gram representation of the response and the N-gram coverage statistics metric associated with the training dataset; andgenerating the trust score based on the first score and the second score.Item 13. The computer program product of item 8, wherein the topic modeling for constructing the LKAs comprises applying latent Dirichlet allocation (LDA) topic modeling to the processed training dataset.Item 14. The computer program product of item 8, wherein the processing the training dataset comprises: performing text preprocessing on the training dataset, wherein the text preprocessing includes tokenization, lemmatization, and stopword removal.Item 15. A system for providing a Generative Artificial Intelligence (GenAI) system with trustworthiness evaluation comprising:Attorney Docket No.: 035006-872001 WO Customer Number: 76615 at least one programmable processor; anda non-transient machine-readable medium storing instructions that, when executed by the processor, cause the at least one programmable processor to perform operations comprising:processing a training dataset that the GenAI was trained upon;constructing a plurality of latent knowledge anchors (LKAs) by applying topic modeling on the processed training dataset, wherein the plurality of LKAs comprises domain-specific knowledge representations within the processed training dataset and specialized domain- specific vocabulary;constructing a plurality of coverage statistical measures over a specialized domain specific vocabulary where the statistical measures quantify a statistical relevance of relationships of co-occurrence between tokens on the vocabulary;training a neural network classifier using the LKAs and the processed training data;evaluating, using the neural network classifier and statistical coverage measures, a response generated by the GenAI to generate a trust score, wherein the trust score indicates an alignment level of the response to the training dataset given a specialized domain-specific vocabulary and / or a comprehensiveness of the response compared to the training dataset;in response to the trust score exceeding a dynamically defined threshold, presenting the response to a user; andin response to the trust score not exceeding the threshold, issuing a command for the GenAI to conduct additional actions.Item 16. The system of item 15, wherein the additional actions comprise generating an additional response, wherein generating an additional response comprises issuing a prompt to the GenAI to refine or supplement the response.Attorney Docket No.: 035006-872001 WO Customer Number: 76615 Item 17. The system of item 15, wherein training the neural network classifier comprises:expanding the training dataset with out-of-domain samples to introduce negative example;generating LKA vectors for in-domain samples and out-of-domain training datasets, wherein an in-domain LKA space is formed by consolidating the LKA vectors for the indomain samples;training the neural network classifier using the in-domain and out-of-domain dataset, wherein the classifier is optimized to maximize a separation between in-domain and out-of-domain responses.Item 18. The system of item 15, wherein the operations further comprise:extracting N-gram sequences from the processed training dataset;calculating a frequency of occurrence of each N-gram within the training dataset to establish an N-gram statistical baseline; andgenerating a N-gram coverage statistics metric associated with the training dataset, wherein the N-gram coverage statistics metric is utilized in calculating the trust score.Item 19. The system of item 18, wherein the operations further comprise:converting the response to N-gram representation and LKA vector representations;generating, using the trained classifier, a first score indicating the alignment level of the response with an in-domain LKA space;calculating a second score based on the N-gram representation of the response and the N-gram coverage statistics metric associated with the training dataset; andgenerating the trust score based on the first score and the second score.Attorney Docket No.: 035006-872001 WO Customer Number: 76615Item 20. The system of item 15, wherein the topic modeling for constructing the LKAs comprises applying latent Dirichlet allocation (LDA) topic modeling to the processed training dataset.Item 21. A computer- implemented method for determining reliability of an output generated by a generative Al model, the method comprising:receiving an output generated by the generative Al model;processing the output to generate:an n-gram representation anda latent knowledge anchor (LKA) representation comprising an LKA vector, wherein the LKA representation is based on LKAs learned from an in-domain corpus using topic modeling;generating an alignment score by applying a trained classifier in inference mode to the LKA vector to estimate whether the output aligns with an in-domain LKA space;generating a coverage score by:identifying n-grams of the output that match an in-domain vocabulary, andretrieving, from a database, precomputed n-gram coverage statistics and precomputed LKA summarization statistics to determine a topic-conditioned coverage metric for the output; andcomputing a reliability score for the output by combining at least the alignment score and the coverage score; andproducing, modifying or discarding the output, responsive to the reliability score.Attorney Docket No.: 035006-872001 WO Customer Number: 76615 Item 22. An efficient computer-implemented method for evaluation of reliability of a Generative Artificial Intelligence (Gen Al) response, the method comprising:in response to receiving a response generated by a GenAI system:converting the response into an N-gram representation and mapping the response to an LKA vector;generating an LKA alignment score by applying a trained neural network classifier to the LKA vector in an inference mode; andgenerating, in parallel to generating the LKA alignment score, an N- gram coverage score by identifying which N-grams in the response exist in a pre-stored bounded vocabulary, retrieving their corresponding coverage statistics from one or more storage media to compute a mean coverage, and standardizing the mean coverage using a pre-stored summarization statistics for a relevant LKA; andcombining the LKA alignment score and the N-gram coverage score into a trust score indicating a reliability of the response,wherein during a modeling stage, the following operations are performed:processing an in-domain training corpus to extract part-of-speech (POS) filtered N-grams and converting the N-grams into a numeric term frequency-inverse document frequency (TF-IDF) representation to build a bounded vocabulary of a defined size;training a topic model using the bounded vocabulary to discover a plurality of hidden topics represented as Latent Knowledge Anchors (LKAs);assigning coverage statistics to the N-grams in the bounded vocabulary representing a frequency of appearance across the training corpus and storing the coverage statistics in the one or more storage media; andidentifying a predefined number of relevant N-grams for the LKAs, computing summarization statistics including a mean and a standard deviation for the LKAs basedAttorney Docket No.: 035006-872001 WO Customer Number: 76615 on the relevant N-grams, and storing said summarization statistics in the one or more storage media, andproducing, modifying or discarding the output, responsive to the trust score.

Claims

Attorney Docket No.: 035006-872001 WO Customer Number: 76615 CLAIMS1. A computer-implemented method for determining reliability of an output generated by a generative Al (GenAI) system, the method comprising:receiving an output generated by the GenAI system;processing the output to generate:an N-gram representation, anda latent knowledge anchor (LKA) representation comprising an LKA vector, wherein the LKA representation is based on a plurality of LKAs learned from an in-domain corpus using topic modeling;generating an alignment score by applying a trained classifier, in an inference mode, to the LKA vector to estimate whether the output aligns with an in-domain LKA space;generating a coverage score by:identifying one or more N-grams of the output that match a pre-stored bounded vocabulary, andretrieving, from a database, precomputed N-gram coverage statistics and precomputed LKA summarization statistics to determine a topic-conditioned coverage metric for the output; andcomputing a reliability score for the output based on, at least in part, one of the alignment score and the coverage score; andproducing, modifying or discarding the output, responsive to the reliability score.

2. The computer- implemented method of claim 1, wherein:the alignment score is an LKA alignment score; and / orthe coverage score is an N-gram coverage score.Attorney Docket No.: 035006-872001 WO Customer Number: 76615 3. The computer-implemented method of claim 1 or 2, wherein the GenAI system comprises a GenAI model and wherein in response to receiving the output generated by the GenAI model, the output is converted into the N-gram representation and is mapped to the LKA vector.

4. The computer-implemented method of any one of the preceding claims, wherein:the alignment score is an LKA alignment score, and / orthe trained classifier is a trained neural network classifier.

5. The computer- implemented method of any one of the preceding claims,wherein generating the coverage score comprises:generating, in parallel to generating the alignment score, the coverage score by identifying N-grams in the output that exist in the pre-stored bounded vocabulary,retrieving coverage statistics corresponding to the identified N-grams from the database to compute a mean coverage, andstandardizing the mean coverage using pre-stored summarization statistics for a relevant LKA.

6. The computer-implemented method of any one of the preceding claims, wherein computing the reliability score comprises combining the alignment score and the coverage score into a trust score indicating a reliability of the output generated by the GenAI system.

7. The computer-implemented method of any one of the preceding claims, the method further comprising performing the following operations during a modeling stage prior to receiving the output generated by the GenAI system for evaluation:processing the in-domain corpus to extract part-of-speech (POS) filtered N-grams and converting the filtered N-grams into a numeric term frequency-inverse document frequency (TF-IDF) representation to build an in-domain bounded vocabulary of a defined size;storing the bounded vocabulary in one or more storage media as the pre- stored bounded vocabulary;Attorney Docket No.: 035006-872001 WO Customer Number: 76615 training a topic model using the bounded vocabulary to discover a plurality of hidden topics represented as the plurality of LKAs;assigning coverage statistics to the N-grams in the bounded vocabulary representing a frequency of appearance across the in-domain corpus and storing the coverage statistics in the database; andidentifying a predefined number of relevant N-grams for the LKAs,computing summarization statistics including a mean and a standard deviation for the LKAs based on the relevant N-grams, andstoring said summarization statistics in the database.

8. The computer-implemented method of any one of the preceding claims, the method further comprising:processing a training dataset that the GenAI system was trained upon, the training dataset including the in-domain corpus;constructing the plurality of LKAs by applying the topic modeling on the processed training dataset, the plurality of LKAs comprising domain-specific knowledge representations within the processed training dataset and the pre-stored bounded vocabulary;constructing a plurality of coverage statistical measures over the pre- stored bounded vocabulary where the statistical measures quantify a statistical relevance of relationships of cooccurrence between one or more tokens in the pre-stored bounded vocabulary; andtraining a neural network classifier using the LKAs and the processed training dataset.

9. The computer-implemented method of claim 8, wherein computing the reliability score for the output comprises:evaluating, using the trained classifier and statistical coverage measures, the output generated by the GenAI system to generate the reliability score, wherein the reliability score indicates an alignment level of the output to the training dataset given the pre-stored bounded vocabulary and / or a comprehensiveness of the response compared to the training dataset.Attorney Docket No.: 035006-872001 WO Customer Number: 76615 10. The computer-implemented method of claim 6, wherein the output generated by the GenAI system is further processed, responsive to the reliability score, wherein:in response to the trust score meeting a dynamically defined threshold, the generated output is presented to a user; and / orin response to the trust score missing the dynamically defined threshold, a command is issued for the GenAI system to conduct additional actions.

11. The computer- implemented method of claim 10, wherein the additional actions comprise generating an additional output, wherein generating an additional output comprises issuing a prompt to the GenAI system to refine or supplement the generated output.

12. The computer-implemented method of claim 8, wherein training the neural network classifier comprises:expanding the training dataset with out-of-domain samples to introduce negative examples;generating LKA vectors for in-domain samples and out-of-domain training datasets, wherein an in-domain LKA space is formed by consolidating the LKA vectors for the indomain samples;training the neural network classifier using the in-domain samples and the out-of-domain datasets, wherein the classifier is optimized to maximize a separation between indomain and out-of-domain responses.

13. The computer- implemented method of claim 8, further comprising:extracting a plurality of N-grams from the training dataset;calculating a frequency of occurrence of the N-grams within the training dataset to establish an N-gram statistical baseline; andgenerating an N-gram coverage statistics metric associated with the training dataset, wherein the N-gram coverage statistics metric is utilized in calculating the reliability score.Attorney Docket No.: 035006-872001 WO Customer Number: 76615 14. A computer program product, for determining reliability of an output generated by a generative Al system, comprising a non-transient machine-readable medium storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations according to the method of any one of claims 1 to 13.

15. A system for determining reliability of an output generated by a generative Al system comprising:at least one programmable processor; anda non-transient machine-readable medium storing instructions that, when executed by the processor, cause the at least one programmable processor to perform operations according to the method of any one of claims 1 to 13.

Images