Fact checking llm outputs curing hallucinations using ai overread

Abstract

Systems and methods for fact checking large language model outputs include inputting data to a first trained large language machine learning model at a foundation layer to generate vectorized data; submitting a retrieval query to the trained large language machine learning model and encoding the retrieval query with a structure corresponding to the vectorized data; performing a semantic search for matches between the vectorized data and the retrieval query; inputting the retrieval query and matches of the semantic search to the trained large language machine learning model; generating, at the trained large language machine learning model, a summary; determining, at the trained large language machine learning model, one or more parts of the summary for fact checking by a user; inputting the parts of the summary for fact checking into the large language machine learning model to retrieve references associated with the summary; and displaying the summary and the references.

Description

FACT CHECKING LLM OUTPUTS CURING HALLUCINATIONS USING Al OVERREADCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the filing date of provisional U.S. Patent Application No. 63 / 552,057, entitled “Fact Checking LLM Outputs Curing Hallucinations using Al Overread,” filed on February 9, 2024, the entire contents of which is hereby expressly incorporated herein by reference.TECHNICAL FIELD

[0002] The present disclosure is generally directed to methods and systems for using machine learning models for use in precision medicine, and, more particularly, to using multimodal large language models having multiple layers and optionally deploying integrated checks for unvalidated responses.BACKGROUND

[0003] The background description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventor, to the extent it is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure.

[0004] With recent advancements in large language models (LLMs), various techniques have been proposed to use LLMs in a healthcare context. However, one of the main challenges with using an LLM in a healthcare application is how to improve accuracy on specific tasks, particularly those that require domain knowledge. LLMs are typically trained on historical data and typically lack the ability to incorporate more current information in real-time. This is particularly problematic in a healthcare context, as patient healthcare records may be continuously updated with new information.

[0005] Consequently, LLM outputs should be fact-checked to account for such issues to ensure that the model's predictive capabilities remain aligned with the latest and most definitive medical data.BRIEF SUMMARY

[0006] In one implementation, a computing system for analyzing multimodal data using one or more large language machine learning models, the computing system comprising: one or more processors; and one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing systemto: input medical image data to a trained image-based machine learning model and generating medical image data embeddings; input genomic data to a trained genomics machine learning model and generating genomic data embeddings; input text data to a trained large language machine learning model, and generating a mapping of extruded data; and feed the medical image data embeddings, the genomic data embeddings, and the mapping of extruded data to trained large language machine learning model at a foundation layer, the trained large language machine learning model at the foundation layer being trained to generate one or more of a data and analytics report, a diagnostic analysis report, a large language model agent, or user data for display in a user interface at an output device.

[0007] In yet another implementation, a computer-implemented method for analyzing multimodal data using one or more large language machine learning models, the computer- implemented method comprising: inputting, via one or more processors, medical image data to a trained image-based machine learning model and generating medical image data embeddings; inputting, via the one or more processors, genomic data to a trained genomics machine learning model and generating genomic data embeddings; inputting, via the one or more processors, text data to a trained large language machine learning model, and generating a mapping of extruded data; and feeding, via the one or more processors, the medical image data embeddings, the genomic data embeddings, and the mapping of extruded data to trained large language machine learning model at a foundation layer, the trained large language machine learning model at the foundation layer being trained to generate one or more of a data and analytics report, a diagnostic analysis report, a large language model agent, or user data for display in a user interface at an output device.

[0008] In various implementations, the computing system and / or computer-implemented method performs postproduction monitoring of an output of the trained large language machine learning model at the foundation layer.

[0009] In various of these implementations, the computing system and / or computer- implemented method assesses sources of data bias in the medical image data embeddings, the genomic data embeddings, and / or the mapping of extruded data; and determines a retraining algorithm for one of more of the trained image-based machine learning model the trained genomics machine learning model, the trained large language machine learning model, and trained large language machine learning model at a foundation layer.

[0010] In various of these implementations, the computing system and / or computer- implemented method assesses metadata in the medical image data embeddings, the genomic data embeddings, and / or the mapping of extruded data.

[0011] In various of these implementations, the computing system and / or computer- implemented method presents a user with a unvalidated response error tool; displays to the user one or more of the data and analytics report, the diagnostic analysis report, the large language model agent, and the user data; and receives user assessment of the output.

[0012] In various of these implementations, the computing system and / or computer- implemented method applies a reverse Retrieval-Augmented Generation (RAG) process in which the output and an indication of relevant source material associated with the output is presented to a user.

[0013] In various of these implementations, the reverse RAG process includes: performing a pre-processing, in the trained large language machine learning model at the foundation layer, on the medical image data embeddings, the genomic data embeddings, and / or the mapping of extruded data to convert the embeddings a vectorized data; submitting a retrieval query to in the trained large language machine learning model at the foundation layer, and, in the in the trained large language machine learning model at the foundation layer, encoding the retrieval query with a structure corresponding to the vectorized data; performing a semantic search for matches between the vectorized data and the retrieval query; and presenting the results of the semantic search to the user.

[0014] In various of these implementations, the reverse RAG process includes: performing, in the trained large language machine learning model at the foundation layer, summarization of retrieved data corresponding to one of more of the trained image-based machine learning model the trained genomics machine learning model, and the trained large language machine learning model; generating, at the trained large language machine learning model at the foundation layer, a summary containing the summarization; determining, at the trained large language machine learning model at the foundation layer one or more outputs of the trained large language machine learning model at the foundation layer for fact checking by a user; and displaying the one or more outputs to a user.BRIEF DESCRIPTION OF THE FIGURES

[0015] The figures described below depict various implementations of the system and methods disclosed therein. It should be understood that each figure depicts one implementation of a particular implementation of the disclosed system and methods, and that each of the figures is intended to accord with a possible implementation thereof. Further, wherever possible, the following description refers to the reference numerals included in the following figures, in which features depicted in multiple figures are designated with consistent reference numerals.

[0016] The figures depict preferred implementations for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative implementations of the systems and methods illustrated herein may be employed without departing from the principles of the invention described herein.

[0017] FIG. 1A depicts an example computing environment 100 in which the techniques disclosed herein may be implemented, according to some embodiments.

[0018] FIG. 1 B illustrates an example implementation of a multimodal LLM configuration.

[0019] FIG. 2 illustrates an example multimodal LLM architecture in accordance with the present techniques.

[0020] FIG. 3 illustrates an example process that allows for the integration of imaging-based Al in the clinical workflow leading to robust validation.

[0021] FIG. 4 illustrates data analysis in an existing radiology clinical workflow process.

[0022] FIG. 5A depicts an example postproduction architecture provides access to all the metadata produced as part of image data, routing and algorithm execution.

[0023] FIG. 5B illustrates an example image report with a machine learning model classified cancer region identified in the image report.

[0024] FIG. 5C illustrates an example report with identified cancer diagnosis data.

[0025] FIGS. 6A-6C depict an example dashboard for viewing metadata, EHR data, and / or an algorithm training set as part of a postproduction monitoring process, in accordance with embodiments described herein.

[0026] FIGS. 7A and 7B illustrate an example sequence diagram for validation of Al output as part of the radiology workflow implemented with the foundation model in accordance with an example of the present techniques.

[0027] FIG. 8 illustrates an example training workflow for a foundation model in accordance with the present techniques, indicating a preprocessing operation.

[0028] FIG. 9 illustrates an example process flow for developing and validating a foundation model of multimodal, multi-layer LLM's in accordance with an example.

[0029] FIG. 10 illustrates an example training process for LLAMA2 (7B parameter LLM) finetuning to develop the foundation models in accordance with the present techniques.

[0030] FIG. 11 illustrates another example of pretraining processes that may be performed, in accordance with various examples of the present invention.

[0031] FIG. 12 depicts examples of challenges associated with validating LLMs and multimodal LLM applications.

[0032] FIG. 13 illustrates an example architecture that includes a postproduction monitoring layer for implementation of postproduction monitoring processes.

[0033] FIGS. 14A and 14B illustrates an example CURE process implemented in an integral part of the text generation validation.

[0034] FIGS. 15A and 15 B illustrate an example generated text from a text-based LLM model, where the generated text includes references for fact checking, including hallucinations.

[0035] FIG. 16 illustrates an example of postproduction monitoring of Al performance through a continuous monitoring process.

[0036] FIGS. 17A and 17B illustrate an example radiology multimodal LLM development with generated output report, in accordance with an example of the present techniques.

[0037] FIGS. 18A and 18B illustrates a generated discharge summary reports with operative notes summary of patient data that may be generated, according to some embodiments.

[0038] FIGS. 19A and 19B illustrate examples of a prototype platform presenting data in other formats to clinicians and healthcare professionals.

[0039] FIG. 20 depicts a combined block and logic diagram 2000 for training various machine learning models of the multimodal LLM architecture, in which the techniques described herein may be implemented, according to some embodiments.

[0040] FIG. 21 depicts an example workflow that may be monitored in a postproduction monitoring process.

[0041] FIG. 22 depicts an example sequence diagram of monitoring IT-related issues.

[0042] FIG. 23 depicts an example sequence diagram of monitoring user-related issues.

[0043] FIG. 24 depicts a flow diagram of an example method for analyzing multimodal data, according to some embodiments described herein.DETAILED DESCRIPTIONOverview

[0044] The present techniques provide methods and systems using large language models (LLMs) for, inter alia, for analyzing multimodal data using one or more large language machine learning models. The methods and systems include, for example, providing medical image datato a trained image-based machine learning model and generating medical image data embeddings, providing genomic data to a trained genomics machine learning model and generating genomic data embeddings; and providing text data to a trained large language machine learning model, and generating a mapping of extruded data. These trained machine learning models may sit in a first layer of a multilayer multimodal LLM architecture that includes a foundation layer that is fed the various model embeddings for generating one or more reports, diagnostic assessments, or generated LLM agents as discussed herein.

[0045] LLMs can play an invaluable role in healthcare applications. LLMs may be integrated into clinical decision support systems to assist doctors in diagnosing diseases, providing realtime recommendations, and streamlining patient data entry. LLMs also excel in patient education, simplifying complex medical information, and improving patient engagement. They can manage medical records by automating transcription and summarization, ultimately saving healthcare provider’s time. In telemedicine, LLMs can be used to analyze patient data, generate alerts, and assist with remote monitoring. Moreover, LLMs can be embedded in chatbots or virtual assistants to perform administrative tasks, schedule appointments, and offer basic medical advice.

[0046] In addition to their textual capabilities, there is a growing need for LLMs to encode multimodal information in healthcare applications. Healthcare data often involves a combination of text, images, and even audio, making it essential for LLMs to process and understand diverse data types. Incorporating these multimodal capabilities enables LLMs to analyze medical images, such as X-rays or MRIs, in conjunction with clinical notes, improving diagnostic accuracy. Furthermore, healthcare data may also involve voice data from patient interviews or medical consultations, and incorporating these capabilities enables LLMs to enhance the depth of patient interactions and enabling more comprehensive healthcare support. The ability to encode and interpret multimodal information empowers LLMs to offer a more holistic approach to healthcare, making them indispensable tools in this ever-evolving field.

[0047] In the healthcare industry, it is envisioned that there will be a growing demand for the development of an extensive repository of specialized LLM-like models tailored to handle specific tasks. While classical Al approaches may be better suited for certain tasks within the healthcare domain, LLM models may be desirable as as orchestrators, coordinating the actions of smaller, specialized models.

[0048] With recent advancements in large language models (LLMs), various techniques have been proposed to use LLM’s in a healthcare context. However, one of the main challenges withusing an LLM in a healthcare application is how to improve accuracy on specific tasks that require domain knowledge, logical reasoning, and / or common sense.

[0049] There are two basic strategies for improving accuracy in LLMs: fine-tuning and distillation. Fine-tuning is a method that adapts a pre-trained LLM to a specific task by updating its parameters on a task-specific dataset. Fine-tuning can help LLMs learn task-relevant features and reduce the gap between the pre-training and the target domains. However, fine- tuning also has some drawbacks, such as overfitting, forgetting, and instability. Overfitting occurs when the LLM memorizes the task-specific data and fails to generalize to new examples. Forgetting occurs when the LLM loses some of the general knowledge and skills that it acquired during pre-training. Instability occurs when the LLM produces inconsistent or contradictory outputs due to the sensitivity of its parameters. Furthermore, fine-tuning may be very timeconsuming and require vast amounts of computing resources, which may be impractical in some cases. Additionally, fine-tuned LLMs may be vulnerable to hallucinations, i.e. , generating made-up information that may be false. Such hallucinations may be particularly harmful in a healthcare context, as false information may harm patients.

[0050] In contrast, distillation compresses a large and complex LLM into a smaller and simpler one by transferring its knowledge and capabilities. Distillation can help LLMs reduce their computational cost, memory footprint, and inference time, while maintaining or even improving their task accuracy. However, distillation also has some challenges, such as information loss, alignment, and diversity.

[0051] Information loss refers to the amount of knowledge and skills that the LLM loses during the compression process. Information loss can be measured by comparing the performance of the original and the distilled LLMs on various tasks and metrics. Alignment refers to the degree of similarity and consistency between the outputs of the original and the distilled LLMs. Alignment can be measured by comparing the probabilities, embeddings, or representations of the two LLMs. Diversity refers to the ability of the distilled LLM to generate diverse and novel outputs, rather than copying or repeating the outputs of the original LLM. Diversity can be measured by calculating the entropy, diversity, or novelty scores of the generated outputs.

[0052] Model distillation is a technique for compressing a large and complex language model (LLM) into a smaller and simpler one, while preserving its knowledge and capabilities. Model distillation typically involves two steps: teacher-student training and student evaluation. Teacher-student training is the process of teaching the smaller LLM, called the student, to mimicthe behavior of the larger LLM, called the teacher. Student evaluation is the process of testing the performance of the student LLM on the target tasks and metrics.

[0053] Model distillation can be beneficial for several reasons. First, model distillation can reduce the resource requirements of LLMs, such as computation, memory, storage, and energy consumption. This can make LLMs more accessible, affordable, and scalable for various applications and devices. Second, model distillation can improve the robustness and generalization of LLMs, by removing redundant or noisy information and retaining the essential and transferable knowledge. This can enhance the performance of LLMs on different domains and tasks, especially for low-resource or out-of-distribution scenarios.

[0054] There are different ways of implementing teacher-student training, depending on how the student LLM learns from the teacher LLM. One way is output distillation, which trains the student LLM to match the output probabilities of the teacher LLM. While output distillation is simple and effective, it may lose some of the fine-grained information and diversity of the teacher LLM. Another way is feature distillation, which trains the student LLM to match the intermediate features or representations of the teacher LLM. Feature distillation is more complex and challenging, but it may capture more of the semantic and syntactic information and diversity of the teacher LLM. A third way is hybrid distillation, which combines output distillation and feature distillation to balance the advantages and disadvantages of both methods.

[0055] There are as number of conventional implementations of model distillation, with various levels of success.

[0056] - Fastformer (https: / / github.com / microsoft / fastformer) is a tool for distilling transformerbased LLMs into smaller and more efficient LLMs. Fastformer uses an attention mechanism, called fast attention, which approximates the self-attention matrix with low-rank matrices and reduces the computation and memory costs. Fastformer also uses a progressive training method, which gradually increases the model complexity and the input length during the distillation process. Fastformer can achieve competitive results on several natural language understanding and generation tasks, such as sentiment analysis, machine translation, and text summarization, while reducing the model size by 2 to 4 times and the inference time by 6 to 8 times.

[0057] - TinyBERT (https: / / github.com / huawei-noah / Pretrained-Language- Model / tree / master / TinyBERT): TinyBERT is a tool for distilling BERT-based LLMs into smaller and faster LLMs. TinyBERT uses a four-stage learning framework, which includes general distillation, task-specific distillation, data augmentation, and intermediate layer adaptation. TinyBERT can achieve comparable or even better performance than BERT on various naturallanguage understanding tasks, while reducing the model size by 7.5 times and the inference time by 9.4 times.

[0058] - DistilBERT (https: / / github.com / huggingface / transformers / tree / master / examples / research_projects / distillatio n): DistilBERT is a tool for distilling BERT-based LLMs into smaller and more efficient LLMs. DistilBERT uses output distillation to train a student LLM with fewer layers and attention heads than the teacher LLM. DistilBERT can retain 97% of the performance of BERT on the GLUE benchmark, while reducing the model size by 40% and the inference time by 60%.

[0059] - MINILM (https: / / github.com / microsoft / unilm / tree / master / minilm): MINILM is a tool for distilling large pre-trained LLMs, such as BERT, RoBERTa, and ALBERT, into smaller and more efficient LLMs. MINILM uses a hybrid distillation method, which consists of multi-task learning, vocabulary alignment, soft-position embedding, and knowledge distillation. MINILM can achieve state-of-the-art results on several natural language generation tasks, such as text summarization and question answering, while reducing the model size by 5 to 6 times and the inference time by 4 to 5 times. MINILM is licensed under the MIT license.

[0060] Yet another strategy for improving the accuracy of LLMs and accounting for new or additional contextual information involves retrieval augmented generation (RAG). RAG takes an input prompt and augments the prompt with contextual information to improve output generation. However, hallucinations may still persist even with the use of RAG.

[0061] Reinforcement learning is a type of machine learning that learns from its own actions and feedback, rather than from labeled data or rules. Reinforcement learning involves an agent, which is an autonomous artificial intelligence program or system that is designed to perceive and learn from its environment and take actions to achieve a specific goal or set of goals, and an environment, which is the world that the agent interacts with. The agent can perform actions in the environment, and receive rewards or penalties based on the outcomes of its actions. The goal of reinforcement learning is to find a policy, i.e., a strategy or a rule, that maximizes the expected cumulative reward over time.

[0062] One way to find a good policy is by using policy gradient methods, which are a family of reinforcement learning algorithms that directly optimize the policy parameters by following the gradient of the expected reward. Policy gradient methods can handle stochastic policies, which are policies that output a probability distribution over actions, rather than a single deterministic action. Stochastic policies are useful for exploration, which is the process of trying out different actions to discover new and potentially better states in the environment.

[0063] However, policy gradient methods may have some drawbacks, such as high variance, slow convergence, and sensitivity to the step size. These drawbacks can make policy gradient methods unstable and inefficient, especially when dealing with large and complex policy networks, such as LLMs. To overcome these drawbacks, Proximal Policy Optimization (PPO) was proposed as a novel policy gradient method that improves the performance and robustness of policy optimization.

[0064] PPO, in turn, generally refers to a reinforcement learning algorithm that optimizes a stochastic policy by using a surrogate objective function. PPO balances exploration and exploitation by clipping the probability ratio between the current and old policies, preventing large policy updates that could harm the performance.

[0065] instructGPT is a framework that leverages PPO to fine-tune large language models (LLMs) with natural language instructions. instructGPT allows users to specify the desired behavior of the LLM for a given task or domain, without requiring any labeled data or code. instructGPT uses the instructions as rewards for the reinforcement learning agent, which adapts the LLM to maximize the expected reward.

[0066] PPO and instructGPT can be used together to fine-tune LLMs for various downstream tasks, such as text summarization, sentiment analysis, question answering, and more.

[0067] As another example, Direct Preference Optimization (DPO) is a technique for reinforcement learning in LLMs that aims to overcome some of the limitations of fine-tuning and distillation. DPO is based on the idea of learning from human preferences, rather than from task-specific labels or rewards. DPO collects human feedback on the outputs of an LLM and uses it to optimize the LLM's parameters towards producing more preferred outputs. DPO does not require any task-specific data or supervision, and can be applied to any LLM and any natural language generation task.

[0068] DPO generally contains two main components: a preference model and a policy optimization algorithm. The preference model is a neural network that learns to predict the probability of a human preferring one output over another, given a context and a query. The preference model is trained on pairwise comparisons of LLM outputs collected from human raters. The policy optimization algorithm is a variant of Proximal Policy Optimization (PPO), a popular reinforcement learning method that updates the LLM's parameters using gradient ascent on the preference model's predictions. PPO ensures that the LLM's outputs remain close to the original distribution, while improving their quality and diversity.

[0069] DPO has several advantages over fine-tuning and distillation. First, DPO does not need any task-specific data or supervision, which reduces the annotation cost and increases the applicability of LLMs to different domains and tasks. Second, DPO does not suffer from overfitting, forgetting, or instability, as it preserves the general knowledge and skills of the LLM, while enhancing its task-relevant features. Third, DPO does not incur any information loss, alignment, or diversity issues, as it compresses the LLM in a way that reflects human preferences and expectations. Zepher is one of the several early LLMs tuned with DPO with good performance: https: / / arxiv.org / abs / 2310.16944

[0070] Yet, despite the various proposed techniques mentioned above, there remains a desire for more a accurate LLM architecture for use in healthcare applications.

[0071] As the use of Al applications in radiology workflows increases, the importance of effective post-production monitoring becomes paramount. Unlike traditional imaging tools, Al algorithms encounter challenges with generalizability and radiologist adoption. A 2016 report by the American College of Radiology Commission on Human Resources already flagged radiologist burnout as a major concern, with heavy workloads and strict time constraints being significant contributors. In this context, Al tools that include a human-in-the-loop approach should be closely monitored to ensure they lessen the radiologist's load rather than add to it. Current models employed in radiology, although initially effective, may become "stale" in a dynamic setting, struggling to adjust to changes such as protocol updates, new scanners, demographic shifts, and novel reconstruction methods. This is complicated further by Al algorithms’ tendency towards inconsistent results, which acts as a barrier to wider acceptance, as highlighted by some studies.

[0072] Moreover, concept drift represents a major hurdle, as examined by some studies, and the intricacies of clinical workflow tools pose additional challenges to Al algorithm integration with radiologist review mechanisms and healthcare summary generation. Additionally, drift may occur in generating healthcare summaries and / or scoring summary and reference pairs. The difficulties in interpretability and pinpointing out-of-distribution data are significant barriers to effective post-production monitoring. Yet, this process is useful for maintaining the safety of Al algorithms for patient care and to prevent overloading radiologists. By continuously evaluating an Al algorithm's performance after its deployment, healthcare providers can help ensure its diagnostic precision, its capacity to improve patient outcomes, and its adaptability to evolving medical standards. Additionally, effective post-production monitoring can inform the development of more resilient Al algorithms and strategies for continuous learning. Forexample, human-in-the-loop and / or reinforcement learning techniques may be used to correct any concept drift.

[0073] Human errors are often attributed to cognitive overload, visual fatigue, or biases that can lead to oversight or misinterpretation of subtle imaging findings. For example, a radiologist might miss a small lesion due to the vast amount of images they need to review, or they may misinterpret an image based on previous experience with similar cases. These errors are variable and can be influenced by a multitude of human factors, including fatigue, distraction, or even the time of day.

[0074] On the other hand, Al errors typically stem from limitations within the algorithms themselves or the data they were trained on. Al may fail to recognize a rare pathology that it was not sufficiently trained to detect, or it could misinterpret an image due to an anomaly that falls outside its programmed parameters. Unlike humans, Al does not get tired or emotionally affected, but it can be consistently wrong in predictable ways if there is a systemic issue with its training data or algorithmic design. Because Al models in healthcare often inform critical decision-making processes their outputs must be monitored for consistency and correctness post-deployment. The actions taken on Al findings must also be recorded and scrutinized to optimize patient outcomes and refine clinical workflows.

[0075] Furthermore, Al algorithms may struggle with generalizability, performing well on data similar to the training set but poorly on new datasets, especially if these datasets come from different patient populations or are acquired using different imaging techniques. Unlike human radiologists who can draw on their training and experience to adapt to new information, Al algorithms need to undergo retraining or fine-tuning to accommodate such changes.

[0076] In the domain of post-production monitoring of Al applications, especially within the dynamic landscape of radiology, other factors may significantly impact the performance of Al models. One such factor is the software version changes. With each update, there is a potential for alterations in the model's behavior, necessitating rigorous testing and validation to ensure continued accuracy and reliability.

[0077] Education on Al usage and workflow optimization is also vital, especially in the nascent stages of Al implementation in radiology. To establish robust post-production monitoring that reflects the unique nature of Al algorithms, a multidisciplinary team is frequently essential.

[0078] Post-production monitoring plays a critical role in evaluating the value of Al analytics in clinical workflows. By continuously analyzing the performance and outcomes of Al tools aftertheir deployment, healthcare providers can gain a deeper understanding of their effectiveness and efficiency in real-world settings. This ongoing assessment allows for the identification of areas where Al significantly enhances patient care, such as by improving diagnostic accuracy, reducing wait times, or personalizing treatment plans. Moreover, post-production monitoring provides valuable insights into the cost-effectiveness of these Al solutions. By comparing the improvements in healthcare outcomes and operational efficiencies against the financial investment made in these tools, healthcare organizations can determine if the price paid for Al technology truly reflects its value. This evaluation is valuable in helping ensure that investments in Al are not just technologically sound, but also economically viable and beneficial to patient care. Ultimately, this leads to more informed decisions about the integration and scaling of Al tools in healthcare settings, helping ensure that they deliver high value for the investment made.

[0079] The safety and success of Al algorithms in production depends on the postproduction workflow. Lack of monitoring workflow and poor implementations can lead to errors and hinder highly optimized workflows. When it comes to radiology focused Al applications the following monitoring elements should be considered: the data, the model, the complexity of the workflow, technical and human factors (specifically for applications that require human in the loop).

[0080] However, the vast amount of metadata that Al models generate and consume must be managed effectively. To resolve such issues, automation is key in handling this data at scale, with minimal errors and reduced manual overhead. Moreover, model monitoring should also extend to scenarios where no output is readily available. This silent performance may indicate underlying issues within the Al model or data input problems that should be identified and rectified to prevent erroneous clinical decisions. The translation of medical information, from imaging to patient notes, should be executed with post-production needs in mind. This includes confirming that the data formats are consistent, comprehensive, and conducive to ongoing monitoring and analysis. The availability of a robust toolset is important for effective postproduction monitoring. These tools must provide the capabilities for detailed performance analytics, real-time alerts, and trend analysis to anticipate and address potential model drifts or anomalies. Further complications arise from a lack of ground truth to verify the output of the LLM. In such cases, the time lapse between the initial diagnostic procedure and the final entry of the pathology report into the patient's record can span several weeks. This delay inherently impacts the timeliness of the ground truth data available for training and validating Al models. The extended timeframe for obtaining definitive diagnoses makes it difficult to rapidly iterate and refine classification algorithms. Consequently, post-production monitoring should account for these lags and incorporate mechanisms to update the Al system when final outcomes arerecorded. Maintaining the relevance and accuracy of these classification algorithms requires meticulous tracking and integration of delayed ground truth into the system, helping to ensure that the model's predictive capabilities remain aligned with the latest and most definitive medical data. This necessitates a flexible and patient-centric approach to Al development in healthcare, where algorithms are designed to adapt and learn from ground truths that emerge on a non- immediate basis.

[0081] To resolve such issues, mechanisms to monitor the input quality as compared to the training and testing sets can act as an indicator of potential model performance degradation. By leveraging LLMs, organizations can automate the labor-intensive process of sifting through reports, thereby enhancing efficiency and potentially uncovering insights that might be missed by human review. However, there are notable drawbacks. LLMs may struggle with contextspecific nuances or rare clinical scenarios not well-represented in their training data, leading to inaccuracies. Additionally, the complexity and cost of implementing and maintaining these models can be significant. Integrating LLM outputs with existing clinical workflows and systems, may also be challenging, and such integration can be resource-intensive.

[0082] Consequently, post-production monitoring should account for these issues, such as managing resources, lag between initial diagnostic and final entry of a diagnostic report, and incorporate mechanisms to update the Al system when final outcomes are recorded.Maintaining the relevance and accuracy of these classification algorithms may involve meticulous tracking and integration of delayed ground truth into the system, helping to ensure that the model's predictive capabilities remain aligned with the latest and most definitive medical data. This, in turn, may necessitate a flexible and patient-centric approach to Al development in healthcare, where algorithms are designed to adapt and learn from ground truths that emerge on a non-immediate basis.

[0083] In various examples, the present techniques incorporate multimodal LLMs into clinical practice through two components. First is the deployment of specific toolsets for various tasks. Second, in various examples, the present techniques integrate with clinical tools, which may not only involve post-production monitoring but also the adaptation of a wide array of specialized Al applications for healthcare applications. This includes, for example, advanced Al ECG models for a diverse range of conditions such as atrial fibrillation, ejection fraction prediction, amyloidosis, hypertrophic cardiomyopathy (HCM), and various valvular diseases, all integrated into a comprehensive clinical dashboard. In various examples, these factors are important for a smooth and effective integration of multimodal LLMs into the healthcare ecosystem.

[0084] The present techniques may further include one or more client computing devices, or applications that may be installed on client computing devices, which enable users to access one or more trained models by providing input prompts that are processed by one or more trained models. These prompts may be propagated to the trained models for processing. The client computing device may receive responses from these models, and convey those responses to the user, optionally after performing pre-processing and / or post-processing operations.

[0085] As discussed herein, the systems and methods of the present disclosure may be used to develop and refine LLMs through specialized tools, to enhance patient care and healthcare overall. In particular, some embodiments of the present disclosure provide for an operating system (OS) powered by generative Al that allows for integrating specialized tools that encompasses expertise of healthcare provides that can be integrated with and establish robust collaborations with systems and tools provided by external vendors. The systems and methods can thus provide can function as an OS, co-pilot, that utilizes LLM agents to interface with existing models and electronic medical records (EMR), etc. determining the most suitable trained LLM models to be used for each patient's data. More specifically, in the present disclosure, in various examples, systems and methods may allow for tailored LLM model provisioning based on the patient’s data and with provisioning managed through a single operating system, which may reduce computer resource demand, reduces network bandwidth demands, while ensuring an optimal integration of Al insights (including generative Al) to be used for patient care.

[0086] As discussed herein, the systems and methods of the present disclosure provide several features through a generative Al-based OS. Such features may include Medical Language Interface (MLI) that is an OS service that provides a specialized interface for healthcare professionals, using medical terminologies and jargon, e.g., through a specialized LLM tool. Healthcare workers may be able to interact with the LLM tool in these embodiments using conversational language typical in medical settings, facilitating tasks like accessing patient records, scheduling appointments, or querying medical databases. Further, the disclosed generative Al-based OS may support a Clinical Task Automation and Decision Support (CTADS) service, which is an OS feature for automating routine administrative tasks such as patient data entry, appointment scheduling, and insurance processing. Additionally, CTADS may assist in clinical decision-making by providing evidence-based recommendations, drug interaction checks, and diagnostic support, all through natural language queries. Another feature provided by the disclosed generative Al-based OS is an Integrated Medical KnowledgeBase (IMKB), that leverages LLMs trained on extensive medical literature and clinical guidelines. Through the feature, the OS can serve as a dynamic medical knowledge resource, offering e.g., it could offer real-time medical information retrieval, patient education materials, and up-to-date clinical guidelines to healthcare providers. Yet another feature is Personalization for Patient Care (PPC), in which the generative Al-based OS may be personalized to individual patient profiles, considering their medical history, preferences, and specific health conditions. This personalization may enhance patient engagement and support tailored health advice, medication management, and treatment planning. Further, as described herein, the generative Al-based OS may allow for Cross-Platform Integration in Healthcare Ecosystem (CPIHE), which allows for hosting systems, LLM tools, etc. to integrate with various healthcare platforms and electronic health record (EHR) systems, enabling seamless data exchange and coordination of care across different healthcare providers, laboratories, and pharmacies.

[0087] In accordance with the above, and with the disclosure herein, the present disclosure includes improvements in computer functionality or in improvements to other technologies at least because the disclosure describes a generative Al-based OS that overcomes unique techniques challenges in the healthcare field. These challenges include enhancing accuracy and reliability in clinical recommendations, maintaining patient confidentiality and data security, and complying with healthcare regulations (like HIPAA in the United States). The disclosed system may also consider ethical issues such as bias in Al algorithms and the impact on the patient-provider relationship. More specifically, generative Al-based OS in some embodiments provides improved intelligence and / or predictive ability of a hosting server or computing device through a multilayered trained machine learning model that allows for personalizing machine learning tools, in particularly LLM tools to the patient based on patient and / or other data.

[0088] Thus, the architectures of the present techniques may be designed to fact-check LLM outputs through a reverse RAG process, in which the generated output becomes an input to an LLM to retrieve relevant references to “show” its reasoning in generating the output.Furthermore, a validation environment may route information efficiently to users (e.g., clinical and technical reviewers) such that the users may assess an output (e.g., a summary) generated by the LLM. The generated output and retrieved references may be scored to indicate the quality of the references. Additionally, users may correct the output in real-time. The scoring information and corrected output may be used for further training and / or retraining of the LLM, if desired, to improve the LLM’s ability to generate accurate outputs.Example Computing Environment

[0089] FIG. 1A depicts an example computing environment 100 in which the techniques disclosed herein may be implemented, according to some implementations. The environment 100 may include computing resources for training and / or operating machine learning models to perform simulated debate in a clinical decision-making context, in some implementations.

[0090] The computing environment 100 may include a client computing device 102, a server computing device 104, an electronic network 106, a context electronic database 110, and a model electronic database 112. The computing environment may further include one or more cloud application programming interfaces (APIs) 114. The components of the computing environment 100 may be communicatively connected to one another via the electronic network 106, in some implementations.

[0091] The client computing device 102 may implement, inter alia, operation of one or more applications for performing multimodal LLM analysis using one or more machine learning models (e.g., large language machine learning models). In some implementations, the client computing device 102 may be implemented as one or more computing devices (e.g., one or more servers, one or more laptops, one or more mobile computing devices, one or more tablets, one or more wearable devices, one or more cloud-computing virtual instances, etc.). In some implementations, a plurality of client computing devices may be part of the environment 100 - for example, a first user may access a client computing device 102 that is a laptop, while a second user accesses the client computing device 102 that is a smart phone, while yet a third user accesses a client computing device 102 that is a wearable device.

[0092] The client computing device 102 may include one or more processors 120, one or more network interface controllers 122, one or more memories 124, an input device 126, an output device 128 and a client API 130. The one or more memories 124 may have stored thereon one or more modules 140 (e.g., one or more sets of instructions).

[0093] In some implementations, the one or more processors 120 may include one or more central processing units, one or more graphics processing units, one or more field- programmable gate arrays, one or more application-specific integrated circuits, one or more tensor processing units, one or more digital signal processors, one or more neural processing units, one or more RISC-V processors, one or more coprocessors, one or more specialized processors / accelerators for artificial intelligence or machine learning-specific applications, one or more microcontrollers, etc.

[0094] The client computing device 102 may include one or more network interface controllers 122, such as Ethernet network interface controllers, wireless network interface controllers, etc. The network interface controllers 122 may include advanced features, in some implementations, such as hardware acceleration, specialized networking protocols, etc.

[0095] The memories 124 of the client computing device 102 may include volatile and / or nonvolatile storage media. For example, the memories 124 may include one or more random access memories, one or more read-only memories, one or more cache memories, one or more hard disk drives, one or more solid-state drives, one or more non-volatile memory express, one or more optical drives, one or more universal serial bus flash drives, one or more external hard drives, one or more network-attached storage devices, one or more cloud storage instances, one or more tape drives, etc., and / or combinations thereof.

[0096] As noted, the memories 124 may have stored thereon one or more modules 140, for example, as one or more sets of computer-executable instructions. In some implementations, the modules 140 may include additional storage, such as one or more operating systems (e.g., Microsoft Windows, GNU / Linux, Mac OSX, etc.). The operating systems may be configured to run the modules 140 during operation of the client computing device 102 - for example, the modules 140 may include additional modules and / or services for receiving and processing data from one or more other components of the environment 100 such as the one or more cloud APIs 114 or the server computing device 104. The modules 140 may be implemented using any suitable computer programming language(s) (e.g., Python, JavaScript, C, C++, Rust, C#, Swift, Java, Go, LISP, Ruby, Fortran, etc.).

[0097] The modules 140 may include a model configuration module 142, an API module 144, an input processing module 146, an authentication / security module 148, a context module 150 and a cataloguing module 152, in some implementations. In some implementations, more or fewer modules 140 may be included. The modules 140 may be configured to communicate with one another (e.g., via inter-process communication, via a bus, via sockets, pipes, message queues, etc.).

[0098] The model configuration module 142 may include one or more sets of computerexecutable instructions (i.e., software, code, etc.) for performing the functionalities described herein. The model configuration module 142 may enable one or more machine learning models (e.g., large language machine learning models, image analysis models, etc.) to be stored, for example in the memory 124 or in the context electronic database 110.

[0099] In some implementations, the model configuration module 142 may be omitted from the modules 140, or its access may be restricted to administrative users only. For example, insome implementations, one or more of the modules 140 may be packaged into a downloadable application (e.g., a smart phone app available from an app store) that enables registered but non-privileged (i.e. , non-administrative) users to access the environment 100 using their consumer client computing device 102. In other implementations, one or more of the client computing device(s) 102 may be locked down, such that the client computing device 102 is controlled hardware, accessible only to those who have physical access to certain areas.

[0100] The API module 144 may include one or more sets of computer executable instructions for accessing one or more remote APIs, and / or for enabling one or more other components within the environment 100 to access functionality of the client computing device 102. In some implementations, the API module 144 may enable other client applications (i.e., not applications facilitated by the modules 140) to connect to the client computing device 102, for example, to send queries or prompts, and to receive responses from the client computing device 102. The API module 144 may include instructions for authentication, rate limiting and error handling.

[0101] As noted, the client computing device 102 may enable one or more users to access one or more trained models by providing input prompts that are processed by one or more trained models. The input processing module 146 may perform pre-processing of user prompts prior to being input into one or more models, and / or post-processing of outputs output by one or more models. For example, the input processing module 146 may process data input into one or more input fields, voice inputs or other input methods (e.g., file attachments) depending upon the application. The input processing module 146 may receive inputs directly via the input device 126, in some implementations.

[0102] In some implementations, the input processing module 146 may perform postprocessing of input received from one or more trained models. In some implementations, postprocessing (and / or pre-processing) may include implementing content moderation mechanisms, to prevent misuses of trained models or inappropriate content generation. The input processing module 146 may include instructions for handling errors and for displaying errors to users (e.g., via the output device 128). The input processing module 146 may cause one or more graphical user interfaces to be displayed, for example to enable the user to enter information directly via a text field.

[0103] The authentication / security module 148 may include one or more sets of computerexecutable instructions for implementing access control mechanisms for one or more trained models, ensuring that the model can only be accessed by those who are authorized to do so, and that the access of those users is private and secure.

[0104] Generally, trained models, utilize state information in order to meaningfully carry on a dialogue with a user or with another trained model. For example, if a user prompts a trained model with a question such as “What is the patient’s medical history relevant for arrhythmia?” followed by a second prompt “And how about cancer?” the model should understand that, in context, the second query relates to the first query, insofar as the user is asking about the medical history for the same patient.

[0105] However, language models (e.g., large language models (LLMs)) are generally stateless, meaning that after they process a prompt, they have no internal record or memory of the information that was input, or the information that was generated as part of the language model’s processing. Thus, some embodiments may add statefulness to models using context information. This may be implemented using sliding context windows, wherein a predetermined number of tokens (e.g., 4096 maximum tokens in the case of GPT 3.5, equivalent to about 3000 words) may be “remembered” by the LLM and can be used to enrich multiple sequential prompts input into the LLM (for example, when the LLM is used in a chat mode).

[0106] The context module 150 may include one or more sets of computer-executable instructions for maintaining state of the type found in this example, and other types of state information. The context module 150 may implement sliding window context, in some implementations. In other implementations, the context module 150 may perform other types of state maintaining strategies. For example, the context module 150 may implement a strategy in which information from the immediately preceding prompt is part of the window, regardless of the size of that prior prompt.

[0107] In some implementations, the context module 150 may implement a strategy in which one or more prior prompts are included in each current prompt. This prompt stuffing technique, or prompt concatenation, may be limited by prompt size constraints — once the total size of the prompt exceeds the prompt limit, the model immediately loses state information related to parts of the prompt truncated from the prompt.

[0108] The cataloguing module 152 may include one or more sets of computer-executable instructions for identifying, retrieving, analyzing, and / or cataloguing documents for a particular user (e.g., a patient). Depending on the implementation, the cataloguing module 152 may perform outside records cataloguing by developing a shared mapping of available data across scanned documents, databases (e.g., a patient healthcare information exchange database), radiology images, and / or other such documents. As further examples, the documents may include particular database and / or application-specific documents (e.g., process management documents and patient healthcare information documents) updated at least every 24 hours. Infurther implementations, the cataloguing module 152 accesses databases at least once every 24 hours for obtaining patient visit data to preschedule Al document processing. In some implementations, the cataloguing module accesses and / or retrieves documents for thousands of patients weekly. In further implementations, the cataloguing module 152 interfaces and / or otherwise integrates with one or more databases and / or other document repositories (e.g., Epic, process management database (e.g., directly through the process management database API or through a Longitudinal Patient Record API), patient healthcare information exchange database, Clarity through Denodo, Mayo Clinic Cloud (Cloud App Factory & Al Factory 2.0, which may be offered via a public, private, and / or hybrid cloud platform), etc.).

[0109] In further implementations, the mapping may include the data types present and their relevant dates. In some implementations, the cataloguing module 152 may focus on the specific data types used by the a particular entity (e.g., a breast clinic, a pulmonary clinic, etc.) for a particular purpose (e.g., new consult referrals), such as screening mammogram, diagnostic mammogram, biopsy procedure report, biopsy pathology report and post-biopsy imaging (e.g., an indication that a patient underwent mammogram on a given date and an indication of the associated report, an indication that the patient underwent biopsy on a different data and an indication of the report, an indication that an ultrasound report is absent, etc.). In further implementations, the cataloguing module 152 may implement techniques for reliably finding these documents among scanned records and extracting their relevant clinical date. In some implementations, the cataloguing module 152 includes and / or utilizes new data pipelines to access the data (e.g., including metadata) from a database (e.g., a radiology database).

[0110] In further implementations, the cataloguing module 152 additionally performs outside records summarization. As such, the cataloguing module 152 may enable users to generate customized, traceable records summaries to further streamline workflows. Similarly, the cataloguing module 152 may additionally perform outside records clean-up for removal of duplicated scanned documents, de-rotation of scanned pages (e.g., orientation matching), and / or Al-assisted translation of medical records in another language (e.g., Mandarin, Spanish, Arabic, etc.). In various implementations, these records summarizations generated by the cataloguing module 152 may be tailored for a multidisciplinary care team, e.g., a care team that includes surgical and medical oncology, radiology, radiation oncology, allied health staff, etc. By performed records summarization on multimodal data types, a more comprehensive synthesis and summary of records data may be performed without assistance from particular care team groups.

[0111] In further implementations, the cataloguing module 152 is communicatively coupled to and / or includes an exposed microservice for internal use (e.g., an API) for connection to other applications, programs, and / or other such projects that use outside medical records. In further implementations, at least some of the processes of the cataloguing module 152 may be implemented through an exposed microservice in external electronic health records computing systems, such as EPIC, (e.g., using SMART on an FHIR APP).

[0112] In still further implementations, the cataloguing module 152 may perform performance evaluation of the functionalities described herein. In some implementations, evaluation may focus on at least three dimensions: (1) appropriate data science metrics for the given task, (2) clinical usability and utility, and / or (3) system evaluation and testing.

[0113] The server computing device 104 may include one or more processors 160, one or more network interface controllers 162, one or more memories 164, an input device (not depicted), an output device (not depicted) and a server API 166. The one or more memories 164 may have stored thereon one or more modules 170 (e.g., one or more sets of instructions).

[0114] In some implementations, the one or more processors 160 may include one or more central processing units, one or more graphics processing units, one or more field- programmable gate arrays, one or more application-specific integrated circuits, one or more tensor processing units, one or more digital signal processors, one or more neural processing units, one or more RISC-V processors, one or more coprocessors, one or more specialized processors / accelerators for artificial intelligence or machine learning-specific applications, one or more microcontrollers, etc.

[0115] The server computing device 104 may include one or more network interface controllers 162, such as Ethernet network interface controllers, wireless network interface controllers, etc. The network interface controllers 162 may include advanced features, in some implementations, such as hardware acceleration, specialized networking protocols, etc.

[0116] The memories 164 of the server computing device 104 may include volatile and / or non-volatile storage media. For example, the memories 164 may include one or more random access memories, one or more read-only memories, one or more cache memories, one or more hard disk drives, one or more solid-state drives, one or more non-volatile memory express, one or more optical drives, one or more universal serial bus flash drives, one or more external hard drives, one or more network-attached storage devices, one or more cloud storage instances, one or more tape drives, etc.

[0117] As noted, the memories 164 may have stored thereon one or more modules 170, for example, as one or more sets of computer-executable instructions. In some implementations, the modules 170 may include additional storage, such as one or more operating systems (e.g., Microsoft Windows, GNU / Linux, Mac OSX, etc.). The operating systems may be configured to run the modules 170 during operation of the server computing device 104 - for example, the modules 170 may include additional modules and / or services for receiving and processing data from one or more other components of the environment 100 such as the one or more cloud APIs 114 or the client computing device 102. The modules 170 may be implemented using any suitable computer programming language(s) (e.g., Python, JavaScript, C, C++, Rust, C#, Swift, Java, Go, LISP, Ruby, Fortran, etc.).

[0118] In some implementations, the modules 170 may include a data collection module 172, a data pre-processing module 174, a model pretraining module 176, a fine-tuning module 178, a model training module 180, a checkpointing module 182, a hyperparameter tuning module 184, a validation and testing module 186, an auto-prompting module 188, a model operation module 190 and an ethics and bias module 192. In some implementations, more or fewer modules 170 may be included. The modules 170 may be configured to communicate with one another (e.g., via inter-process communication, via a bus, via sockets, pipes, message queues, etc.). The modules 170 may respond to network requests (e.g., via the API 166) or other requests received via the network 106 (e.g., via the client computing device 102 or other components of the environment 100).

[0119] The data collection module 172 may be configured to collect information used to train one or more modules of the multimodal LLM architectures described herein. In general, the information collected may be any suitable information used for training a large language model. For example, the data collection module 172 may collect data via web scraping, via API calls / access, via database extract-transform-load (ETL) processes, etc. Sources accessed by the data collection module 172 include social media websites, books, websites, academic publications, web forums / interest sites (e.g., Reddit, Facebook, bulletin boards, etc.), etc. The data collection module 172 may access data sources by active means (e.g., scraping or other retrieval) or may access existing corpuses. The data collection module 172 may include sets of instructions for performing data collection in parallel, in some implementations. The data collection module 172 may store collected data in one or more electronic databases, such as a database accessible via the cloud APIs 114 or via a local electronic database (not depicted). The data may be stored in a structured and / or unstructured format. In some implementations, the data collection module 172 may store large data volumes used for training one or moremodels (i.e. , training data). For example, the data collection module 172 may store terabytes, petabytes, exabytes or more of training data.

[0120] In some implementations, the data collection module 172 may retrieve data from one or more databases as described herein. For example, the data collection module 172 may process the retrieved / received data and sort the data into multiple subsets based on information included within the database. For example, the data collection module 172 may receive one or more sets of unstructured text (e.g., transcripts of one or more historical meeting minutes of a clinical review board). The data collection module 172 may chunk the data according to time (e.g., hourly, daily, quarterly, etc.).

[0121] Further, to achieve the multimodal functionality of the present invention, the data module collection module 172 may include data of different modalities, including, as shown and further described herein (for example, in reference to FIG. 2), medical image data such as 2D images and 3D images, captured video data, genomic data, where genomic data may present any number of omic data, signal data from monitoring equipment in a healthcare system, as well as text data.

[0122] The model preprocessing module 174 may include instructions for pre-processing data collected by the data collection module 172. In particular, the model preprocessing module 174 may perform text extraction and / or cleaning operations on data collected by the data collection module 172. The data pre-processing module 174 may perform preprocessing operations, such as lexical parsing, tokenizing, case conversions and other string splitting / munging. In some implementations, the data collection module 172 may perform data deduplication, filtering, annotation, compliance, version control, validation, quality control, etc.In some implementations, one or more human reviewers may be looped into the process of preprocessing data collected by the data pre-processing module 174. For example, a distributed work queue may be used to transmit batch jobs and receive human-computed responses from one or more human workers. Once pre-processed, the data pre-processing module 174 may store copied and / or modified copies of the training date in an electronic database.

[0123] In some implementations the data pre-processing module 174 may include instructions for parsing the unstructured text received by the data collection module 172 to structure the text. For example, when the text relates to clinician notes regarding a patient, the data pre-processing module 174 may generate a time series data structure in which each set of notes is represented by one or more timestamps, and at each timestamp, text associated with various speakers, determinations (e.g., treatment, recommendation, etc.) is labelled. The data pre-processing module 174 may also label the data according to the identity of one or morespeaker and / or one or more topic. For example, the time series data may be labeled according to one or more speakers associated with textual speech, in a language transcript form. The time series may include one or more keywords associated with the transcript. In some implementations, the present techniques may use a separate trained text summarization module to generate keywords used for this purpose. In this way, the data pre-processing module 174 may generate structured data corresponding to unstructured meeting minutes, such that the structured data is enriched with information about the meeting that is suitable for training. This structured data may be processed by downstream processes / modules.

[0124] Generally, the present techniques may train one or more models to perform language generation tasks that include token generation. Both training inputs and model outputs may be tokenized. Herein, tokenization refers to the process by which text used for training is divided into units such as words, subwords or characters. Tokenization may break a single word into multiple subwords (e.g., “LLM” may be tokenized as “L” and “LM”). The present techniques may train one or more models using a set of tokens (e.g., a vocabulary) that includes many (e.g., thousands or more) of tokens. These tokens may be embedded into a vector. This vector of token or “embeddings” may include numerical representations of the individual tokens in the vocabulary in high-dimensional vector space. The modules 170 may access and modify the embeddings during training to learn relationships between tokens. These relationships effectively represent semantic language meaning.

[0125] In some implementations, a specialized database (e.g., a vector store, a graph database , etc.) may be used to store and query the embeddings. Embedding databases may include specialized features, such as efficient retrieval, similarity search and scalability. For example, the server computing device 104 may include a local electronic embedding database (not depicted). In some implementations, a remote embedding database service may be used (e.g., via the cloud APIs 114). Such a remote embedding database service may be based on an open source or proprietary model (e.g., Milvus, Pinecone, Redis, Postgres, MongoDB, Facebook Al Similarity Search (FAISS), etc.). The server computing device 104 may include instructions (e.g., in the data collection module 172) for adding training data to one or more specialized databases, and for accessing it to train models.

[0126] The present techniques may include language modeling, wherein one or more deep learning models are trained by processing token sequences using a large language model architecture. For example, in some implementations, a transformer architecture may be used to process a sequence of tokens. Such a transformer model may include a plurality of layers including self-attention and feedforward neural networks. This architecture may enable themodel to learn contextual relationships between the tokens, and to predict the next token in a sequence, based upon the preceding tokens. During training, the model may be provided with the sequence of tokens and it learns to predict a probability distribution over the next token in the sequence. This training process may include updating one or more model parameters (e.g., weights or biases) using an objective function that minimizes the difference between the predicted distribution and a true next token in the training data. Particular techniques for training are discussed in more detail below.

[0127] Alternatives to the transformer architecture may include recurrent neural networks, long short-term memory networks, gated recurrent networks, convolutional neural networks, recursive neural networks, and other modeling architectures.

[0128] In some implementations, the modules 170 may include instructions for performing pretraining of a language model (e.g., an LLM), for example, in a pretraining module 176. The pretraining module 176 may include one or more sets of instructions for performing pretraining, which as used herein, generally refers to a process that may span pre-processing of training data via the data pre-processing module 174 and initialization of an as-yet untrained language model. In general, a pre-trained model is one that has no prior training of specific tasks. For example, the model pretraining module 176 may include instructions that initialize one more model weights. In some implementations, model pretraining module 176 may initialize the weights to have random values. The model pretraining module 176 may train one or more models using unsupervised learning, wherein the one or more models process one or more tokens (e.g., preprocessed data output by the data pre-processing module 174) to learn to predict one or more elements (e.g., tokens). The model pretraining module 176 may include one or more optimizing objective functions that the model pretraining module 176 applies to the one or more models, to cause the one or more models to predict one or more most-likely next tokens, based on the likelihood of tokens in the training data. In general, the model pretraining module 176 causes the one or more models to learn linguistic features such as grammar and syntax. The pretraining module 176 may include additional operations, including training, data batching, hyperparameter tuning, and / or model checkpointing.

[0129] The model pretraining module 176 may include instructions for generating a model that is pretrained for a general purpose, such as general text processing / understanding. This model may be known as a “base model” in some implementations. The base model may be further trained by downstream training process(es), for example, those training processes described with respect to the fine-tuning module 178. The model pretraining module 176 generally trains foundational models that have general understanding of language and / orknowledge. Pretraining may be a distinct stage of model training in which training data of a general and diverse nature (i.e., not specific to any particular task or subset of knowledge) is used to train the one or more models. In some implementations, a single model may be trained and copied. Copies of this model may serve as respective base models for a plurality of finetuned models.

[0130] In some implementations, base models may be trained to have specific levels of knowledge common to more advanced agents. For example, the model pretraining module 176 may train a medical student base model that may be subsequently used to fine tuned to generate another agent model such as an internist model, a surgeon model, a resident model, etc. In this way, the base model can start from a relatively advanced stage, without requiring pretraining of each more advanced model individually. This strategy represents an advantageous improvement, because pretraining can take a long time (many days), and pretraining the common base model only requires that pretraining process to be performed once.

[0131] FIGS. 10 and 11 illustrate examples of pretraining processes that may be performed by the pretraining module 175, in accordance with various examples of the present invention.

[0132] The modules 170 may include a fine-tuning module 178. The fine-tuning module 178 may include instructions that train the one or models further to perform specific tasks. For example, the fine-tuning module 178 may train each of a plurality of agents to generate one or more outputs that are based on each respective agents’ personality or characteristics.Specifically, the fine-tuning module 178 may include instructions that train one or more models to generate respective language outputs (e.g., text generation), summarization, question answering or translation activities based on characteristics of a user and / or corpus of documents.

[0133] Continuing the example, the fine-tuning module 178 may include sets of instructions for retrieving one or more structured data sets, such as time series generated by the data preprocessing module 174. These structured data sets may be sorted by time and / or speaker to train one or more machine learning models (e.g., one or more language models) that may be used within the environment 100 to simulate debate. For example, the fine-tuning module 178 may include instructions for configuring an objective function for performing a specific task, such as generating text that is similar to text found within the corpus of training data associated with a particular individual by role. For example, the fine-tuning module 178 may include instructions for fine-tuning a pathologist model, based on a base language model. These fine-tuning instructions may select statements of pathologists from one or more databases including acorpus of data (or another data source). A medical resident model may be fine-tuned by the fine-tuning module 178, wherein the base model is the same used to fine-tune the pathologist model. The fine-tuning module 178 may train many (e.g., hundreds or more) additional models.

[0134] In some implementations, the fine-tuning module 178 may include user-selectable parameters that affect the fine-tuning of the one or more models. For example, a “caution” bias parameter may be included that represents medical conservativeness. This bias parameter may be adjusted to affect the cautiousness with which the resulting trained model (i.e. , agent) approaches medical decision-making. Additional models may be trained, for additional personas / tasks, as discussed below.

[0135] In some implementations, to manage complexity of fine-tuning and other machine learning operations of the server computing device 104, one or more open source frameworks may be used. Example frameworks include TensorFlow, Keras, MXNet, Caffe, SciKit learn, PyTorch. Specifically for training and operating language models, frameworks such as OpenLLM and LangChain may be used, in some implementations. The fine-tuning module 178 may use an algorithm, such as stochastic gradient descent, or another optimization technique to adjust weights of the pretrained model.

[0136] Fine-tuning may be an optional operation, in some implementations. In some implementations, training may be performed by the training module 180 after pretraining by the model pretraining module 176. In some implementations, the model training module 180 may perform task-specific training like the fine-tuning module 178, on a smaller scale or with a more tailored objective. For example, whereas the fine-tuning module 178 may fine tune a model to learn knowledge corresponding to a surgeon, the model training module 180 may further train the model to learn knowledge of a plastic surgeon, an orthopedic surgeon, etc. Depending on the implementation, the model may be trained as and / or using proprietary models and / or techniques (e.g., the PALM2 / text-bison foundation model, MedPALM model, and / or other such LLMs).

[0137] FIG. 2 illustrates an example multimodal LLM architecture in accordance with the present techniques in which the fine-tuning module 178 may be implemented for postprocessing of outputs from a multimodal LLM.

[0138] The training module 180 may include one or more submodules, including the checkpointing module 182, the hyperparameter tuning module 184, the validation and testing module 186 and the auto-prompting module 188. The checkpointing module 182 may perform checkpointing, which is saving of a moders parameters. The checkpointing module 182 may store checkpoints during training and at the conclusion of training, for example, in the modelelectronic database 112. In this way, the model may be run (e.g., for testing and validation) at multiple stages and its training parameters loaded, and also retrained from a checkpoint. In this way, the model can be run and trained forward without being re-trained from the beginning, which may save significant time (e.g., days of computation). The hyperparameter tuning module 184 may include hyperparameters such as batch size, model size, learning rate, etc. These hyperparameters may be adjusted to influence model training. The hyperparameter tuning module 184 may include instructions for tuning hyperparameters by successive evaluation. The validation and testing module 186 may include sets of instructions for validating and testing one or more machine learning models, including those generated by the model pretraining module 176, the fine-tuning module 178 and the model training module 180. The auto-prompting module 188 may include sets of instructions for performing auto-prompting of one or more models. Specifically, the auto-prompting module 188 may enrich a prompt with additional information. The auto-prompting module 188 may include additional information in a prompt, so that the model receiving the prompt has additional context or directions that it can use. This may allow the auto-prompting module 188 to fine-tune a base model using one-shot of few-shot learning, in some implementations. The auto-prompting module 188 may also be used to focus the output of the one or more models.

[0139] In some implementations, the training module 180 may include instructions for training one or more additional machine learning models, such as supervised or unsupervised machine learning models. For example, as discussed below, in some implementations, the present techniques may include processing imaging data to enrich debate over a patient’s care. In that case, the patient’s imaging data may be processed by a model (e.g., a convolutional neural network) and the results processed further (e.g., by a language model) and / or provided to the client computing device 102. The training module 180 may train such a supervised model separately from training one or more language models. Further, the server computing device 104 may select one or more trained models at runtime based on data about a specific patient, based upon data contained in a prompt or based on other conditions that may be preprogrammed into the server computing device 104.

[0140] In some implementations, the training module 180 may train multi-modal models. For example, the training module 180 may train a plurality of models each capable of drawing from multimodal data types such as written text, imaging data, laboratory data, real-time monitoring data, pathology images, etc. In some cases, the training module 180 may train a single model capable of processing the multimodal data types. In some implementations, a trained multimodal model may be used in conjunction with another model (e.g., a large language model) toprovide non-text data interactions with users. Non-text data may be analyzed and integrated into the debate functions discussed herein. Further details regarding training the models are discussed herein.

[0141] The model operation module 190 may be a multimodal LLM module that operates one or more trained models. Specifically, the model operation module 190 may initialize one or more trained models, load parameters into the model(s), and provide the model(s) with inference data (e.g., prompt inputs). In some implementations, the model operation module 190 may deploy one or more trained model (e.g., a pretrained model, a fine-tuned model and / or a trained model) onto a cloud computing device (e.g., via the API 166). The model operation module 190 may receive one or more inputs, for example from the client computing device 102, and provide those inputs (e.g., one or more prompts) to the trained model. In some implementations, the API 166 may include elements for receiving requests to the model, and for generating outputs based on model outputs. For example, the API 166 may include a RESTful API that receives a GET or POST request including a prompt parameter. The model operation module 190 may receive the request from the API 166, and pass the prompt parameter into the trained model and receive a corresponding input. For example, the prompt parameter may be “Medical history relevant for arrythmia”. The prompt output may be “The patient has a history of atrial fibrillation and an atrial septic defect.”

[0142] The model operation module 190 may operate models in different modes. For example, in a first mode, the model operation module 190 may receive a prompt input via the client computing device 102, and provide that input to each of a plurality of agents for processing. The output of each agent may be collected and transmitted back to the client computing device 102 for display. The outputs may be labeled according to an identifier of each model (e.g., “pathologist,” “surgeon,” “medical student,” etc.). In the first mode, the model operation module 190 may receive additional inputs from the client computing device 102 that enable the user to interact with the one or more trained models in a question-answer format.

[0143] In a question-answer implementation, the user may ask follow up questions. For example, the user may enter a prompt such as “Tell me about patients who were similarly situated. How did they respond to SBRT? To surgery?” As noted, the server computing device 104 may have access to historical patient data, and the server 104 (e.g., the data preprocessing module 174) may include instructions for retrieving additional data from knowledge databases regarding patients to provide additional context to the one or more language models. These knowledge databases may include the electronic healthcare records database discussed above, as well as external sources such as academic papers, case studies, transcripts, etc. In someimplementations, the language models may be trained using this additional data ahead of time, and may not retrieve the data at runtime. For example, the training data for the liver cancer example may include the KRAS mutation status of the patient, their chemotherapy records, and outcomes for surgery, radiation, and other approaches. For example, FIG. 1B illustrates an example implementation of the multimodal LLM configuration of the operation module 190.

[0144] Thus, as discussed herein, some implementations use multi-modal modeling. The data preprocessing module may, for example, process and understand image data, audio data, video data, etc. The server computing device 104 may interpret and respond to queries that involve understanding content from these different modalities. For example, the server computing device 104 may include an image processing module (not depicted) including instructions for performing image analysis on images provided by users, or images retrieved from patient EHR data. In some implementations, the server computing device 104 may generate outputs in modalities other than text. For example, the server computing device 104 may generate an audio response, an image, etc. Combining multi-modal data may enable the present models to perform more comprehensive analysis of patient conditions, based on information processed in multiple different modes simultaneously.

[0145] The operating module 190 may include a set of computer-executable instructions that when executed by one or more processors (e.g., the processors 160) cause a computer (e.g., the server computing device 104) to perform retrieval-augmented generation. Specifically, the operating module 190 may perform retrieval-augmented generation based upon inputs or queries received from the user. This allows the operating module 190 to tailor responses of a model based on the specific input and context, such as the medical issue under discussion. For example, one or more models may be pre-trained, fine-tuned and / or trained as discussed above. During that training, the model may learn to generate tokens based on general language understanding as well as application-specific training. Such a model at that point may be static, insofar as it cannot access further information when presented with an input query.

[0146] When the model is used at runtime, however, such as when deployed in the environment 100, the operating module 190 may perform retrieval operations, such as searching or selecting information from a document, a database, or another source. The operating module 190 may include instructions for processing user input and for performing a keyword search, a regular expression search, a similarity search, etc. based upon that user input. The operating module 190 may input the results of that search, along with the user input, into the trained model. Thus, the trained model may process this additional retrieved information to augment, or contextualize, the generation of tokens that represent responses tothe user’s query. In sum, the retrieval augmented generation that may be applied in some embodiments may allow the model to dynamically generate outputs that are more relevant to the user’s input query at runtime. Information that may be retrieved may include data corresponding to a patient (e.g., patient demographic information, medical history, clinical notes, diagnoses, medications, allergies, immunizations, laboratory results, oncology information, radiation and imaging information, vitals, etc.) and additional training information, such as medical journals, notes or speech transcripts from symposia or other meetings / conferences, etc.

[0147] The present techniques may trigger retrieval augmented generation by processing a prompt, in some implementations. For example, a prompt may be processed by the input processing module 146 of the client computing device 102, prior to processing the prompt by the one or more generative models. The input processing module 146 may trigger retrieval augmented generation based on the presence of certain inputs, such as patient information, or a request for specific information, in the form of keywords. The input processing module 146 may perform entity recognition or other natural language processing functions to determine whether the prompt should be processed using retrieval augmented generation prior to being provided to the trained model.

[0148] As discussed above, prompts may be received via the input processing module 146 of the client computing device 102 and transmitted to the server computing device 104 via the electronic network 106. In some implementations, the output of the model may be modulated prior to being transmitted, output, or otherwise displayed to a user.

[0149] For example, the ethics and bias module 192 may process the prompt input prior to providing the prompt input to the trained model, to avoid passing objectionable content into the trained model. In some implementations, the ethics and bias module 192 may process the output of the trained model, e.g., to avoid providing objectionable output. It should be appreciated that trained language models may be unpredictable, and thus, processing outputs for ethical and bias concerns may be important in some implementations (e.g., such as in a medical context). Notwithstanding, medical professionals may use the present techniques for purposes of thought experimentation, i.e. , specifically to generate provocative debate. In doing so, the present techniques may advantageously assist practitioners with analyzing all possible treatment options, including those that may be discarded as too obvious, too expensive, too time-consuming, or having other seemingly undesired properties. Ultimately, the present techniques may be used to augment and solidify human decision making, rather than as a substitute for such deliberate thinking.

[0150] The client computing device 102 and the server computing device 104 may communicate with one another via the network 106. In some implementations, the client computing device 102 and / or the server computing device 104 may offload some or all of their respective functionality to the one or more cloud APIs 114. In some implementations, the one or more cloud APIs 114 may include one or more public clouds, one or more private clouds and / or one or more hybrid clouds. The one or more cloud APIs 114 may include one or resources provided under one or more service models, such as Infrastructure as a Service (laaS), Platform as a Service (PaaS), Software as a Service (SaaS), and Function as a Service (FaaS). For example, the one or more cloud APIs 114 may include one or more cloud computing resources, such as computing instances, electronic databases, operating systems, email resources, etc. The one or more cloud APIs 114 may include distributed computing resources that enable, for example, the model pretraining module 176 and / or other of the modules 170 to distribute parallel model training jobs across many processors.

[0151] In some implementations, the one or more cloud APIs 114 may include one or more language operation APIs, such as OpenAI, Bing, Claude. ai, etc. In other implementations, the one or more cloud APIs 114 may include an API configured to operate one or more open source models, such as Llama 2.

[0152] The electronic network 106 may be a collection of interconnected devices, and may include one or more local area networks, wide area networks, subnets, and / or the Internet. The network 106 may include one or more networking devices such as routers, switches, etc. Each device within the network 106 may be assigned a unique identifier, such as an IP address, to facilitate communication. The network 106 may include wired (e.g., Ethernet cables) and wireless (e.g., Wi-Fi) connections. The network 106 may include a topology such as a star topology (devices connected to a central hub), a bus topology (devices connected along a single cable), a ring topology (devices connected in a circular fashion), and / or a mesh topology (devices connected to multiple other devices). The electronic network 106 may facilitate communication via one or more networking protocols, such as packet protocols (e.g., Internet Protocol (IP)) and / or application-layer protocols (e.g., HTTP, SMTP, SSH, etc.). The network 106 may perform routing and / or switching operations using routers and switches. The network 106 may include one or more firewalls, file servers and / or storage devices. The network 106 may include one or more subnetworks such as a virtual LAN (VLAN).

[0153] The environment 100 may include one or more electronic databases, such as a relational database that uses structured query language (SQL) and / or a NoSQL database or other schema-less database suited for the storage of unstructured or semi-structured data.

[0154] The present techniques may store training data, training parameters and / or trained models in an electronic database such as the database 112. Specifically, one or more trained machine learning models may be serialized and stored in a database (e.g., as a binary, a JSON object, etc.). Such a model can later be retrieved, deserialized and loaded into memory and then used for predictive purposes. The one or more trained models and their respective training parameters (e.g., weights) may also be stored as blob objects. Cloud computing APIs may also be used to stored trained models, via the cloud APIs 114. Examples of these services include AWS SageMaker, Google Al Platform and Azure Machine Learning.

[0155] In operation, a user may access a prompt graphical user interface via the client computing device 102. The prompt graphical user interface may be configured by the model configuration module 142 and generated by the input processing module 146, and displayed by the input processing module 146 via the output device 128. Specifically, the model configuration module 142 may be configured to have one or more digital panel objects each comprising one or more trained models as digital agent objects. The model configuration module 142 may configure the graphical user interface to accept prompts and display corresponding prompt outputs generated by one or more models processing the accepted outputs. The input processing module 146 may be configured to transmit the prompts input by the user via the network electronic network 106 to the API 166 of the server computing device 104. The API 166 may process the user inputs via one or more trained models.

[0156] At the time the user accesses the prompt graphical user interface, one or more models may already be trained, including pretraining and fine-tuning. These trained models may be selectively loaded into the one or more agent objects based on configuration parameters, and / or based upon the content of the user’s input prompts.

[0157] In some implementations, the user may engage in a question-answer session with the client computing device 102. In some implementations, the user may provide a prompt and receive an output (e.g., a transcript) of a full discussion or debate between one or more models.

[0158] It will be understood that, although the various modules are depicted as belonging to one of the server computing device 104 or the client computing device 102, the system 100 may include a single device including the described module (e.g., the server computing device 104) that is accessed for remote computing and / or other use cases by one or more client devices (e.g., the client computing device 102, output device 128, etc.).

[0159] FIG. 1 B illustrates an example implementation of the multimodal LLM configuration of the operation module 190. The multimodal LLM configuration includes at least two layers of trained machine learning models, a first layer having trained imaging machine learning modelwith embedding, trained genomics machine learning model with embedding, a text embedding layer implemented in a text-based LLM with requests and responses, and an instruction layer. The first layer including the trained imaging machine learning model, trained genomics machine learning model, and text embedding layer may be in a first layer that is separate from the foundation layer. The instructions may be embedded by the word embedding layer 192a. The trained imaging machine learning model may encode images (e.g., X-rays, CT scans, MRIs, ultrasound scans, etc.) into a numerical format in the model encoding operation 194a and / or embed the images using the embedding model 194b. Encoding may convert the images into numerical values. Embedding may likewise convert the images into a numerical format (e.g., convert the image into one or more vector representations) and also capture semantic relationships between data. In some embodiments, embedding may include reducing the dimensionality of the data. The trained genomics machine learning model with embedding may also encode genomic data in an encoding operation 196a and / or embed genomic data with an embedding model 196b. The requests and responses may be embedded (e.g., generate a mapping of extruded data) by a word embedding layer 192b, e.g. a text embedding layer. The embeddings of each of the trained machine learning models in the first layer feed a foundation LLM 198 for implementing various processes and methods herein.

[0160] FIG. 2 illustrates a multi-layered LLM having a foundation model in accordance with the present application, which may be used in various personalized healthcare applications 218. An orchestration layer 216 may be used to coordinate the use of the multi-layered LLM in the healthcare applications 218. In some embodiments, the multi-layered LLM may be used for fine-tuned generative tasks 212 or for tool use 214. An example foundation model is described herein in a radiology application. More particularly, the multi-layered LLM is described in the context of rheumatoid arthritis (RA) analysis and prediction to assist in precision medicine decision making. By way of background, RA is a chronic inflammatory disorder primarily affecting joints, often leading to pain, swelling, and potentially joint deformity. Treatment options aim to reduce symptoms and improve quality of life. One of the mainstay treatments is Methotrexate, a disease-modifying antirheumatic drug (DM ARD). It works by suppressing the immune system to reduce inflammation, thereby slowing RA progression. However, Methotrexate poses challenges, including potential side effects like liver toxicity, lung problems, and increased susceptibility to infections.

[0161] Traditionally, Large Language Models (LLMs) have been adept at navigating the complexities of human languages, which involve a relatively small set of letters forming thousands of words, crafted into sentences and paragraphs. In contrast, the language ofbiology, particularly genomics, operates with a more limited alphabet: four nucleotides in DNA (A, T, G, C) and RNA (A, U, G, C), yet these are arranged into vastly more complex sequences.

[0162] This genomic language, while seemingly simpler due to its smaller alphabet, poses a significantly greater challenge for Al-driven models. For instance, the human genome encompasses over 3 billion nucleotides, and even the genomes of organisms as small as coronaviruses contain around 30,000 nucleotides. The primary difficulty lies in segmenting this immense sequence into meaningful and interpretable units.

[0163] To address these such issues, the multi-layered LLM of FIG. 2 may analyze diverse data types 202, including clinical notes, patient-reported symptoms, imaging, laboratory results, and critically (in some examples), genomic data, to provide a more informed and personalized approach to patient management. The data 202 may be used in vector searches 206 such that the vectors representing the data 202 may be compared to vectors representing laboratory knowledge, nursing knowledge, and / or patient records to retrieve data relevant to the data 202.

[0164] The muti-layered LLM configuration can leverage an LLM's advanced reasoning capabilities to interpret complex medical and genetic information, with the ability to tailor treatment insights to each patient's unique response. The inclusion of imaging data allows for a nuanced assessment of disease progression, enhancing our understanding beyond standard clinical metrics. The integration of genomic data is particularly groundbreaking, opening avenues for predicting individual responses to a treatment such as Methotrexate based on specific genetic markers.

[0165] This comprehensive approach aims to significantly enhance treatment strategies, reduce adverse effects, and improve outcomes in RA management. Such advancements could also be pivotal in addressing other chronic conditions, where treatment response varies widely among individuals.

[0166] As described herein, there are various components to development and testing of a multi-layered LLM configuration like that shown in FIGS. 1 B and 2, in particular for radiology applications. In the illustrated example in FIG. 2, the multi-layered LLM configuration includes in a first layer 204 having a series of trained machine learning models, e.g., formed of encoding and decoding layers and concatenation layers or in the case of a text a tokenizer. For example, the first layer includes a trained genomic machine learning model trained to identify potential genetic markers influencing treatment response, e.g., for RA. The first layer includes a 2D image machine learning model trained to analyze 2D medical images and a 3D image machine learning model trained to analyze 3D medical images, both for identify cancer tissue or other pathologies in image data. Similarly, a trained video machine learning model may be provided.The architecture in FIG. 2 may further include a fine-tuned publicly available medical domain LLM with specific data relevant to clinical content. The first layer may include (e.g., either developed or adapted) a trained imaging embedding machine learning model trained to integrate radiographic data, providing a comprehensive view of disease progression. Additionally, the first layer includes a trained text embedding LLM model that provides textual data to a second layer of the foundation model. The second layer 210 is an LLM (e.g., an application and / or enterprise-specific LLM model) and includes healthcare system specific embedding models.

[0167] This multi-modal, multi-layer LLM architecture can be used in fine-tuned generative tasks 212 or for tool use 214 to provide clinicians with valuable insights. For example, the multimodal, multi-layer LLM architecture may be used to enable clinicians to identify patients less likely to respond to, for example, Methotrexate, thus facilitating more targeted and effective treatment strategies.

[0168] To validate the multi-modal, multi-layer LLM architecture and assess performance, a robust validation dataset may be applied implementing measures to capture the system's performance accurately. In some examples, the dataset encompasses a diverse range of patient profiles to ensure the system's applicability across various demographics and disease severities. In particularly, datasets may be provided, and compared against comparative studies and clinical trial simulations, to test architecture outputs for Rheumatoid arthritis (RA) treatment (in an example), offering personalized, data-driven care options, and setting a precedent for the application of Al in personalized medicine.

[0169] The foundation model allows for evaluating the created app as part of the clinical workflow. Evaluating algorithms may be used as part of the clinical workflow to better assessment the usability and real-life performance of the foundation model, and such evaluation can provide insights on how to structure education materials for patients, postproduction monitoring, etc. Further still, as noted above, the foundation model allows for applications generated for a certain imaging modality to be extended to other applications. For example, the toolsets and processes created can be used to fast-track application creation in other imaging modalities.

[0170] While foundation models (FMs) offer an ability to tackle a wide array of reasoning and comprehension tasks, there are numerous challenges around developing suitable LLM models and for healthcare applications. For example, extensive and varied training medical imaging datasets may be desirable when building instruction tailored multimodal LLMs. These datasets are expensive to create due to the nature of the information. For instance, to design a VisualQuestion Answering system: X-ray images, question and answer sets and grounding information are typically used to allow for interpretability. Most of the publicly available models today are based on small scale models designed leveraging publicly available models. A further challenge is the complexity of medical data. For example, current vision and language encoders struggle to capture the intricacies of medical data. This creates a desire for custom training such models to encode the domain knowledge. LLM model evaluation is another challenge. Rigorous assessment of multimodal LLMs in the medical field is daunting. These models typically generate open-ended responses, making it challenging to evaluate the factual correctness and completeness of free-text responses, especially during large-scale evaluations requiring domain expertise. Interpretability is yet another challenge. Healthcare-focused applications typically demand the integration of mechanisms that ensure interpretability of findings. While attention mechanisms like grad-CAMs have been used, they often lack sufficient specificity. Therefore, the ability to precisely identify the regions utilized for generating the system's answers becomes a significant advantage. Incorporation of prior information presents another unique challenge. Within clinical workflows, accessing historical patient information when available holds significant importance. Designing LLMs that can effectively account for previous patient data presents a valuable aspect, albeit one fraught with challenges — particularly when dealing with missing information and the seamless integration of this capability into clinical workflows. Medical image size and resolution is yet another challenge for developing effective foundation models, as the majority of the state-of-the-art techniques subsample the images in sizes that hinder the ability to detect subtle disease.

[0171] Various examples of training a multimodal, multi-layer LLM architecture in accordance the present techniques are described herein. At an initial stage, this disclosure discusses the advantages of an in-domain LLM. While various open-source biomedical LLMs, such as BioMegatron and GatorTron exist, they are primarily trained on PubMed Central articles, and a small amount of EHR data.

[0172] The present techniques, in various examples, may leverage Vicuna 7B, an LLM from LMSYS. Additionally, to enrich this LLM with medical knowledge, some embodiments have developed pretraining that leverages sources, including PMC article abstracts, radiology, discharge summaries, medical terminology from Wikipedia, and clinical notes. However, other LLMs are within the scope of this disclosure.

[0173] Further, some embodiments may separately train a vision encoder specialized for X- ray images, such as chest X-rays (CXRs). For this, some embodiments may leverage a diverse set of X-ray images performed in a healthcare institution and employ contrastive learning as thetraining approach. For example, an autoencoder based on ViT-B model consisting of 86.6M parameters may be trained. To help ensure that the foundation model of the present techniques would be useful in case of small pathologies, in some examples, the training includes medical images having an input size of 2048x2048. Current, architectures downsample the data to sizes smaller that 518x518, hindering use in real life applications. To align the output of the image vision encoder with the text an adapter would be utilized, for example by having multiple LLM agents in the same layer of the foundation model. This architecture may allow for mapping visual data to the corresponding language space more efficiently. Further the architecture can also be used to automate report generation. In an example implementation, the vision encoder may be trained by leveraging more that 1.5 million X-ray images for pretraining purposes.These X-ray images may originate, for example, from across an entire healthcare system (locations, departments, etc.) to capture a diverse population and imaging equipment.

[0174] From the training, the multimodal, multi-layer LLM architecture in accordance the present techniques may be used develop an application centered on chest X-rays and tube placement. For example, application development with the foundation model of the present techniques can be performed using a visual question-answering paradigm. When prompted appropriately, the foundation model will identify and provide a comprehensive list of "appliances" (including tubes and lines) while assessing their positional adequacy and flagging any relevant complications. Existing toolsets, such as MAIRA, can be extended to include multiple views and timepoints as input. Accordingly, one feature and advantage of this methodology is to provide validation studies that can be performed to capture the performance of the foundation model. For the development of the final product a finetuning dataset will be created. To enhance foundation model interpretability, visual grounding may also be part of the model design, which will allow the users to better assess the output of the system.

[0175] A valuable aspect of validating Al algorithms involves acknowledging and addressing the differences that may arise between the dataset used during training and the data encountered in real-world production environments. Training datasets are often curated and may not fully represent the complexity and variability of real-world data. This disparity can lead to a phenomenon known as "dataset shift," where the model's performance degrades because the production data differs significantly from the training data in distribution, quality, or context. For instance, an Al model trained in healthcare on data from a specific demographic may not perform as well when applied to a more diverse population. Additionally, due to the complexity of the clinical workflow evaluating the algorithm as integrated in the workflow can provide valuable information that cannot be captured in offline validation studies.

[0176] Multimodal LLM - Radiology Foundation Model

[0177] Radiology has been advanced through recent advent of trained machine learning models, such as CLIP (Contrastive Language-Image Pretraining) and multi-modal Large Language Models (LLMs). These Al-driven tools offer unprecedented capabilities in analyzing and interpreting medical images, bringing a new dimension to diagnostics and patient care. Multi-modal LLMs, for example, excel in interpreting complex data by combining text, images, and other forms of data. CLIP may be particularly desirable in some applications for its ability to understand and link visual content with natural language descriptions. In radiology, CLIP can be used to match radiographic images with textual descriptions, aiding radiologists in identifying subtle patterns or anomalies that might be missed by the human eye.

[0178] The systems and methods of the present techniques may use Al models, such as CLIP, to rapidly process large volumes of imaging data, reducing the time radiologists spend on image analysis. This efficiency not only accelerates the diagnostic process but also reduces the likelihood of human error, potentially leading to better patient outcomes. In particular, some embodiments may use these models can analyze radiographic images in conjunction with patient histories, lab results, and clinical notes, leading to more comprehensive and accurate diagnoses

[0179] The systems and methods of the present techniques, in particular, multi-layered LLM having a foundation model may offer numerous advantages over conventional Al models. A foundation model can be extended to differ image modalities, such as forming a trained X-ray machine learning model capable of accommodating diverse inputs, including various views and follow-up scans from the same patient. LLM models across the different layers of the foundation model can be adapted to domain specific applications. The entire layer multi-layer architecture may be adaptable to allow for fast-track development of Radiology specific applications. Additionally, the embeddings models created may be able to be used by multimodal LLM models.

[0180] For example, the foundation model may be used to develop an application centered on chest X-rays and tube placement. A multimodal radiation LLM may be provided, for example, for determining Tubes Placement in Sequential X-Ray Imaging. In an example, this multimodal radiation LLM is designed to validate the proper placement of lines and tubes in patients undergoing treatment requiring frequent radiological monitoring. This advanced Al- driven tool uniquely integrates two distinct inputs: serial chest X-ray images taken at different timepoints, and a textual prompt detailing the clinical context or specific queries about the placement of medical devices. In some examples, the multimodal radiation LLM will follow avisual question-answering paradigm. When prompted appropriately, it will identify and provide a comprehensive list of "appliances" (including tubes and lines) while assessing their positional adequacy and flagging any relevant complications. Thus, leveraging the foundation model and domain specific LLM system may allow for generation of an X-ray focused application that meets performance expectations with less data while performing good in out of distribution cases.

[0181] FIGS. 17A and 17B illustrate an example radiology multimodal LLM development with generated output report, in accordance with an example of the present techniques. In FIG. 17A, a series of embeddings are used to determine if the tube and / or line position changes between different input X-ray images taken at two different timepoints, where embedding outputs are concatenated and provided to the trained radiology multimodal LLM for generating the report in FIG. 17B. A query 1702 may include text and radiology images such as X-rays. The X-rays may be passed to an X-ray encoder and adaptor 1704 to embed the X-ray images into embeddings 1706. The text of the query 1702 may also be embedded into embeddings 1706. The embeddings 1706 may then be concatenated passed to an LLM 1708, which may be a foundational model, to generate a report 1710, which is shown in FIG. 17B.

[0182] The present techniques may allow for integrating the application development using the foundation model in a clinical workflow and allow for creating a post-production mechanism for monitoring and performance evaluation. FIG. 3, for example, illustrates a process, e.g., based on what has been termed “FAST” data analysis within an existing radiology clinical workflow process (FIG. 4), that allows for the integration of imaging-based Al in the clinical workflow leading to robust validation. The "FAST” process has validated 12 analytics (FIG. 4). As part of the application development process in the foundation model, the technology required to enable such translation was created. The FAST process of FIG. 4 is compatible with various medical device software and requirements, as shown in the architecture of FIG. 5A. .

[0183] FIG. 5A depicts a postproduction architecture that provides access to all the metadata produced as part of image data, routing and algorithm execution, as well as the ability to combine this information with EHR datasets. The postproduction architecture may include a digital imaging and communications in medicine (DICOM)ZAI orchestrator 502, Al analytics 504, a picture archiving and communication system (PACS) 506, an algorithm / imaging metadata database 510, an EHR database 512, an algorithm training set database 514, one or more APIs 518, and an application 520.

[0184] Al analytics 504 may be used to analyze images by using a foundational model, for example. The images may include different types of medical images. For example, a radiologydepartment may analyze X-ray images, as shown in FIG. 5B which illustrates an image report with a machine learning model classified cancer region identified in the image report. The classified cancer region may be a part of the Al analytics 504. Additionally, Al analytics 504 may be used to generate a report with identified cancer diagnosis data, as illustrated in FIG. 5C. The DICOM / AI orchestrator 502 may define formats for medical images, specify how to exchange medical images and related, and manage the storing and transmission of images. The DICOM / AI orchestrator 502 may be used in conjunction with the PACS 506 to store, retrieve, transmit, and present images such that the software for such monitoring is aligned with medical device requirements. The DICOM / AI orchestrator 502 may also be used to manage metadata associated with the images (e.g., images stored using PACS 506, algorithms used in Al analytics 504). A user 508 may provide feedback on the algorithm / imaging metadata 510 as part of a post-production monitoring mechanism. EHR data from the EHR database 512 and algorithm training datasets 514 may be combined with the information from the algorithm / imaging metadata 510. The metadata, EHR datasets, and algorithm training set may be accessed by an application 520 via one or more APIs 518. In some embodiments, the application 520 may present a dashboard (e.g., the dashboard depicted in FIGS. 6A, 6B, and 6C) for viewing the metadata, EHR data, and / or the algorithm training set as part of the postproduction monitoring process.

[0185] As discussed above, FIG. 5B illustrates an image report with a machine learning model classified cancer region identified in the image report. The image in the image report may be stored, retrieved, transmitted, and / or presented by using the DICOM / AI orchestrator 502 and / or the PACS 506. The classified cancer region may be identified as part of the Al analytics 504, and may be identified by using a trained machine learning model, such as a foundational model.

[0186] FIG. 5C illustrates a generated report with identified cancer diagnosis data, include Al generated physical characteristics data 502, contextual element data 506, and a predictive analysis data 504 showing eGFR future prediction disease scoring. Additional findings data may also be included.

[0187] FIGS. 6A, 6B, and 6C present respective portions a dashboard, such as a dashboard from the application 520, that provides a concise summary of one of the deployed analytics. Complementing this, the monitoring system may also record all support requests, issues and complaints associated with the deployed algorithms. The dashboard depicted in FIGS. 6A, 6B, and 6C may be used in post-production monitoring to monitor the performance of an Al algorithm.

[0188] Thus, the architectures of the present techniques may be designed to capture information from the algorithm, technical infrastructure, and the users. Additionally, this information can be leveraged in combination with information retrieved from the electronic health record (EHR). The validation environment is to route information efficiently to both clinical and technical reviewers. Clinical reviewers can assess the impact of Al performance on patient outcomes and workflow integration, while technical reviewers can address underlying algorithmic or data issues. This dual-review structure ensures that feedback is evaluated from multiple perspectives, fostering a holistic approach to continuous improvement.

[0189] Further, the robust post-production architecture, that captures performance of the foundation model, may also offer, as part of the workflow, that a radiologist is notified when an Al result is available and retrieve the automatically generated report (FIGS. 6A-6C) and secondary capture that will contain the grounding information that corresponds to the generated report. A user would be able to accept, reject or modify the report as well as provide corrected grounding information in case of report modification, as described in FIGS. 21-23).

[0190] FIGS. 7A and 7B illustrate an example sequence diagram for validation of Al output as part of the radiology workflow implemented with the foundation model in accordance with an example of the present techniques (e.g., a postproduction monitoring process). The results may be delivered leveraging a PACS viewer. A postproduction database may collect information from all key technologies utilized in the workflow. The validation of the Al output may include input from a radiologist (e.g., via a computing device) 702, a modality 704 (e.g., device used to acquire an image), a DICOM image and message router 706 (e.g., LB Compass), a retrieval and workflow manager 708 (e.g., navigator), orchestrator 710, a multimodal LLM 712 (e.g., a foundational model), a PACS viewer 714, PACS 716, and a postproduction database 718.

[0191] A series of operations for algorithm processing 720 may occur. At operation 722, an image from the modality 704 may be transmitted via a DICOM C-store command to the DICOM image and message router 706. At operation 724, the DICOM image and message router 706 may transmit the image via a DICOM C-store command to the PACS viewer 714 for viewing of the image. The image may also be transmitted via a DICOM C-store command to the orchestrator 710 at operation 726. At operation 728, the orchestrator triggers analysis of the image by the multimodal LLM 712. The multimodal LLM 712 may analyze the image at operation 730. At operation 732, the orchestrator 710 may transmit a message via a DICOM C- store command that may include a secondary capture (SC) image (e.g., an image in a non- DICOM format) to signify that the process has started. At operation 734, the DICOM image and message router may transmit a DICOM SC to signify the process has started to the PACSviewer 714. At operation 736, the PACS viewer may display a button so that a user using the PACS viewer may view Al analysis results from the multimodal LLM 712. At operation 738, the multimodal LLM 712 may generate a DICOM standard report. At operation 740, the multimodal LLM 712 may transmit an output to the orchestrator 710. At operation 742, the orchestrator may transmit the output via a DICOM C-store command to the image and message router 706. At operation 744, the orchestrator 744 may log information about the execution of the multimodal LLM 712 in the postproduction database 718. At operation 746, the image and message router 706 may transmit the output to the PACS viewer 714. At operation 748, the Al analysis may be made available to the Al results button in the PACS viewer 714 such that selection of the Al results button retrieves the Al results / report. At operation 750, the PACS viewer 714 may receive a request to update the Al results with the contents of the standard report. At operation 752, the standard report may arrive and be included as part of the Al results.

[0192] A series of operations for radiologist reading 754 may occur. At operation 756, a radiologist 702 may interact with a computing device to open an exam including radiology images. At operation 758, the PACS viewer may display the images for viewing. At operation 760, the radiologist may review the images. At operation 762, the radiologist may select the Al results button. The PACS viewer 714 may display the Al generated report including the Al results and the standard report at operation 764. At operation 766, the radiologist 702 may use the computing device to review the generated report. At operation 768, the radiologist 702 may use the computing device to accept, reject or correct the report (e.g., validation). If the report is accepted, the PACS viewer 714 may transmit a DICOM standard report to the DICOM image and message router 706 at operation 770. The DICOM image and message router 706 may transmit postproduction information (e.g., whether the report was accepted, rejected, or corrected along with corrections) to the post production database 718 at operation 772.

[0193] The foundation model in the present techniques may be extended to support imaging modalities that generate 3D and 4D output. For example, a new image vision encoder may be incorporated in the toolset developed and used to generate applications for CT and MR imaging. Additionally, the foundation model may be used to generate additional X-ray image focused applications aiming to disease classification, disease progression and reporting error detection. One advantage of leveraging these foundation models may be that the algorithm development will require less data than traditional Al approaches and that the out-of-distribution performance may be improved.

[0194] Turning now to aspects of the present techniques that integrate text based LLMs in the foundation model architecture to achieve the multimodal, multi-layer LLM architectures herein. It is known that LLMs represent a paradigm shift in the landscape of artificial intelligence, offering unprecedented capabilities in natural language processing and understanding. Characterized by their vast size and complexity, these models are trained on extensive corpuses of textual data, enabling them to generate, interpret, and contextualize language with remarkable proficiency. The emergence of LLMs, epitomized by models like GPT (Generative Pretrained Transformer) and BERT (Bidirectional Encoder Representations from Transformers), has catalyzed a transformative impact across various sectors, with healthcare standing out as a particularly promising domain for their application.

[0195] Some embodiments may utilize LLMs in healthcare to process and synthesize vast amounts of complex, unstructured data - a hallmark of medical information. From parsing intricate clinical notes to interpreting vast repositories of biomedical literature, LLMs can distill actionable insights, aiding in decision-making processes that are central to patient care and medical research. Their applications extend across several dimensions of healthcare: in clinical decision support, they can analyze patient histories and suggest potential diagnoses; in medical research, they assist in the rapid review of literature, facilitating the discovery of novel therapeutic approaches; in personalized medicine, LLMs contribute to understanding patientspecific genetic information, tailoring treatments to individual genetic profiles. Furthermore, some embodiments hold significant promise in public health, where they can monitor and analyze patterns in health-related data, playing a crucial role in outbreak prediction and management.

[0196] This versatility of LLMs, combined with their continuously improving accuracy and sophistication, positions some embodiments to as a significant technology in the future of healthcare. By augmenting human expertise with deep, data-driven insights, some embodiments may enhance the efficiency, precision, and personalization of medical care, ultimately contributing to improved health outcomes and the advancement of medical science. However, realizing this potential necessitates navigating a landscape replete with technical, ethical, and practical challenges, underscoring the need for a concerted effort to integrate these powerful tools responsibly into the healthcare ecosystem.

[0197] The systems and methods of the present techniques may offer several advantages over conventional systems. These include, for example, creating healthcare specific benchmark and finetune datasets to perform better in clinical tasks. The present techniques can be used to compare fine tuning of LLM models approaches versus build from scratch approaches to createa healthcare domain 7B parameter LLM. Training within domain, high quality data can improve the quality and performance of LLMs trained from scratch in healthcare specific applications. Further the present techniques may be used to train 30B parameter models. Further still, creation of a flexible validation environment to validate LLMs and capture input for further finetuning and improvement will allow fast validation of new solution and increased the quality of the solution generated.

[0198] Research has shown that when it comes to modeling performance, custom-built LLMs, i.e., those tailored for specific tasks or domains, can outperform general-purpose LLMs like ChatGPT, which are designed for a wide array of applications. Examples of this include BloombergGPT, which is specialized for finance, and LLMs that are tailored for medical question answering.

[0199] FIG. 8 illustrates an example training workflow for a foundation model in accordance with the present techniques, indicating a preprocessing operation. The general process of creating an LLM, includes pretraining and finetuning, which is further discussed below with reference to FIG. 20. The term "pre" in "pretraining" refers to the initial phase where a model like an LLM is trained on a large, diverse dataset to develop a broad understanding of language. This pretrained model then serves as a foundational resource that can be further refined through finetuning, a process where the model is specifically trained on a narrower dataset that is more specific to particular tasks or domains.

[0200] In FIG. 8, training data 802 may be used for training an LLM. The training data 802 may include raw, unlabeled text data, and may include internet text, books, research articles, and proprietary material (e.g., material belonging to an enterprise utilizing the foundational model). The training data 802 may also include datasets from Medical Journal and Publications, Clinical Guidelines and Protocols, Textbooks, Patient Education Materials, Drug Databases, Clinical Case Studies, Healthcare Policies, Continuing Medical Education Resources, Guidelines, and Nutrition and Lifestyle Information. Such sources of data may be incorporated alongside EHR datasets to provide a comprehensive knowledge base. EHR datasets may be anonymized to ensure that sensitive patient information is protected while maintaining data utility. This initial data identification and mapping phase sets the foundation for subsequent stages of LLM development. In some embodiments, the training data 802 may include structured data.

[0201] The text data 802 may undergo a preprocessing stage 804 before it is used in training the foundational model 806. The preprocessing stage 804 may include preprocessing the collected data and tokenization. Data preprocessing may include cleaning and formatting thedatasets to ensure uniformity and quality. This may include tasks such as removing irrelevant information, standardizing data formats, addressing any inconsistencies and deduplication.

[0202] Tokenization is often a valuable operation in natural language processing where text is divided into smaller units, typically words or sub-words to facilitate machine learning tasks. Tokenization techniques that are specifically tailored for medical text may be employed, as the domain often involves complex medical terms and abbreviations.

[0203] Additionally, during this preprocessing stage 804, the techniques may address data privacy concerns, particularly when dealing with EHR datasets. Anonymization techniques may be applied to protect patient confidentiality while retaining data integrity.

[0204] A foundational model 806 may be trained on the training data 802 that has undergone a preprocessing stage 804. To train the foundational model, a set of subscale experiments may be performed leveraging smaller models to determine the best dataset choices and weighting. Quality filters, document de-duplication and diversification (D4), concept clusters, and hyperparameter tuning may be experimented with to enhance model performance. The training process may further include using small-scale models to create a scaling law that predicts the performance of the final large-scale model. This predictive approach ensures that the model's performance aligns with expectations before committing to computationally intensive runs. The training process may include pretraining of the foundational model 806 (e.g., a 7B Llama 2 based model) leveraging replay datasets as well as healthcare provided specific datasets.

[0205] The model 806 may continue to be fine-tuned after the pretraining process.Finetuning is a valuable step in refining the LLM's performance for healthcare tasks. The model may be fine-tuned on medical question-answering datasets, cleaned-up differential diagnoses (DDx), and other low-resource task-specific datasets. Fine-tuning ensures that the LLM is well- aligned with the medical domain. In some embodiments, fine-tuning may include training the model 806 on labeled data 808.

[0206] As part of the training process, the model performance of the model 806 may be evaluated by selecting representative evaluation criteria and datasets. A pretrained 7B parameter model may be finetuned and compared against a model trained from scratch using an equal amount of computing resources. This thorough evaluation helps determine which approach leads to superior results across various metrics. Evaluating the model performance may be part of the postproduction monitoring process.

[0207] The foundational model 806 may include a LLM utilizing a transformer architecture. The transformer architecture, core component of the LLM, comprises two main modules: anencoder and a decoder. The encoder module processes the input text, transforming it into numerical representations or vectors that encapsulate the contextual information of the input. Subsequently, the decoder module utilizes these encoded vectors to produce the output text. For instance, in a translation task, the encoder encodes text from the source language into vectors, and the decoder deciphers these vectors to generate text in the target language. Both the encoder and decoder consist of multiple layers interconnected by a self-attention mechanism. Questions regarding input preprocessing and encoding will be comprehensively addressed in subsequent sections through step-by-step implementations.

[0208] One element of many transformers and LLMs is the self-attention mechanism. This mechanism empowers the model to assess the significance of various words or tokens within a sequence relative to one another. It facilitates the model in capturing long-range dependencies and contextual relationships within the input data, thereby enhancing its capacity to generate coherent and contextually relevant output.

[0209] Another challenge that arises from the use of publicly available or commercially available LLMs is the legal and ethical implications of their training material and outputs. One of the main sources of training material for LLMs is the vast corpus of text available on the internet. However, this raises several issues regarding the quality, reliability, and copyright of the data. The internet contains a mixture of factual, fictional, and misleading information, which may affect the accuracy and validity of the LLMs' outputs. Moreover, some of the data may be protected by intellectual property rights or contain personal or confidential information, which poses legal and ethical risks for the LLMs' developers and users. Therefore, careful selection, filtering, and preprocessing of the training data may be useful to help ensure that the LLMs are trained on relevant, reliable, and permissible data.

[0210] While recent LLMs like MedPalm and GPT-4 have excelled in medical questionanswering benchmarks, these assessments primarily involve multiple-choice exams and may not reflect real clinical environments. The present techniques, however, deploy a foundation model that alleviates the substantial burden on clinicians who spend significant time on data- related tasks, contributing to professional burnout. Tasks like summarizing patient histories or generating diagnoses are delegated to LLM models via instructions grounded in EHRs. However, current evaluations fail to capture the diversity and intricacies of real-world clinical data.

[0211] Examples of using the present techniques to create healthcare specific benchmark and finetune datasets will now be described.

[0212] Fine-tuning and benchmark datasets may be specified in the development of some LLMs. These datasets serve as the foundation upon which LLMs are trained and evaluated, ensuring their performance aligns with the specific clinical tasks they are designed to handle. However, for some implementations, it may be important to note that many publicly available datasets predominantly focus on multiple-choice tasks, which may not accurately represent the diverse and nuanced challenges encountered in clinical practice.

[0213] To address this limitation, the foundation model in some embodiments of the present techniques may emphasize the creation of benchmark datasets tailored to distinct clinical categories. These categories encompass a wide range of healthcare tasks, including retrieval and summarization of critical information, developing care plans, performing calculations and scoring, offering diagnosis support / differential diagnosis, and facilitating information extraction from patient records. Each category mirrors the complexities of real-world clinical scenarios, ensuring that LLMs are not only well-equipped but also rigorously tested across various healthcare domains.

[0214] Furthermore, some of these tasks exhibit substantial interobserver variability, underscoring the value of incorporating input and expertise from multiple medical doctors during dataset curation for some implementations. This collaborative effort may help ensures that the benchmark datasets encompass the intricacies and nuances inherent in clinical decisionmaking, ultimately enhancing the LLMs' ability to provide valuable insights and support to healthcare professionals. In essence, the present techniques aim to help bridge the gap between LLM development and the intricacies of clinical practice by creating benchmark datasets that closely mimic the diverse challenges faced by medical practitioners in some embodiments.

[0215] Additionally, parts of these datasets can be used for finetuning in the foundation model. For example, the present techniques can be used to create an infrastructure that allows maintenance and continuous improvement of these datasets. Each dataset is going to be accompanied by a data card capturing the characteristics of data. The value of this dataset is not only limited in the products that will be developed as part of the generative Al-based operation systems for a healthcare provided, while also allowing for onboarding similar solutions for 3rd party vendors.

[0216] As noted above, in various examples, the present techniques compare fine tuning versus build from scratch approach in creating a healthcare domain 7B parameter LLM.

[0217] As discussed herein, developing specialized medical LLMs may benefit from a systematic approach. We now describe an example process for creating LLMs tailored tohealthcare applications, focusing on model pretraining, finetuning, optimization experiments, and the decision-making process when choosing between starting from scratch or finetuning from open-source checkpoints, as may be implemented using the process of FIGS. 7A and 7B.

[0218] FIG. 9 illustrates an example process flow for developing and validating a foundation model of multimodal, multi-layer LLM’s in accordance with an example. A use case (e.g., radiography) may be identified at operation 902. Datasets for training and benchmarking related to the use case may be identified at operation 904. The datasets may undergo preprocessing operation 906.

[0219] During an experiment phase, data that will be analyzed y the model may be prepared at operation 908. A model may be selected at operation 910, which may include selecting the size of the model and the architecture. The model may be pretrained at operation 912 on the data.

[0220] The model may undergo further training and validation. At operation 914, the model may receive prompts. At operation 916, a RAG process may be applied to the prompt such that the prompt may be used to retrieve relevant contextual information, which may be passed to the model along with the prompt. At operation 918, the model may be fine-tuned with prompts and other information. At operation 920, the model may undergo a validation operation using a benchmark dataset 922. After being validated, the model may be integrated into applications at operation 924.

[0221] At operation 926, the model and / or application using the model may be deployed for actual use. The model may undergo post-production monitoring at operation 928. Based on the monitoring, the model may undergo further fine-tuning at operation 918.

[0222] FIG. 10 illustrates an example training process for LLAMA2 (7B parameter LLM) finetuning to develop the foundation models in accordance with the present techniques, labeled Mayo LLM vO and Mayo LLM v1. As shown, the training process in FIG. 10X can be used to map out publicly available datasets and evaluate copyright status and quality. In particular, the process can be used to preprocess GAP-Replay datasets. Further, mapped out internal datasets were used, which included identifying data stewards and creating workflows for data retrieval. In the illustrated example, LLAMA2 (7B parameter LLM) was selected the model to evaluate the dataset composition for domain adaptation based on small scale experimentation and scaling laws. However, other LLM models may be used for analysis. Benchmarking datasets were defined to implement desired metrics.

[0223] Training the foundation model may include storing a version of the LLAMA 2 model (LLAMA 2 checkpoint 1002). The LLAMA 2 model may be pretrained at operation 1004 on nonprotected health information (PHI) to create a first foundation model 1006 (Mayo LLM vO). Indomain datasets 1012, including guidelines, training material, policies, procedures, textbooks, and / or PHI data may be used to further train the model at operation 1008, which may include small-scale experimentation and / or further pretraining of the model. The further training of the model at operation 1008 may generate a second foundation model 1010 (Mayo LLM v1). Both the first foundation model 1006 and second foundation model 1010 may be fine-tuned and undergo a performance evaluation operation 1014. During the performance evaluation operation 1014, the first foundation model 1006 and second foundation model 1010 may be evaluated against public benchmarks to generate a validation report for each of the models 1006, 1010.

[0224] Early published studies highlight the challenges of smaller models like GPT-2 and BERT in multi-step reasoning. While other studies have found that larger models, such as GPT- 3 and PaLM, show potential for reasoning, these larger models are known to be limiting for various reasons. Attempts at reasoning in LLMs and known limitations are briefly discussed.

[0225] Reasoning plays a pivotal role in the efficacy of LLMs across various applications, underpinning their ability to solve complex problems and make informed decisions. This capability is particularly crucial in fields like healthcare and business, where decision-making is based on a nuanced understanding of diverse factors and potential outcomes. Beyond decision support, reasoning is integral to enhancing natural language understanding in LLMs, enabling them to interpret nuances, infer meanings, and understand implications in a human-like manner. Such sophistication in understanding and response enhances the trust and reliability, especially in applications involving direct human interaction such as educational tools, customer service bots, or personal assistants. Furthermore, reasoning empowers LLMs with the ability to generalize knowledge across different domains through transfer learning, adapting to new tasks with minimal retraining. Finally, the aspect of ethical and fair decision-making in LLMs is greatly bolstered by reasoning, as it ensures that responses and decisions are not only logical but also unbiased and considerate of ethical implications. Therefore, incorporating reasoning into LLMs significantly enhances their overall functionality, making them more adaptable, trustworthy, and aligned with human cognitive processes.

[0226] Currently, there are several approaches for improving reasoning in LLMs: fully supervised finetuning and prompting, in-context learning, chain of thoughts, rational engineering and problem decomposition.

[0227] Fully supervised finetuning involves training small language models on datasets with explicit reasoning. For example, a GPT model has been fine-tuned using the CoS-E dataset to improve common sense question-answering. However, this method has drawbacks since it requires a specific dataset and the abilities do not generalize in out of context applications.

[0228] For prompting and in-context learning, large models like GPT-3 have shown promising few-shot learning abilities through in-context learning, where they can "reason" through problems using a query and examples. However, multi-step reasoning still is an issue. Studies suggest that the full capabilities of these models haven't been fully explored yet.

[0229] To enhance explicit reasoning, chain-of-thought (CoT) prompting has been proposed. This method involves presenting models with examples that articulate intermediary steps in the thinking process in natural language, thereby encouraging the model to follow a similar reasoning path.

[0230] Rational engineering involves refining rationale examples to improve LLM reasoning. Some studies advocate for complexity-based prompts to create deeper rationales, enhancing LLM performance.

[0231] To address complex issues, especially those needing compositional generalization, breaking the problem into simpler subparts (i.e. , problem decomposition) is effective. This approach, which tackles each smaller issue individually, helps in solving the overall complex problem more efficiently.

[0232] Towards these directions, the present techniques may be used to create a larger model based on the finding on experiments performed above to train a 30B parameter model on approximately 1 trillion (1T) tokens (FIG. 11). Toward this end, FIG. 11 illustrates an example process for training a 30B parameter LLM.

[0233] As shown in FIG. 11 , a 30 billion (30B) LLM may use in-domain datasets 1108 (e.g., guidelines, training material, policies, procedures, textbooks, PHI data, public data) in a pretraining operation 1102. At operation 1104, the 30B parameter LLM may undergo small scale experiments to generate a foundation model 1106 (Mayo LLM v1). The foundation model 1106 may be finetuned for a specific task and undergo a performance evaluation operation 1110. During the performance evaluation operation 1110, the model 1106 may be evaluated against public benchmarks and enterprise-specific benchmarks (e.g., Mayo generated benchmarks) to generate validation reports.

[0234] Post-production monitoring of healthcare apps plays an important role in ensuring the seamless integration of Al models into clinical workflows and their ongoing effectiveness in real-world healthcare scenarios. When developing an IT infrastructure for this purpose, one of the primary objectives is to establish a robust API that allows for the integration of finely-tuned models designed for specific healthcare tasks. These models, which may include machine learning algorithms or artificial intelligence applications, are tailored to address the unique requirements of clinical practice.

[0235] The integration of these models into the clinical workflow is significant step. Before such integration can occur, medical professionals may desire prospective studies to evaluate their real-life accuracy and efficacy. These studies may help identify any potential discrepancies between the model's performance in controlled environments and its performance in actual clinical settings, thereby ensuring that the technology meets the high standards required in healthcare.

[0236] Post -production monitoring is not a one-size-fits-all process; instead, it should be to be tailored to each specific healthcare application. To achieve this, the relevant stakeholders may be identified and involved. This includes clinicians, who play a pivotal role in providing input on the integration requirements. Collaborating closely with healthcare professionals ensures that the interface aligns with what clinicians need and expect to see in their daily practice. By engaging stakeholders from the outset and maintaining ongoing feedback loops, healthcare app developers can refine their solutions and continuously improve the quality of care delivered through these innovative technologies. Overall, post-production monitoring of healthcare apps may serve as a bridge between Al development and its real-world application in clinical settings, with the goal of enhancing patient outcomes and healthcare delivery.

[0237] As shown in FIG. 12, validating LLMs and multimodal LLM applications presents several challenges. LLMs often contain a vast number of parameters, leading to scalability concerns. Privacy and security issues arise when dealing with sensitive data. Selecting appropriate evaluation metrics can be a complex task, and LLMs may inherit biases from their training data. Domain-specific validation can be intricate, and interpretability is limited, making human baselines less applicable. Overfitting, adversarial attacks, and the need for robustness further complicate the validation process. Ongoing research efforts aim to develop improved evaluation methodologies, domain-specific benchmarks, and methods to enhance fairness, interpretability, and reliability in LLM validation. In healthcare, where domain-specific expertise and large datasets are necessary for validation, the complexity is compounded by cost and the availability of qualified physicians (FIG. 12). Validating LLMs may include algorithmic evaluation on proxy metrics, use case-grounded algorithmic evaluations, user studies with proxy use cases or users, and user studies with real world use cases and users.

[0238] When evaluating Natural Language Understanding (NLU), some aspects include assessing the model's comprehension of context, its ability to recognize user intent, and its capacity to resolve ambiguity. For Natural Language Generation (NLG), the focus may be on evaluating coherence, factual accuracy, and language quality, including grammar, style, and fluency. The perplexity metric may be employed to gauge language comprehension, where lower values indicate better language understanding. Benchmark tests like GLUE and SuperGLUE may provide comprehensive assessments of NLU systems. Task-specific metrics, such as BLEU for translation and ROUGE for summarization, may be used to measure the quality of model-generated text against human references. Adversarial testing may be used to identify vulnerabilities in the model by subjecting it to inputs designed to confuse or mislead.The evaluation of Out-of-Distribution (OOD) performance may be used to test the model's ability to handle novel data, while stress testing ensures model stability under extreme conditions, ensuring accuracy in challenging scenarios.

[0239] In the domain of Error Reporting and Resolution, components may encompass automated error detection, involving the implementation of systems to identify and report data extraction errors. In this way, a resolution workflow may be established, encompassing root cause analysis and corrective actions. User Validation and Feedback may also be performed, including User Acceptance Testing (UAT), where healthcare professionals assess the app's clinical utility and usability. A Feedback Loop may be established to allow users to report inaccuracies or issues in data extraction, facilitating ongoing improvement efforts.

[0240] In the realm of Real-World Performance Testing, a comprehensive approach may include Pilot Studies. This may involve selecting diverse healthcare settings for phased app implementation and establishing objective metrics. Comparison Studies entail benchmarking the app's performance against established methods, assessing data accuracy, and evaluating its impact on clinical decisions. User Feedback and Clinical Integration involve gathering input from healthcare professionals and studying usability, while Continuous Monitoring and Adjustment encompass real-time monitoring, iterative improvements, and scalability assessments. Outcome Evaluation focuses on assessing clinical impacts and conducting costbenefit analysis, while Long-Term Studies assess the sustained performance of the app and post-implementation review for long-term adherence and user satisfaction.

[0241] Metrics for assessing an algorithm's accuracy and reliability may encompass Precision and Recall, measuring its ability to identify relevant patient history information and the accuracy of those identifications. The F1 Score combines these metrics into a single accuracy score, while the False Positives / Negatives Rate tracks incorrect identifications. Performance andefficiency metrics may include Processing Time to gauge extraction speed, Throughput indicating capacity, and System Uptime and Availability ensuring operational readiness. User experience metrics may comprise User Satisfaction Score, Adoption Rate, and Ease of Use and Learning Curve, offering insights into user satisfaction and adaptability. Compliance and security metrics may involve monitoring HIPAA Compliance Incidents and ensuring Audit Trail Integrity. Clinical impact metrics may assess how the algorithm influences clinical decisions and error rates within clinical contexts. Continuous improvement metrics may involve evaluating the effectiveness of the feedback loop and monitoring the response to version upgrades. Data quality metrics may encompass Data Completeness and Data Consistency, evaluating the comprehensiveness and consistency of captured patient history information.

[0242] To evaluate LLMs to ensure that their outputs are trustworthy and suitable for various applications, including healthcare, the present techniques provide a systematic approach that includes a variety of techniques. The evaluation of an LLM may begin with an output analysis, i.e., comparing the LLM’s responses to similar queries to see if it consistently reasons in a stable manner. Inconsistencies may indicate issues with reasoning. Next, the LLM’s outputs may be examined for logical coherence, ensuring that its conclusions logically follow from the provided input data, reinforcing the reliability of its reasoning.

[0243] Evaluation of an LLM may include comparative evaluation, e.g., engaging domain experts, such as healthcare professionals, to evaluate the reasoning and conclusions presented by the LLM to provides valuable insights into the quality of reasoning. Additionally, the LLM’s reasoning may be compared against established benchmarks or standards within the field. Such quantitative assessment helps gauge the LLM’s performance objectively.

[0244] The evaluation of an LLM may also include scenario-based testing, e.g., development of specific scenarios or test cases, including challenging situations or those with known outcomes, to observe how the LLM handles diverse scenarios. Practical testing sheds light on the LLM’s reasoning capabilities. Conducting 'what-if' analyses by altering input variables provides insight into how the LLM adapts its reasoning, and reveals the LLM’s flexibility and adaptability.

[0245] The evaluation of an LLM may also include explanation evaluation, e.g., assessing whether the LLM provides clear and understandable explanations for its reasoning.Transparent explanations enhance user trust. The explanations may be evaluated for depth and detail to. ensure they are sufficient for users to comprehend the basis of the LLM's conclusions. Checking for transparency and traceability may be achieved through a CURE and / or reverse RAG process (as shown in FIGS. 14B and 15).

[0246] The evaluation of an LLM may also include a feedback loop, e.g., gathering feedback from users about their perceptions of the LLM's reasoning quality (such as shown in FIGS. 5A, 7A-7B, 21-23). User insights and feedback serve as a valuable resource for improvement, which may be used to make necessary adjustments and enhancements to the LLM, continuously improving its reasoning capabilities.

[0247] The evaluation of an LLM may also include checks for transparency and traceability, i.e., checking if the LLM can accurately reference sources or data it used in its reasoning process. The LLM's ability to trace its decision-making path may be evaluated, providing transparency into its reasoning process for better understanding and auditability. Checking for transparency and traceability may be achieved through a CURE and / or reverse RAG process (as shown in FIGS. 14B and 15).

[0248] The evaluation of an LLM may further include error analysis, e.g., conducting a comprehensive analysis to understand which aspect of the LLM's reasoning process was flawed, which aids in error mitigation. The LLM may be continuously monitored for patterns in errors to identify any systemic issues in the LLM's reasoning, allowing targeted improvements.

[0249] Finally, evaluation of an LLM may include evaluating its real-world performance, e.g., evaluating how the LLM's reasoning influences actual decisions and outcomes in real-world scenarios, which provides insights into its practical utility. The LLM’s performance may be continuously monitored over an extended period to detect and address evolving issues in its reasoning, ensuring sustained reliability.

[0250] FIG. 13 illustrates an example architecture that includes a postproduction monitoring layer for implementation of such processes. An EHR database 1302 may store patient data (e.g., electronic healthcare records). The patient data may be passed to a foundational model LLM 1304 (e.g., production LLM) to generate output such as a data and analytics report, a diagnostic analysis report, a large language model agent, and / or user data. The output may be presented in a user interface 1306 (e.g., ANIMATE / MAYA, described in more detail with reference to FIGS. 18A, 18B, 19A, and 19B). A monitoring layer 1308 may be used to monitor the performance of the foundational model 1304 by receiving metadata from the model 1304 and receiving user feedback and / or monitoring user interactions via the user interface 1306. The data gathered from monitoring the model may be viewed on one or more monitoring dashboards 1310 and / or may be communicated to other applications via one or more data science APIs 1312. The data gathered from model may also be used for continuous improvement of the model at operation 1314.Example Use Case - Hallucination and Quality Measurements of Al Generated notes

[0251] As the use of generative Al extends into more clinical tasks, establishing a framework to evaluate and validate these models for effectiveness and safety may become important. This often involves human oversight, particularly to mitigate risks like hallucination and incorrect conclusions derived by Al. To address this, in various examples, the systems and methods of the present techniques include a Checker for Unvalidated Response Errors (termed “CURE”), which is a tool designed to enable human experts to quickly assess the performance of generative Al models. The CURE tool may present the user with the Al-generated output alongside potential references, allowing the expert to compare the Al's conclusions with the original sources and make an informed judgment on their validity. Such assessment of generative Al models may be part of a post-production monitoring process.

[0252] To streamline this process, in some examples, the present techniques employ a reverse Retrieval-Augmented Generation (RAG) process. In this reverse RAG process, the system first generates text and then seeks out relevant source material, contrary to the traditional RAG approach where source material guides text generation. This reverse process may aid in pinpointing the origin of specific information within the Al-generated content. FIGS. 14A and 14B illustrates an example CURE process implemented in an integral part of the text generation validation.

[0253] FIG. 14A illustrates an example RAG process. At a preprocessing operation 1402, all documents 1404 in an enterprise archive are passed to an LLM 1406 to embed chunks of the document into numerical values 1408 (e.g., vectors) and stored in a database. At operation 1409, a query 1410 may be passed to the LLM 1406 to embed the query into numerical values 1412 (e.g., a vector), similarly to the documents 1404. At a semantic search operation 1414, the numerical values 1412 corresponding to the query 1410 may be matched with relevant vectors from the database. For example, in FIG. 14A, the vector 1408a is similar to the vector 1412. At a retrieval operation 1416, relevant documents 1418 are retrieved. The relevant retrieved documents 1418 will augment the prompt 1412 when passed to another LLM for other applications.

[0254] FIG. 14B illustrates an example fact checking process. Retrieved documents 1418 may be passed to an LLM 1420, which may be a foundational LLM, to generate a summary 1422. The summary 1422 may correspond to a prompt such as prompt 1410 of FIG. 14A (“Medical history relevant for arrhythmia.”) Summary 1422 may include one or more hallucinations (e.g., incorrect conclusions generated by the LLM 1420). The summary 1422 may be passed to an LLM 1424 to split the summary 1422 into concepts that require fact checking 1426. The concepts 1426 may be used to find potential references from the retrieveddocuments 1418 by using a semantic search, and pair the references with the concepts 1426.This process is called a reverse RAG process.

[0255] Additionally, the text-based LLM model of the foundation model may be programmed to not only generate text but also to provide explanations and scores for how each specific concept in the document was developed. This feature offers a 'Confidence Level' for each segment of the generated text, enabling a more nuanced understanding of the Al's performance. The integration of a novel reverse RAG process with an intuitive user interface (Ul), such as the one depicted in FIG. 15, forms the cornerstone of CURE that may be used to facilitate desired confidence levels. For example, the Ul may be designed to be user-friendly, facilitating quick and efficient evaluation by clinicians. It displays both the Al-generated content and its corresponding references in a clear, accessible manner, allowing for swift cross- referencing and decision-making. In some embodiments, the explanation and scoring functions may be provided by a second LLM model (e.g., another text-based LLM model of the foundation model). The reverse RAG process increases accuracy of the LLM model that generates text by providing a mechanism for the text-generating LLM model to “fact-check” the generated text.

[0256] Various techniques such as ensemble evaluators, meta-learning approaches, and human-in-the-loop (HITL) validation may be used to improve the accuracy of the scoring model. Ensemble evaluators may include using multiple models which vote or aggregate their scores to evaluate the summary generated by an LLM (i.e., the foundational model), rather than only one model. For example, one model may be used to address adherence to a topic (i.e., a content relevance evaluator) and another can be used to examine logical argument flow (i.e., a structure evaluator) to generate a score for the generated text. In some embodiments, HITL techniques such as reinforcement learning with human feedback (RLHF) may be used to improve the performance of the scoring model, as described below in to FIG. 20.

[0257] This combination of reverse RAG technology and a streamlined Ul provides a robust set of guardrails for clinical use of Al models. It may not only accelerate the deployment of these models to clinicians, but may also help ensure a safer and more reliable application in critical healthcare settings. By bridging the gap between Al innovation and clinical practice, CURE stands as an important tool in the responsible and effective utilization of generative Al in healthcare.

[0258] FIG. 15A illustrates an example generated text from a text-based LLM model, where the generated text includes references for fact checking, including hallucinations. A summary 1502 may be generated by a foundational model (e.g., model 1420 in FIG. 14B). The summary 1502 may be compared against an original reference document 1506 for fact checking. Thefoundational model and / or a fact-checking model may retrieve reference documents and / or portions of reference documents for the comparison with the summary. Portions of the summary 1502 that do not match the document and may be hallucinations may be highlighted, such as section 1504. Changes may be made to sections of the summary 1502 by clinicians. In addition, with the present techniques, any change done by the clinicians to the generated content will be saved and used for future fine-tuning of these models and the improvement of the CURE tool algorithm. In some embodiments, a second machine learning model (e.g., a fact-checking model) may be used to generate a score for the generated text. The score may indicate the quality of the summary and its associated reference, e.g., whether the reference explains and / or provides support for a fact included in the summary. For example, the score may be on a scale of 0 to 10, in which a score of 0 indicates that the text is highly inaccurate and a score of 10 indicates a that the text is highly accurate and / or a high degree of confidence that the text is accurate. In some embodiments, the foundational model may generate the score for the generated output text and / or output text-reference pairing.

[0259] Further, the CURE tool may be made become available to the users leveraging the infrastructure herein, for example, the infrastructure created by the ANIMATE platform described below. That is, the CURE tool Ul may be made available across a healthcare network. In this way, the CURE tool can be instrumental in monitoring modifications made by clinicians to automatically generated notes. By comparing these notes with the revisions made by humans any data drift and potential decline in the algorithm's performance over time may be tracked and evaluated using the present techniques. Additionally, similar metrics, suitably adapted for generative tasks, may be employed to assess the performance of these new models over time.

[0260] FIG. 15B depicts an example of user assessing the generated text in real-time as part of the fact checking process. Text may be highlighted and / or color-coded based on a score generated by a second LLM (e.g., a fact-checking model). For example, a score below a threshold may indicate that a chunk of generate text is not well-supported by a displayed reference and / or is a hallucination, and may require additional review, which causes the text to be highlighted in the CURE Ul. A user may edit the text via the CURE tool Ul to correct errors in the generated summary and / or mark text as valid. Editing the text may remove any highlighting or color-coding of the text in the CURE tool Ul. The user may also score the generated text, which may be used to fine-tune and / or retrain a second LLM that generates the score.

[0261] FIG. 16 illustrates an example of postproduction monitoring of Al performance through a continuous monitoring process. At operation 1602, an algorithm may be developed. At operation 1610, the expected performance of the algorithm may be evaluated. At operation 1604, the algorithm may be put into practice in the real world. At operation 1606, sources of bias may influence the model, causing altered algorithm performance at operation 1608. The altered algorithm performance may be compared against the expected performance. The algorithm may undergo re-training to account for the altered performance.

[0262] As described in some examples, the systems and methods herein provide a platform, termed “ANIMATE” configured to be the enterprise standard of translation engine / delivery framework to integrate Al and generative Al solutions that focus on the patient and the disease rather than a particular task. The ANIMATE platform may contextualize Al results with relevant patient data and information from other tools to promptly identify previously unsuspected risks, evaluate known diseases, and guide therapies. A feature and advantage the ANIMATE platform in some embodiments is to reduce the time from initial presentation to definitive therapy for healthcare patients and to reduce the clerical burden on healthcare staff. FIGS. 18A and 18B illustrates a generated discharge summary reports with operative notes summary of patient data (including healthcare data, treatment data, diagnostic data, etc.) that may be generated with from the CURE tool implemented in the ANIMATE platform. FIG. 18A depicts a discharge summary 1802 with source notes 1804. FIG. 18B depicts an LLM-generated summary note 1806. In some embodiments, the ANIMATE platform may be used in connection with or as the CURE U I for fact-checking the summary note 1806.

[0263] FIGS. 19A and 19B illustrate example screenshots of a prototype platform presenting data in other formats to clinicians and healthcare professionals. Such information may be monitored in a post-production monitoring process.In some examples, the present techniques include systems and methods that allow for in-house development using a new platform, termed the GenAI App Store. This platform is not just a tool for model development and sharing within a healthcare system (such as that of Mayo Clinic), it is a foundation for future expansion and collaboration. With core services like CURE (Checker for Unvalidated Response Errors), which assesses Al-generated content for hallucinations and quality, the platform can ensure the safe and effective use of Al in clinical settings. The platform allows clinicians to validate Al conclusions against original sources, integrating a reverse Retrieval-Augmented Generation process and user-friendly interface for efficient evaluation. The scalability of this platform means that, eventually, it could support the use of external modelsand allow other healthcare providers to benefit from Mayo's Al advancements, significantly enhancing patient care through collaborative innovation.Example Training of a Machine-Learning Model

[0264] FIG. 20 depicts a combined block and logic diagram 2000 for training various machine learning models of the multimodal LLM architecture, in which the techniques described herein may be implemented, according to some embodiments. Some of the operations in FIG. 20 may represent hardware and / or software components, other operations may represent data structures or memory storing these data structures, registers, or state variables (e.g., 2012), and other operations may represent output data (e.g., 2025). Input and / or output signals may be represented by arrows labeled with corresponding signal names and / or other identifiers. The methods and systems may include one or more servers 2002, 2004, 2006, such as the server 104 or an external computing device.

[0265] The training techniques illustrated in FIG. 20 may be used for LLMs at various layers of the multimodal LLM architecture, including for example, at a first layer and at the foundation model layer. For example, a machine learning model (e.g., models 2015, 2050) may be a foundational model that is trained to generate a healthcare summary for a particular patient. The healthcare summary may be an examination summary, a discharge summary, a preoperation summary, a post-operation summary, etc. In some embodiments, a machine learning model may be trained to retrieve references that are relevant to the output. For example, a machine learning model may be trained to retrieve references relevant to generated output text (e.g., via a reverse RAG process as described in FIGS. 14-15, 25-28). In some embodiments, a machine learning model may be a fact-checking machine learning model (which may be a model 2015) and may be trained to generate a score for summary and reference pairs.

[0266] In one implementation, the server 2002 may fine-tune a pretrained language model 2010. The pretrained language model 2010 may be obtained by the server 2002 and be stored in a memory, such as memory 122. The pretrained language model 2010 may be loaded into a machine learning training module, such as the MLTM 142, by the server 2002 for retrain ing / fine- tuning. A supervised training dataset 2012 may be used to fine-tune the pretrained language model 2010 wherein each data input prompt to the pretrained language model 2010 may have a known output response for the pretrained language model 2010 to learn from. The supervised training dataset 2012 may be stored in a memory of the server 2002, e.g., the memory 122 or the training database 126. In one implementation, the data labelers may create the supervised training dataset 2012 prompts and appropriate responses. The pretrained language model 2010 may be fine-tuned using the supervised training dataset 2012 resulting in the supervisedfine-tuning (SFT) machine learning model 2015 which may provide appropriate responses to user prompts once trained. The trained SFT machine learning model 2015 may be stored in a memory of the server 2002, e.g., memory 122.

[0267] In some embodiments, the server 2002 may fine-tune the pretrained language model 2010 using a set of vectors associated with a set of training data. In some instances, the set of training data may include prompts associated with questions and documents, and responses associated with the prompts. Creating the set of vectors may include (1) splitting the text of the prompts, associated questions and / or associated documents into semantic clusters, and (2) encoding the semantic clusters as the set of vectors. The semantic clusters may be one or more words, a portion of a word, or a character. A distance between the vectors (e.g., a cosine distance, a Euclidean distance) may depend on a relevance between the semantic clusters corresponding to the vectors.

[0268] In some embodiments, data may be added to the supervised training dataset 2012 for further training, fine-tuning, and / or retraining. In one embodiment, additional output-response pairs with scores may be stored and added to the supervised training dataset 2012 to train / fine- tune / retrain a model (e.g., a text-based foundational LLM) that generates text (e.g., healthcare summaries) to generate more accurate output text, to train / fine-tune / retrain a fact-checking model to generate more accurate references for an output, and / or to train / fine-tune / retrain a fact-checking model to generate more accurate scores for an output-reference pair. For example, for an output-reference pairing including “the patient has been diagnosed with arrythmia” and a doctor’s note indicating the arrythmia diagnosis from an EHR associated with the patient, the machine learning model may generate a score of 10 from a scale of 0-10. A data labeler may accept the score, and the output-reference pairing along with the score may be stored for further training / retraining / fine-tuning. In another example, for an output-reference pairing including “she also enjoys a secret hobby of salsa dancing, which she believes contributes to her heart health” and a doctor’s note indicating the arrythmia diagnosis from an EHR associated with the patient, the fact-checking machine learning model may generate a score of 0, indicating that the reference does not explain the fact, and that the output text is likely a hallucination, and the output-reference pairing along with the score may be stored for further training / retraining / fine-tuning. In some embodiments, a human-in-the-loop approach may be leveraged to provide feedback to the generated scores. For example, for an outputreference pairing including “she also enjoys a secret hobby of salsa dancing, which she believes contributes to her heart health” and a doctor’s note indicating the arrythmia diagnosis from an EHR associated with the patient, the fact-checking machine learning model may generate ascore of 8. A data labeler may review the output, reference, and score, and change the score to 0 to indicate that the reference does not explain the fact. The output-reference pair and the new score (i.e. , 0) may be stored for future training / retraining / fine-tuning. In some embodiments, changes to the generated text itself may be saved and used for further training / retraining / fine- tuning. For example, for an output-reference pairing including “the patient has not been diagnosed with arrythmia” and a doctor’s note indicating the arrythmia diagnosis from an EHR associated with the patient, the machine learning model may generate a score of 10. Rather than changing the score, a reviewer may edit the text to read “the patient has not been diagnosed with arrythmia.” The edited text, reference, and score may be stored for future training / retraining / fine-tuning. In some embodiments, if the score generated by the factchecking machine learning model is below a threshold score, the output text may be highlighted in a user interface (e.g., a CURE Ul) for a human labeler to review.

[0269] In one implementation, training the machine learning model 2050 may include the server 2004 training a reward model 2020 to provide as an output a scaler value / reward 2025. The reward model 2020 may be required to leverage Reinforcement Learning with Human Feedback (RLHF) in which a model (e.g., machine learning model 2050) learns to produce outputs which maximize its reward 2025, and in doing so may provide responses which are better aligned to user prompts.

[0270] Training the reward model 2020 may include the server 2004 providing a single prompt 2022 to the SFT machine learning model 2015 as an input. The input prompt 2022 may be provided via an input device (e.g., a keyboard) via the I / O module of the server, such as I / O module 146. The prompt 2022 may be previously unknown to the SFT machine learning model 2015, e.g., the labelers may generate new prompt data, the prompt 2022 may include testing data stored on training database 126, and / or any other suitable prompt data. The SFT machine learning model 2015 may generate multiple, different output responses 2024A, 2024B, 2024C, 2024D to the single prompt 2022. The server 2004 may output the responses 2024A, 2024B, 2024C, 2024D via an I / O module (e.g., I / O module 146) to a user interface device, such as a display (e.g., as text responses), a speaker (e.g., as audio / voice responses), and / or any other suitable manner of output of the responses 2024A, 2024B, 2024C, 2024D for review by the data labelers.

[0271] The data labelers may provide feedback via the server 2004 on the responses 2024A, 2024B, 2024C, 2024D when ranking 2026 them from best to worst based upon the promptresponse pairs. The data labelers may rank 2026 the responses 2024A, 2024B, 2024C, 2024D by labeling the associated data. The ranked prompt-response pairs 2028 may be used to trainthe reward model 2020. In one implementation, the server 2004 may load the reward model 2020 via the machine learning module (e.g., the machine learning module 140) and train the reward model 2020 using the ranked response pairs 2028 as input The reward model 2020 may provide as an output the scalar reward 2025.

[0272] In one implementation, the scalar reward 2025 may include a value numerically representing a human preference for the best and / or most expected response to a prompt, i.e., a higher scalar reward value may indicate the user is more likely to prefer that response, and a lower scalar reward may indicate that the user is less likely to prefer that response. For example, inputting the “winning” prompt-response (i.e., input-output) pair data to the reward model 2020 may generate a winning reward. Inputting a “losing” prompt-response pair data to the same reward model 2020 may generate a losing reward. The reward model 2020 and / or scalar reward 2025 may be updated based upon labelers ranking 2026 additional promptresponse pairs generated in response to additional prompts 2022.

[0273] In one example, a machine learning model may be trained to generate a healthcare summary in response to a prompt. A data labeler may provide to the SFT machine learning model 2015 as an input prompt 2022, “Medical history relevant for arrythmia.” The input may be provided by the labeler via the user device 102 over network 110 to the server 2004 running a chatbot application utilizing the SFT machine learning model 2015. The SFT machine learning model 2015 may provide as output responses including healthcare information to the labeler via the user device 102. For example, in response to the prompt “Medical history relevant for arrythmia,” the model 2015 may generate the following responses: (i) “the patient has been diagnosed with arrythmia” 2024A; (ii) “the patient has a history of atrial fibrillation and an atrial septal defect” 2024B; and (iii) “she also enjoys a secret hobby of salsa dancing, which she believes contributes to her heart health” 2024C. The data labeler may rank 226, via labeling the prompt-response pairs, prompt-response pair 2022 / 2024B as the most preferred answer; prompt-response pair 2022 / 2024A as a less preferred answer; and prompt-response 2022 / 2024C as the least preferred answer. The labeler may rank 2026 the prompt-response pair data in any suitable manner. The ranked prompt-response pairs 2028 may be provided to the reward model 2020 to generate the scalar reward 2025.

[0274] In another example, a machine learning model may be trained to retrieve references that support and / or explain generated text, such as a healthcare summary. For example, generated text such as “the patient has been diagnosed with arrhythmia” may be provided to an SFT machine learning model 2015 to retrieve references explaining the fact. The SFT machine learning model 2015 may retrieve and provide as references to the labeler via the user device102. For example, in response to the generated text such as “the patient has been diagnosed with arrhythmia,” the SFT machine learning model 2015 may provide : (i) a patient-provided intake form indicating arrhythmia diagnosis 2024A; (ii) a doctor’s note indicating arrhythmia diagnosis 2024B; and (iii) a blood test result 2024C. The data labeler may rank 226, via labeling the prompt-response pairs, prompt-response pair 2022 / 2024B as the most preferred answer; prompt-response pair 2022 / 2024A as a less preferred answer; and prompt-response 2022 / 2024C as the least preferred answer.

[0275] In another example, a machine learning model may be trained to score generated output text and reference pairs. For example, an output-reference pair such as “the patient has been diagnosed with arrhythmia” along with a doctor’s note indicating arrhythmia diagnosis may be provided to an SFT machine learning model 2015 to generate a score for the outputreference pair. The SFT machine learning model 2015 may provide as scores to the labeler via the user device 102. For example, in response to the output-reference pair of “the patient has been diagnosed with arrhythmia” and a doctor’s note indicating arrhythmia diagnosis, the machine learning model 2015 may generate: (i) a score of “8” 2024A; (ii) a score of “10” 2024B; and (iii) a score of “0” 2024C. The data labeler may rank 226, via labeling the prompt-response pairs, prompt-response pair 2022 / 2024B as the most preferred answer; prompt-response pair 2022 / 2024A as a less preferred answer; and prompt-response 2022 / 2024C as the least preferred answer.

[0276] While the reward model 2020 may provide the scalar reward 2025 as an output, the reward model 2020 may not generate a response (e.g., text). Rather, the scalar reward 2025 may be used by a version of the SFT machine learning model 2015 to generate more accurate responses to prompts, i.e., the SFT model 2015 may generate the response such as text to the prompt, and the reward model 2020 may receive the response to generate a scalar reward 2025 of how well humans perceive it. Reinforcement learning may optimize the SFT model 2015 with respect to the reward model 2020 which may realize the configured machine learning model 2050.

[0277] In one implementation, the server 2006 may train the machine learning model 2050 (e.g., via the machine learning module 140) to generate a response 2034 to a random, new and / or previously unknown input 2032. To generate the response 2034, the machine learning model 2050 may use a policy 2035 (e.g., algorithm) which it learns during training of the reward model 2020, and in doing so may advance from the SFT model 2015 to the machine learning model 2050. The policy 2035 may represent a strategy that the machine learning model 2050 learns to maximize the reward 2025. As discussed herein, based upon prompt-response pairs,a human labeler may continuously provide feedback to assist in determining how well the machine learning model’s 2050 responses match expected responses to determine the rewards 2025. The rewards 2025 may feed back into the machine learning model 2050 to evolve the policy 2035. Thus, the policy 2035 may adjust the parameters of the machine learning model 2050 based upon the rewards 2025 it receives for generating good responses. The policy 2035 may update as the machine learning model 2050 provides responses 2034 to additional prompts 2032.

[0278] In one implementation, the response 2034 of the machine learning model 2050 using the policy 2035 based upon the reward 2025 may be compared using a cost function 2038 to the SFT machine learning model 2015 (which may refrain from using a policy) response 2036 of the same prompt 2032. The cost function 2038 may be trained in a similar manner and / or contemporaneous with the reward model 2020. The server 2006 may compute a cost 2040 based upon the cost function 2038 of the responses 2034, 2036. The cost 2040 may reduce the distance between the responses 2034, 2036, i.e. , a statistical distance measuring how one probability distribution is different from a second, in one implementation the response 2034 of the machine learning model 2050 versus the response 2036 of the SFT model 2015. Using the cost 2040 to reduce the distance between the responses 2034, 2036 may avoid a server overoptimizing the reward model 2020 and deviating too drastically from the human- intended / preferred response. Without the cost 2040, the machine learning model 2050 optimizations may result in generating responses 2034 which are unreasonable but may still result in the reward model 2020 outputting a high reward 2025.

[0279] In one implementation, the responses 2034 of the machine learning model 2050 using the current policy 2035 may be passed by the server 2006 to the rewards model 2020, which may return the scalar reward 2025. The machine learning model 2050 response 2034 may be compared via the cost function 2038 to the SFT machine learning model 2015 response 2036 by the server 2006 to compute the cost 2040. The server 2006 may generate a final reward 2042 which may include the scalar reward 2025 offset and / or restricted by the cost 2040. The final reward 2042 may be provided by the server 2006 to the machine learning model 2050 and may update the policy 2035, which in turn may improve the functionality of the machine learning model 2050.

[0280] To optimize the machine learning model 2050 over time, RLHF via the human labeler feedback may continue ranking 2026 responses of the machine learning model 2050 versus outputs of earlier / other versions of the SFT machine learning model 2015, i.e., providing positive or negative rewards 2025. The RLHF may allow the servers (e.g., servers 2004, 2006) tocontinue iteratively updating the reward model 2020 and / or the policy 2035. As a result, the machine learning model 2050 may be retrained and / or fine-tuned based upon the human feedback via the RLHF process, and throughout continuing usage may become increasingly efficient.

[0281] Although multiple servers 2002, 2004, 2006 are depicted in the example block and logic diagram 200, each providing one of the three steps of the overall machine learning model 250 training, fewer and / or additional servers may be utilized and / or may provide the one or more operations of the machine learning model 2050 training. In one implementation, one server may provide the entire machine learning model 2050 training.

[0282] FIG. 21 depicts an example workflow that may be monitored in a postproduction monitoring process. The performance of the algorithm, IT-related issues, and user-related issues may all be monitored in the postproduction monitoring process. In FIG. 21, a clinician may submit a query, such as a question about an abdominal CT scan, to an application that utilizes Al for analysis by the Al algorithm (e.g., an algorithm utilized in a foundational model, such as an algorithm for generating body composition measurements from abdominal CT scans). A radiologist may utilize a tool (e.g., Records of Computed Knowledge Expressed by neural nets (ROCKET)) to view the analytic data generated by the algorithm in a user interface to generate feedback on the results. The radiologist may accept the data, reject the data, or route the data to an image analyst for rework and / or further analysis. Such feedback may be used to determine the performance of the algorithm.

[0283] FIG. 22 depicts an example sequence diagram of monitoring IT-related issues. Data from various sources such as DICOM and other tools (e.g., ROCKET) may be collected by a data analytics application (e.g., Splunk). The data gathered by the data analytics application may be transmitted to an Al analytics monitoring service. Data from other sources such as electronic healthcare record management systems (e.g., EPIC) and electronic healthcare record databases (e.g., UDP) may also be transmitted to the Al analytics monitoring service. For example, various healthcare orders may be transmitted to the Al analytic monitoring service from the EHR management system, and metadata, ICD10 codes, radiologist reports, and / or demographic data may be transmitted to the Al analytic monitoring service from the EHR database for monitoring. Such data may be monitored by the Al analytic monitoring service to determine the overall number of radiology images or scans processed vs. the number of scans or images reported as failed to process. Furthermore, such monitoring may capture if failures have affected the overall workflow.

[0284] FIG. 23 depicts an example sequence diagram of monitoring user-related issues. A data scientist may review results weekly and identify trends in failures, monitor performance, new imaging protocols, and scanners and submit the results to an Al analytics monitoring service. A radiology support assistant (RSA) may also review results daily and submit the results to an Al analytics monitoring service. The RSA may notify a radiologist of any errors or issues, and the radiologist may likewise notify the RSA of any errors or issues related to the review of the output data generated by an Al algorithm. Errors and issues may include a criticality score and may include: an ordering clinician placing an order with the results being available in ROCKET, but the radiologist does not review the data, or reposts a value calculated by the Al analytics (e.g., a skeletal muscle mass index for an Al algorithm that generates body composition measurements from abdominal CT scans) (major critical score); an ordering clinician is placing an order with the results being available in ROCKET, and the radiologist reviews the results but does not click accept / reject and copies the values to the report (minor score); an ordering clinician placing an order with the results are available in ROCKET, and the radiologist reviews the results but does click accept / reject and types the wrong values to report (major score); the ordering clinician placing an order with the results are not available in ROCKET but the radiologist dictates the report without waiting for the results of the Al analysis (minor score); and / or the ordering clinician placing an order but the results are not available in ROCKET due to technical failure, and the radiologist dictates the report without waiting for the results (major). Performance metrics captured by the review of the Al analysis by the radiologist and / or RSA may include an overall performance of the Al algorithm based on the number of accepted values vs. the number of reject values, a comparison of the algorithm-generated values vs. a reported or actual value, the overall execution time, a comparison between the amount of time for the algorithm to calculate a value vs. when a radiologist approves a report, a number of orders per ordering physician, the quality of the radiologist reading Al results, and / or failures due to the algorithm not finding a correct marker in an image (e.g., not finding an L3 disc in an abdominal CT scan). The RSA may record the issue for immediate service. Such monitoring of results may also catch errors where no output is available.

[0285] FIG. 24 depicts a flow diagram of an example method for fact checking large language model outputs.

[0286] At operation 2402, the method may include inputting source data (e.g., text data, video data, audio data, DICOM data, etc.) to a first trained large language machine learning model at the foundation layer, to generate vectorized data. At operation 2404, the method may include submitting a retrieval query to the trained large language machine learning model at thefoundation layer, and, in the trained large language machine learning model at the foundation layer, encoding the retrieval query with a structure corresponding to the vectorized data.

[0287] At operation 2406, the method may include performing a semantic search for matches between the vectorized data and the retrieval query. At operation 2408, the method may include inputting the retrieval query and one or more results of the search to the trained large language machine learning model. At operation 2410, the method may include generating, at the trained large language machine learning model at the foundation layer, a summary in response to the retrieval query.

[0288] At operation 2412, the method may include determining, at the trained large language machine learning model at the foundation layer, one or more parts of the summary for fact checking by a user. At operation 2414, the method may include inputting the parts of the summary for fact checking into the large language machine learning model to retrieve one or more references associated with the parts of the summary. At operation 2416, the method may include displaying the summary and the one or more references to a user. In some embodiments, the summary and references may be displayed in an unvalidated response error tool. In some embodiments, the method may further include receiving, by the unvalidated response error tool, a user assessment of the summary. The method may include fine-tuning the second trained machine learning model based on the user assessment of the summary.Advantageously, the fine-tuned model may be stored on a computer readable storage media for future use at operations 2402-2416 in some embodiments. Additionally or alternatively, statistics may be generated from the assessments in some embodiments, which may be displayed in a dashboard or other report (not shown) to the relevant stakeholders.

[0289] In some embodiments, the method may further include generating, at the trained large language machine learning model, pairings between the parts of the summary and the one or more references. The method may in determining, at a second trained machine learning model, a score for each pairing and displaying the score for each pairing. In some embodiments, the parts of the summary may be color-coded based on the score. In some embodiments, the method may include identifying the reference paired with the part of the summary in response to a user interaction with the part of the summary.

[0290] FIG. 25 depicts an example of a retrieval augmented generation (RAG) process. Context documents 2502 may be passed to an LLM 2506 to embed chunks of the document into numerical values (e.g., vectors). A query 2504 for retrieval may also be passed to the LLM 2506 to embed the query into numerical values, similarly to the context documents 2502. The LLM 2506 performs a semantic search such that the vectors corresponding to the query 2504may be matched with vectors corresponding to relevant context documents 2502. The relevant context documents 2508 are retrieved and augment the prompt 2504 when passed to another LLM for other applications.

[0291] FIG. 26 depicts an example of summary generation after retrieval. Relevant contextual documents and / or chunks are retrieved according to a RAG process at operation 2602 (described further in detail in FIG. 25). The retrieved documents / chunks 2604 may be passed to an LLM 2606 and used to generate a summary 2608. The summary 2608 may include hallucinations, i.e., incorrect, fabricated, and / or nonsensical information in the response.

[0292] FIG. 27 depicts an example of a fact checking process. A response generated by an LLM (e.g., the LLM 2606 of FIG. 26) may be broken into chunks and include concepts to fact check 2702. The concepts to fact check 2702 may be passed to an LLM 2704 and may be embedded into vectors. The vectors may be used to retrieve potentially relevant document / chunks 2706. The potentially relevant documents / chunks 2706 may be used for fact checking as described in FIG. 28.

[0293] FIG. 28 depicts further operations in an example fact checking process. Generated text (e.g., a summary generated in response to an input query) may be paired with potential references. The pairs 2802 of generated text and references may be passed to an LLM 2804. The LLM 2804 may generate scores 2806 for the pairs 2802 on how well the references explain the fact(s) from the generated text. For example, the LLM 2804 may rank or score the references on a scale of 1-10. The generated text, references, and / or scores may be displayed in an unvalidated error response tool (e.g., a CURE user interface) at operation 2808. A user may interact with the unvalidated error response tool to assess the generated text.Aspects of the Disclosure

[0294] Aspects of the techniques described in the present disclosure may include any of the following aspects, either alone or in combination:

[0295] Example 1. A computer-implemented method for fact checking large language model outputs, the method comprising: inputting text data to a first trained large language machine learning model at the foundation layer to generate vectorized data; submitting a retrieval query to the trained large language machine learning model at the foundation layer, and, in the trained large language machine learning model at the foundation layer, encoding the retrieval query with a structure corresponding to the vectorized data; performing a semantic search for matches between the vectorized data and the retrieval query; inputting the retrieval query and one or more results of the search to the trained large language machine learning model; generating, atthe trained large language machine learning model at the foundation layer, a summary in response to the retrieval query; determining, at the trained large language machine learning model at the foundation layer, one or more parts of the summary for fact checking by a user; inputting the parts of the summary for fact checking into the large language machine learning model to retrieve one or more references associated with the parts of the summary; and displaying the summary and the one or more references to a user.

[0296] Example 2. The computer-implemented method of example 1 further comprising: generating, at the trained large language machine learning model, pairings between the parts of the summary and the one or more references; determining, at a second trained machine learning model, a score for each pairing; and displaying the score for each pairing.

[0297] Example 3. The computer-implemented method of example 1 or 2, wherein the summary and references are displayed in an unvalidated response error tool.

[0298] Example 4. The computer-implemented method of any of examples 1 through 3, further comprising: receiving, by the unvalidated response error tool, a user assessment of the summary.

[0299] Example 5. The computer-implemented method of any of examples 1 through 4, further comprising: fine-tuning the second trained machine learning model based on the user assessment of the summary.

[0300] Example 6. The computer-implemented method of any of examples 1 through 3, wherein the parts of the summary are color-coded based on the score.

[0301] Example 7. The method of any of examples 1 through 3, further comprising: identifying the reference paired with the part of the summary in response to a user interaction with the part of the summary.

[0302] Example 8. A computing system for fact checking large language model outputs, the system comprising: one or more processors; and one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing system to: input text data to a first trained large language machine learning model at the foundation layer, to generate vectorized data; submit a retrieval query to the trained large language machine learning model at the foundation layer, and, in the trained large language machine learning model at the foundation layer, encoding the retrieval query with a structure corresponding to the vectorized data; perform a semantic search for matches between the vectorized data and the retrieval query; input the retrieval query and one or more results of the search to the trained large language machine learning model; generate, at the trained largelanguage machine learning model at the foundation layer, a summary in response to the retrieval query; determine, at the trained large language machine learning model at the foundation layer one or more parts of the summary for fact checking by a user; input the parts of the summary for fact checking into the large language machine learning model to retrieve one or more references associated with the parts of the summary; and display the summary and the one or more references to a user.

[0303] Example 9. The computing system of example 8, the one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, further cause the computing system to: generate, at the trained large language machine learning model, pairings between the parts of the summary and the one or more references; determine, at a second trained machine learning model, a score for each pairing; and display the score for each pairing.

[0304] Example 10. The computing system of example 8 or 9, wherein one or more of the summary, references and / or score are displayed in an unvalidated response error tool.

[0305] Example 11. The computing system of any of examples 8 through 10, the one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, further cause the computing system to: receive, by the unvalidated response error tool, a user assessment of the summary.

[0306] Example 12. The computing system of any of examples 8 through 11 , the one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, further cause the computing system to: fine-tune the second trained machine learning model based on the user assessment of the summary.

[0307] Example 13. The computing system of examples 8 through 10, wherein the parts of the summary are color-coded based on the score.Example 14. The computing system of examples 8 through 10, the one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, further cause the computing system to: identify the reference paired with the part of the summary in response to a user interaction with the part of the summary.

[0308] Example 15. A computer program product for fact checking medical model outputs, the computer program product comprising one or more computer readable storage mediums having program instructions embodied therewith, the program instructions executable by a processing component to cause the processing component to: input healthcare data to one or more trained large language machine learning models at a foundation layer, to generatevectorized data; submit a medical query to the one or more trained large language machine learning models, and, in one or more trained large language machine learning models, encode the retrieval query with a structure corresponding to the vectorized data; perform a semantic search for matches between the vectorized data and the retrieval query; input the medical query and the matches to the one or more trained large language machine learning models; generate, at the one or more trained large language machine learning models, a summary in response to the retrieval query; determine, at the one or more trained large language machine learning models, one or more parts of the summary for fact checking by a health professional; input the parts of the summary for fact checking into the one or more large language machine learning models to retrieve one or more references associated with the parts of the summary; display the summary and the one or more references to the health professional; receiving, by an unvalidated response error tool, an assessment of the summary from the health professional; fine-tune the trained machine learning model based on the user assessment of the summary to generate an updated one or more large language machine learning models; and store the updated one or more updated large language models onto the one or more computer readable storage mediums.Additional Considerations

[0309] The various embodiments described above can be combined to provide further embodiments. All U.S. patents, U.S. patent application publications, U.S. patent application, foreign patents, foreign patent application and non-patent publications referred to in this specification and / or listed in the Application Data Sheet are incorporated herein by reference, in their respective entireties, for all purposes. Implementations of the embodiments can be modified if necessary to employ concepts of the various patents, applications, and publications to provide yet further embodiments.

[0310] These and other changes can be made to the embodiments in light of the abovedetailed description. In general, in the following claims, the terms used should not be construed to limit the claims to the specific embodiments disclosed in the specification and the claims but should be construed to include all possible embodiments along with the full scope of equivalents to which such claims are entitled. Accordingly, the claims are not limited by the disclosure.

[0311] The following considerations also apply to the foregoing discussion. Throughout this specification, plural instances may implement operations or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations be performed in the order illustrated.These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.

[0312] It should also be understood that, unless a term is expressly defined in this patent using the sentence "As used herein, the term " " is hereby defined to mean . . . " or a similar sentence, there is no intent to limit the meaning of that term, either expressly or by implication, beyond its plain or ordinary meaning, and such term should not be interpreted to be limited in scope based on any statement made in any section of this patent (other than the language of the claims). To the extent that any term recited in the claims at the end of this patent is referred to in this patent in a manner consistent with a single meaning, that is done for sake of clarity only so as to not confuse the reader, and it is not intended that such claim term be limited, by implication or otherwise, to that single meaning. Finally, unless a claim element is defined by reciting the word "means" and a function without the recital of any structure, it is not intended that the scope of any claim element be interpreted based on the application of 35 U.S.C. § 112(f).

[0313] Unless specifically stated otherwise, discussions herein using words such as "processing," "computing," "calculating," "determining," "presenting," "displaying," or the like may refer to actions or processes of a machine (e.g., a computer) that manipulates or transforms data represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other machine components that receive, store, transmit, or display information.

[0314] As used herein any reference to "one implementation" or "an implementation" means that a particular element, feature, structure, or characteristic described in connection with the implementation is included in at least one implementation. The appearances of the phrase "in one implementation" in various places in the specification are not necessarily all referring to the same implementation.

[0315] As used herein, the terms "comprises," "comprising," "includes," "including," "has," "having" or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, unless expressly stated to the contrary, "or" refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

[0316] In addition, use of "a" or "an" is employed to describe elements and components of the implementations herein. This is done merely for convenience and to give a general sense of the invention. This description should be read to include one or at least one and the singular also includes the plural unless it is obvious that it is meant otherwise.

[0317] Upon reading this disclosure, those of skill in the art will appreciate still additional alternative structural and functional designs for implementing the concepts disclosed herein, through the principles disclosed herein. Thus, while particular implementations and applications have been illustrated and described, it is to be understood that the disclosed implementations are not limited to the precise construction and components disclosed herein. Various modifications, changes and variations, which will be apparent to those skilled in the art, may be made in the arrangement, operation and details of the method and apparatus disclosed herein without departing from the spirit and scope defined in the appended claims.

Claims

What is claimed is:1 . A computer-implemented method for fact checking large language model outputs, the method comprising: inputting data to a first trained large language machine learning model at a foundation layer to generate vectorized data; submitting a retrieval query to the trained large language machine learning model at the foundation layer, and, in the trained large language machine learning model at the foundation layer, encoding the retrieval query with a structure corresponding to the vectorized data; performing a semantic search for matches between the vectorized data and the retrieval query; inputting the retrieval query and one or more matches of the semantic search to the trained large language machine learning model; generating, at the trained large language machine learning model at the foundation layer, a summary in response to the retrieval query; determining, at the trained large language machine learning model at the foundation layer, one or more parts of the summary for fact checking by a user; inputting the parts of the summary for fact checking into the large language machine learning model to retrieve one or more references associated with the parts of the summary; and displaying the summary and the one or more references to the user.

2. The computer-implemented method of 1 further comprising: generating, at the trained large language machine learning model, pairings between the parts of the summary and the one or more references; determining, at a second trained machine learning model, a score for each pairing; and displaying the score for each pairing.

3. The computer-implemented method of claim 1 or 2, wherein the summary and references are displayed in an unvalidated response error tool.

4. The computer-implemented method of claim 3, further comprising: receiving, by the unvalidated response error tool, a user assessment of the summary.

5. The computer-implemented method of claim 4, further comprising: fine-tuning the second trained machine learning model based on the user assessment of the summary to generate an updated second trained large language machine learning model.

6. The computer-implemented method of claim 3, wherein the parts of the summary are color-coded based on the score.

7. The computer-implemented method of claim 3, further comprising: identifying the reference paired with the part of the summary in response to a user interaction with the part of the summary.

8. A computing system for fact checking large language model outputs, the system comprising: one or more processors; and one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing system to: input text data to a first trained large language machine learning model at the foundation layer, to generate vectorized data; submit a retrieval query to the trained large language machine learning model at the foundation layer, and, in the trained large language machine learning model at the foundation layer, encoding the retrieval query with a structure corresponding to the vectorized data; perform a semantic search for matches between the vectorized data and the retrieval query; input the retrieval query and one or more results of the search to the trained large language machine learning model; generate, at the trained large language machine learning model at the foundation layer, a summary in response to the retrieval query; determine, at the trained large language machine learning model at the foundation layer one or more parts of the summary for fact checking by a user; input the parts of the summary for fact checking into the large language machine learning model to retrieve one or more references associated with the parts of the summary; and display the summary and the one or more references to a user.

9. The computing system of claim 8, the one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, further cause the computing system to: generate, at the trained large language machine learning model, pairings between the parts of the summary and the one or more references; determine, at a second trained machine learning model, a score for each pairing; and display the score for each pairing.

10. The computing system of claim 9, wherein one or more of the summary, references and / or score are displayed in an unvalidated response error tool.

11. The computing system of claim 10, the one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, further cause the computing system to: receive, by the unvalidated response error tool, a user assessment of the summary.

12. The computing system of claim 11 , the one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, further cause the computing system to: fine-tune the second trained machine learning model based on the user assessment of the summary.

13. The computing system of claim 10, wherein the parts of the summary are color- coded based on the score.

14. The computing system of claim 10, the one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, further cause the computing system to: identify the reference paired with the part of the summary in response to a user interaction with the part of the summary.

15. A computer program product for fact checking medical model outputs, the computer program product comprising one or more computer readable storage mediums having programinstructions embodied therewith, the program instructions executable by a processing component to cause the processing component to: input healthcare data to one or more trained large language machine learning models at a foundation layer, to generate vectorized data; submit a medical query to the one or more trained large language machine learning models, and, in one or more trained large language machine learning models, encode the retrieval query with a structure corresponding to the vectorized data; perform a semantic search for matches between the vectorized data and the retrieval query; input the medical query and the matches to the one or more trained large language machine learning models; generate, at the one or more trained large language machine learning models, a summary in response to the retrieval query; determine, at the one or more trained large language machine learning models, one or more parts of the summary for fact checking by a health professional; input the parts of the summary for fact checking into the one or more large language machine learning models to retrieve one or more references associated with the parts of the summary; display the summary and the one or more references to the health professional; receiving, by an unvalidated response error tool, an assessment of the summary from the health professional; fine-tune the trained machine learning model based on the user assessment of the summary to generate an updated one or more large language machine learning models; and store the updated one or more updated large language models onto the one or more computer readable storage mediums.