System and method for generating and deploying task-specific agents

Abstract

This application describes, in particular, a method and system for building and deploying agents. An exemplary method includes receiving a request for an agent configured to perform a specific task. A first agent type is identified from a set of agent types based on the requirements for performing a specific task, and each agent type in the set of agent types corresponds to a respective language model. A model component having the first agent type is generated. An implementation component is generated, and the implementation component is configured to communicatively combine the model component with a set of component components based on the requirements for performing a specific task, and the set of components includes a set of data sources, a set of tools, and / or a set of output components. The agent, including the model component and the implementation component, is deployed to a work environment.

Description

[Technical Field]

[0001] Related applications This application claims priority to U.S. Provisional Application No. 63 / 505,018, filed on 30 May 2023, entitled "AI-Enabled Clinical Assistant," and to U.S. Provisional Application No. 63 / 515,532, filed on 25 July 2023, entitled "Systems and Methods for Generating and Deploying Task-Specific Agents," each of which is incorporated herein by reference in its entirety.

[0002] The disclosed embodiments generally relate to the use of task-specific orchestration, including, but not limited to, the generation, combination, and deployment of task-specific machine learning agents. [Background technology]

[0003] Many professions require complex thinking, where numerous factors must be considered when selecting solutions to a given situation, hypothesizing new factors and solutions, and testing those new factors and solutions to confirm their effectiveness. For example, an oncologist considering a particular patient's cancer status must optimally consider many different factors when assessing the patient's cancer status, as well as many other factors when formulating and implementing an optimized treatment plan. These factors may include, for example, the patient's family history, past medical history, current diagnosis, genomic / molecular profile of the patient's hereditary DNA and tumor DNA, currently accepted national guidelines for standard treatment within that cancer subtype, recently published research on the patient's condition, available clinical trials relevant to the patient, available drugs, and other therapeutic interventions that may be good options for the patient, as well as data from similar patients. In addition, cancer and cancer treatment research is rapidly evolving, and researchers must continuously utilize data, new research, and new treatment guidelines to conduct crucial deliberation on new factors and treatments when diagnosing cancer status and optimizing treatment plans.

[0004] In particular, it is no longer reasonable for oncologists to be familiar with every new research in the field of cancer treatment. Similarly, it is extremely difficult for oncologists to manually analyze the medical records and outcomes of thousands or millions of cancer patients every time they make a specific treatment recommendation for a particular patient they are treating. As a first problem, oncologists often do not even have access to health information from healthcare institutions other than their own. In the United States, the enforcement of a federal law known as the Health Insurance Portability and Accountability Act of 1996 (commonly known as "HIPAA") has imposed significant limitations on one healthcare provider's ability to access the health records of another. In addition, healthcare systems face administrative, technical, and financial challenges in making their data available to third parties and integrating it with similar data from other healthcare systems. When multiple institutions are responsible for developing a single integrated repository, significant disagreements can arise regarding the structure of one or more data dictionaries, how to access the data, which individuals or other providers are permitted to access the data, and the amount of data that can be accessed. Furthermore, the scope of data that can be searched is overwhelming for oncologists who wish to perform manual reviews. Every patient possesses health information containing hundreds or thousands of data elements. When sequencing information is included in health information accessed and analyzed using next-generation sequencing, the amount of health information that can be analyzed increases dramatically. For example, a single FASTQ or BAM file produced during whole-exome sequencing, even though it contains sequencing only of a patient's exome—a mere fraction of the entire human genome—occupies gigabytes of memory.

[0005] Tools are continuously being developed to help oncologists diagnose cancer conditions, select and implement optimized treatments, and explore and consider new cancer condition factors, new cancer conditions (e.g., diagnoses), new treatment factors, new therapies, and new efficacy factors. For example, large cancer databases are being developed and maintained for oncologists to access and manipulate to explore diagnostic and treatment options, as well as new insights and treatment hypotheses. Computers enable access to and manipulation of cancer data and its derivatives.

[0006] Traditional computers and workstations function well as data access and operation interfaces, but they have several drawbacks. First, using computer interfaces often requires oncologists to click multiple times across different interfaces to find specific information. This is a cumbersome and time-consuming process, and often fails to achieve the desired outcome of receiving answers to the questions the oncologist is trying to ask. Furthermore, oncology and research data activities often involve a series of questions or requests (hereinafter "requests") that increasingly focus on more detailed data responses, where the oncologist / researcher needs to repeatedly input additional information to define the next level of requests because the intermediate results are not particularly interesting.

[0007] Recently, large-scale language models (LLMs) such as ChatGPT by OpenAI have become increasingly popular for using human language to answer a variety of questions. However, LLMs suffer from a lack of robustness in performance, particularly when used to evaluate niche information or answer specific niche questions in a medical context. For example, traditional LLMs are trained on vast amounts of data that do not adequately represent the niche subject, leading to inaccuracies or omissions in the evaluations and outputs provided by the LLM. Furthermore, LLMs tend to be over-tuned when trained on niche datasets, causing their performance to degrade with unseen data excluded from the training dataset.

[0008] Furthermore, LLMs are resource-intensive, requiring significant computational and financial resources for training and adoption. For example, while traditional LLMs may have a context window with a capacity of 8,000 to 100,000 tokens, they cannot maintain the accuracy and precision of their output when processing large numbers of tokens. See Li, Dacheng et al., 2023, "How Long Can Context Length of Open-Source LLMs Truly Promise?", NeurIPS, Workshop on Instruction Tuning and Instruction. [Overview of the project]

[0009] Therefore, the inventors of this application recognized the need for a system and method that enables users to query medical information (and other types of information) using natural language, an intuitive interface, and supplementary questions.

[0010] This disclosure, in particular, describes generating, deploying, and interacting with machine learning agents (e.g., machine learning orchestrations). For example, agents may be generated and deployed using agent building components (e.g., within a control plane) and / or agent building user interfaces. Deployed agents may be stored in an agent host (and optionally updated via agent building components). Agents may be task-specific, and agent generation may involve identifying appropriate machine learning models and model configurations for specific tasks. Agents may be configured via configuration files and communicatively coupled to one or more datasets, one or more tools (e.g., formatting, data access, and / or analysis tools), and / or one or more output components. In this way, the systems and methods described herein enable users without programming expertise to generate, deploy, and interact with agents to acquire and analyze medical (and other types of) data. Identifying the correct agent type for a given task can improve performance and reduce computer and storage costs. Furthermore, the systems and methods described herein restrict the data that agents can access and edit, thereby improving security and preventing data corruption and unauthorized access.

[0011] According to some embodiments, the Disclosure provides systems and methods for instructing a user by generating, modifying, and / or deploying task-specific machine learning models, such as within a user interface of a client device. In some embodiments, the user interface is accessible through a display device, such as via internet address access, which in turn presents a library of agents and / or agent modules for building customized and / or predetermined agents, for deploying customized and / or predetermined agents, and for running customized and / or predetermined agents. The user interface further allows for the customization of data that may be provided to the agent via a collection of documents.

[0012] In some embodiments, the Disclosure provides systems and methods for generating, configuring, and / or maintaining an architecture for an agent (sometimes also referred to as an agent module or agent component). In some embodiments, the architecture includes a framework of multiple interconnected nodes that enable the agent to run, such as over a communication network on a remote client device. In some embodiments, the system and method maintains the agent architecture, such as multiple interconnected nodes including input nodes (e.g., initial terminal nodes) and output nodes (e.g., final terminal nodes) that can be deployed and / or run within a cloud architecture. In some embodiments, the system and method maintains conditional logic associated with the agent, thereby enabling the agent to perform certain functions, such as how API calls are handled to adapt the execution and embedding of the agent within other cloud-based software systems.

[0013] According to several embodiments, the Disclosure provides a system and method for a framework for agent configuration to structure how an agent responds to various prompts received from different users, using one or more coarse-grained logic, such as the arrangement of nodes in a node architecture of interconnected nodes, and / or one or more fine-grained logic, such as modifiable weights for each node in the node architecture. In some embodiments, an agent receives a prompt that includes structured data and / or a text file as input, which is then applied to the node architecture according to the coarse-grained and fine-grained logic. For example, information from a prompt may be provided to a node associated with a first model trained on tumor screening in a plurality of interconnected blocks, and ground truth associated with the structured data is transformed into a form acceptable for ingestion by the first model.

[0014] According to some embodiments, the Disclosure provides a system and method for a user interface framework that grants access rights to secure data based on user credentials, allows users to access different user interface elements based on tools and / or external services, and enables the creation and / or storage of personalized agent modules, such as collaboration between third-party users.

[0015] According to some embodiments, a method for configuring a task-specific agent includes: (i) receiving a request from a user for an agent configured to perform a particular task; (ii) identifying a first agent type from a set of agent types in response to the request, based on one or more requirements for performing a particular task, wherein each agent type in the set of agent types corresponds to a respective language model; (iii) generating a model component having the first agent type, wherein generating the model component includes generating a set of operation instructions for the model component; (iv) generating an implementation component for the agent, wherein the implementation component is configured to communicatively combine the model component into a set of components, based on one or more requirements for performing a particular task, wherein the set of components includes one or more of a set of data sources, a set of tools, and a set of output components; and (v) deploying the agent into a working environment, wherein the agent includes the model component and the implementation component.

[0016] According to some embodiments, a method for identifying a target includes (i) receiving a request from a user to identify a target that satisfies a set of criteria; (ii) obtaining a set of protocols from the request via a language model component; (iii) generating one or more structured queries based on the set of protocols via a language model component; (iv) sending one or more structured queries to one or more databases via a language model component; and (v) receiving a set of targets that satisfy the set of criteria from one or more databases in response to the sending of one or more structured queries.

[0017] According to some embodiments, computing systems are provided, such as cloud computing systems, server systems, personal computer systems, and / or other types of electronic devices. The computing system includes a control circuit and a memory for storing one or more sets of instructions. One or more sets of instructions include instructions for performing any of the methods described herein.

[0018] According to some embodiments, a non-temporary computer-readable storage medium is provided. The non-temporary computer-readable storage medium stores one or more sets of instructions for execution by a computing system. One or more sets of instructions include instructions for performing any of the methods described herein.

[0019] Therefore, devices and systems are disclosed together with methods for providing clinical support by generating, deploying, and using agents. Such methods, devices, and systems may complement or replace conventional methods, devices, and systems for providing clinical support.

[0020] The features and advantages described herein are not necessarily exhaustive, and several additional features and advantages will be apparent to those skilled in the art, particularly in light of the drawings, specifications, and claims provided herein. Furthermore, it should be noted that the language used herein has been selected primarily for readability and instructional purposes and is not necessarily chosen to describe or surround the subject matter described herein.

[0021] To enable a more detailed understanding of this disclosure, a more specific description could be provided by referring to the features of various embodiments, some of which are illustrated in the accompanying drawings. However, the accompanying drawings merely illustrate the relevant features of this disclosure and should not be considered limiting, as a person skilled in the art will understand when reading this disclosure that the description may accept other valid features. [Brief explanation of the drawing]

[0022] [Figure 1] A block diagram illustrating an exemplary platform according to some embodiments. [Figure 2A] A block diagram illustrating an exemplary client device according to some embodiments. [Figure 2B] A block diagram illustrating an exemplary client device according to some embodiments. [Figure 3] A block diagram illustrating an exemplary server system according to some embodiments. [Figure 4] A block diagram illustrating an exemplary database according to some embodiments. [Figure 5A] Illustrates an exemplary user interface for querying a database according to some embodiments. [Figure 5B] Illustrates an exemplary user interface for querying a database according to some embodiments. [Figure 6] Illustrates an exemplary architecture for deploying agents according to some embodiments. [Figure 7A] Illustrates an exemplary process for model selection according to some embodiments. [Figure 7B] Illustrates an exemplary process for model selection according to some embodiments. [Figure 7C] Illustrates an exemplary process for model selection according to some embodiments. [Figure 7D] Illustrates an exemplary process for model selection according to some embodiments. [Figure 7E] Illustrates an exemplary process for model selection according to some embodiments. [Figure 8A] Illustrates an exemplary model architecture according to some embodiments. [Figure 8B]Several exemplary model architectures are illustrated using various embodiments. [Figure 9A] This document illustrates exemplary processes for data vectorization and query processing using several embodiments. [Figure 9B] This document illustrates exemplary processes for data vectorization and query processing using several embodiments. [Figure 9C] This document illustrates exemplary processes for data vectorization and query processing using several embodiments. [Figure 9D] This document illustrates an exemplary process for patient queries using several embodiments. [Figure 10A] This paper illustrates exemplary user interfaces for agent-based search in several embodiments. [Figure 10B] This paper illustrates exemplary user interfaces for agent-based search in several embodiments. [Figure 10C] This paper illustrates exemplary user interfaces for agent-based search in several embodiments. [Figure 11A] This paper illustrates exemplary user interfaces for interacting with a digital assistant, based on several embodiments. [Figure 11B] This paper illustrates exemplary user interfaces for interacting with a digital assistant, based on several embodiments. [Figure 11C] This paper illustrates exemplary user interfaces for interacting with a digital assistant, based on several embodiments. [Figure 12A] This paper illustrates exemplary user interfaces for interacting with a digital assistant, based on several embodiments. [Figure 12B] This paper illustrates exemplary user interfaces for interacting with a digital assistant, based on several embodiments. [Figure 12C]This paper illustrates exemplary user interfaces for interacting with a digital assistant, based on several embodiments. [Figure 12D] This paper illustrates exemplary user interfaces for interacting with a digital assistant, based on several embodiments. [Figure 13] This flowchart illustrates exemplary methods for deploying agents in several embodiments. [Figure 14] This document illustrates exemplary workflows for implementing and / or interacting with an agent, based on several embodiments. [Figure 15] This document illustrates another exemplary workflow for implementing and / or interacting with an agent, using several embodiments. [Figure 16] This document illustrates another exemplary workflow for implementing and / or interacting with an agent, using several embodiments. [Figure 17] Several exemplary agent modules are illustrated according to various embodiments. [Figure 18A] This document illustrates exemplary user interactions with an exemplary orchestration creator application in several embodiments. [Figure 18B] This document illustrates exemplary user interactions with an exemplary orchestration creator application in several embodiments. [Figure 18C] This document illustrates exemplary user interactions with an exemplary orchestration creator application in several embodiments. [Figure 18D] This document illustrates exemplary user interactions with an exemplary orchestration creator application in several embodiments. [Figure 18E] This document illustrates exemplary user interactions with an exemplary orchestration creator application in several embodiments. [Figure 18F]This document illustrates exemplary user interactions with an exemplary orchestration creator application in several embodiments. [Figure 18G] Several exemplary workflows are illustrated using various embodiments. [Figure 19A] This document illustrates exemplary workflow representations of orchestration configured based on user prompts, using several embodiments. [Figure 19B] This document illustrates exemplary workflow representations of orchestration configured based on user prompts, using several embodiments. [Figure 19C] This example illustrates a comparison of accuracy between an agent architecture solution and a single-model solution. [Figure 20] Examples of various logic functions implemented in some embodiments of this disclosure are illustrated below. [Figure 21] This flowchart illustrates exemplary ways of interacting with task-specific orchestration through several embodiments. [Figure 22] This flowchart illustrates exemplary methods for deploying task-specific machine learning models through several embodiments. [Figure 23] This flowchart illustrates exemplary methods for constructing task-specific machine learning models through several embodiments. [Figure 24] This flowchart illustrates typical methods for performing clinical tasks using several embodiments. [Figure 25] This flowchart illustrates exemplary methods, in several embodiments, that enable third-party access to and use of the agent module. [Figure 26] This flowchart illustrates an exemplary method for selecting a model for a clinical task, using several embodiments. [Figure 27] This flowchart illustrates exemplary methods for verifying data compatibility through several embodiments.

[0023] According to common practice, various features illustrated in the drawings are not necessarily depicted to actual size, and similar features can be shown throughout this specification and the drawings using the same reference numerals. [Modes for carrying out the invention]

[0024] This disclosure, in particular, describes a platform for generating, deploying, and using task-specific orchestrations (e.g., task-specific agents) that include task-specific machine learning models (e.g., language models, translation models, and other types of models) for specific tasks and / or within specific domains. The platform may include a plurality of individual task-specific orchestrations that can operate independently or in combination to return accurate and relevant information (e.g., target cohorts, clinical trial information, and / or identification of members of a target population). In some embodiments, each task-specific orchestration (or agent) may include one or more machine learning models, such as language models, that have been trained and / or fine-tuned on a particular domain. The platform may also include one or more composite orchestrations (e.g., composite agents) that instruct a plurality of task-specific orchestrations configured for different tasks and combine the results.

[0025] In some embodiments, the platform acts as an operating system for implementing task-specific orchestrations for performing clinical tasks. The platform may include one or more of the following exemplary components: For example, a gene sequencing component with downstream molecular bioinformatics may operate to retrieve relevant biomarkers from DNA, RNA, or derivatives thereof of a specimen (e.g., a tumor biopsy) to report sequencing results to the ordering physician. In another embodiment, a pathology imaging component may operate on cell and / or slide-level images to identify relevant biomarkers from cells within an imaged specimen. In yet another embodiment, a radiography component may operate on larger images of the body through various radiography techniques to identify the presence or longitudinal progression of a tumor. Each of these components may include, or communicate with, a corresponding agent to identify and / or report information relevant to a user query or request.

[0026] In one embodiment, an agent may be configured by a user using a user interface (e.g., a console in a web or desktop application) and deployed to various environments (e.g., alpha, beta, client, and / or production environments). Each environment may be linked to a different source, have different privileges, and / or have different authenticated users. In some embodiments, precision medicine principles are employed to customize the user interface, such as modifications based on a set of subjects (e.g., patients) associated with the application's users. Environments may be defined by access to data sources and / or users. Agent configurations may be stored in a control plane. The agent itself may run within an appropriate workload plane (e.g., a data plane), which may not have access to the control plane. In this embodiment, an agent builder in the control plane is configured to push configurations into various environments. For example, this synchronization may be fast enough for a user to configure an agent and immediately test the configuration in an interactive console in a working environment. An exemplary architecture includes two components: an agent builder in a control plane that hosts a user interface (UI) for configuring agents, and an agent host in a workload plane that hosts a UI and API for interacting with deployed agents. When an agent configuration is changed or an agent version is deployed, the agent builder may notify the agent host in each environment so that the updated agent can be deployed. For example, this may be done via pub / sub messages to a topic of agent configurations or via a simple HTTP request. In some embodiments, the agent builder utilizes a cognitive architecture that includes memory modules and behavioral spaces. For example, the cognitive architecture organizes agents in three dimensions: their informational memory (e.g., divided into working memory and long-term memory), their behavioral spaces (e.g., divided into internal and external behaviors), and their decision-making procedures (e.g., structured as interactive loops with planning and execution).

[0027] In another embodiment, after deployment, the agent may receive a user query (e.g., requesting information about a clinical trial), generate a Structured Application Programming Interface (API) call, use the generated API call to query a remote server to retrieve relevant results, reformat the relevant information, and return it to the user. In some embodiments, each action is performed by a different agent builder block component (sometimes also referred to as a builder block, block, or node). In some embodiments, the agent is configured for multiple types of tasks. In these embodiments, the agent may identify the intent of the user's query (e.g., searching for a clinical trial or identifying an adverse event) and respond accordingly. In some embodiments, the agent is configured for only one type of task (e.g., a task-specific agent). In some of these embodiments, the agent does not identify the user's intent (e.g., the agent may assume an intent). In some embodiments, the agent receives intent from different components or systems. The agent may also interface with other agents to obtain additional information for the user query (e.g., patient records or relevant guidelines). In some embodiments, the agent includes a pre-trained language model (e.g., trained in a specific domain and / or using a specific database). In some embodiments, the agent queries an unstructured database (e.g., in addition to or instead of generating API calls).

[0028] The platform or its components may be used in conjunction with any medical field such as oncology, endocrinology (e.g., diabetes), mental health (e.g., depression and related pharmacogenetics), and cardiovascular disease (e.g., to assist physicians in the treatment of any relevant disease state within those fields). For example, the platform may also include a cardiology-based component (e.g., including one or more agents) that operates on electrocardiogram (ECG) data to identify patients at high risk of cardiovascular disease. In another embodiment, the platform may include a data curation component (e.g., including one or more agents) that takes raw (e.g., unstructured) data and structures it into a common and useful format as a repository of clinical data (e.g., a multimode database) on which other agents, models, and / or components can operate. In another embodiment, the platform may be configured to search within clinical data to identify relevant patient cohorts and / or generate insights and / or analyses. In another embodiment, the platform may be configured to monitor electronic health records (EHRs) to identify care gaps and / or reminders to physicians to take action for each patient. In this way, the platform functions as a physician-assisted documentation management tool to identify issues / events that physicians have not manually compiled documentation for, ensuring that patients and other subjects receive timely care. The platform may be configured to track and / or catalog relevant therapies (e.g., on-label and / or off-label use) for a set of disease conditions. The platform may also track and / or catalog relevant clinical trials (e.g., multiple countries and / or multiple authorities) for a set of disease conditions.

[0029] As will be discussed below, the platform may include an AI-enabled support user interface (which may be described herein as a clinical assistant or digital assistant) that provides access to patient insights. The AI-enabled support user interface may use one or more task-specific orchestrations, each including language models and / or other types of machine learning.

[0030] In some embodiments, the platform includes a hub component that enables physicians to order, track, and view trial results and export patient data. In some embodiments, the hub component provides insights into genomic mutations, therapeutic implications, and clinical trial referencing. The hub component may be used in conjunction with an AI-enabled clinical assistant to enable physicians to interact using conversational language, including natural language input, supplementary questions, and comments. The platform may also include a peer-to-peer messaging component for physicians and other healthcare professionals to share knowledge, insights, and / or perspectives in medical fields such as molecular oncology (for example, because it is relevant to patient care). The messaging component may be used in conjunction with an AI-enabled clinical assistant to engage in conversations on the messaging component and optionally learn from them. For example, an AI-enabled clinical assistant may be called upon in a conversation to provide insights and / or data on a particular topic or conversation. The platform may also include an electronic health record (EHR) interface component (e.g., including one or more agents) configured to enable physicians and optionally other users to view, edit, and / or search EHRs. The EHR interface component can be communicatively coupled with one or more services and / or databases to retrieve updated information and reports (e.g., via push notifications). The EHR interface component can be used in conjunction with an AI-enabled clinical assistant to search, edit, summarize, and / or correct EHRs. The platform may also include research analytics components (e.g., including one or more agents) that provide anonymized patient / clinical data and insights.

[0031] Here, embodiments are referenced, the embodiments illustrated in the accompanying drawings. Numerous specific details are provided in the following description to allow for a full understanding of the various embodiments described. However, it will be apparent to those skilled in the art that the various embodiments described can be practiced without these specific details. In other cases, well-known methods, procedures, components, circuits, and networks are not described in detail so as not to unnecessarily obscure the aspects of the embodiments.

[0032] Figure 1 is a block diagram illustrating platform 100 according to several embodiments. In some embodiments, platform 100 is an AI platform (e.g., the AI ​​platform described above). Platform 100 includes one or more client devices 102 that are communicably coupled to a server system 106 via one or more networks 104. According to some embodiments, platform 100 further includes, or communicates with, one or more external services 110 and one or more external databases 108. In some embodiments, one or more networks 104 include public communication networks, private communication networks, or a combination of both public and private communication networks. For example, one or more networks 104 can be any network (or combination of networks) such as the Internet, other wide area networks (WANs), local area networks (LANs), virtual private networks (VPNs), metropolitan area networks (MANs), peer-to-peer networks, and / or ad-hoc connections. In some embodiments, platform 100 includes only a subset of the components shown in Figure 1. For example, platform 100 may include only one of either client device 102 or server system 106.

[0033] In some embodiments, the client device 102 is associated with one or more users. In some embodiments, each user is authenticated separately (e.g., assigned a separate / unique authentication token). In some embodiments, the client device 102 is a personal computer, a mobile electronic device, a wearable computing device, a laptop computer, a tablet computer, a mobile phone, a feature phone, a smartphone, a speaker, a television (TV), and / or any other electronic device capable of interacting with the user (e.g., an electronic device having an I / O interface). The client device 102 may be wirelessly and / or wired (e.g., directly via an interface such as an HDMI interface) coupled to other components of the platform 100.

[0034] In some embodiments, the client device 102 transmits and receives information such as queries and results through the network 104. For example, the client device 102 may transmit queries or requests to the server system 106, external services 110, and / or external database 108 through the network 104. In another embodiment, the client device 102 may receive results and other responses from the server system 106, external services 110, and / or external database 108 through the network 104. In some embodiments, two or more client devices 102 communicate with each other (e.g., retransmitting and responding to queries and requests). Two or more client devices 102 may communicate via the network 104 or directly (e.g., via a wired connection or via a peer-to-peer wireless connection).

[0035] In some embodiments, the server system 106 includes a plurality of electronic devices coupled to communicate with one another. In some embodiments, the plurality of electronic devices are juxtaposed (e.g., within a data center), while in other embodiments, the plurality of electronic devices are geographically separated from one another. In some embodiments, the server system 106 stores and provides clinical data and / or patient data. In some embodiments, the server system 106 trains, exposes, and / or utilityes one or more agents and / or language models. In some embodiments, the server system 106 uses one or more agents and / or language models to receive and respond to queries and requests from client devices 102. In some embodiments, the server system 106 includes a plurality of nodes and / or clusters configured to handle different types of tasks and / or requests and queries from different geographical locations.

[0036] In some embodiments, the client device 102 and / or the server system 106 communicate with the external service 110 and / or the external database 108 via an application programming interface (API). In some embodiments, the external service 110 and / or the external database 108 are maintained / operated by a third party relative to the platform 100. In some embodiments, the external service 110 includes agents, location services, time services, web-enabled services, and / or services that access information stored outside the platform 100. In some embodiments, the external database 108 includes one or more medical databases, clinical databases, subject databases, research databases, and / or general knowledge databases. In some embodiments, the external database 108 includes one or more of the databases shown in Figure 4.

[0037] Figure 2A is a block diagram illustrating a client device 102 according to several embodiments. The client device 102 includes one or more central processing units (CPUs) 202, a user interface 204, one or more network (or other communication) interfaces 214, memory 218, and one or more communication buses 217 for interconnecting these components. In some embodiments, the client device 102 includes a processor or other control circuit (for example, in addition to or instead of the CPU 202). For example, the client device 102 may include one or more GPUs and / or DPUs (for example, to perform machine learning tasks). The communication bus 217 optionally includes circuitry (sometimes called a chipset) that interconnects and controls communication between system components. Optionally, the client device 102 includes a location detection component such as a global navigation satellite system (GNSS) (e.g., GPS (Global Positioning System), GLONASS, Galileo, BeiDou) or other geolocation receiver, and / or location detection software for determining the location of the client device 102.

[0038] In some embodiments, the client device 102 includes, but is not limited to, one or more sensors, including an accelerometer, gyroscope, compass, magnetometer, optical sensor, short-range wireless communication transceiver, barometer, humidity sensor, temperature sensor, proximity sensor, rangefinder, and / or other sensors / devices for sensing and measuring various environmental conditions.

[0039] The user interface 204 includes an output device 206 and an input device 212. In some embodiments, the input device 212 includes a keyboard, mouse, trackpad, and / or touchscreen. In some embodiments, the user interface 204 includes a display device including a touch-sensitive surface, in which case the display device is a touch-sensitive display. In client devices with a touch-sensitive display, a physical keyboard is optional (for example, a soft keyboard may be displayed when keyboard input is required). In some embodiments, the output device 206 includes a speaker and / or a connection port for connecting to a speaker, earphone, headphones, or other external listening device. In some embodiments, the input device 212 includes a microphone and / or a speech recognition device for capturing voice (e.g., speech from the user).

[0040] In some embodiments, one or more network interfaces 214 include wireless and / or wired interfaces for receiving and / or transmitting data to and from other client devices 102, server systems 106, and / or other devices or systems. Data communication may be performed using any of the following custom or standard wireless protocols (e.g., NFC, RFID, IEEE 802.15.4, Wi-Fi, ZigBee, 6LoWPAN, Thread, Z-Wave, Bluetooth, ISA100.11a, WirelessHART, MiWi, etc.). Furthermore, data communication may be performed using any of the following custom or standard wired protocols (e.g., USB, Firewire, Ethernet, etc.). For example, one or more network interfaces 214 may include a wireless interface 216 for enabling wireless data communication with other client devices 102, systems, and / or other wireless (e.g., Bluetooth-enabled) devices. Furthermore, in some embodiments, the wireless interface 216 (or different communication interfaces of one or more network interfaces 214) enables data communication with other WLAN-enabled devices and / or server systems 106 (via one or more networks 104).

[0041] Memory 218 includes high-speed random-access memory such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices, and may include non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory 218 optionally includes one or more storage devices located remotely from the CPU 202. Memory 218, or alternatively, the non-volatile memory solid storage device within Memory 218, includes a non-temporary computer-readable storage medium. In some embodiments, Memory 218 or the non-temporary computer-readable storage medium of Memory 218 may contain the following programs, modules, and data structures, or Operating System 220, including procedures for handling various basic system services and performing hardware-dependent tasks. A network communication module 222 for connecting the client device 102 to one or more other computing devices connected to one or more networks 104 via one or more network interfaces 214 (wired or wireless), User interface module 224 receives commands and / or inputs from the user (e.g., from input device 212) via user interface 204 and provides outputs via user interface 204 (e.g., output device 206). • Stores a subset or superset of the agent library 226, including multiple agent modules 6102 (e.g., agent building blocks and / or generated agents). In some embodiments, the agent library 226 operates in conjunction with an assistant module in a server system 106 (e.g., an assistant module 316). In some embodiments, the agent library 226 contains the following modules (or sets of instructions), or o Includes one or more subsets or supersets of Model 228 that engage with the user and / or perform specific tasks in the further progression of user requests or queries. In some embodiments, Model 228 includes one or more large language models such as GPT-3, GPT-4, BioGPT, and PaLM-2, Model 228 comprises an interface module 230 that enables communication with other applications, components, and devices (e.g., via APIs or structured queries). In some embodiments, the interface module 230 may include, as discussed herein, agents (e.g., task-specific orchestrations), task-specific orchestration creator applications, one or more orchestration libraries for selecting orchestrations to perform a task (e.g., an orchestration marketplace), • A web browser application 234 for accessing, browsing, and interacting with websites. Other applications such as word processing, calendar processing, mapping, weather, stockpiling, time management, virtual digital assistant, presentations, numerical calculations (spreadsheets), drawing, instant messaging, email, telephone, video conferencing, photo management, video management, digital music player, digital video player, 2D games, 3D (e.g., virtual reality) games, e-book reader, and / or applications for training support, 236, and • One or more data modules 240 for storing and / or accessing data such as medical data, clinical data, patient data, and user data. In some embodiments, one or more data modules 240 are: *One or more medical databases 242 for storing medical data (for example, regarding therapies, drugs, treatments, patients, cohorts and / or diseases), *Includes one or more user databases 244 for storing user data such as user preferences, user settings, and other metadata.

[0042] In some embodiments, one or more agent modules 6102 are configured to engage with the user in an integrated conversational manner using natural language dialogs and / or to call external services to retrieve information or perform various actions as needed.

[0043] Referring to Figure 2B, in some embodiments, the platform 100 provides a system for managing and deploying a plurality of agent modules 6102, including managing them through various blocks (e.g., agent builder blocks) implemented in the form of one or more nodes 6108. In some embodiments, each agent module 6102 is associated with a defined information domain and / or task-specific capabilities, which allows for the incorporation of a particular agent module 6102 based on information determined from user-provided prompts and / or based on the user's selection of an agent module 6102. In some embodiments, agent module 6102-1 is configured for a first specific task of generating a summary report of a patient's medical records; a second agent module 6102-2 is configured for a second specific task of guiding a patient through a care plan; a third agent module 6102-3 is configured for a third specific task of creating patient care guidelines based on a patient's health profile; a fourth agent module 6102-4 is configured for a fourth specific task of identifying patients requiring follow-up in a hospital; a fifth agent module 6102-5 is configured for a fifth specific task of identifying changes in standard treatment for disease settings; a sixth agent module 6102-6 is configured for a sixth specific task of evaluating unstructured data associated with a patient to identify a cohort of similar patients; and a seventh agent module 6102-7 is configured for a seventh specific task of phenotypic analysis of subjects, or a combination thereof. However, the disclosure is not limited thereto. In some embodiments, the agent library 226 is located in one or more client devices 102 and / or server systems 106.

[0044] In some embodiments, each agent module 6102 provides extensive content and functionality that end users can engage in and / or configure for such engagements, ranging from simple static responses to an advanced knowledge system that facilitates automated conversations and data analysis leading to integrated transactions with solutions and external systems, through one or more nodes 6108 associated with the agent module 6102. In summary, one or more nodes 6108 form part or all of the node architecture 6106 associated with the agent module 6102, which defines the rules for traversing between nodes. In some embodiments, each agent 6102 has a corresponding node architecture 6106, which provides a one-to-one relationship between the agent module 6102 and the node architecture 6106. In some embodiments, each agent module 6102 supports the generation of additional agent modules 6102 that engage with one or more models 228 and / or nodes 6108 of the node architecture 6106 of each agent module 6102 or different agent modules 6102. In some embodiments, each agent module 6102 supports the selection of agent modules 6102 within a library of agent modules, defining flexible integration of these agent modules 6102 into various system architectures. However, this disclosure is not limited thereto.

[0045] In some embodiments, each agent module 6102 provides a defined scope for involvement in the workflow. Thus, in some embodiments, each agent module 6102 is configured to assist end users in asking questions and / or resolving problems, or in fulfilling specific requests for information retrieval, such as through a conversational communication framework. Some embodiments provide the ability to create, manage, and operate agent modules 6102, for example, by using a user interface-based agent module builder or similar, making it possible to create, edit, or delete agent modules 6102 via a user interface.

[0046] Some embodiments provide a user interface-based agent module designer to assist in the creation and editing of agent modules 6102 and / or workflows associated with various agent modules 6102 (workflows may also be referred to as assemblies or orchestrations). In some embodiments, this workflow appears as a node architecture containing multiple interconnected nodes. In some embodiments, the agent module designer includes the ability to define the name of an agent module 6102, create an agent module 6102, edit an agent module 6102, delete individual nodes 2210 associated with an agent module 6102, expand and / or collapse branches of a node 6108, view and edit the conditional logic of a node 6108, and verify node crossing (for example, when one or more nodes 6108 connect to different nodes 6108).

[0047] In some embodiments, node 6108 of agent module 6102 reflects one or more decision points within agent module 6102, such as one or more predetermined decision points. In some embodiments, agent module 6102 evaluates data such as graphical data from client device 102 (e.g., prompts provided by the user on client device 102, outputs from different agent modules 6102, etc.) by parsing and / or evaluating incoming data such as recognized keywords, phrases, ground truth labels. For example, based on the detection of recognized features, agent module 6102 may process information associated with the data received from client device 102 in a specific direction within a plurality of interconnected nodes 6108, such as from node 6108-1 associated with agent module 6102-1 to node 6108-2 associated with agent module 6102-1, and / or from node 6108-1 associated with agent module 6102-1 to node 6108-2 associated with agent module 6102-1. Therefore, in some embodiments, the use of one or more nodes 6108 associated with each agent module 6102 within a plurality of interconnected nodes 6108 is analogous to sequentially traversing a decision tree having different nodes 6108 associated with different agent modules 6102, each different agent module 6102 evaluating information based on relevant conditional logic and advancing the information within the plurality of interconnected nodes 6108. However, the disclosure is not limited thereto. In some embodiments, each node within the plurality of interconnected nodes 6108 includes conditional logic that, for example, can perform data evaluation, data retrieval, data generation, or a combination thereof, based on an evaluation of the information input to each node 6108.In some embodiments, each node within a plurality of interconnected nodes 6108 takes some action, such as generating a message and / or sending information to another node 6108 in the same agent module 6102 as the respective node, or to a different node 6108 in a different agent module 6102, or similar.

[0048] In some embodiments, the corresponding node architecture 6106 associated with one or more agent modules 6102 defines, at least partially, conditional logic 6112 for performing a particular clinical task. For example, each node 6108 may contain a corresponding logic 6112, which defines a workflow for handling one or more tasks assigned to each node 6108. In some embodiments, the conditional logic of the node architecture 6106 is executed in a first order of a first set of interconnected nodes 6108 from a set of interconnected nodes 6108, based on the corresponding logic 6112 of each node 6108 in the set of interconnected nodes 6108. Thus, the logic 6112, when collectively combined through the interconnected nodes of the node architecture 6106, allows for a detailed configuration of each node 6108 that defines the conditional logic of the node architecture. For example, referring briefly to Figure 20, the logic 6112 may contain one or more AND, OR, XOR, and / or NOT operations within the logic 6112. As an example, the corresponding logic 6112 requires the presence of the first condition, but does not require the second or third condition.

[0049] In some embodiments, the node array includes one or more data source nodes 6108 associated with a specific task of retrieving data elements from a remote data source (e.g., an external database 108). In some embodiments, the corresponding logic 6112 enables connection to the corresponding database, for example, by using an access token associated with the corresponding agent module 6102, communicating at least a portion of the retrieved data to one or more nodes 6108, and / or executing one or more queries to identify / analyze such data. In some embodiments, each node architecture 6106 includes at least one input node that forms the initial terminal node in the order of the nodes 6108. In some embodiments, the node architecture includes multiple paths traversing from input to output nodes, such as paths in a branching tree. In some embodiments, each respective node 6108 represents a computational process, such as a function, input, output, or similar, which is realized when data is applied to the node 6108. Furthermore, since each node is interconnected by an edge to at least one other node 6108, the output from one node 6108 can be supplied as an input to a different node 6108 to form a chain, sequence, or node within the node architecture 6106.

[0050] In some embodiments, memory 218 includes one or more modules not shown in Figures 2A and 2B. For example, memory 218 may include one or more agent modules (e.g., retriever components) that are separate from agent library 226. In some embodiments, client device 102 includes one or more standalone agents (e.g., running and operating on client device 102) and / or one or more dependent agents (e.g., operating in conjunction with components on a remote device such as server system 106). In some embodiments, one or more agents are generated / trained on server system 106 and deployed on client device 102.

[0051] Figures 2A and 2B illustrate client device 102 according to several embodiments, but are intended more to functionally describe the various features that may be present in the client device than to be schematic diagrams of the embodiments described herein. In practice, as will be recognized by those skilled in the art, items shown separately may be combined, and some items may be separated.

[0052] Figure 3 is a block diagram illustrating an exemplary server system 106 according to several embodiments. According to some embodiments, the server system 106 includes one or more CPUs 302, one or more user interfaces 304, one or more network interfaces 306, memory 310, and one or more communication buses 308 for interconnecting these components. In some embodiments, the server system 106 includes other types of control circuits and / or processors (for example, in addition to or instead of the CPUs 302). For example, the server system 106 may include one or more GPUs or DPUs for machine learning tasks.

[0053] Memory 310 includes high-speed random-access memory such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices, and may include non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory 310 optionally includes one or more storage devices located remotely from one or more CPUs 302. Memory 310, or alternatively, the non-volatile solid-state memory devices within Memory 310, includes a non-temporary computer-readable storage medium. In some embodiments, Memory 310 or the non-temporary computer-readable storage medium of Memory 310 contains the following programs, modules and data structures, or • Operating System 312, which includes procedures for handling various basic system services and for performing hardware-dependent tasks. A network communication module 314 is used to connect the server system 106 to other computing devices connected to one or more networks 104 via one or more network interfaces 306 (wired or wireless). • It stores a subset or superset of Assistant Module 316, which interacts with the user (e.g., a remote user) in an integrated conversational manner using natural language dialogs, and calls external services at appropriate times to obtain information or perform various actions. In some embodiments, Assistant Module 316 operates in conjunction with the agent library on the client device 102 (e.g., Agent Library 226). In some embodiments, Assistant Module 316 stores the following modules (or sets of instructions), or *One or more agents 318 (for example, any of the agents described herein, such as retriever agents and target population membership agents) configured to perform a specific task or to perform a task within a specific domain. *One or more interface modules 320 that enable agent 318 to communicate with other agents, applications, components, and devices (for example, via APIs or structured queries), and • Includes a subset or superset of one or more server data modules 330 for storing and / or accessing data (e.g., clinical data and user data). In some embodiments, one or more server data modules 330 are: *One or more medical databases 332 for storing medical data (for example, regarding therapies, drugs, treatments, patients, cohorts and / or diseases), *Includes one or more agent databases 334 for storing agent data such as settings, training, commands, and other metadata.

[0054] In some embodiments, the server system 106 includes a web or hypertext transfer protocol (HTTP) server, a file transfer protocol (FTP) server, and web pages and applications implemented using Common Gateway Interface (CGI) scripts, PHP hypertext preprocessor (PHP), Active Server Pages (ASP), hypertext markup language (HTML), extended markup language (XML), Java, JavaScript, asynchronous JavaScript and XML (AJAX), XHP, Javelin, Wireless Universal Resource File (WURFL), and similar.

[0055] In some embodiments, memory 310 includes one or more modules not shown in Figure 3. For example, memory 310 may include one or more agent modules (e.g., retriever components) separate from assistant module 316. In some embodiments, server system 106 includes one or more standalone agents (e.g., operating in conjunction with components on remote devices such as client device 102) and / or one or more dependent agents (e.g., running and operating in server system 106). In some embodiments, memory 310 includes an agent library (e.g., agent library 226).

[0056] Figure 3 illustrates server systems 106 according to several embodiments, but is intended more to functionally describe the various features that may be present in the server system than to serve as a schematic diagram of the structures of the embodiments described herein. In practice, as will be recognized by those skilled in the art, items shown separately may be combined, and some items may be separated. For example, some items shown separately in Figure 3 may be implemented on a single server, and a single item may be implemented by one or more servers. In some embodiments, the clinical database and / or agent database 334 is stored on a device accessed by the server system 106 (e.g., an external database 108). The actual number of servers used to implement the server system 106, and how features are allocated among them, will vary from implementation to implementation and will optionally depend in part on the amount of data traffic the server system will handle during peak and average usage periods.

[0057] Each of the modules identified above and stored in memories 218 and 310 corresponds to a set of instructions for performing the functions described herein. The data, modules, or programs (e.g., sets of instructions) identified above do not need to be implemented as separate software programs, procedures, or modules; therefore, various subsets of these modules can be combined or rearranged in different ways in various implementations. In some embodiments, memories 218 and 310 optionally store subsets or supersets of each of the modules and data structures identified above. Furthermore, memories 218 and 310 optionally store additional modules and data structures not described above.

[0058] Figure 4 is a block diagram illustrating a system database 400 according to several embodiments. In some embodiments, at least a portion of the system database 400 is optionally stored in a client device 102 (e.g., as medical database 242), a server system 106 (e.g., as medical database 332), and / or an external database 108, which is advantageously enabled at and / or near the client device 102, such as via a communication network. However, the disclosure is not limited thereto.

[0059] In some embodiments, the system database 400 includes molecular report genome datasets and clinical datasets 402 and / or non-patient specific knowledge databases (KDBs) 404, as in some embodiments such as the knowledge database 404 in Figure 14. In some embodiments, the datasets 402 include, among other data, genome, transcriptome, epigenome, microbiome, clinical, memorized modified proteome, omics, organoids, imaging, and cohort and trend datasets. For example, cohort selection, search, analysis, and study datasets may include data on tumor of unknown origin (TUO) predictors, metastasis predictors, and survival analysis. As an example, imaging datasets may include radiographic imaging data, immunohistochemical imaging data, positron emission tomography (PET) data, and / or single-photon emission computed tomography (SPECT) imaging data. Imaging datasets may include data on nodule identifiers, tracking, and / or longitudinal analysis.In some embodiments, KDB404 may include, as illustrated, provider panels 406 (e.g., information related to gene panels supported by service providers operating the system), drug classes (e.g., drug class-specific information (e.g., whether a particular class of drugs acts on pancreatic cancer, or drugs thought to be included in a particular drug class, etc.)), specific genes 408, immunological outcomes (e.g., information related to treatment based on the outcomes of a particular immunobiomarker), specific drugs, drug class-mutalytic interactions, mutation-drug interactions, delivery methods (e.g., questions about the process carried out by the service provider), clinical trials, general immunology, clinical symptoms such as clinical diseases, terminology sheets (e.g., definitions of industry-specific terms), provider coverage (e.g., information about provider tests and results), provider samples (e.g., information about the types of samples that may be processed by the provider), knowledge (e.g., scripted questions and answers to various frequently asked questions not included in other subdatabases), radiation (e.g., information related to suitable radiation therapy depending on a particular cancer condition), clinical Guidelines (e.g., national guidelines on cancer classification, accepted treatments, etc.) and clinical trial questions and answers (e.g., including separate sub-databases for specific information types, including information on the location and administrator of the clinical trial). Organizing KDB404 into sub-databases makes it easier to manage those databases as the information within them evolves over time, and may allow for the addition of new sub-databases related to other defined information types. In some embodiments, the clinical datasets 402 and / or KDB404 are arranged in a different manner than shown in Figure 4 (e.g., in different sub-databases and / or different organizational schemes). Advantageously, by utilizing multiple datasets associated with different domains of subject matter and / or applying a classification system to the datasets, the knowledge database provides a storage system for data, such as medical records and clinical documents, which one or more agent modules 6102 can ingest based on task-specific requirements associated with each domain or classification.Furthermore, in some embodiments, the knowledge database 404 is enabled to store such data with anonymization controls in order to enable training and / or analysis of the stored data without the risk of disclosing confidential and / or privileged information.

[0060] Figures 5A and 5B illustrate exemplary user interfaces for searching databases (e.g., data module 240 in Figure 2A, data module 330 in Figure 3, and / or database 400 in Figure 4) according to several embodiments. In some embodiments, the user interface 500 shown in Figure 5A includes a first set of user interface elements to enable the user to search multiple types of data and apply specific filters to identify result sets, such as searching for a first classification of data from data modules 240 associated with one or more external databases 108 and / or agent modules 6102. The user interface 500 includes affordances 502 (e.g., advanced filters) for manually applying filters. In some embodiments, the user interface 500 also includes one or more user interface elements, such as user interface element 504, for interacting with the agent to search (e.g., to identify cohorts with text). In some embodiments, a prompt user interface is presented within the user interface 500, which may be a user interface for interacting with a general language model and / or one or more task-specific orchestrations. For example, a user might enter a text prompt such as "Patients over 45 years of age diagnosed with colorectal cancer and treated with atezolizumab, durvalumab, or interferon," and the agent module 6102 will provide a response according to the corresponding node architecture 6106 associated with the agent module 6102 (which may be visually represented in other user interfaces of the application, e.g., a workflow representation). In some embodiments, the agent module 6102 generates one or more nodes 6108 of the node architecture in response to the prompt, such as one or more filters based on the text prompt.In a non-limiting embodiment, if the first prompt includes multiple graphical data of chest X-rays from a single human, the agent module 6102 may provide the multiple graphical data to a first filter of node 6110-1 associated with a screening chest X-ray modality for a first biomarker. Furthermore, in some embodiments, the agent module 6102 compares one or more features associated with the multiple graphical data to generate a second filter for identifying or generating a corresponding target identifier in a library of target identifiers uniquely associated with the target. However, the disclosure is not limited thereto. In some embodiments, the user interface 500 is updated to indicate the filter applied by the agent in response to a text prompt.

[0061] In some embodiments, the user interface 500 includes an agent window or section where the user can ask the agent questions regarding filters, data, and / or modalities. The user interface 500 shown in Figure 5B includes an agent section 520 where the user can prompt an agent module 6102 and view information returned and / or generated by agent module 6102 or another (connected) agent module 6102 other than the module that received the prompt. In this way, the time for data analysis can be reduced (e.g., via a generative AI-driven human-in-the-roof assistant), thereby reducing the computational load on the client device 102 and / or the server system 106. The user interfaces described herein enable the user to make feasibility assistance requests, including (i) requests to create feasibility from scratch, (ii) editing, adjusting, and / or validating saved query logic, and / or (iii) answering questions from the user during a cohort splitting workflow. In some embodiments, the solution architect can answer questions or generate queries based on user input.

[0062] For example, the agent module 6102 may be configured to solve problems when using applications, unknown / niche datasets, and / or user interfaces, reducing the number of user inputs required and helping to automatically identify response data from very large collections of data (e.g., reducing the number of follow-up and query modifications required to identify response data). For example, the agent module 6102 can simplify documentation navigation and discovery by increasing user-driven cohort splitting via a human-in-the-loop feasibility assistant and providing an LLM-driven chatbot (e.g., one incorporating knowledge center content). In some embodiments, some or all of the knowledge database 404 is searched, indexed, and / or analyzed to associate user-generated free text with corresponding outputs from task-specific agent modules 6102. In some embodiments, data from the knowledge database 404 is used to train machine learning models using agent modules 6102.

[0063] In some embodiments, the system (e.g., platform 100 or its components) determines (e.g., using a different machine learning model than the model receiving the prompt) that a prompt provided by the user contains a natural language description of a cohort (e.g., a patient cohort containing a set of one or more patients) that the user wants to build. In some embodiments, the cohort is explicitly identified through the natural language description. In some embodiments, the cohort is derived from the natural language description by parsing the prompt, such as by applying the prompt to node 6108-1 of node architecture 6106, which is configured to preprocess (e.g., parse) one or more parts of the description as a request to apply filtering behavior to the cohort. In some embodiments, node 6108-1 identifies the intent and / or one or more commands from the request. In some embodiments, the request triggers a modification of affordance 502 on user interface 500 (e.g., to display a filter funnel on user interface 500). In this way, the application user can interact with the data source more effectively and efficiently by using natural language prompts to trigger actions that require multiple user inputs to multiple different user interface elements, and / or by navigating different user interfaces (e.g., a first set of user interfaces for determining filtering actions based on natural language prompts, and a second set of user interfaces for implementing filtering actions (e.g., within different web or desktop applications)). In some such embodiments, this analysis advantageously allows the user to modularly browse, understand, and modify any of the filters, thereby enabling efficient identification of patient cohorts.Furthermore, because the agent module 6102 is associated with the data module 240, and is updated with the latest information through a communication network or local updates such as the latest medical records (e.g., live collection), the agent module 6102 identifies one or more cohorts or subjects based on real-time information associated with the subjects by filtering the data stored in these databases (e.g., unseen data / protected data). In some embodiments, the agent module 6102 provides processed output to communicate the inference to the user through a response displayed on the client device 102, which may be presented within the user interface elements of the user interface 500.

[0064] In some embodiments, the agent module is configured to construct a funnel call (e.g., JSON) and apply one or more filters to the call, such as a first node associated with a first filter and / or a second node associated with a second and a third filter. For example, a single node 6108 may be generated, which has a parameter 6110 for any possible filter and a corresponding logic 6112 for applying data to the possible filters and / or combinations of filters associated with node 6108. In this embodiment, parameter 6110 is shown as optional, and the function supplies it to model 228. In another embodiment, for any filter, node 6108 is generated if parameter 6110 matches the filter's input, and the response (e.g., JSON) is appended to a larger funnel object call.

[0065] As a simple example, if the user enters the prompt "Who is Leo DiCaprio's girlfriend? What is her current age raised to the power of 0.43?", the agent module may provide the following processing result. > Entering new PlanAndExecute chain... steps=[Step(value='Search for Leo DiCaprio's girlfriend on the internet.”),Step(value='Find her current age.'),Step(value='Raise her current age to the 0.43 power using a calculator or programming language.'),Step(value='Output the result.'),Step(value=”Given the above steps taken,respond to the user's original question.”)] > Entering new AgentExecutor chain... Action: ... { “action”:“Search”, “action_input”:“Who is Leo DiCaprio's girlfriend?” } ... Observation: DiCaprio broke up with his girlfriend Camila Morrone, 25, in the summer of 2022, after dating for four years. He's since been linked to another famous supermodel, Gigi Hadid. The power couple were first supposedly dating in September after being spotted getting cozy during a party at New York Fashion Week. Thought: Based on the previous observation, I can provide the answer to the current objective. Action: ... { “action”:“Final Answer”, “action_input”:“Leo DiCaprio is currently linked to Gigi Hadid.” } ... Finished chain. **** Step: Search for Leo DiCaprio's girlfriend on the internet. Response:Leo DiCaprio is currently linked to Gigi Hadid. > Entering new AgentExecutor chain... Action: ...

[0066] Example 1 - Exemplary agent processing output Embodiment 1 shows a portion of the processing output (e.g., the first step). The processing output continues until agent module 6102 obtains a response to each of the questions in the original query. In some embodiments, each step is performed by a different node 6108 and / or agent module 6102. In this way, the user can learn to interact with agent module 6102, understand the rationale behind the responses, and understand where the data is obtained by agent module 6102 that triggers the responses. This allows the user to modify its prompts and / or instruct the agent to modify its response process to obtain the desired results, or modify agent module 6102 itself, such as the order of nodes in node architecture 6106, one or more parameters 6110 of each node in node architecture 6106, one or more logic of node 6108, or similar.

[0067] Figure 6 illustrates exemplary system architectures for deploying agents (e.g., agent module 6102) according to several embodiments. The architecture 600 shown in Figure 6 includes an agent builder component within the control plane of the client device 102. The control plane acts as a data supervisor, coordinating communication between different components and collecting data from the data plane (e.g., the work environment presented on the display of the client device 102). In some embodiments, the control plane is located above the data plane (e.g., the work environment) and enforces rules over the data plane, thereby partitioning the data plane and preventing unauthorized or unauthenticated control of the data plane from insecure client devices that are not associated with a portion of the data plane. However, this disclosure is not limited thereto. In some embodiments, the agent builder hosts a user interface for configuring the agent module, such as by configuring a corresponding node architecture 6106 associated with the agent module 6102. In some embodiments, the agent builder component is communicatively coupled to an agent library 226 in the control plane that stores multiple agent modules 6102, such as agent module 6102, and to an agent host in the working environment (e.g., via a configuration pubsub component). The agent module 6102 in the working environment may be communicatively coupled to the agent library 226 in the working environment, a document index (e.g., one or more data sources such as knowledge database 404 and / or external database 108), and a large language model (e.g., model 228). In some embodiments, the agent library 226 includes a user interface and API for interacting with deployed agents. In some embodiments, the large language model in Figure 6 represents a node 6108 (e.g., a task-specific agent) as described herein.

[0068] In some embodiments, the agent builder includes a front-end and a back-end. In some embodiments, the agent builder front-end includes an access component (e.g., a management console such as user interface 1830 in Figure 18D, which may be a home user interface presented to the user when access credentials to the application are provided), an agent list (e.g., an agent library such as user interface 1830 in Figure 18E, which may include multiple task-specific orchestrations accessed by the user based on the access credentials provided to the application), agent builder components (e.g., a first representation of node architecture 6106 (e.g., a form builder representation) and a second representation of node architecture 6106 (e.g., a workflow representation), either or both of user interfaces 1812 and 1822, respectively), and / or a data source management component. In some embodiments, the agent builder back-end includes a database layer, API services, and / or a configuration publisher component. In some embodiments, the agent builder front-end and back-end run on separate electronic devices.

[0069] In some embodiments, the agent host includes a frontend and a backend. In some embodiments, the agent host frontend includes an access component, an agent list, an interaction console, and / or a document console. In some embodiments, the agent host backend includes a websocket for interactive users, a database layer, API access to deployed agents, tools and / or custom chain implementations, a document loader, and / or configuration subscription components. In some embodiments, the agent host frontend and backend run on separate electronic devices.

[0070] In some embodiments, the agent builder component is configured to generate, deploy, and / or update one or more agent modules 6102 and / or corresponding node architectures 6106 to form one or more working environments (e.g., one or more workload planes). In some embodiments, each agent module 6102 is associated with an agent type. In some embodiments, the agent type includes a type of model 228 and / or conditional logic 6112, such as an implementation configuration. For example, agent module 6102 includes a language model associated with a first node 6108 and corresponding type-specific logic, thereby further associating agent module 6102 with specific domains, such as a first configuration implementation for applying a prompt to model 228 when the prompt is associated with a first modality, and a second configuration implementation when the prompt is associated with a second modality different from the first modality. In some embodiments, the logic 6112 is specified in the corresponding agent module 6102 configuration file, which is advantageous as it is possible to configure the logic after applying various prompts to the agent module 6102 and / or after configuring the logic 6112 using multiple client devices (e.g., end users). However, the disclosure is not limited thereto.

[0071] In some embodiments, agent module types include transformation agent modules (e.g., performing functions such as data transformation, regular expressions, and string templates), authentication agent modules, language model agent modules (e.g., applying inputs to large language models), data collection agent modules (e.g., RAG modules), super agent modules (e.g., configured to recognize other agent types and their capabilities and instantiate and / or delegate to appropriate agent modules), sequential agent modules (e.g., sequentially linking multiple models and / or tools), tool usage agent modules, coding agent modules (e.g., configured to generate code in a particular programming language), and classification agent modules (e.g., configured to determine intent, domain, or other classifications for user input). In some embodiments, language model agent modules provide / store contextual information such as conversation history, user preferences, subject details, and similar. In some embodiments, data collection agent modules are connectable to external data sources (e.g., external service 110 and / or external database 108). In some embodiments, sequential agent modules include recursive agent modules (e.g., repeating and / or refining outputs until certain criteria are met). In some embodiments, the super agent module is configured to compare available agent module types and recommend a specific agent module type for a particular situation / purpose. In some embodiments, the coding agent module is configured to generate code for a new agent module based on user input (e.g., natural language input). In some embodiments, the classification agent module is a component of the routing agent module. For example, the classification agent module determines the intent / domain of the input, and the routing agent module routes the input to downstream components according to the determined intent / domain.In some embodiments, a sequential agent module is a component of a routing agent module. For example, a routing agent module coordinates the operation of multiple components and / or modules (e.g., data transmission and timing). In some embodiments, each agent module is generated / provided with guardrails (e.g., to enhance privacy, security, data type, etc.). In some embodiments, an agent module is configured to recognize whether data is protected health information (PHI) and take appropriate action. For example, an agent module may disable information sharing options when providing PHI.

[0072] In some embodiments, different agent module types are associated with different domains within multiple domains (e.g., different subjects, data types, and / or data classes) (e.g., trained, instructed, and / or combined). For example, in some embodiments, the multiple domains form an input space that defines a vast amount of data associated with various subjects. In some embodiments, the input space defines an N-dimensional space of data obtained from multiple data sources, where N is a positive integer such as 2, 3, 4, or 10. In some embodiments, each domain within the multiple domains defines a classification or subset of data, such as one or more specific datasets in the system database 400 in Figure 4. However, the disclosure is not limited thereto.

[0073] As a non-limiting embodiment, consider a first input space associated with multiple medical records, where each medical record contains multiple text data and multiple graphical data associated with the corresponding patient. Thus, multiple domains collectively defined by the information obtained from the multiple medical records allow for the classification of information and the training of agent modules 6102 of the classified domains, such as a first domain associated with the statin drug class and a second domain associated with the glucagon-like peptide (GPL) agonist drug class in Figure 4. In a non-limiting embodiment, agent module 6102-1 is associated with a first domain for generating a summary report of a patient's medical records; agent module 6102-2 is associated with a second domain for guiding subjects such as patients or healthcare professionals associated with patients through care planning; agent module 6102-3 is associated with a third domain for creating patient care guidelines based on a patient's health profile; agent module 6102-4 is associated with a fourth domain for identifying patients requiring hospital follow-up; agent module 6102-5 is associated with a fifth domain for identifying changes in standard treatment for disease settings; and / or agent module 6102-6 is associated with a sixth domain for evaluating unstructured data associated with patients to identify cohorts of similar patients.

[0074] An exemplary agent type is a database integration agent module (e.g., agent module 6102) associated with one or more data source nodes 6108. The exemplary database integration agent may also be an adverse effects agent configured to access the FDA label database and interpret adverse effects information from the database. The configuration of the tool-using agent may include a custom prompt for Model 228 and one or more data sources that the agent database integration module can access and / or use.

[0075] Another exemplary agent type is a custom chain agent module (e.g., a super agent module) that takes an input prompt, analyzes the prompt (e.g., parses the prompt into one or more commands and / or tokens), and sends information from the parsed prompt (e.g., commands and / or tokens) to Model 228 or other components, such as node 6108 of a custom chain agent module or a different node 6108 of a different agent module 6102. For example, agent module 6102 may retrieve data from different databases (e.g., external database 108, knowledge database 404, etc.), and the data may be retrieved in a variety of different formats and / or structures, such as unstructured text, structured text, tables, charts, graphical data, and / or similar. In some embodiments, agent module 6102 reformats and / or reconstructs the data retrieved from the database for application to Model 228 and / or different agent modules 6102. In some embodiments, agent module 6102 evaluates and / or obtains an optimal set of parameters for inputting data into model 228 and / or different agent modules 6102, and / or translates the data retrieved from the database based on the optimal set of parameters. In some embodiments, the retrieved data is reconstructed into a homogeneous dataset (for example, different hospitals may use different codes for the same procedure, which are homogenized into uniform coding by agent module 6102, for example). The configuration of a custom chain agent module 6102 may include a sequence of nodes 6108 associated with the custom chain agent module 6102, and / or a sequence of other nodes 6108 associated with other agent modules 6102 used by the custom chain agent module 6102, and / or a definition of the corresponding chain object.In this way, the agent module 6102 can be considered a configuration of a specific agent type for a specific task through a plurality of interconnected nodes 6108 that form the node architecture 6106 of the agent module 6102 (represented, for example, as database objects). One embodiment of the super-agent module is described with respect to the workflow representation in Figures 19A and 19B. Thus, the super-agent module 6102 enables the decomposition of complex evaluation and logic into inference paths through a plurality of interconnected nodes 6108, thereby making it computationally less burdensome to reach accurate and precise responses. In some embodiments, the agent module 6102 is accessible via an interactive console and / or an application programming interface (API), as illustrated in Figures 18A-18F and Figures 19A and 19B.

[0076] A simplified, exemplary agent module configuration is shown in Example 2. { “id”:“1234” / / primary key “name”:“adverse-effects”, “description”:“Looks up drug adverse effects from FDA labels”, “type”:“chatbot-agent-with-tools”, / / this particular type corresponds to a model prompted to choose from available tools “data_classification”:“INTERNAL”, “owners”:[example owner], “visibility”:“INTERNAL”, “required_roles”:[“EXAMPLE_READER”], “data_stores”:[ ], }

[0077] Example 2 - Exemplary Agent Configuration In some embodiments, one or more parts of the agent configuration are stored in a separate version table (linked, for example, by the agent ID). In this way, the agent configuration can be edited without affecting the deployed agent version. A simplified exemplary agent version configuration is shown in Example 3. { “id”:“1234” / / foreign key linking this version to an agent “version”:52, / / editing the agent config creates a new version “console_deployed_environments”:{“alp”:true,“bet”:true,“prd”:false}, “api_deployed_environments”:{“alp”:true,“bet”:true,“prd”:false}, / / whether to enable API access in addition to interactive access / / agent-type-specific configuration object “config”:{ “tools”:[{“name”:“fda-label-adverse-effects-highlights”}] “llm”:{ “model”:“xxxx” “temperature”:{ “value”: 0; “editable”:true, / / whether this value is editable in the interaction console }, “max_tokens”:{“value”:1024,“editable”:false} } } }

[0078] Example 3 - Exemplary Agent Version Configuration In an exemplary scenario, a user configures an agent in the console and then deploys it to one or more environments (e.g., a workload plane and / or a control plane). In this scenario, the agent configuration is stored in the control plane (e.g., as shown in Figure 6). As shown in Figure 6, the agent itself runs in the appropriate working environment, and the working environment does not have access to the control plane. The agent builder in the control plane is configured to push the configuration to the various environments (e.g., via the configuration pubsub component shown in Figure 6). In some embodiments, when the agent configuration is changed or an agent version is deployed, the agent builder notifies the agent hosts in each environment so that the updated agent can be deployed. This may be via a pubsub message to the agent configuration topic or via a simple HTTP request.

[0079] Architecture 600 allows for flexibility to support various deployment strategies for each agent module 6102. For example, some end users, such as those who will use agent module 6102 interactively without engineering support, expect that agent module 6102 to operate entirely within a production work environment. In some embodiments, an administrator, such as the creator of agent module 6102, can choose a deployment style suitable for their application by restricting agent module 6102 to one or more domains, one or more databases 108, one or more services 110, or a combination thereof. For example, a first user might want to adopt a user interface that includes one or more user interface elements described with respect to the application (e.g., user interface 500) by embedding the components directly within a web page, while a second user might want to interact with an API configured to receive user requests and provide responses in the form of data structures, and the second user might integrate into a different user interface element not associated with the application.

[0080] In some embodiments, the user of the agent builder user interface within the control plane is provided with a creation access token, which can also be requested from the creation agent host. In some embodiments, an integrated user interface is presented to the user, showing them both the agent builder and the interactive console, with multiple input functions visualized through representations, without making the user aware of the difference between the control plane and the working environment. For example, a user who wants to test agent module 6102 in a low-level environment may be provided with a link to open agent module 6102 in a new tab or frame of the application. In some embodiments, an authentication request is presented, and an access token is obtained by agent module 6102 for that environment. In some embodiments, the user interface includes an indication of which environment is currently active.

[0081] In some embodiments, the data module 240 (e.g., document index) shown in Figure 6 includes one or more of the following: a static corpus, a dynamic corpus, an embedding model (e.g., model 228), a chunking strategy, a storage backend, a data classifier (e.g., public, internal, or secret), and / or visibility settings (e.g., private, public, or role-based restriction). In some embodiments, the data module maintains an index of data that may be temporary or permanent. In some embodiments, data elements associated with data files (e.g., documents) are evaluated through a chunking process, embeddings are generated for chunks produced from the chunking process, and embeddings are inserted into a database (e.g., embedding 6240 in Figure 14). In some embodiments, the data module 240 includes a set of ingestion parameters (e.g., about the number of documents to ingest and / or a measure of similarity). In some embodiments, the data module 240 corresponds to a set of databases (e.g., a medical database), such as database 400 in Figure 4. In some embodiments, the parameters associated with each agent module 6102 and / or node 6108 of model 228 include selecting one or more document indexes to be ingested via data module 240. In some embodiments, embeddings are created and siloed for future use. In some embodiments, each embedding is associated with one or more access control lists (ACLs).

[0082] A tool is a mechanism that allows an agent module to integrate with other components and with the outside world. In some embodiments, tools are made available to the agent module as agent builder blocks. Some tools may be general-purpose, while others may be custom for specific integrations. Different agent module types may have different access to tools; for example, a language chain agent may consist of a set of available tools, and the model may be configured to select when and how to use them, while the language chain may follow a fixed step sequence. In some embodiments, the agent configuration defines when and how tools are invoked. As an example, tools may be configured with a fixed base URL so that the agent cannot make authentication requests to other services. In some embodiments, tools are configured to authenticate using the end user's access token, rather than granting access roles to the agent's machine user. In some embodiments, tools are restricted to certain endpoints and / or methods (e.g., GET requests only), and as a result, tools are restricted to performing administrator tasks on behalf of users without administrator privileges (e.g., write permissions).

[0083] In some embodiments, the tool has parameters specified when configuring the agent module and / or parameters that may be specified by the agent module itself at the time of the call (e.g., parameter 6110). An exemplary tool is an authentication request tool configured to fetch an internal URL using a user's access token. The authentication request tool may include the following parameters: name, description, base URL, and / or input parameters (which may be specified by the agent, for example). For example, the exemplary authentication request tool may have an order identifier as an input parameter. Another exemplary tool is an external request tool that fetches an external URL. The parameters of the external request tool may include name, description, base URL, and / or input parameters. Another exemplary tool is an email tool that sends an email. The parameters of the email tool may include recipient, subject, and / or body.

[0084] As an example, a task-specific agent module includes: (i) an agent module configured to send emails summarizing which customers are having problems with orders and / or identifying opportunities for retraining; (ii) an agent module configured to generate data tables, JSON schemas, and other data transactions; (iii) an agent module configured to find and / or provide summaries of orders within customer groups having specific flags (e.g., using timestamps for order creation timing); (iv) an agent module to identify behavioral changes in ordering habits and adjust orders accordingly (e.g., increasing delays and / or canceling orders); (v) an agent module to generate inclusion / exclusion criteria from protocol documents, generate structured queries (e.g., SQL queries) from structured lists, and / or generate specifications (e.g., YAML specifications) from structured lists of inclusion / exclusion criteria; and (vi) an agent module to answer questions about specific tests based on information in protocols and / or other test materials or documents.

[0085] In one embodiment, an agent module 6102 configured to identify and / or evaluate adverse effects receives a user query regarding adverse effects associated with a particular drug. In this embodiment, the agent module 6102 parses the query to identify the drug name from the query and applies the drug name to one or more nodes 6108 to obtain a set of adverse effects associated with the drug. In this embodiment, the agent module 6102 provides a response with a description of the set of adverse effects.

[0086] The following Example 4 shows a simplified configuration example of an adverse action agent. extract_drug_name=LLMChain( prompt=extract_drug_name_prompt, llm=llm, output_key='drug_name', verbose=true, ) highlights_lookup=FdaLabelChain( fda_labels_client,verbose=true) sample_document=LLMChain( prompt=answer_prompt, llm=llm, output_key=”text”, verbose=true, ) combined=SequentialChain( chains=[extract_drug_name,highlights_lookup,analyze_document], input_variables=["query"] output_variables=[“text”,“fda_label_web_url”,“product_name”,“generic_name”], verbose=true, return_all=true, )

[0087] Example 4 - Exemplary Composition of an Adverse Agent The following Example 5 shows an exemplary chain definition for agent module 6102. { “type:“chain”, “chain”:{ “type”:“SequentialChain”, “chains”:[ { “type”:“LLMChain”, [ka] “output_key”:“drug_name” }, { “type”:“FdaLabelChain” }, { “type”:“LLMChain”, [ka] “function_key”:“text” }, ], “input_variables”:〔“query”〕, “output_variables”: [“text”, “fda_label_web_url”, “product_name”, “generic_name”] } }

[0088] Example 5 - Definition of an Exemplary Adverse Agent Chain In some embodiments, the type or classification of agent module 6102 is selected for a particular task based on an analysis of a set of different types or classifications. In some embodiments, the analysis includes comparing label-dependent spectra from the outputs of a pre-trained model 228. For example, the comparison may be performed using the Jensen-Shannon (JS) divergence of the principal component analysis (PCA) decomposed output spectra. In some situations, a model 228 more suitable for a downstream task has a larger JS divergence. The JS divergence is described in Menendez et al., 1997, "The Jensen-Shannon divergence," Journal of the Franklin Institute 334(2), pp. 307-314, which is incorporated herein by reference in its entirety for all purposes. Furthermore, some models 228 have a greater information capacity in the intermediate layers. The greater information capacity can be determined by measuring the dimensionality of the PCA-reduced spectra coming from the layer outputs. In some embodiments, a decomposed spectrum selector of a pre-trained model is configured to perform the above analysis.

[0089] Those skilled in the art will understand that there are numerous pre-trained and fine-tuned deep language models (DLMs) available. However, the performance of each model for downstream fine-tuning tasks can vary considerably. For this reason, a model selection process (e.g., heuristics) can save time and effort compared to training several models and then choosing the best-performing one.

[0090] In some situations, models better suited to downstream tasks are better at separating data according to the label of each individual data point. This can be verified by examining the label-dependent statistics in the output of the task-dependent output header. Even if downstream training has not been performed, this can be done by examining the label-dependent spectrum of data coming from the output of a pre-trained model. A useful metric for determining label-dependent spectral separation is JS divergence. Often, spectra are multidimensional, so JS divergence can be calculated and summed along the dimensionality of the spectrum. This can be problematic, as higher-dimensional outputs are inherently favored simply because they contribute more dimensions to the sum. Simple JS divergence not only favors higher-dimensional outputs but also fails to consider the correlation between outputs.

[0091] To circumvent this problem, the spectrum can be decomposed into the first N principal components necessary to explain 99% of the cumulative explanatory variance, where N is a positive integer. In some embodiments, N is 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 10-20, 20-50, 10-100, or 1-1000. A pre-trained DLM with a higher PCA-reduced JS divergence may lead to better downstream classification performance because it has an intrinsic advantage in distinguishing label-dependent data. For example, there is a correlation between PCA-reduced JS divergence and the macroscopic F1 score of the DLM on test data. Therefore, in some scenarios, the choice of pre-trained model has a significant impact on the final performance.

[0092] The macroscopic F1 score for DLM on test data can be derived from the DLM's precision and recall. Precision is the accuracy of the positive predictions made by the DLM. This can be considered as the ratio of true positive (TP) predictions to the total number of positive predictions (true positives + false positives). Recall (also called sensitivity or true positive rate) measures the DLM's ability to correctly identify positive instances. This can be considered as the ratio of true positive predictions to the total number of actual positive instances (true positives + false negatives). The F1 score can then be calculated as the harmonic mean of precision and recall.

number

[0093] The correlation between PCA reduction JS divergence and macroscopic F1 of the test data also applies to image modalities. In the computer vision domain and modalities, a larger PCA reduction JS divergence in the model output spectrum indicates better downstream performance. Furthermore, in some scenarios, the layers that yield the highest PCA dimension lead to the strongest results. For example, a fully exemplary model when trained yields a macroscopic test F1 score of 0.76, while the same model with only up to 8 layers (e.g., the maximum PCA dimension) yields a macroscopic test F1 score of 0.86, which is quite good.

[0094] In some embodiments, PCA reduced JS divergence analysis is used for (i) selecting agent module 6102 and / or model 228 for a particular task (e.g., selecting the appropriate agent module), (ii) identifying the appropriate dimensionality of agent module 6102 and / or model 228 (e.g., reduced dimensionality), (iii) optimizing one or more node parameters of agent module 6102 and / or model 228 for best use (e.g., selecting layers), (iv) optimizing one or more inputs of agent module 6102 and / or model 228 (e.g., which combination of inputs provides the best initial divergence), (v) creating embeddings to identify whether a combination of agent module 6102 and / or model 228 is beneficial, (vi) deterministic pruning of agent module 6102 and / or model 228, or (vii) combinations thereof. For example, optimizing the configuration of agent module 6102 and / or model 228 for best use may include selecting the nodes and / or layers of the model that have the highest dimensionality after reduction.

[0095] In an exemplary scenario, the user obtains a set of labeled data for training a classifier. The user may split the data into training, validation, and test datasets. A set of pre-trained agent modules 6102 and / or models 228 is identified by a first agent module 6102 (e.g., a transformer, a convolutional neural network, and / or a recurrent neural network) that can fit the task. Each of the agent modules 6102 and / or models 228 is run across the validation dataset, and the spectrum from the last layer of the pre-trained model and / or agent module may be examined (e.g., before entering the classification head). This yields a tensor of shape (N, D), where N is the number of examples in the validation set and D is the dimensionality of the output from the last layer of the pre-trained model and / or agent. To remove linear dependencies, a 99% PCA reduction may be applied to the output, yielding a new tensor of shape (N, D_pca). JS divergence can be calculated between class labels (e.g., in a one-for-the-remainder manner for each component and the sum). Then, a pre-trained agent module 6102 and / or model 228 that yields the highest sum JS divergence can be selected. The selected agent module 6102 and / or model 228 may correspond to a specific type or classification. In some embodiments, the agent modules 6102 and / or model 228 in the set of pre-trained models have 2, 3, 4, 5, 6, 7, 8, 9, 10, 10-20, 20-30, or more than 30 nodes and / or layers.

[0096] Furthermore, parts of agent module 6102 and / or model 228 may be identified as useful for specific datasets and / or information domains, such as a first agent for use in the genome domain, a second agent module for use in the microbiome domain, a third agent for use in the imaging domain, and a fourth agent module for use in the drug domain. 第The spectrum from the zero elements may be obtained, a 99% PCA reduction may be performed, and then the dimensions may be recorded. For example, the nodes and / or layers of agent module 6102 and / or model 228 with the largest PCA reduction dimension may be selected, and all nodes and / or layers after selection may be discarded. The output of the selection may be fed to the classification head (which can then perform fine-tuning of agent module 6102 and / or model 228). Figure 7A shows an exemplary agent module 6102 with multiple hidden interconnected nodes forming layers and having PCA reduction JS divergence and PCA reduction dimensionality measurement points. Figure 7B shows another exemplary agent module 6102 and / or model 228 with multiple interconnected nodes forming layers, where nodes and / or layers may be discarded after selected nodes and / or layers have been identified. Figure 7C shows a graph of PCA reduction dimension versus number of hidden layers for exemplary agent module 6102 and / or model 228.

[0097] As shown in Figure 7C, an exemplary model 第 The eighth hidden layer has the highest PCA reduction dimension. Figures 7D and 7E show a spectral comparison between the reduced eighth hidden layer model and the full exemplary model for a specific dataset. Table 1 below shows the reduced dimensions in this example after each model has been trained on the listed datasets. 第 This section shows a comparison between the 8 hidden layer model and the complete model. [Table 1] [Table 1-1]

[0098] As shown in Table 1, the reduced 8-layer model outperformed the full model on these datasets. Furthermore, the reduced model was computationally less expensive than the full model.

[0099] Figures 8A and 8B illustrate exemplary agent module 6102 architectures (e.g., agent module architecture 800 and agent module architecture 850) in several embodiments. In some embodiments, the agent module architectures of Figures 8A and 8B illustrate a workflow of a number of corresponding interconnected nodes; however, this disclosure is not limited thereto. For ease of explanation, the exemplary agent module 6102 architectures depicted in Figures 8A and 8B are described in relation to the devices, systems, and techniques described in relation to Figures 1 to 6. However, those skilled in the art will recognize that the same or similar operations can be performed using a variety of alternative and / or additional devices, systems, and techniques.

[0100] Figure 8A shows an agent module architecture 800 in which a client device 102 sends a query describing "What is PFS?" (e.g., the text of a prompt provided by the user, or other data based on the determined intent of one or more parts of the prompt text) to a server system 106 which includes a backend component (e.g., an implementation component of agent module 6102). In some embodiments, the backend component includes or sends queries to an embedding component (e.g., an embedding API 802, embedding 6240 in Figure 8C, embedding 6240 in Figure 14, etc.) which may be configured to generate embeddings based on one or more parts of the query text received from the client device 102 and / or another agent module 6102. In some embodiments, the query embedding is compared to an embedding in a vector database 6240 (e.g., to determine and provide vector similarity 804), and the result is sent to the backend component as context with the query vector and / or a different agent module 6102, according to the logic 6112 of the node architecture. In some embodiments, the agent module 6102 sends context, queries, and optionally chat history to a node 6108-related model 228 component (e.g., a large language model). The model 228 outputs a response to the agent module 6102, such as a terminal node, which transmits the response to a client device 102, according to some embodiments.

[0101] In some embodiments, embeddings are generated from data in knowledge database 404 and stored in vector database 6240 (as illustrated, for example, in Figures 8A and 14). In some embodiments, documents from knowledge database 404 are divided into snippets, which are tokenized and added to vector database 6240. In some embodiments, snippets of consecutive text are extracted from each patient file and stored as chunks (snippets, where the terms chunk and snippet are used interchangeably). In some embodiments, each chunk contains 100 to 1000 characters. In some embodiments, the chunking (dividing) algorithm duplicates the chunks. In an exemplary chunking algorithm, each chunk consists of 512 characters, and each chunk has 128 characters of overlap with another chunk extracted from data in the knowledge database.

[0102] In some embodiments, the document is divided into chunks and / or snippets based on a fixed character length (with any duplicates), a fixed token length (with any duplicates), and / or section-based division (e.g., identification of section headings and their division). In some embodiments, prompts for agent module 6102 include the context of the taken-up patient, inclusion / exclusion criteria, and questions for determining whether the patient meets the criteria.

[0103] Figure 8B shows an embodiment similar to the embodiment in Figure 8A, but in Figure 8B, data from a knowledge database (e.g., a question and answer dataset) is used to fine-tune model components (e.g., a model API (e.g., for model 228), a task-specific machine learning model, etc.) according to several embodiments. In some embodiments, the query embedding process is replaced or complemented by other types of search (e.g., Elasticsearch). In some embodiments, other types of search are performed on a database 108 containing document snippets. In some embodiments, one or more conversation histories of chat history associated with a user of a client device 102 are retrieved from the client device 102 or the server system 106. For example, in some embodiments, each agent module 6102 is associated with a data module 240 containing one or more conversation histories of various prompts and / or responses from multiple users.

[0104] In some embodiments, vector similarity is based on cosine distance, Euclidean distance, Manhattan distance, Jacquard distance, correlation distance, chi-squared distance, Mahalanobis distance, and / or semantic comparison of embeddings.

number

number

[0105] In some embodiments, queries, context, and chat history are formatted (e.g., vectorized) and input into model 228. In some embodiments, responses from model components are parsed and formatted by agent module 6102, and the formatted responses are transmitted to client device 102.

[0106] Figures 9A to 9C illustrate exemplary processes for data vectorization and querying according to several embodiments. Figure 9A shows the process of generating embeddings from a document (e.g., a PDF document 902). In some embodiments, the document is obtained from a medical database (e.g., a clinical trial protocol document from a clinical trial database), which may be a live collection of medical data (e.g., including means for obtaining real-time updates to the data source). In some embodiments, the embeddings are also generated from metadata corresponding to the document and stored in a vector database 904.

[0107] Figure 9A also illustrates the process of generating question embeddings and identifying relevant chunks of a document using cosine similarity with vectors in the vector database 6240. It will be understood that instead of using cosine similarity search, distance metrics such as the distance metric described above can be used. Figure 9B shows the question and relevant chunks input into a Large Language Model (LLM), which then generates an answer (for example, the relevant chunks are used as context by the LLM to answer the question).

[0108] Figure 9C shows an example where a query vector is generated from a query via an embedded API. The query vector is used to identify a similar vector, identified as “patient” in a first vector database. The similar vector from the first vector database (and / or the query vector) is used to identify a second similar vector in a second vector database. The query vector and the second similar vector (and optionally the first similar vector) are provided to the language model via prompts. The language model outputs an answer to the query, which is optionally reformatted and transmitted to the user. In the example in Figure 9C, the query is “What is the reason for Linda Watson’s order cancellation?” and the language model outputs the status reason as the answer.

[0109] As discussed above, in some embodiments, the agent module (e.g., the backend component) is configured to perform intent matching and / or parameter extraction for user queries and requests. In some embodiments, intents are assumed (e.g., the agent module is configured for a specific task). In some embodiments, the agent module extracts domain-specific parameters. For an exemplary query, "show patients with MSI high, TMB less than 20, which have been diagnosed with central neurocytoma in the past four months," the extracted parameters might be ["mis":"high", "tmb":"{"lt""20"}, "diagnosis":"central neurocytoma", "date_range":{...}].

[0110] In some embodiments, the agent module is configured to automatically add structured queries (e.g., SQL queries) from user queries and transmit those structured queries to a structured database. For example, the agent module may retrieve a specific schema, retrieve inclusion and exclusion criteria, and generate a structured query for the database based on the criteria identified from the query and the schema of the database being searched. In some embodiments, the structured query is transmitted to another agent module or component to interact with one or more structured databases. For example, the user query "how many patients are older than 18?" may be translated into the SQL query "SELECT COUNT(*)FROM demographic WHERE age > 18".

[0111] Example 6 below shows an example of configuration parameter 6110 for one or more nodes 6108 of agent module 6102 for a process to find patients who meet cohort inclusion / exclusion criteria (for example, using the process shown in Figure 9B). { “cache_path”: “cohort_name”: 'n_patients': 'embedding_batch_size': 'device': 'doc_chunk_size': 'doc_chunk_overlap': 'text_seperators': 'embedding_model': 'instructor_embed_instruction': 'instructor_query_instruction': 'prompt_template_name': 'generator_queries': 'vector_store_db_name': 'generator_model': 'n_retrieved_docs': 'temperature': 'top_p': 'tok_k': 'max_output_tokens': 'output_parser': 'chunking_config_hash': 'embedding_config_hash': }

[0112] Example 6 - Exemplary Configuration Parameters Given the enormous amount of text contained within the EHR real-world data (RWD) warehouse, processing entire patient clinical notes within a model's context window (e.g., LLM) becomes impractical. In some embodiments, this challenge is addressed by implementing an ingestion-enhanced generation (RAG) approach to identify relevant portions of EHR text, e.g., relevant portions of unstructured clinical notes. The RAG approach has proven to be more efficient and effective than providing a model with a larger context window. In some embodiments, RAG is a two-step process involving ingesting relevant documents from a corpus (e.g., a large corpus with thousands or millions of documents) and then feeding the ingested documents into a model for analysis and response generation.

[0113] In some embodiments, clinical notes from an EHR are divided into individual segments, also referred to herein as snippets (e.g., chunks, as illustrated in Figure 9A). One exemplary method for segmenting unstructured clinical data (e.g., clinical notes) includes tokenizing the unstructured clinical data to obtain multiple tokens, and segmenting the multiple tokens to obtain multiple segments (snippets), for example, each segment within the multiple segments having approximately the same number of tokens.

[0114] In some embodiments, individual snippets are evaluated to determine whether they contain information relevant to determining whether a subject has a target medical condition. In some embodiments, the evaluation is performed by natural language processing. In some embodiments, the evaluation is performed based on pattern recognition of regular expressions (Regex) associated with the target medical condition. In some embodiments, using Regex avoids introducing bias through additional hyperparameter tuning and narrows the focus of assessing the model's ability to diagnose diseases. However, other incorporation models can be used instead of or in addition to Regex. For example, in some embodiments, snippets are evaluated using term frequency-inverse document frequency. In some embodiments, snippets are evaluated using Cohere reranking. In some embodiments, snippets are evaluated using instructor embeddings.

[0115] In some embodiments, snippets are captured by using a model (e.g., LLM) to identify portions of medical records containing information related to a target medical condition. In some embodiments, the model is prompted to identify any portion of a medical record related to an indicator of disease diagnosis. In some embodiments, the identified portion (e.g., snippet) is defined as being within a specific range of characters. For example, in some embodiments, the identified portion must be X to Y characters long, where X is the minimum length and Y is the maximum length. In some embodiments, the identified portion (e.g., snippet) is defined as being within a specific range of token lengths. For example, in some embodiments, the identified portion must be X to Y tokens long, where X is the minimum length and Y is the maximum length. In some embodiments, the identified portion (e.g., snippet) must satisfy a relevance threshold. For example, in some embodiments, a set of candidate portions is identified and ranked in terms of their relevance to each other and to the medical condition, and the top X number of candidate portions are selected for capture. In some embodiments, the ranking is limited to portions taken from a single document within the medical record. In some embodiments, the ranking is applied to multiple documents within a medical record.

[0116] While the RAG approach reduces the amount of text processed by the model, RWD clinical notes often contain many text pages. As a result, the Regex retriever is likely to return a large number of snippets that it determines contain information relevant to determining whether a subject has a target medical condition, which may exceed the model's context window. In some embodiments, a map reduction approach is employed to address this problem. Map reduction allows for parallel execution of the model on individual snippets, improving efficiency and reducing processing time. It also facilitates handling a large number of identified snippets by distributing the processing load across multiple iterations. By generating individual outputs for each snippet, the chain can extract specific information that contributes to a more comprehensive final result.

[0117] Therefore, in some embodiments, each identified snip is presented as context to the model, along with a set of instructions to facilitate decision-making. In some embodiments, the model is asked through prompts to indicate whether the snippet indicates that the subject is PH. In some embodiments, the prompts instruct the model to respond in the form of yes or no, or yes, no, or unclear. In some embodiments, the prompts further instruct the model to support its answer with evidence. In some embodiments, by prompting the model to support its answer with evidence, the model essentially summarizes the relevant portion of the snippet and reduces the context supplied to a second model (e.g., a map reduction model chain).

[0118] In some embodiments, the prompt includes a statement that guides the model. For example, again referring to the example of phenotypic analysis of pulmonary hypertension, in some embodiments, the prompt instructs the model to count "possible" cases of PH as "no" answers. In some embodiments, the prompt instructs the model to count clinical notes of a medical history of PH as "yes" answers. In some embodiments, the model is further provided with examples of evidence indicating the presence of the target medical condition. In some embodiments, the model is further provided with examples of evidence not indicating the presence of the target medical condition. In some embodiments, the model is further provided with evidence indicating the absence of the target medical condition. In some embodiments, the prompt includes a chain of thought (CoT) phrase. The use of CoT enhances the reasoning by the model.

[0119] Next, the outputs generated by the model for each snippet are aggregated to formulate a final decision. In some embodiments, aggregation is performed using a model. In some embodiments, the model is provided with the output from the snippet evaluation as context and the same command prompts as for the evaluation of each individual snippet. In some embodiments, the model is provided with the output from the snippet evaluation as context, but with different command prompts than for the evaluation of each individual snippet. For example, in some embodiments, the model is asked whether any of the outputs from the snippet evaluation indicate a positive diagnosis for the target medical condition. In some embodiments, aggregation is a maximum aggregate function that checks whether any of the individual snippet queries returned a positive diagnosis, and if so, assigns a positive label to the entire patient.

[0120] In some embodiments, the snippet evaluation and aggregation steps are performed using the same model. In some embodiments, the snippet evaluation and aggregation steps are performed by the model after a single prompt asking the model whether the subject has a medical condition based on the evidence contained within the snippet. In some embodiments, the snippet evaluation and aggregation steps are performed in series so that the model is provided with separate prompts for the two steps. In some embodiments, the snippet evaluation and aggregation steps are performed using different models.

[0121] In some embodiments, a user prompt is received in the API along with instructions to ingest snippets and then present them to an AI component in response to the user prompt. In some embodiments, the API receives a prompt concerning a first subject or group of subjects. In some embodiments, medical records concerning the subject or group of subjects have already been parsed (snippeted), and the snippets are stored in a curated database. In some embodiments, the snippeted records are also sorted to identify, for example, snippets related to a target medical condition in the curated database. In some of these cases, the API ingests pre-sorted snippets from the database and presents them to the AI ​​component. In other embodiments, if the medical records are not snippeted, the API ingests the medical records and instructs a module (e.g., a natural language processing module) to parse the medical records, snippet them, optionally sort the snippets, and identify those snippets related to a target medical condition. Similarly, in some embodiments where medical records are snippeted but not sorted, the API takes in the snippets and instructs a module (e.g., a natural language processing module) to identify those snippets related to a target medical condition. The API then presents the identified snippets to an AI component (e.g., a model such as an LLM) either in parallel (e.g., via separate instances of the AI ​​component) or sequentially, with each snippet asking the AI ​​component whether it indicates that the subject has a target medical condition, and optionally requesting that it provide the rationale for the answer. The AI ​​component generates an answer for each snippet and optionally generates a quadratic logic (inference) for each answer. The API also includes instructions for aggregating the component's answers regarding whether the subject has a target medical condition into a final answer.In some embodiments, the API requests the model to aggregate component answers and optional quadratic logic so that the AI ​​component does not provide component answers externally, but rather returns a single answer to the subject, which is returned as a response to the API prompt containing the query.

[0122] Figure 9D illustrates an exemplary process for patient querying in several embodiments. In the embodiments of Figure 9D, document information from document 972 (e.g., patient document) is provided to systems 970, 974, and 976 (e.g., agent module or assembly). In some embodiments, document 972 includes one or more types of documents (e.g., PDF document, image document, text document, word processor document, etc.). In some embodiments, document 972 includes one or more data collections corresponding to one or more databases.

[0123] In some embodiments, system 970 (e.g., a first agent module) is configured to extract text from document 972 (e.g., using various text recognition and extraction techniques). System 970 can be considered an embodiment of a concept-specific ingestion assembly. The extracted text is harmonized (e.g., using models 980 and 982). In some embodiments, model 980 is configured to convert image data into text data (e.g., extracting text chunks that may contain lossy OCR content). In some embodiments, model 982 is a document classification model (e.g., configured to identify chunks with relevant information and discard chunks without relevant information). The extracted text is chunked (e.g., using the processes described above with respect to Figures 9A-9C) and stored as embeddings in a vector database. System 970 also retrieves query embeddings from user queries, identifies similar embeddings in the vector database, and outputs text chunks relevant to the user queries. In some embodiments, user queries are generated based on inclusion / exclusion (IE) criteria. In some embodiments, the system 970 is configured to return text chunks corresponding to a specific patient. For example, the system 970 retrieves a patient identifier from a user query and outputs only the text chunks mapped to that patient identifier.

[0124] In some embodiments, system 974 (e.g., a second agent module) is configured to retrieve a patient's embedded representation from document 972 and identify patients similar to those identified or characterized in the input query. In some embodiments, system 974 is configured to identify patients who satisfy a set of IE criteria (e.g., a set of criteria provided in the input query or by another agent module). In some embodiments, system 974 is configured to identify patient clusters associated with a set of IE criteria (or similar criteria).

[0125] According to some embodiments, system 976 (e.g., a third agent module) is configured to classify documents 972 and identify a subset of documents having classifications that match (or are similar to) classifications extracted from an input query. In some embodiments, for the identified subset of documents, system 976 is configured to provide a document identifier, a document link, and / or the document itself.

[0126] In the embodiment shown in Figure 9D, the outputs of systems 970, 974, and 976, along with information from user queries, are provided to model 984 (e.g., a large language model configured to understand user queries). In some embodiments, model 984 is configured to provide an output that incorporates information from each of systems 970, 974, and 976 (e.g., a summary of information from the other systems). In some embodiments, model 984 is configured to provide natural language output to the user.

[0127] Figures 10A–10C illustrate exemplary user interfaces for agent-based search in several embodiments. In the embodiments of Figures 10A–10C, a trial identifier is selected, and a query for a trial is entered by the user. The trial identifier and query are received by an agent module (e.g., a clinical trial agent module), which provides a natural language response. In some embodiments, the trial query is received without a trial identifier (e.g., the query is received with a target identifier and / or other information about the trial). In some embodiments, the agent module identifies the corresponding trial based on the information in the query. For example, a first agent module identifies the trial, and a second agent module applies the query to the trial (and optionally format the response for the user).

[0128] In some embodiments, the digital assistant (including, for example, a super-agent module and a cohort-building agent module) is configured to enable the user to efficiently and accurately narrow down a set of subjects (e.g., patients) within a target dataset and / or verify that the set of subjects is a desired cohort. For example, the digital assistant is configured to assist the user in determining whether (i) an existing cohort is accurate (e.g., a set of subjects that respond to the user's research question), (ii) whether an existing cohort is missing some subjects from a patient dataset, and / or (iii) whether the user has identified complete and accurate patient records for the set of subjects. In some embodiments, the user interacts with the digital assistant via a search user interface. In some embodiments, the digital assistant is configured to translate natural language input from the user into a structured query (e.g., by identifying intent and / or query parameters).

[0129] In some embodiments, the digital assistant (or the cohort-building agent module of the digital assistant) includes one or more templates and / or tools to assist the user. For example, cohort-builder templates and / or table-builder templates may be provided (as illustrated, for example, in Figures 11A-11C). These templates can provide information that the user can use when asking the digital assistant to assist with their tasks. For example, using a table-builder template, the user can select from their delivery and then populate a primary table that can be used for their analysis with various data concepts they want to extract across multiple tables in the delivery. Using the templates, the digital assistant is configured to determine which delivery and therefore which columns are available to the user, thereby increasing the likelihood that the task of creating a new table for the user will be successfully completed. As described above, in some embodiments, the digital assistant is or includes a super-agent module configured to take in user queries and identify the appropriate agent module to address the query (e.g., identify a cohort-building agent module to answer cohort-related queries).

[0130] In some embodiments, the digital assistant includes an ensemble of agent modules. In some embodiments, the digital assistant includes a front-end agent module (e.g., including a language model) configured to identify commands and / or tokens within a user query. In some embodiments, the digital assistant includes a routing agent module configured to route a subset of commands and / or tokens to the appropriate agent module (e.g., based on query intent, previous interactions, and / or information about a particular user or subject). In some embodiments, the digital assistant (and / or one or more of the agent modules of the digital assistant) accesses information about a patient or subject associated with a query or command, and the digital assistant tailors its response to the patient or subject (e.g., personalizes the response). In some embodiments, the digital assistant maintains information about previous interactions (e.g., previous commands, queries, and responses) and uses the maintained information to inform subsequent interactions. In some embodiments, the digital assistant uses information about previous interactions as contextual information for the query. In some embodiments, the digital assistant uses information about a patient or subject (e.g., age, sex, medical history, current medications, etc.) as contextual information for the query. In some embodiments, the digital assistant uses information about the user (e.g., a healthcare professional) as contextual information for the query (e.g., the user's area of ​​study, current patient, medical specialty, healthcare network, access to medical devices, etc.).

[0131] Figures 11A to 11C illustrate exemplary user interfaces for interacting with a digital assistant in several embodiments. Figure 11A shows an interface for interacting with a digital assistant. The interface in Figure 11A includes affordances for opening a cohort builder tool or a table builder tool. Figure 11B shows a digital assistant interface including a section with a cohort builder tool. Figure 11C shows a digital assistant interface including a section with a table builder tool. As illustrated in Figures 11A to 11C, the digital assistant may be provided with various tools selected by the user (e.g., the cohort builder in Figure 11B and the table builder in Figure 11C). In some embodiments, the digital assistant provides the user with different tools based on the user's requests and / or queries. In some embodiments, each tool corresponds to a different agent module of the digital assistant. In some embodiments, a single agent module includes multiple tools. In some embodiments, the digital assistant (or the super-agent module of the digital assistant) identifies a tool from a pre-generated set of tools based on the user's intent and provides the user with access to the tool. In some embodiments, the digital assistant provides information about the selected tool about the query, user, patient, subject, and / or other contextual information (for example, to pre-populate one or more fields of the tool and / or configure the tool otherwise). In some embodiments, the digital assistant communicates the tool with the selected information using a routing agent module.

[0132] Figures 12A to 12D illustrate exemplary user interfaces for interacting with a digital assistant according to several embodiments. In some embodiments, the digital assistant consists of one or more agent modules (e.g., task-specific agent modules). In some embodiments, the digital assistant is communicatively coupled with one or more agent modules (e.g., task-specific agent modules). For example, the digital assistant may include an API agent module (e.g., a front-end module) for receiving / parsing user queries and commands, a super-agent module for identifying appropriate agent modules and / or tools to respond to each query and / or command, and a routing agent module for transmitting information to the identified modules and tools. In some embodiments, the super-agent module is configured to compile information from the identified modules and tools to generate a complete response to each query or command. In some embodiments, the API agent module is configured to translate the complete response into a natural language response and provide the natural language response to the user. In some embodiments, the digital assistant includes one or more transformation agent modules for transforming data into the format and / or structure required by a particular tool. In some embodiments, the conversion agent module is configured to validate the data to ensure it is appropriate for the corresponding tool (e.g., to provide error information when the data is inappropriate). In some embodiments, the digital assistant is communicatively coupled to two or more databases (e.g., external database 108 and / or database 400 in Figure 4). In some embodiments, the digital assistant includes one or more interface agent modules configured to interact with the corresponding databases.

[0133] Figure 13 is a flowchart illustrating exemplary method 1300 for deploying an agent module according to several embodiments. Method 1300 is implemented in a computing system (e.g., a client device, a server system, and / or a service platform) having one or more processors (e.g., CPU 202 and / or 302) and memory (e.g., memory 218 and / or 310). In some embodiments, the memory stores one or more programs configured to be executed by one or more processors. At least some of the operations shown in Figure 13 correspond to instructions stored in computer memory or a computer-readable storage medium. In some embodiments, the computing system is a platform 100, a client device 102, and / or a server system 106.

[0134] The computing system receives requests from users for agent modules configured to perform specific tasks (1310). For example, a specific task could be customer support assistance, cohort building, or code writing. In some embodiments, the request includes one or more requirements for performing a specific task. In some embodiments, the request identifies a desired agent type for the agent. Examples of tasks include identifying inclusion and exclusion criteria from protocol documentation and generating structured queries based on the identified inclusion and exclusion criteria. Another exemplary task includes generating an inclusion and exclusion criteria table (or sheet) for storage in a database, and / or generating a set of inclusion and exclusion criteria (e.g., as YAML annotations). Other exemplary tasks include identifying trial inclusion and / or exclusion, patient discovery and feasibility, and medical documentation generation.

[0135] As described above, exemplary agent types for agent modules include, among other things, (i) API broker agents configured to interact with APIs in response to natural language input, (ii) document search agents configured to interact with databases and / or indexes in response to natural language input, (iii) model broker agents with access and knowledge to interact with machine learning models, (iv) template document generation agents configured to generate content in the form of templates, (v) co-pilot assistant agents embedded in applications and configured to assist application users using a knowledge base and / or question-and-answer pairs, and (vi) schema-specific data products, answer questions, and specific (vii) a data product generation agent configured to search for data products and / or create new data products; (viii) a composite agent configured to act as an orchestrator for multiple subordinate agents; (ix) a natural language search agent configured to enable free-response questions and summaries with multi-level conversational support; (ix) an SQL database agent configured to query one or more SQL databases, tables, and views in response to natural language input; (x) a vertex database agent configured to query one or more vector databases in response to natural language input; and (xi) a feasibility agent configured to search a multimodal library of patient data for a specific set of attributes provided in natural language.

[0136] The computing system identifies a first agent type from a set of agent types based on one or more requirements for performing a particular task (1320). For example, the first type of agent may be a tool-using agent, a chain agent (e.g., a super-agent), a routing agent, and / or a translation-enabled agent. In some embodiments, each agent type in the set of agent types corresponds to a respective language model. In some embodiments, each agent type corresponds to a different model (e.g., a different type of model, a different size of model, and / or a different fine-tuned model).

[0137] The computing system generates a model component having a first agent type (1330). In some embodiments, generating a model component includes generating a set of operation instructions for the model component. In some embodiments, generating a model component includes fine-tuning and / or training the model of the model component.

[0138] The computing system generates implementation components for the agent (1340), which are configured to communicatively combine model components into a set of components based on one or more requirements for performing a particular task. In some embodiments, the set of components includes one or more of the following: a set of data sources, a set of tools, and a set of output components.

[0139] The computing system deploys the agent to a working environment (e.g., a test environment or a manufacturing environment) (1350). In some embodiments, the agent consists of a model component and an implementation component. An exemplary agent is configured to receive (i) clinical trial prompts, (ii) historical feasibility studies, (iii) clinical data, and / or (iv) therapy data as input and to output a set of subjects identified for a clinical trial. Another exemplary agent is configured to assist a clinical development researcher. The agent (e.g., an implementation component) is configured to identify user intent from user queries / requests, (i) potential query extensions, (ii) provide corresponding concepts, interfaces, and / or datasets, (iii) provide query validation, (iv) identify data sources to be searched, (v) suggest and / or apply filters, and / or (vi) provide other user guidance. Exemplary user intents include identifying a particular cohort (e.g., a breast cancer cohort or a colorectal cancer cohort) and, given a particular drug target and indication, identifying expression and / or performing a particular analysis.

[0140] An exemplary task may be coding, in which the agent may be configured to generate code in a specific coding language based on natural language input from the user. For example, the user specifies a specific coding language, input, and desired output of the code. Another exemplary task may be workspace manipulation, in which the agent may be configured to manipulate data within the workspace and / or provide explanations, suggestions, and / or recommendations for manipulating the data. Another exemplary task may be interpreting claim data, in which the agent may be configured to interpret (unstructured) claim data such as notes and descriptions, identify trends, and / or make predictions based on the claim data. The agent may also be configured to understand therapies and / or treatment trends, markets, and / or landscapes (for example, based on historical claims data). For example, the agent may be configured to identify gaps in commercialization based on historical claims data. In some embodiments, historical claims data is obtained from a set of databases. In some embodiments, the set of databases is controlled / maintained by a separate entity and / or has a different format and / or structure. As will be discussed in detail below, Figures 14 to 18 illustrate various workflows for implementing and / or interacting with an agent module in several embodiments.

[0141] Figure 13 shows several logical stages in a specific order, but the order-independent stages may be rearranged, and the other stages may be combined or separated. The orderings and groupings presented herein are not exhaustive, as several rearrangements or other groupings not specifically mentioned will be obvious to those skilled in the art. Furthermore, it should be recognized that various stages may be implemented in hardware, firmware, software, or any combination thereof.

[0142] Figure 14 illustrates exemplary workflows for implementing and / or interacting with an agent, according to several embodiments. The exemplary workflow includes an administrative flow from a management application 6212 (e.g., an administration page, portal, or user interface) to various components such as the internal platform 6214, the secure platform 6216, and the search and analytics component 6218. According to some embodiments, the components interact with the component library 6220. Components 6214, 6216, and 6218 may be instances of platform 6222. The exemplary workflow also includes a flow from an OS console 6202 (e.g., a user page, portal, or user interface) to the management console 6204, the agent builder component 6206, the transformation component 6208, and / or the data product component 6210. For example, the flow may be based on the user's intent (e.g., to build an agent, use an agent, perform administrator operations, and / or access data products). The data source 6228 may be any of the data sources described herein. In some embodiments, the data source 6228 includes information from the knowledge database 404 (or any other database described herein). In some embodiments, an agent 6224 (e.g., consisting of one or more agent modules) is used to interact with the user. The agent 6224 consists of an agent building block 6226 and may use an API integration 6230, a model 6232, a conversational agent component 6234, a RAG component 6236, and an embedding 6240 (e.g., a vector of data from the knowledge database 404).

[0143] Figures 15 and 16 illustrate exemplary workflows for implementing and / or interacting with an agent, according to several embodiments. As shown in Figure 15, a user may submit a prompt (e.g., via voice and / or text input). Automatic speech recognition (ASR) may be performed on the prompt (e.g., using an agent module) to identify the intent of the prompt. After the intent is identified, an appropriate performer (e.g., an agent module, tool, or ensemble thereof) is selected and passed the intent and one or more parameters (and / or other contextual information). The performer may be communicatively coupled to one or more external databases 108 (e.g., knowledge database 404). In this way, the performer may retrieve and analyze response information from the external databases 108. After the performer has determined the response, the response may be converted (e.g., into voice and / or text data) and then provided to the user. Figure 16 illustrates a similar embodiment that receives a prompt 3601 and generates a response 3606. In the embodiment shown in Figure 16, the service may include one or more agent modules and / or one or more tools (selected, for example, by the super-agent module). In some embodiments, the client application includes a user interface (e.g., a web interface). In some embodiments, the client application includes a digital assistant (e.g., as described above with respect to Figures 11 and 12).

[0144] Figure 17 illustrates examples of an ingestion-enhancement-generation (RAG) agent module according to several embodiments. As shown in Figure 17, the RAG agent module may include an embedding model (configured, for example, to generate embeddings from patient documents), a prompt interface, a generation model, and one or more transformation (e.g., an analyzer) components.

[0145] Figures 18A to 18F illustrate exemplary sequences of user interaction with an exemplary agent builder application 1800 (e.g., an application for creating task-specific orchestrations, including composite orchestrations (e.g., chains of agents)) according to several embodiments. As discussed herein, the agent builder application 1800 may include various user inputs for modifying each orchestration (e.g., agent module 6102) associated with the user of the agent builder application 1800. According to some embodiments, the user may access the agent builder application 1800 by providing user credentials from a client device 102 to a server system 106.

[0146] In some embodiments, aspects of the agent builder application 1800 are modified based on information about the user accessing the agent builder application. For example, data collection available to the user (e.g., accessible via user input directed to the data collection user interface element 1806) accesses the agent builder application 1800 based on specific credentials provided by the user. In this way, the system makes precision medicine principles applicable by providing the user with user-specific interactions (e.g., patient data about a patient cohort). In some embodiments, different users may have different levels of access to the tools and data available within the platform and / or agent framework. For example, a first set of users (e.g., consumers) may access their respective user interfaces associated with the home user interface element 1802, while a second set of users (e.g., builders) may also access user interfaces associated with the user interface element 1804 (e.g., agent tabs). In this way, users can interact with the agent builder application 1800 without explicitly determining whether they are authorized to use a particular aspect of the agent builder application 1800.

[0147] As will become apparent to those skilled in the art after reading the description of the series of interactions illustrated by Figures 18A to 18F, the embodiments described herein provide users with technical improvements by providing efficient and effective solutions for users to generate solutions to complex clinical tasks or sets of clinical tasks, which may otherwise require the user to provide multiple different inputs to multiple components and / or applications.

[0148] Figure 18A shows a user interface 1812 of an agent builder application 1800 in several embodiments. The user interface 1812 includes a form builder user interface element 1814 for interacting with (e.g., instantiating and / or configuring) an agent module 6102 in several embodiments. In some embodiments, the user interface 1812 includes a global user interface element that resides within each different user interface of the agent builder application, as described herein. For example, user interface 1812 includes user interface elements for accessing different user interfaces of agent builder application 1800 (for example, user interface element 1802 for accessing the home user interface, user interface element 1804 for accessing the agent builder user interface, user interface element 1806 for accessing the data viewing user interface, such as user interface 500, as described in relation to Figures 5A and 5B, and user interface element 1808 for browsing a list of task-specific orchestrations (e.g., task-specific agents) available to the user accessing agent builder application 1800). For example, global user interface elements may include prompt user interface element 1810 for starting a chat session with each agent module of agent builder application 1800.

[0149] The user interface 1812 includes several user interface elements for modifying the orchestration 1850 (a task-specific orchestration, which may include, for example, agent module 6102 and / or agent architecture 6106) according to several embodiments. For example, the user interface 1812 includes a user interface element 1814 for naming the orchestration 1850 and a user interface element 1816 for providing a description of the orchestration. In some embodiments, other users who have access to data associated with the orchestration 1850 may access and / or implement the orchestration by selecting it from an agent library (e.g., agent library 226). According to some embodiments, the user interface 1812 also includes a template selector section for interacting with multiple user interface elements corresponding to different default orchestrations, which the user can select and provide to the orchestration 1850 with an initial node architecture 6106 (for example, user interface element 1818A for creating a task-specific orchestration for interacting with a general-purpose machine learning model, user interface element 1818B for interacting with a task-specific orchestration that includes a machine learning model (e.g., a general-purpose machine learning model and / or a task-specific machine learning model) trained on specific data (e.g., from data collection that is continuously updated in real time), and user interface element 1818C for interacting with a task-specific orchestration previously created within a task-specific orchestration creator application).

[0150] As illustrated in the symbol block diagram of Figure 18A, an orchestration 1850 (e.g., a task-specific agent) is instantiated according to a user who provides inputs directed to their respective user interface elements 1818B in order to interact with the task-specific orchestration using data provided by the user (e.g., medical documents, live collected data). In some embodiments, the orchestration 1850 includes one or more agent-level configurations 1852 (e.g., orchestration-level attributes, agent settings) and one or more block-level configurations 1854 (e.g., node-level attributes, model settings).

[0151] As shown in Figure 18A, when the orchestration is instantiated based on user input, user-specific data 1828 is provided to the orchestration 1850. In some embodiments, based on the fact that the orchestration 1850 is instantiated by user input directed to user interface element 1818B, each machine learning model of the task-specific agent is trained (e.g., automatically, without further input provided by the user) on clinician-specific patient data (e.g., precision medicine) based on data associated with each user accessing the task-specific orchestration creator application 1800.

[0152] Figure 18B illustrates the user interface 1820 (task builder form user interface elements) of the agent builder application 1800, which includes multiple user interface elements for modifying the orchestration 1850 instantiated in Figure 18A. According to some embodiments, the user interface 1820 of the task-specific orchestration creator application 1800 includes a set of user interface elements (e.g., form user interface view 1822, workflow user interface view 1824) for toggling each representation of the task-specific orchestration 1850. The user interface 1820 includes a set of user interface elements 1826A for modifying aspects of the orchestration 1850. According to some embodiments, several interactions with the set of user interface elements 1826A modify the agent-level configuration 1852, while other interactions with the set of user interface elements 1826A modify the block-level configuration 1854. For example, a user selection of an agent type (e.g., API broker agent) may modify one or more agent-level configurations 1852 without causing any modification to any block-level configuration 1854.

[0153] Figure 18C illustrates another embodiment of the user interface 1820 of the agent builder application 1800, which includes a set of user interface elements 1826B for modifying the orchestration 1850. For example, the set of user interface elements 1826B includes user interface elements for configuring a machine learning model (e.g., a large-scale language model) of the orchestration 1850, which may result in modifications to the block-level configuration 1854 of the orchestration 1850. According to some embodiments, the task-specific machine learning model is the agent module 6102 of the orchestration 1850.

[0154] Figure 18D illustrates an embodiment of the user interface 1830 (e.g., a workflow editor) of the agent builder application 1800, including a workflow representation 1832 of the orchestration 1850. As indicated by user interface elements 1822 and 1824 (where user interface element 1824 is shown as being selected instead of user interface element 1822), the workflow representation 1832 can be presented based on user input toggles between a form view and a workflow view. In some embodiments, Figure 18D illustrates a transition from Figure 18C in response to a user selection of the workflow UI element 1824.

[0155] As depicted in Figure 18D, the workflow representation 1832 is constructed to represent the node architecture 6106 of the orchestration 1850, which can be based on agent-level configuration 1852 and block-level configuration 1854. In some embodiments, all workflow representations 1832 configurable by the agent builder application 1800 include blocks 1834A and 1834B, respectively, representing the inputs and outputs of the orchestration 1850. For example, the input represented by block 1834A may include the text content of a prompt and / or embedding generated based on the text content of a prompt (as described with respect to the exemplary applications and orchestrations described with respect to Figures 18A and 18B). The workflow representation 1832 may also include one or more blocks representing machine learning models (e.g., block 1834C representing a large language model configured in Figure 18C). In some embodiments, the workflow representation 1832 is an interactive representation (e.g., a drag-and-drop representation) in which the user can select an output and then select an input to combine the inputs into an output (or vice versa). According to some embodiments, each input and output has a corresponding data type (e.g., indicated by color and / or label). In some embodiments, the system provides recommended building blocks (e.g., agent modules, models, tools, and / or other types of building blocks) based on user prompts. In some embodiments, the system provides a list of available building blocks, and the user may add them to agent modules by dragging and dropping the building blocks into a workflow representation.

[0156] In some embodiments, the agent building blocks described herein include data building blocks, operator building blocks, and / or tool building blocks. Non-limiting embodiments of data building blocks include agent list blocks (e.g., retrieving a list of available agents), input blocks (e.g., accepting values ​​from a user), message blocks (e.g., returning recent messages (and optionally related metadata) from a conversation), output blocks (e.g., returning responses such as messages or documents), history blocks (e.g., returning message history), ingest blocks (e.g., ingesting data such as documents from a database or collection), and semantic blocks (e.g., identifying semantically similar documents and / or texts). Non-limiting embodiments of operator blocks include storage blocks (configured, for example, to store bits of data and / or set common data values ​​of various types), array blocks (configured, for example, to convert inputs into arrays (e.g., combines)), map blocks (configured, for example, to perform subassemblies on inputs in an array and return a resulting array), JSON blocks (configured, for example, to convert input text into objects via JSON parsing and optionally validate against a provided scheme), XML blocks (configured, for example, to convert input text into objects via XML parsing and optionally validate), status blocks (configured, for example, to provide information about the execution status), template blocks (configured, for example, to output text according to a given template), and tool blocks (configured, for example, to wrap assemblies that are consumed by other blocks).Non-exclusive embodiments of toolblocks include, for example, an agent toolblock (configured to work with an agent module), a similarity block (configured to provide similarity scores to documents), a web block (configured to act as an HTTP interface), and a model tool interface block (configured to work with a model and a tool, e.g., to request the model to use a tool).

[0157] The workflow representation 1832 in Figure 18D illustrates an exemplary super-agent module configured to understand agent building blocks and assemblies (e.g., which building blocks are available, what each building block is configured to do, which data collections are available, and how to connect the individual building blocks and collections). In the embodiment of Figure 18D, a large language model block 1834C provides prompts that provide the assembly (orchestration) with functionality to meet user requirements.

[0158] Figure 18E illustrates another embodiment of user interface 1830 that includes a workflow representation (e.g., workflow representation 1832 after instantiation of several additional blocks, e.g., including one or more nodes). As shown in Figure 18E, user interface 1830 may include several different user interface elements corresponding to each of the nodes of a task-specific orchestration. For example, the representation of orchestration 1850 shown in user interface 1830 includes block user interface element 1850A corresponding to user input to task-specific orchestration 1842. The representation of task-specific orchestration further includes block user interface element 1850B corresponding to the data input component of task-specific orchestration 1842 (e.g., the specification of the relevant document for use in one or more other nodes of the task-specific orchestration). The representation of a task-specific orchestration further includes a block user interface element 1850C corresponding to a templated prompt preprocessing component of the task-specific orchestration 1842, the templated prompt preprocessing component receiving prompts from the user and determining output prompts (e.g., transformed and / or reformatted prompts) to provide to the task-specific machine learning model of the task-specific orchestration (e.g., a large language model represented by block user interface element 1850E).

[0159] As illustrated by the exemplary representation of task-specific orchestration 1842 in Figure 18E, different blocks representing different nodes of task-specific orchestration 1842 may include their respective visual indicators (e.g., visual indicator 1852B for block user interface element 1850B) that differ from other visual indicators of other blocks, in the representation shown in Figure 18E. In some embodiments, a first visual indicator (e.g., a red bar, an alert symbol, or other type of indicator) indicates that a block lacks a connection (or other integration requirement) that would otherwise integrate well with orchestration 1850. In some embodiments, the visual indicator is used to indicate the block type (e.g., a tool block, a model block, a routing block, etc.). Figure 18E includes blocks 1834D (e.g., a template block) and 1834E (e.g., an output block).

[0160] Figure 18F illustrates another embodiment of the user interface 1830 in which the user provides prompts for text input within the chat user interface 1838, and the chat user interface is configured to facilitate interaction between the user and the orchestration 1850 that the user modifies via the user interface 1830. According to some embodiments, when the user edits the orchestration 1850 and / or when data in a data source that electronically communicates with the orchestration 1850 is updated, the user interface 1838 responds to the update of the orchestration 1850 and / or the data that electronically communicates with the orchestration 1850. For example, the user may provide a first prompt for a first configuration of the orchestration 1850 and receive a first response. Next, after editing orchestration 1850 (for example, by adding a block corresponding to another machine learning model), the user may provide a second prompt for a second configuration of the orchestration, the second prompt being identical to the first prompt, but orchestration 1850 providing a different response to the prompt based on the modifications to the configuration of orchestration 1850.

[0161] Figure 18G illustrates a workflow 1890 for generating and executing an orchestration according to several embodiments. According to some embodiments, a first block (e.g., block 1892) is tasked with retranslating a user query into a statement requesting functionality (e.g., a tool) that satisfies the user query. In some embodiments, the statement requesting functionality is provided to an agent builder module (e.g., module 1894), e.g., a super-agent module, configured to generate an assembly (orchestration). For example, the agent builder module may have access to a block assembly language and a list of available building blocks. In some embodiments, the agent builder module is configured to generate a JSON assembly. In some embodiments, the generated assembly is passed to a validation module configured to validate, streamline, and / or revise the generated assembly. In some embodiments, if the generated assembly fails validation, it is returned to the agent builder module for revision. In some embodiments, the validated assembly is passed to an execution block (e.g., block 1896) configured to execute the assembly. For example, the validated assembly and user query (or follow-up query) are provided to an execution block to obtain a response to the user query. In some embodiments, the response is provided to a formatting block (e.g., block 1898) (e.g., to generate a natural language response).

[0162] In some embodiments, the superagent module includes a machine learning model (e.g., a large language model) with corresponding prompts describing building blocks, assemblies, agents, and / or data collection. In some embodiments, the superagent module is provided with prompts including a list of building block types, as well as descriptions of each functionality, input type, and output type. In some embodiments, the superagent module is provided with prompts indicating guidelines for generating assemblies, orchestrations, and / or agent modules. In some embodiments, the superagent module is provided with prompts including one or more exemplary assemblies, orchestrations, and / or agent modules.

[0163] Figures 19A and 19B illustrate exemplary workflow representations 1900 of a composite orchestration (e.g., a super-agent or chain agent) configured to construct an orchestration (e.g., each different agent module 6102, including other composite orchestrations (e.g., a chain of agents)) based on user prompts, according to several embodiments. In some embodiments, the composite orchestration is configured to determine one or more other orchestrations (e.g., agent modules 6102) to use to provide responses based on input received by an application (e.g., from a client device 102 or a user). In this way, the composite orchestration may be configured to generate responses to prompts corresponding to multiple task-specific and / or domain-specific tasks without being trained (tuned) for each individual task or domain of the multiple task-specific and / or domain-specific tasks.

[0164] In some embodiments, one or more blocks of the workflow representation 1900 representing a composite orchestration are configured to provide multi-agent routing capability for the composite orchestration. For example, a parsed JSON block for a composite orchestration contains a list of agent IDs that the composite orchestration can select to create a task-specific orchestration based on a prompt provided by the user. In this embodiment, a list of agents to use block is configured to generate an array of agent names and descriptions from the list of agent IDs. A system prompt template among the routing agents is configured to format the array for use by the LLM. Another agent to use block is configured to receive an agent ID and invoke the agent corresponding to the identified agent ID.

[0165] In some embodiments, by using a list of agent IDs, the agent builder application can create a message to pass to the machine learning model, which, along with a user-provided prompt, allows the composite orchestration to determine which agent is appropriate to respond to a particular query (e.g., a query determined based on the identified intent of the user prompt).

[0166] According to some embodiments, after a set of task-specific orchestrations (e.g., task-specific agents) has been selected for use by a composite orchestration, a routing agent may be invoked by the composite orchestration to determine the order and / or flow of data across different composite orchestrations (e.g., output orchestrations of the composite orchestration) generated by the composite orchestration.

[0167] Therefore, in some embodiments, the composite orchestration is configured to receive input prompts from the user and, based on the input prompts, generate an entire node architecture (e.g., node architecture 6106) to most effectively provide the user with a response based on the prompts. In some embodiments, after the composite orchestration has determined the node architecture, the user is presented with a workflow editor that includes the composite orchestration generated by the composite orchestration and one or more options for editing or revising the generated composite orchestration.

[0168] In this way, agents (agent modules) can be generated and deployed without engineering assistance. For example, an end user (e.g., a medical professional) can generate and deploy an agent by selecting user interface elements (e.g., clicking a button) and updating values ​​for the agent (e.g., generating a configuration file). This low-code / no-code editor allows agents to be developed with functionality equivalent to traditional programming languages, but without the need to manually enter / edit code. In addition, the agent builder includes security measures such as user authentication and authorization requirements, enforcement of data types between components, and data anonymization. In some embodiments, the agent builder allows the user to select a specific model or model version. In some embodiments, the agent builder recommends a specific model or model version based on the user's expressed intent, system capabilities, and / or relevant data sources. In some embodiments, the agent builder utilizes an assembly language that is Turing complete (and independent of any type attributable to an agent module). For example, agent modules are generated and deployed without the need to classify or assign module types.

[0169] An exemplary cohort building process involves receiving a user query in a cohort agent module. In this embodiment, the cohort agent module maps the query to a set of cohort criteria (e.g., inclusion / exclusion (IE) criteria). In some embodiments, the cohort agent module uses a model (e.g., an LLM trained to understand the mapping of queries to IE criteria) to map the query to the set of cohort criteria. In this embodiment, the cohort agent module maps the IE criteria to a set of filters. In some embodiments, the cohort agent module uses a second model (e.g., an LLM trained to understand the mapping of IE criteria to filters) to map the IE criteria to a set of filters. In this embodiment, the cohort agent module identifies each filter value for the set of filters. In some embodiments, the cohort agent module uses a third model (e.g., an LLM trained to understand filter value mapping) to identify each filter value.

[0170] In some embodiments, the cohort building process further includes a second cohort agent module configured to receive user queries and identify specific concepts (e.g., drug, assay, diagnostic, etc.) from the user queries. In this embodiment, the second cohort agent module is configured to identify one or more filters (and corresponding filter values) for each specific concept. In some embodiments, the second cohort agent module uses a model to identify one or more filters (and corresponding filter values) for a particular concept. For example, the second cohort agent module may use an LLM trained to understand diagnostic and / or drug concepts to identify the corresponding filters. In some embodiments, the second cohort agent module includes a block configured to examine an ontology tree in detail to match specific concepts in user queries with ontologities.

[0171] In some embodiments, the cohort building process further includes a third agent module configured to receive and match filters and filter values ​​from the cohort agent module. For example, the third agent module may include a model (e.g., LLM) trained to understand filter overlap and extension. In some embodiments, the third agent module is configured to provide the matched filters and corresponding filter values ​​to the user (e.g., the user who submitted the user query).

[0172] Figure 19C illustrates a comparison of accuracy between an agent architecture solution (e.g., an agent module configured for cohort building) and a single model solution (e.g., a general-purpose large-scale language model). In the single-model solution, the model is provided with a task description and an example illustrating the task of identifying criteria mentioned in user queries. In the agent architecture solution, the agent module is provided with a task description and example (e.g., as in the single-model solution) and is also prompted to generate additional information about the criteria. This additional information is used to interpret the output from the agent module's model (e.g., a language model). Furthermore, the agent module in this embodiment is configured to use relational information from a medical database (and / or dataset) and map the model output to information from the medical database (and / or dataset) to identify cohort requirements / criteria. In this way, the agent module provides improved accuracy compared to the single model, as illustrated in Figure 19C.

[0173] As shown in Figure 19C, the agent architecture solution offers improved accuracy compared to a single-model solution for several different criterion types. The agent modules described herein can improve accuracy while reducing size and computational requirements. Furthermore, the generic model has no guardrails (e.g., hallucinates), and users cannot specify specific data collection, request specific output formats, or request specific processing workflows. In some embodiments, the agent modules described herein are configured to cite sources when providing responses (e.g., to allow the user to reconfirm conclusions in the response). In some embodiments, a validation block is used to determine whether the conclusions in the response match the cited sources. By example, a document similarity block (e.g., using an embedding model) may be used to determine whether the output response is similar to the cited document and / or input document. In some embodiments, the language model used by the agent module is limited to performing low-precision tasks (e.g., tasks with precision below a given threshold), such as fact generation. For example, the language model may only be employed for high-precision tasks such as natural language processing, query parameter extraction, and document formatting. In some embodiments, the agent modules described herein (e.g., from domain-specific and / or curated data collection) are provided only to appropriate documents (e.g., connected to and / or trained on appropriate documents). In some embodiments, the accuracy of the agent modules is tracked over time, and the agent modules are reconfigured and / or limited for specific tasks based on the tracked accuracy.

[0174] Herein, in order to facilitate channel-based question-and-answer engagement with clients through various agent modules 6102, the details of a platform hosting such agent modules 6102 are described along with various exemplary components and workflows, and flowcharts of the system processes and features in several embodiments are disclosed with reference to Figures 21 to 28.

[0175] Figure 21 is a flowchart illustrating a method 2100 for interacting with task-specific orchestration (e.g., an agent or agent module) according to several embodiments. Method 2100 is implemented in a computing system (e.g., a client device, a server system, and / or a service platform) having one or more processors (e.g., CPU 202 and / or 302) and memory (e.g., memory 218 and / or 310). In some embodiments, the memory stores one or more programs configured to be executed by one or more processors. At least some of the operations shown in Figure 21 correspond to instructions stored in computer memory or a computer-readable storage medium. In some embodiments, the computing system is a platform 100, a client device 102, and / or a server system 106.

[0176] The computing system obtains medical data from one or more data collections (2102). In some embodiments, the computing system obtains other types of data (e.g., in addition to or instead of medical data). In some embodiments, one or more data collections are obtained from one or more databases (e.g., external database 108, database 404, and / or other types of databases).

[0177] The computing system presents a set of user interface elements (e.g., user interface elements illustrated in Figures 18A-18F and Figures 19A and 19B) selected from a plurality of user interface elements in a user interface displayed on a display, each user interface element in the set of user interface elements representing a task-specific orchestration (e.g., agent module) from a plurality of task-specific orchestrations, each task-specific orchestration containing one or more machine learning models that are fine-tuned (or otherwise trained) for a particular task or domain based on medical data from one or more data collections (2104). In some embodiments, the computing system includes a display. In some embodiments, the computing system is communicatively coupled to the display.

[0178] In response to the user's selection of each user interface element representing each task-specific orchestration, the computing system provides at least a portion of the medical data from one or more data collections to each selected task-specific orchestration and presents the user with different user interfaces for communicating with each selected task-specific orchestration (e.g., selected agent modules) (2106). In some embodiments, the different user interfaces indicate a particular task of the task-specific orchestration. In some embodiments, the different user interfaces indicate at least a portion of the medical data.

[0179] In accordance with receiving prompts provided by the user through different user interfaces, the computing system presents a response object, which is generated by a selected task-specific orchestration based on (i) the prompts provided by the user and (ii) at least some of the medical data from one or more data collections (2108). In some embodiments, the response object is presented on the user interface (in addition to, or instead of, being presented on different user interfaces, for example).

[0180] Figure 21 shows several logical stages in a specific order, but the order-independent stages may be rearranged, and the other stages may be combined or separated. The orderings and groupings presented herein are not exhaustive, as several rearrangements or other groupings not specifically mentioned will be obvious to those skilled in the art. Furthermore, it should be recognized that various stages may be implemented in hardware, firmware, software, or any combination thereof.

[0181] Figure 22 is a flowchart illustrating method 2200 for deploying task-specific machine learning models in several embodiments. Method 2100 is implemented in a computing system (e.g., a client device, a server system, and / or a service platform) having one or more processors (e.g., CPU 202 and / or 302) and memory (e.g., memory 218 and / or 310). In some embodiments, the memory stores one or more programs configured to be executed by one or more processors. At least some of the operations shown in Figure 22 correspond to instructions stored in computer memory or a computer-readable storage medium. In some embodiments, the computing system is a platform 100, a client device 102, and / or a server system 106.

[0182] The computing system receives prompts from the user, which are associated with one or more commands and multiple tokens (2202). In some embodiments, the prompts provide initial instructions or commands to the agent module 6102, setting the tone and providing a framework for how the agent module 6102 should navigate the corresponding node architecture 6106. In some embodiments, the prompts include one or more commands or questions presented in Model 228 and / or other information such as context, input, or embodiment, which helps to provide better results. For example, referring briefly to Figure 15, the prompts may be received from one or more client devices in text format (e.g., one or more text strings) and / or audio format (e.g., one or more utterances by the user). In some embodiments, the prompts are received in graphical data format. In some embodiments, the audio and / or graphical inputs are converted by the agent module (e.g., a conversion agent module) into text, vectors, embeddings, or other types of representations.

[0183] In some embodiments, prompts and / or context information are received from one or more client devices 102, an external database 108, and / or an external service 110. In some embodiments, a first prompt is received from a first client device 102, which is a mobile remote device such as a smartphone; a second prompt is received from a second client device through a client application running on the second client device; and a third prompt is received from a third client device dedicated to receiving prompts and providing responses to the user.

[0184] In some embodiments, the prompt received from the user does not define the intent of the query and / or one or more search conditions, but rather provides niche information associated with a particular domain of subject matter. Therefore, rather than applying the prompt to a general model (e.g., Model 228), the system deploys a task-specific agent model associated with a task-specific machine learning model that specializes in understanding the context associated with a particular domain, such as by training within a specific domain using the knowledge database 404. In some embodiments, the prompt is associated with one or more commands and / or one or more tokens. In some embodiments, one or more commands are determined from the prompt, such as by parsing the prompt to obtain one or more commands inferred from the prompt. In some embodiments, one or more commands correspond to the intent of the prompt.

[0185] In some embodiments, a prompt is associated with multiple tokens. In some embodiments, a prompt is parsed, for example, by applying the prompt to an input node 6108 of the node architecture 6106, in order to generate multiple tokens. Those skilled in the art will understand that certain language models have a limited context window and function by predicting subsequent or future tokens based on one or more input tokens, such as multiple tokens associated with a prompt. In some embodiments, parsing a prompt into multiple tokens makes it possible to structure the prompt into a form optimized for input to a particular agent module 6102, model 228, and / or node 6108. Advantageously, multiple tokens provide the ability to depict the prompt into one or more commands represented by different subsets of tokens, which may be provided to different nodes 6108 of one or more agent modules 6102.

[0186] In some embodiments, the multiple tokens include 10 to 100,000 tokens. In some embodiments, the multiple tokens include at least 10 tokens, at least 500 tokens, at least 1,000 tokens, at least 5,000 tokens, or at least 50,000 tokens. In some embodiments, the multiple tokens include up to 10 tokens, up to 500 tokens, up to 1,000 tokens, up to 5,000 tokens, or up to 50,000 tokens. In some embodiments, the multiple tokens collectively represent the entire prompt. In some embodiments, the multiple tokens collectively represent less than all of the prompt. In some embodiments, the multiple tokens include one or more character tokens, one or more subword tokens, one or more word tokens, one or more phrase tokens, or a combination thereof.

[0187] Therefore, by associating multiple tokens with a prompt, the system advantageously allows some or all of the multiple tokens to be applied to different agent modules 6102 associated with different subject domains, for example, a first set of tokens among the multiple tokens is used in an application that uses a first agent module 6102 associated with evaluating formulation information of a first drug composition class and a second agent module 6102 associated with evaluating collective phenotypes.

[0188] The computing system identifies a task-specific machine learning model 228-1 within a plurality of task-specific machine learning models (e.g., model 228) according to a first command in one or more commands (2204). In some embodiments, each task-specific machine learning model is associated with at least one node 6108 within a plurality of interconnected nodes that collectively form a node architecture 6106. In some embodiments, each task-specific machine learning model (e.g., each model 228) defines conditional logic 6112 for performing a particular task of the task-specific machine learning model.

[0189] In some embodiments, each node 6108 within a plurality of interconnected nodes 6108 is associated with a corresponding classification within a plurality of classifications. In some embodiments, the plurality of classifications include functions performed by nodes (e.g., data source nodes, input nodes, output nodes, etc.). In some embodiments, the plurality of classifications include one or more data source classifications, one or more machine learning model classifications, and / or one or more conditional logical classifications.

[0190] In some embodiments, the conditional logic 6112 of each task-specific machine learning model is defined at least partially by different users. In some embodiments, a first user modifies one or more parameters associated with a first node 6108 and stores the corresponding agent module 6102 in the server system 106. In some embodiments, a second user, different from the first user, further modifies at least one parameter in the one or more parameters associated with the first node, thereby enabling the second user to either modify the corresponding agent module 6102 stored in the server system 106 and / or generate additional agent modules 6102 stored in the server system 106, thereby enabling the construction of a library of agents 226 through user-generated modifications.

[0191] Advantageously, by associating each task-specific machine learning module with both the logical 6112 and node 6108 via the agent module 6102, the computing system is able to deploy the task-specific agent module 6102, as well as extend the language model by accessing the external database 108 according to the internal data control flow defined by the logical 6112.

[0192] The computing system applies (2206) some or all of a plurality of tokens to a node 6108-1 within at least one node 6108 associated with a first task-specific machine learning model (e.g., model 228). In some embodiments, the computing system communicates via a communication network with a remote device such as the second client device 102-2 of FIG. 15 and / or the server system 106 of FIG. 1, an access token associated with a user, a first task-specific machine learning model, and / or a corresponding agent module 6102. Advantageously, the access token enables the node 6108-1 to access up-to-date information and data stored remotely from the agent module 6102, such as via a first access token stored in the data module 240. In some embodiments, the node 6108-1 obtains a plurality of restricted data, such as a plurality of personal health information or personally identifiable information associated with a patient, from one or more electronic health records via a communication network. In some embodiments, the node 6108-1 incorporates secure data elements in accordance with authentication of an access token from a source other than the first task-specific machine learning model. As a non-limiting example, a given data source such as the client device 102, an external database 108, and / or an external service 110 may issue an access token such as an application program interface (API) key and a token, where the API key identifies a particular client device 102 belonging to a particular entity and the API token indicates the identity of a source message or content.

[0193] In some embodiments, application includes determining a correlation between a first node and a second node. In some embodiments, the correlation is based on an evaluation of one or more restricted data within a plurality of restricted data, such as one or more biomarkers identified within a plurality of restricted data, one or more phenotypes associated with a plurality of restricted data, or the same. In some embodiments, the second node is interconnected with the first node, which allows for direct communication of data between the first and second nodes. In some embodiments, the second node is indirectly interconnected with the first node such that one or more additional nodes interpose between the first and second nodes. In some embodiments, if the correlation between the first and second nodes satisfies a threshold condition of the first node, method 2400 includes generating a plurality of text data distinct from a prompt, the text data responding to the prompt.

[0194] In some embodiments, if the correlation between the first node and the second node does not satisfy the threshold condition of the first node, the computing system repeatedly applies some or all of the tokens from the plurality of tokens to the first node. In some embodiments, the computing system repeats applying some or all of the tokens from the plurality of tokens to the first node for at least 10 iterations. In some embodiments, the computing system repeats the application to the first node until a second threshold condition associated with the first node is satisfied. For example, the second threshold condition may be associated with the maximum allowable character length of the response, and the first threshold condition may be associated with the accuracy and precision of the response.

[0195] In some embodiments, determining the correlation between the first node and the second node includes determining one or more vector embeddings associated with the prompt, such as the embedding 6240 of FIG. 14 and / or one or more vector embeddings of the database 6240 of FIG. 8B. In some embodiments, each vector embedding within the one or more vector embeddings is a predetermined vector embedding, which reduces the computational burden compared to the need to generate such embeddings. However, the present disclosure is not limited thereto. In some examples, determining the correlation between the first node and the second node includes identifying one or more data sources associated with the first node and the second node.

[0196] The computing system passes (2208) some or all of the tokens among the plurality of tokens to the second node. In some embodiments, the prompt advances some or all of the tokens from the first node to the second node or applies the output from the first node based on some or all of the tokens to the second node, thereby proceeding through the node architecture.

[0197] The computing system applies some or all of the tokens among the plurality of tokens to the second node, thereby deploying a task-specific machine learning model (2210).

[0198] FIG. 22 shows several logical stages in a particular order, but stages that are not order-dependent may be reordered, other stages may be combined, or divided. Some reorderings or other groupings not specifically mentioned will be apparent to those skilled in the art, so the orderings and groupings presented herein are not exhaustive. Further, it should be recognized that the various stages may be implemented in hardware, firmware, software, or any combination thereof.

[0199] Figure 23 is a flowchart illustrating method 2300 for configuring a task-specific machine learning model in several embodiments. Method 2300 is implemented in a computing system (e.g., a client device, a server system, and / or a service platform) having one or more processors (e.g., CPU 202 and / or 302) and memory (e.g., memory 218 and / or 310). In some embodiments, the memory stores one or more programs configured to be executed by one or more processors. At least some of the operations shown in Figure 23 correspond to instructions stored in computer memory or a computer-readable storage medium. In some embodiments, the computing system is a platform 100, a client device 102, and / or a server system 106.

[0200] The computing system receives requests from users to modify machine learning models configured to perform specific clinical tasks (2302). In some embodiments, requests are generated by the user by selecting and / or arranging graphical user interface elements within a user interface associated with a corresponding node architecture. In some embodiments, the user interface includes an agent builder component within the control plane of the computer system. In some embodiments, requests include multiple text data, including one or more text strings entered by the user. In some embodiments, specific clinical tasks include generating a summary report of a patient's medical records, guiding a patient through a care plan, creating patient care guidelines based on a patient's health profile, identifying patients requiring hospital follow-up, identifying changes in standard treatment for a disease setting, and / or evaluating unstructured data associated with a patient to identify a cohort of similar patients.

[0201] The computing system incorporates a corresponding node architecture (e.g., node architecture 6106) (2304) that defines conditional logic for performing specific clinical tasks by a machine learning model (e.g., model 228) based on the machine learning model. In some embodiments, the corresponding node architecture 6106 is associated with one or more agent modules 6102, each of which is further associated with one or more machine learning models through a plurality of interconnected nodes 6108 associated with the agent module 6102. In some embodiments, the node architecture 6106 defines conditional logic, such as coarse-grained or high-level logic, for performing specific clinical tasks, such as by using the machine learning model. In some embodiments, the conditional logic of the node architecture is executed according to a first sequence of interconnected nodes 6108 from a plurality of nodes. In some embodiments, each node represents a data object (e.g., executable code) that implements a fine-grained or low-level logic function based on the corresponding conditional logic 6112. In some embodiments, the nodes include input nodes configured to parse data elements and / or receive data from other sources, at least one output node for communicating data to other sources and / or generating new data, and / or one or more intermediate nodes connected between the input nodes and at least one output node, wherein the first set of interconnected nodes includes one or more data source nodes, one or more machine learning model nodes, and one or more conditional logical nodes. In some embodiments, one or more data source nodes use Model 228 to employ an ingestion-enhanced generation (RAG) process on zero-shot information such as component 6236 in Figure 14. For example, the computing system may apply the RAG process to the entire patient record, which would allow the entire patient record to be applied to Model 228 with excessive computational load rather than focusing only on a specific type of clinical note.

[0202] In some embodiments, this RAG process is used to analyze clinical references throughout the entire patient record without requiring predefined sections of interest. However, this disclosure is not limited thereto. In some embodiments, the RAG process utilizes one or more vector embeddings, such as a plurality of predetermined vector embeddings, where each predetermined vector embedding is associated with a corresponding text string or snippet. Advantageously, this RAG approach may be more efficient and effective than providing a larger context window to LLM.

[0203] In some embodiments, input nodes are configured to receive prompts from a user associated with a specific clinical task. In some embodiments, input nodes are initial terminal nodes in a node architecture that receive prompts from the user in raw format (e.g., as instruction data). In some embodiments, output nodes are configured to generate responses to prompts from the user based on a task-specific machine learning model associated with the output node. In some embodiments, each machine learning model node within one or more machine learning model nodes is configured to retrieve information corresponding to the acquired prompt using the corresponding domain associated with the respective machine learning model. In some embodiments, each machine learning model node within one or more machine learning model nodes includes one or more parameters and one or more functions for interacting with other nodes in a plurality of interconnected nodes.

[0204] The computing system generates a representation of the corresponding node architecture 6106 for display on a remote device (e.g., client device 102) (2306). In some embodiments, the representation of the corresponding node architecture 6106 shows a graphical representation of one or more edges connecting each node in a plurality of interconnected nodes (e.g., as illustrated in Figures 19A and 19B). In some embodiments, the representation includes a plurality of input features that allow the user to manage and / or configure each node, such as ordering nodes in a plurality of interconnected nodes, removing nodes from a plurality of interconnected nodes, adding nodes from a plurality of interconnected nodes, and modifying one or more parameters of a node. In some embodiments, the presentation includes a first input feature for configuring conditional logic of the corresponding node architecture that allows for coarse-grained or high-level modifications to the corresponding node architecture 6106. In some embodiments, the first input feature allows the end user to rearrange the order of nodes by changing the position of a first node in the representation (e.g., dragging and dropping a node from a first position to a second position), thereby making a node adjacent to a second node. In some embodiments, the representation includes a second input feature for configuring the parameter 6110 of the corresponding node 6108 in a first set of interconnected nodes, which allows for fine-grained or low-level modifications to the corresponding node architecture 6106. In some embodiments, the second input feature allows the end user to modify how they process graphical data from the first modality based on the end user's authentication and / or authentication privileges, and / or the geographical location of the client device 102 associated with the end user.

[0205] The computing system receives a selection of either a first or a second input feature (2308). In some embodiments, the section is received through the input of client device 102, such as through the mouse and / or keyboard of client device 102. In some embodiments, the selection of either the first or second input feature defines a second order of a second set of interconnected nodes from a plurality of nodes. In some embodiments, the selection modifies the first node such that, when a threshold condition is met, the output of the first node is no longer input to the third node rather than the second node. In some embodiments, the representation modifies the node visualization if, when input to the second node, the output of the first node does not satisfy the logic 6112 of the second node. In an example, if a second node is trained on a domain of knowledge database 404 associated with two-dimensional graphical data, and the first node is configured to output three-dimensional volumetric graphical data, the visualization of the first node and / or the second node is modified to visualize that the output does not satisfy a threshold condition requiring the input of two-dimensional data to the second node. In some embodiments, the selection of the first feature may also allow for the placement and / or generation of a third node interposed between the first and second nodes, the third node being configured to splice the three-dimensional data into two-dimensional data, and then satisfy a threshold condition requiring the input of two-dimensional data to the second node.

[0206] Advantageously, the representation provides a visual way to manage and / or configure the structure of node 6108 and the node architecture through multiple input features, forming various configurations of interconnected nodes without requiring extensive coding or computational knowledge from the end user.

[0207] The computing system updates the conditional logic of the corresponding node architecture according to a second order of a second set of interconnected nodes, thereby configuring how the machine learning model performs a particular clinical task (2310). Conveniently, by updating the conditional logic, other users can access the agent module and benefit from the acquired learning provided by the updated conditional logic.

[0208] In some embodiments, the computing system generates a configuration file for the corresponding node architecture. In some embodiments, the configuration file sets up the working environment for the corresponding node architecture 6106 and one or more type-specific machine learning models (e.g., Model 228) associated with the corresponding node architecture.

[0209] Figure 23 shows several logical stages in a specific order, but the order-independent stages may be rearranged, and the other stages may be combined or separated. The orderings and groupings presented herein are not exhaustive, as several rearrangements or other groupings not specifically mentioned will be obvious to those skilled in the art. Furthermore, it should be recognized that various stages may be implemented in hardware, firmware, software, or any combination thereof.

[0210] Figure 24 is a flowchart illustrating a method 2400 for performing a clinical task according to several embodiments. The method 2400 is performed in a computing system (e.g., a client device, a server system, and / or a service platform) having one or more processors (e.g., CPU 202 and / or 302) and memory (e.g., memory 218 and / or 310). In some embodiments, the memory stores one or more programs configured to be executed by one or more processors. At least some of the operations shown in Figure 24 correspond to instructions stored in computer memory or a computer-readable storage medium. In some embodiments, the computing system is a platform 100, a client device 102, and / or a server system 106.

[0211] Advantageously, by deploying task-specific agents (e.g., agent modules 6102 trained for a particular domain), the computing system can respond to prompts and / or requests for information associated with niche domains that are otherwise too computationally complex to be available to untrained models within the domain.

[0212] The computing system receives a prompt (2402). In some embodiments, the computing system communicates with a machine learning model (e.g., Model 228) trained to assist in performing a clinical task, such as by storing Model 228 in the first computing system or via the communication network 104. In some embodiments, a clinical task is selected from the group consisting of (i) generating a summary report of a patient's medical records, (ii) instructing a patient through a care plan, (iii) creating patient care guidelines based on a patient's health profile, (iii) identifying patients who require hospital follow-up, (v) identifying changes in standard treatment for a disease setting, and (vi) evaluating unstructured data associated with a patient to identify a cohort of similar patients. In some embodiments, the clinical tasks include (i) generating a summary report of the patient's medical records, (ii) instructing the patient through a care plan, (iii) creating patient care guidelines based on the patient's health profile, (iii) identifying patients who require hospital follow-up, (v) identifying changes in standard treatment for the disease setting, and (vi) evaluating unstructured data associated with the patient to identify cohorts of similar patients, or a combination thereof.

[0213] In some embodiments, the clinical task is to generate a summary report of a patient's medical records. In some such embodiments, a machine learning model is trained using medical records of other patients other than the patient on which the report is based. In some embodiments, agent module 6102 generates a clinical summary report of all medical records of the patient that are accessible to agent module 6102, such as a first electronic medical record associated with the patient during a first epoch, retrieved from a first secure database, and a second electronic medical record associated with the patient during a second epoch, retrieved from a second secure database. In some embodiments, the computing system generates a clinical summary report when threshold conditions are met when evaluating some or all of the patient's medical records, such as any time when a significant event occurs in the patient's lifecycle, including upcoming appointments, significant changes in patient care, updates to records that alter previous results on the clinical summary report, or a combination thereof. In some embodiments, generating the report further includes communicating the report to a client device 102 associated with the patient and / or a healthcare professional associated with the patient for viewing on a client device 102. In some embodiments, the clinical task includes literature searches, cohort builders, insurance claims builders, patient queries, or handbook queries.

[0214] In some embodiments, the patient query agent module includes a set of input blocks, a set of template blocks, a set of document ingestion blocks, a set of model blocks, and a set of output blocks. For example, a first input block is configured to receive a user query, and a second input block is configured to receive a patient identifier. In another embodiment, a first template block is configured to generate an object (e.g., a JSON object) for the patient identifier, and a second template block is configured to reformat the user query from the first input block (e.g., to determine the command, intent, and / or domain from the query). A third template block may be configured and used to convert information from a document ingestion block into an object (e.g., a JSON object). A first model block (e.g., an LLM model) may be configured and used to answer questions based on information from a document ingestion block (e.g., to answer yes or no questions). A set of output blocks may be used to output data from the document ingestion block, model block, and / or various template blocks.

[0215] In some embodiments, the report provides the patient directly with one or more real-time clinical summaries via a user interface display on a client device, and the report includes information updated with self-reported outcomes, as well as data from external services and / or databases associated with the subject, such as a connected fitness client application. In some embodiments, the report is configured to provide the patient with a diagnosis, track the patient's health data during a third epoch, and / or visualize the health overview in real time, such as through one or more charts or tables in the report.

[0216] In some embodiments, the clinical task is to direct a patient (or other subject) through a first care plan. In some such embodiments, the machine learning model is trained using a second care plan that is different from the first care plan. In some embodiments, the agent module 6102 is configured to continuously obtain up-to-date data from an external source and maintain a database of one or more medical / clinical criteria, guidelines, regulations, information, or combinations thereof by ensuring accuracy with recent test results. In some embodiments, the agent module 6102 includes one or more nodes 6108 that integrate an overall clinical care guide for the patient and / or healthcare provider, such as by obtaining new medical publications from updated clinical information from the subject and / or additional tests associated with the subject and recommending the next step in the care plan or a gap in the care plan.

[0217] In some embodiments, before generating a natural language response to a prompt, the computing system selects a data repository from among a plurality of repositories based on identification of a domain within a plurality of domains associated with the repository of data. In some embodiments, each respective repository of data from among the plurality of repositories is associated with a corresponding domain within the plurality of domains. In some embodiments, the machine learning model is selected by conditional logic from among a plurality of available machine learning models based on the content of the prompt. In some embodiments, the first node includes corresponding logic 6112 that evaluates an intent inferred from the prompt and identifies a corresponding domain associated with the intent. In some embodiments, generating a report of a patient's medical record includes anonymizing personally identifiable information from the patient's medical record according to one or more rules defined by a task-specific machine learning model.

[0218] [[ID=*]] In some embodiments, generating a patient's medical record report includes determining demographic information associated with the patient. In some embodiments, generating a patient's medical record report includes determining the patient's past medical condition. In some embodiments, generating a patient's medical record report includes determining one or more care plans for the patient. In some embodiments, generating a patient's medical record report includes determining one or more therapies administered to the patient. In some embodiments, generating a patient's medical record report includes determining a summary of specific care instructions for the patient.

[0219] In some embodiments, guiding a patient through a care plan includes evaluating one or more clinical publications associated with different care plans. In some embodiments, guiding a patient through a care plan includes performing a patient assessment. In some embodiments, the assessment includes one or more prompts configured to elicit information from the patient. In some embodiments, the assessment includes a biometric assessment of the patient. In some embodiments, the assessment is configured to elicit responses from the patient to notify the agent module 6102, such as whether the subject initiated a care plan and / or the patient's status, e.g., whether the subject initiated a care plan, whether the patient had an adverse reaction to the care plan, whether the patient strictly followed the care plan, whether the patient noticed an improvement in one or more conditions presented by the subject, or a combination thereof.

[0220] In some embodiments, guiding a patient through a care plan involves generating a patient-specifically configured agent module 6102, which allows the patient to interact with the agent module 6102 in a conversational manner, guiding the patient along the care plan, explaining the next steps in the care plan, responding to prompts about follow-up care, or a combination thereof. For example, after a treatment plan has been determined by the agent module 6102 using a first node 6108, the agent module 6102 may respond to specific patient queries using a second node 6108 that provides personalized guidance and evaluates the treatment plan based on other user-specific needs. In some embodiments, the agent module 6102 can be used by the physician associated with the patient to evaluate the recommendations and / or understand the underlying guidelines and individualized data points that led to the recommendations. In some embodiments, the agent module 6102 provides recommendations and human-verifiable support for decisions that derived the recommendations using logic 6112.

[0221] In some embodiments, developing patient care guidelines based on a patient's health profile includes evaluating one or more clinical publications. In some embodiments, developing patient care guidelines based on a patient's health profile includes determining one or more discrepancies between a first therapy and one or more biometric or health parameters associated with the patient's medical records. In some embodiments, developing patient care guidelines based on a patient's health profile includes generating one or more patient-specific charts.

[0222] In response to receiving a prompt, the computing system responds to the prompt and generates a natural language response based on an analysis by a machine learning model of a repository of data deemed relevant to the prompt (2404). The computing system provides the natural language response to a second computing system separate from the computing system (2406). For example, the computing system displays the natural language response on a remote display.

[0223] Figure 24 shows several logical stages in a specific order, but the order-independent stages may be rearranged, and the other stages may be combined or separated. The orderings and groupings presented herein are not exhaustive, as several rearrangements or other groupings not specifically mentioned will be obvious to those skilled in the art. Furthermore, it should be recognized that various stages may be implemented in hardware, firmware, software, or any combination thereof.

[0224] Figure 25 is a flowchart illustrating Method 2500, which enables third-party access to and use of an agent module, according to several embodiments. Method 2500 is implemented in a computing system (e.g., a client device, a server system, and / or a service platform) having one or more processors (e.g., CPU 202 and / or 302) and memory (e.g., memory 218 and / or 310). In some embodiments, the memory stores one or more programs configured to be executed by one or more processors. At least some of the operations shown in Figure 25 correspond to instructions stored in computer memory or a computer-readable storage medium. In some embodiments, the computing system is a platform 100, a client device 102, and / or a server system 106.

[0225] The computing system receives a user identifier and prompts related to the identified clinical task in the user interface (2502). In some embodiments, the user identifier is received in the prompt (e.g., within the same communication). In some embodiments, the user identifier is received before the prompt (e.g., the user identifier is received when the user connects to the computing system communicatively or when the user authenticates with the computing system).

[0226] The computing system determines a set of task-specific components (e.g., agent modules) and a set of databases to which the user identifier has access (2504). In some embodiments, the set of task-specific components and / or databases is determined based on the access level and / or permission associated with the user identifier. In some embodiments, the databases (and / or data within the databases) are subject to different access control lists. In some of these embodiments, the user identifier is checked against the access control list to determine which data the user is permitted to access.

[0227] A computing system selects task-specific components from a set of task-specific components based on a prompt, using a machine learning model trained to select from a set of task-specific components (e.g., a super-agent module) (2506). For example, the machine learning model identifies intents and / or commands from the prompt and selects task-specific components according to the identified intents and / or commands.

[0228] The computing system, based on prompts, connects task-specific components from a set of databases to the database in a communicative manner (2508). In some embodiments, task-specific components are connected to the database in a communicative manner via connection tools or blocks (e.g., API tools).

[0229] The computing system provides prompts to task-specific components (2510). In some embodiments, the computing system provides one or more commands, intents, and / or tokens corresponding to the prompt, rather than the prompt itself.

[0230] The computing system receives a response to a prompt, and the response is generated by a task-specific component using information from a database (2512). In some embodiments, the response is transformed and / or formatted by the task-specific component and / or a front-end component (e.g., an interface agent module).

[0231] The computing system provides a response to the user (2514). For example, the computing system displays (and / or outputs audibly to the user) the response. In some embodiments, the computing system provides the user with a translated (e.g., summarized) and / or reformatted response. For example, the computing system may provide a natural language version of the response.

[0232] Figure 25 shows several logical stages in a specific order, but the order-independent stages may be rearranged, and the other stages may be combined or separated. The orderings and groupings presented herein are not exhaustive, as several rearrangements or other groupings not specifically mentioned will be obvious to those skilled in the art. Furthermore, it should be recognized that various stages may be implemented in hardware, firmware, software, or any combination thereof.

[0233] Figure 26 is a flowchart illustrating method 2600, in several embodiments, for selecting from task-specific machine learning models to address a clinical task. Method 2600 is implemented in a computing system (e.g., a client device, a server system, and / or a service platform) having one or more processors (e.g., CPU 202 and / or 302) and memory (e.g., memory 218 and / or 310). In some embodiments, the memory stores one or more programs configured to be executed by one or more processors. At least some of the operations shown in Figure 26 correspond to instructions stored in computer memory or a computer-readable storage medium. In some embodiments, the computing system is a platform 100, a client device 102, and / or a server system 106.

[0234] In some embodiments, the computing system selects between 10 task-specific machine learning models and 1,000,000 task-specific machine learning models. In some embodiments, the computing system selects from at least 10 task-specific machine learning models, at least 50 task-specific machine learning models, at least 1,000 task-specific machine learning models, or at least 100,000 task-specific machine learning models. In some embodiments, the computing system selects from up to 10 task-specific machine learning models, up to 50 task-specific machine learning models, up to 1,000 task-specific machine learning models, or up to 100,000 task-specific machine learning models.

[0235] In some embodiments, clinical tasks include generating summary reports of a patient's medical records, instructing patients through care plans, creating patient care guidelines based on a patient's health profile, identifying patients requiring hospital follow-up, identifying changes in standard treatment for a disease setting, or evaluating patient-associated unstructured data to identify cohorts of similar patients.

[0236] The computing system receives a prompt from the user (2602). In some embodiments, the prompt includes a plurality of text data, including one or more text strings entered by the user. In some embodiments, the prompt is provided in text format, such as by entering a text string via a keyboard on the client device 102. In some embodiments, the prompt is received from the user by causing the user to identify one or more data sources for obtaining information. In some embodiments, the prompt from the user is received in the form of one or more data files, such as one or more portable document files, one or more text files, one or more two-dimensional graphical images, one or more three-dimensional graphical images, one or more longitudinal datasets, or a combination thereof.

[0237] In some embodiments, the prompt may include a patient identifier, patient attributes, patient test results, patient diagnosis, or a combination thereof. For example, the prompt may include a patient identifier that the end user is looking for in a clinical setting, which allows the method to retrieve patient-related data elements, such as based on a lookup structure associated with the identifier. In some embodiments, patient attributes may allow the computing system to utilize one or more agent modules 6102 trained on various domains specific to the patient. In some embodiments, patient attributes may include demographic information, the patient's medical status, therapies administered to the patient, patient biometrics, patient allergies, patient lifestyle information, and / or similar.

[0238] In some embodiments, prompts are generated by the user by selecting and arranging graphical user interface elements within a user interface associated with multiple task-specific machine learning models and / or machine learning models. In some embodiments, the user arranges graphical user interface elements, each corresponding to a node 6108 further associated with multiple task-specific machine learning models and / or machine learning models.

[0239] In some embodiments, each task-specific machine learning model is selected from among multiple task-specific machine learning models based on differences in the key component analysis of prompts for each of the multiple task-specific machine learning models. In some embodiments, selecting each task-specific machine learning model from among multiple task-specific machine learning models is based on the identification of a first domain through association with prompts. In some embodiments, selecting each task-specific machine learning model includes generating a task-specific machine learning model with conditional logic configured to respond to prompts. In some embodiments, selecting each task-specific machine learning model includes identifying a first classification of machine learning models and selecting each task-specific machine learning model based on association with the first classification of machine learning models. In some embodiments, selecting each task-specific machine learning model includes forming a first sequence for multiple interconnected nodes.

[0240] In some embodiments, the computing system selects at least two task-specific machine learning models from a group of task-specific machine learning models based on a prompt. In some embodiments, the computing system provides some or all of the prompt to each of the at least two task-specific machine learning models selected from the group of task-specific machine learning models. In some embodiments, the computing system receives information from each of the at least two task-specific machine learning models, and the response corresponds to the combination of information from the at least two task-specific machine learning models.

[0241] In some embodiments, the computing system selects a first task-specific machine learning model from at least two task-specific machine learning models as the initial terminal task-specific machine learning model. In some embodiments, the computing system selects a second task-specific machine learning model from at least two task-specific machine learning models as the final terminal task-specific machine learning model.

[0242] In some embodiments, the computing system provides prompts to a first task-specific machine learning model. In some embodiments, the computing system receives information from the first task-specific machine learning model. In some embodiments, the computing system provides information to a second task-specific machine learning model. In some embodiments, the computing system receives a response to the prompt from the second task-specific machine learning model, the response being generated by the second task-specific machine learning model.

[0243] Upon determining that a prompt requests assistance for a clinical task, the computing system selects a task-specific machine learning model from among multiple task-specific machine learning models, each trained to assist one of multiple clinical tasks, based on the prompt (2604). In some embodiments, determining that a prompt requests assistance for a clinical task further includes parsing the prompt into one or more commands, thereby forming the intent of the prompt requesting assistance for a clinical task. In some embodiments, determining that a prompt requests assistance for a clinical task further includes identifying a first domain among multiple domains associated with the intent of the prompt.

[0244] In some embodiments, determining that a prompt requests assistance with a clinical task further includes applying the prompt to a machine learning model to generate a first response that, unlike the prompt, responds to a prompt from the user. In some embodiments, determining that a prompt requests assistance with a clinical task further includes obtaining a first domain among several domains of the input space associated with the prompt. In some embodiments, determining that a prompt requests assistance with a clinical task further includes evaluating the value of the first response. In some embodiments, if the value of the first response satisfies a threshold condition, the first response is communicated to the user via a communication network. In some embodiments, if the value of the first response fails to satisfy the threshold condition, the method includes identifying a first task-specific machine learning model associated with the first domain.

[0245] In some embodiments, when the value of the first response fails to satisfy a threshold condition, the method includes applying the first response and / or prompt to a first task-specific machine learning model to generate a second response that responds to the prompt, unlike the first response.

[0246] In some embodiments, each task-specific machine learning model is trained on a first domain within a group of domains. In some embodiments, each domain within the group of domains includes at least one task-specific machine learning model trained on its own domain.

[0247] The computing system provides prompts to each task-specific machine learning model selected from among a plurality of task-specific machine learning models (2606). In some embodiments, providing prompts to each task-specific machine learning model involves applying the prompts to a first node in a plurality of interconnected nodes, thereby generating responses that respond to the prompts, unlike the prompts themselves. In some embodiments, the first node is associated with a first domain-specific machine learning model in a plurality of task-specific machine learning models, and each task-specific machine learning model in a plurality of task-specific machine learning models is associated with (i) at least one node in a plurality of interconnected nodes, and (ii) defines conditional logic for performing a particular task. In some embodiments, each node in a plurality of interconnected nodes is connected by an edge to at least one node in a plurality of interconnected nodes.

[0248] The computing system receives responses to prompts, and these responses are generated by machine learning models specific to each task (2608).

[0249] Upon determining that the response addresses a clinical task, the computing system provides the response to the user (2610). In some embodiments, the computing system provides the user with a translated and / or formatted version of the response. For example, the computing system provides a natural language version of the response.

[0250] Figure 26 shows several logical stages in a specific order, but the order-independent stages may be rearranged, and the other stages may be combined or separated. The orderings and groupings presented herein are not exhaustive, as several rearrangements or other groupings not specifically mentioned will be obvious to those skilled in the art. Furthermore, it should be recognized that various stages may be implemented in hardware, firmware, software, or any combination thereof.

[0251] Figure 27 is a flowchart illustrating a method 2800 for verifying data compatibility in several embodiments. The method 2800 is implemented in a computing system (e.g., a client device, a server system, and / or a service platform) having one or more processors (e.g., CPU 202 and / or 302) and memory (e.g., memory 218 and / or 310). In some embodiments, the memory stores one or more programs configured to be executed by one or more processors. At least some of the operations shown in Figure 27 correspond to instructions stored in computer memory or a computer-readable storage medium. In some embodiments, the computing system is a platform 100, a client device 102, and / or a server system 106.

[0252] Upon determining that a prompt received based on user input requests assistance for one or more clinical tasks, the computing system uses a machine learning model (e.g., a super-agent module) trained to select from among multiple task-specific components (e.g., agent modules and / or tools) to select a set of task-specific components from among multiple task-specific components based on the prompt (e.g., based on the intent and / or command of the prompt) (2802).

[0253] The computing system acquires orchestration data (e.g., operational parameters, objectives, domains, tasks, and / or other types of orchestration data) for a set of task-specific components, each task-specific component within the set of task-specific components being configured to support each of one or more clinical tasks (2804).

[0254] The computing system determines from the orchestration data at least one data conformity criterion (e.g., input data type, data quality requirement, or other criterion) for clinical task data related to one or more clinical tasks (2806). In some embodiments, the at least one data conformity criterion corresponds to a transformation applied to the clinical task data.

[0255] The computing system receives clinical task data (2810). For example, clinical task data includes image data, text data, audio data, and / or other types of data. In some embodiments, clinical task data is received from one or more databases (e.g., database 108 and / or database 404). In some embodiments, clinical task data is acquired using one or more interface agent modules.

[0256] In accordance with the determination that clinical task data does not meet at least one data suitability criterion, the computing system provides the user with a notification indicating that one or more clinical tasks cannot be performed using the clinical task data (2812). For example, the notification may indicate that the clinical task data is of the wrong type or is of insufficient quality to be used to perform one or more clinical tasks.

[0257] Figure 27 shows several logical stages in a specific order, but the order-independent stages may be rearranged, and the other stages may be combined or separated. The orderings and groupings presented herein are not exhaustive, as several rearrangements or other groupings not specifically mentioned will be obvious to those skilled in the art. Furthermore, it should be recognized that various stages may be implemented in hardware, firmware, software, or any combination thereof.

[0258] In some embodiments, a task-specific agent module includes one or more machine learning models (e.g., a language model, a transformer model, etc.). In some embodiments, each machine learning model is restricted to a specific set of domains, a specific set of databases, and / or a specific type of data. In some embodiments, each machine learning model is fine-tuned on a specific set of domains and / or a specific set of data. In some embodiments, a task-specific agent module consists of a machine learning model, short-term and / or long-term memory, an action space, a decision tool, and / or action procedures (e.g., internal and / or external actions). In some embodiments, a task-specific agent module consists of a set of blocks that are joined together to perform more complex actions (e.g., each block has set functionality). For example, a set of blocks may include a block configured for document collection, a block configured for document segmentation, a block configured for segment analysis, a block configured for data transformation, and / or a block configured for response formatting. In some embodiments, a first agent module consists of one or more other agent modules. In some embodiments, each block is configured to perform a single function (e.g., document ingestion, string formatting, model invocation, API invocation, data routing, etc.). For example, a string template block may be configured to convert an input string into one or more output variables.

[0259] In some embodiments, task-specific agent modules are configured to ingest and analyze data and generate corresponding content (e.g., articles, summaries, reports, explanations, insights, etc.). In some embodiments, the output of the task-specific agent module is individualized / customized for a specific user or subject (e.g., a patient). For example, a task-specific agent module accesses information about a specific user or subject and uses that information to provide content directed at that user or subject (for example, a task-specific agent module is configured to prepare a letter on behalf of a user or subject using information from one or more databases (e.g., a medical database and / or a legal database) and information about the user or subject. In some embodiments, the information about the user or subject includes information from one or more documents (e.g., patient records) and / or information about previous interactions with the task-specific agent module (or with other task-specific agent modules in the system).

[0260] In some embodiments, a task-specific agent module is configured to interact with the user (for example, a front-end agent module configured to interact using natural language). In some embodiments, a task-specific agent module is configured to take in and summarize documents. In some embodiments, a task-specific agent module is configured to take in, revise, and / or generate code in accordance with user requests and / or prompts. In some embodiments, a task-specific agent module is configured to perform data transformation and / or data validation. For example, a task-specific agent module is configured to convert visual data into text data. In some embodiments, a task-specific agent module is a super-agent module configured to determine the appropriate agent module and / or tool to respond to a particular prompt or type of prompt. In some embodiments, a task-specific agent module is a routing agent module configured to route one or more subprocesses corresponding to a prompt to the appropriate task-specific agent module and / or tool. In some embodiments, a super-agent module is configured to identify multiple domains and route commands and / or requests according to each of those domains.

[0261] In some embodiments, task-specific agent modules include input-output agent modules, input-model-output agent modules (e.g., using LLM or other model components), document capture agent modules (e.g., configured for RAG-based document capture), logical routing agent modules (e.g., incorporating one or more programming languages), super agent modules (e.g., using one or more additional agent modules for a particular task or subtask), recursive agent modules (e.g., configured to recursively perform an action or function until a condition is met), filter agent modules (e.g., configured to identify and / or adjust search filters based on user prompts), and language agent modules (e.g., configured to translate an input language (e.g., a programming language or natural language) into a different output language).

[0262] In some embodiments, the generated task-specific agent module (e.g., task-specific orchestration) is provided to the user via a digital assistant function. For example, the digital assistant is provided with information about the task-specific agent module and is configured to interact with the task-specific agent module (e.g., retrieve and pass information) when a prompt with the appropriate intent is received. In some embodiments, the generated task-specific agent module is provided to the user via a third-party API service. For example, the task-specific agent module is presented within a user interface and is directly selectable by the user of the user interface.

[0263] Various exemplary embodiments and aspects of this disclosure are described below for convenience. These are provided as examples and do not limit the subject art. Some of the examples described below are illustrated with respect to the drawings disclosed herein for illustrative purposes only, without limiting the scope of the subject art.

[0264] (A1) In one embodiment, several embodiments include a method for configuring a task-specific agent (e.g., Method 1300). In several embodiments, the Method is implemented in a computing system (e.g., Platform 100, Client Device 102, or Server System 106). The Method includes (i) receiving a request from a user (e.g., via the Agent Builder in Figure 6) for an agent configured to perform a particular task; (ii) identifying a first agent type from a set of agent types based on one or more requirements for performing the particular task in response to the request, wherein each agent type in the set of agent types corresponds to a respective language model; and (iii) generating a model component having the first agent type, wherein the model component (e.g., a system prompt of a large language model represented by block 1834C is "You are a helpful AI (iv) Generating an implementation component for an agent, wherein the implementation component is configured to communicatively combine model components into a set of components based on one or more requirements for performing a particular task, and the set of components includes one or more of the following: a set of data sources (e.g., the conversation history component of the LLM message builder in Figure 19A), a set of tools (e.g., an array of agent descriptions output by the used agent list component in Figure 19A), and a set of output components; and (v) Deploying the agent to a working environment (e.g., the control plane of client device 102 in Figure 6), wherein the agent includes the model components and the implementation component.In some embodiments, the first agent type is identified based on one or more user characteristics (e.g., data type and / or work performed by the user). In some embodiments, the system identifies a set of agent types based on one or more user characteristics and presents the set of agent types to the user. In some of these embodiments, the system receives from the user a selection of the first agent type from the presented set of agent types.

[0265] For example, specific tasks may include assisting end users in expanding queries (e.g., by comparing embeddings corresponding to user input prompts with one or more vector embeddings in embedding 6240 of Figure 14), providing guidance / needs related to specific conditions or treatments, answering questions about treatment methodologies, assisting end users in identifying suitable data sources for queries or requests, assisting end users in specifying specific target cohorts, and / or providing persona-specific guidance. Exemplary tasks include identifying molecular landscapes, identifying relevant codes, and suggesting subsequent analyses.

[0266] (A2) In some embodiments of A1, the first agent type is selected from a set of agent types (as previously described with respect to Figures 7A-7E, for example) based on the divergence of the major component analysis for each agent type. For example, label-dependent spectra from the output of pre-trained models may be compared using the Jensen-Shannon (JS) divergence of the output spectra decomposed by major component analysis (PCA). Models better suited to downstream tasks may have a larger JS divergence. The final downstream performance of the fine-tuned models may also correlate with the JS divergence. For example, the PAC JS divergence may be used to determine the potential downstream performance of each model for a particular classification task.

[0267] (A3) In some embodiments of A1 or A2, at least one of the components is a second agent, which is configured to perform a second task. For example, a chain of agents (e.g., each orchestration in a composite orchestration) can be configured to communicate with each other to complete a particular task (for example, the block representing the agent router in representation 1900 in Figure 19A shows the connection between the agent router orchestration and the LLM message builder orchestration). In some embodiments, the agent includes two or more models, each of which is configured to operate on different types of data. For example, the output from a large language model represented by block 1834C in Figure 18D can be provided as input to another large language model to perform a different task.

[0268] (A4) In some embodiments of A1 to A3, the set of components includes a set of data sources, and the set of data sources includes a vector database (e.g., vector database 6240). In some embodiments, the set of data sources includes one or more of the following: a document index, a static corpus, a dynamic corpus, a set of document chunks, a set of document embeddings, and document metadata (e.g., document classification). In some embodiments, each data source in the set of data sources has a corresponding application programming interface. As an example, the set of data sources may include an external database 108, a data module 240, a server data module 330, and / or database 400.

[0269] (A5) In some embodiments of A1 to A4, the set of components includes a set of output components, and the set of output components includes an interactive console. In some embodiments, the set of output components includes an interactive user interface, such as a digital assistant user interface (as illustrated, for example, in Figures 10A to 10C, 11A to 11C, and 12A to 12D).

[0270] (A6) In some embodiments of A1 to A5, the set of components includes a set of tools, the set of tools includes parameters and functions for interacting with other components in the set of components. For example, the set of tools provides a mechanism for the agent to integrate with the outside world (e.g., other systems and components). In another embodiment, the tools may include several parameters specified when configuring the agent, and several parameters that may be specified by the agent itself at the time of invocation. The tools may be general-purpose or custom-built for a particular integration. In some embodiments, different agent types have different access to the tools. For example, a first type of agent consists of a set of available tools, and the language model itself can choose when and how to use them. In this embodiment, a second type of agent follows a fixed set of steps, and the corresponding agent configuration defines when and how the tools are invoked. Exemplary tools were described above with respect to Figure 6.

[0271] (A7) In some embodiments of A6, the set of tools includes one or more authenticated request tools configured to fetch URLs using a user access token, external request tools configured to fetch URLs outside the computing system, and email tools configured to send emails.

[0272] (A8) In some embodiments of A1 to A7, the method further includes generating a configuration file for an agent (as shown in, for example, embodiments 4 and 5 above), the configuration file sets up a working environment and one or more type-specific configuration objects for the agent.

[0273] (A9) In some embodiments of A1 to A8, requests are received via an agent builder component in the control plane of the computing system.

[0274] (A10) In some embodiments of A9, the method further includes, after deploying the agent, (i) receiving an agent configuration update in the agent builder component, and (ii) sending the update information from the agent builder component to the agent.

[0275] (A11) In some embodiments of A1 to A12, deploying an agent involves deploying the agent to an agent host (for example, the agent host described above with reference to Figure 6) within the working environment. For example, the agent host includes front-end components (for example, authentication and user interaction with the deployed agent). The agent host may also include back-end components including a database layer, a document loader, listener components (for example, an update listener and / or a subscription listener), a WebSocket, and / or API access.

[0276] (A12) In some embodiments of A1 to A11, a user request to the agent includes an access token (e.g., an Octa token) corresponding to the working environment.

[0277] (A13) In some embodiments of A12, the method further includes authenticating the agent on one or more of a set of data sources using an access token. For example, the agent forwards the end user's access token for authentication instead of granting the agent's machine user an authorization role. In some embodiments, the agent is restricted to certain endpoints and / or access methods (e.g., GET requests only), so the agent cannot be used to perform administrative tasks on behalf of a user with write permissions. In some embodiments, the agent is configured with a fixed base URL, so the agent cannot be used for authenticated requests to other services.

[0278] (A14) In some embodiments of A1 to A13, generating a model component involves selecting several hidden layers for the model of the model component. For example, the number of hidden layers of the model is selected based on the calculated PCA for the output at each hidden layer of the model. For example, the model is reduced as described above with respect to Figures 7A to 7E.

[0279] (A15) In some embodiments of any of A1-A14, the method further includes (i) acquiring one or more documents via an ingestion component, (ii) extracting text from one or more documents, (iii) acquiring a set of text snippets from the text, (iv) generating a set of embeddings for the set of text snippets, and (v) storing the set of embeddings in a set of data sources (as illustrated, for example, in Figure 9A). For example, one or more documents may include one or more PDF documents. In some embodiments, one or more documents are acquired from one or more medical databases (e.g., external database 108, data module 240, server data module 330, and / or database 400). In some embodiments, the medical databases (i) are owned / operated by different entities, (ii) have different data modalities, and / or (ii) have different data structures and / or formats. In some embodiments, the set of text snippets is tokenized. In some embodiments, the document is multimodal (e.g., including text and images), a first embedding component is used to embed a first type of data, and a second embedding component is used to embed a second type of data.

[0280] (A16) In some embodiments of A1 to A15, the method further includes, after deploying the agent, (i) in the agent receiving a user query from a second user; (ii) generating query embeddings from the user query; (iii) identifying one or more embeddings stored in a set of data sources, wherein one or more embeddings are identified based on a similarity score with the query embeddings; (iv) obtaining information corresponding to one or more embeddings; (v) in the agent generating a natural language response to the user query based on the obtained information; and (vi) presenting the natural language response to the second user (for example, as described above with respect to Figures 9A to 9C). In some embodiments, a first subset of embeddings corresponds to data stored in an electronic medical record, and a second subset of embeddings corresponds to metadata of the electronic medical record. In some embodiments, the similarity score is based on cosine similarity.

[0281] (A17) In some embodiments of A16, the natural language response includes the answer, the rationale for the answer, and supporting evidence for the answer. For example, the user query may be "Is Subject X eligible for Test Y?", and the response may be "Yes, because Subject X's EHR indicates that Subject X meets the inclusion and exclusion criteria for Test Y listed below."

[0282] (A18) In some embodiments of A16 or A17, the method further includes determining user intent from a user query and generating query embeddings based on the user intent (for example, as described above with respect to Figures 9A-9C).

[0283] (A19) In some embodiments of A18, the method further includes identifying one or more tools (e.g., an import tool, an embedding tool, and / or a formatting tool) from a set of tools based on the user's intent, and the information corresponding to one or more embeds is obtained using one or more tools.

[0284] (A20) In some embodiments of A16 to A19, the method further includes identifying a set of parameters from a user query and generating query embeddings based on the set of parameters (for example, as described above with respect to Figures 9A to 9C).

[0285] (A21) In some embodiments of A1 to A20, the set of data sources includes a medical database (e.g., database 400). For example, the medical database stores a set of electronic medical records.

[0286] (A22) In some embodiments of any of A1 to A21, the method further includes, after deploying the agent, (i) providing a user interface to an end user, (ii) receiving a query from the end user via the user interface, (iii) generating a response to the query via the deployed agent, and (iv) providing the generated response to the end user (for example, as shown in Figures 12A-12D). In some embodiments, the query includes identifying the type of data to be retrieved and / or identifying one or more databases to be retrieved.

[0287] (A23) In some embodiments of A22, the user interface is a support interface for medical applications (for example, the user interface illustrated in Figures 11A to 11C).

[0288] (A24) In some embodiments of A22 or A23, the query relates to the concatenation of two data tables, and the response includes information from the table concatenated from the two tables. For example, the agent concatenates two tables and provides the concatenated table and / or data from the concatenated table. In some embodiments, the response includes instructions for concatenating the two data tables (as illustrated, for example, in Figures 12C and 12D).

[0289] (A25) In some embodiments of A22 or A23, the query relates to the interpretation of a data source, and the response contains information about the interpretation of the data source. For example, the query relates to understanding the data model, understanding the columns of the data, and / or understanding how the data was derived.

[0290] (A26) In some embodiments of A22 to A25, the response is generated based on query and context data obtained by the agent. In some embodiments, the context data includes the end user's chat history (as illustrated, for example, in Figures 8A and 8B).

[0291] (A27) In some embodiments of A22-A26, the query is a natural language query, the agent is configured to generate a structured query from the natural language query, and the response is generated based on the structured query. For example, Figure 9C shows how a natural language query is converted into a vector (e.g., by an embedded component of the agent), and that vector is used to search a set of vector databases.

[0292] (A28) In some embodiments of A22-A27, the response includes an answer to the query, the rationale for the answer, and supporting evidence for the answer. For example, the query includes a task for an agent, the agent provides a breakdown of the task (as illustrated in Example 1 above, for example), and performs each part using the provided tools.

[0293] (A29) In some embodiments of A22 to A28, the agent invokes a function in response to a query, the function invocation includes one or more filters identified based on the query, and the response is generated based on the information obtained from the function invocation.

[0294] (A30) In some embodiments of A1 to A29, the agent is fine-tuned based on data from a set of data sources. In some embodiments, fine-tuning includes providing exemplary queries and instructions on how to respond to each. In some embodiments, fine-tuning includes providing a set of questions and answers (as illustrated, for example, in Figure 8B).

[0295] (B1) In other embodiments, some embodiments include a method for identifying subjects (for example, as members of a target population). In some embodiments, the method is implemented in a computing system (e.g., platform 100, client device 102, or server system 106). The method includes (i) receiving a request from a user to identify subjects that satisfy a set of criteria; (ii) obtaining a set of protocols from the request via a language model component; (iii) generating one or more structured queries based on the set of protocols via a language model component; (iv) sending one or more structured queries to one or more databases via a language model component; and (v) receiving a set of subjects that satisfy the set of criteria from one or more databases in response to the sending of one or more structured queries.

[0296] (B2) In some embodiments of B1, one or more databases include one or more clinical databases, therapeutic databases, and medical databases (e.g., database 400). In some embodiments, one or more databases include one or more datasets and / or data collections (e.g., document collections). In some embodiments, one or more documents store multiple types of documents (e.g., text documents, images, audio files, etc.).

[0297] (B3) In some embodiments of B1 or B2, one or more structured queries include one or more SQL queries (for example, the structured queries shown in the user interface illustrated in Figure 11C). In some embodiments, one or more structured queries are in a data definition language, a data query language, a data manipulation language, a data control language, and / or a transaction control language.

[0298] (B4) In some embodiments of B1 to B3, the set of protocols is obtained by abstracting one or more criteria specified in the request. In some embodiments, one or more protocols include one or more commands and / or one or more parameters.

[0299] (B5) In some embodiments of B1 to B4, the language model component includes two or more task-specific agents. In some embodiments, the task-specific agents are trained using one or more abstraction sheets and / or ground truths. For example, the task-specific agents may be either the task-specific agent modules or orchestrations described herein.

[0300] (B6) In some embodiments of B1 to B5, the set of criteria includes a set of inclusion criteria and a set of exclusion criteria. In some embodiments, the set of criteria includes one or more criteria about the subject (e.g., age, sex, etc.). In some embodiments, the set of criteria includes one or more criteria about the subject's medical condition.

[0301] (C1) In other embodiments, some embodiments include a method for interacting with task-specific orchestration (e.g., method 2100). In some embodiments, the method is implemented in a computing system (e.g., platform 100, client device 102, or server system 106).The method involves (i) acquiring data (e.g., medical data) from one or more data collections and / or databases (e.g., data from database 404, a set of documents provided to the orchestration represented by block 1826B), and (ii) presenting a set of user interface elements selected from a plurality of user interface elements (e.g., a set of user interface elements 1818A to 1818C in Figure 18A, each of which may correspond to a user command for instantiating a particular task-specific orchestration (e.g., orchestration 1850), wherein each user interface element in the set of user interface elements represents a task-specific orchestration (e.g., agent module) from a plurality of task-specific orchestrations, and each of the task-specific orchestrations fine-tunes for a particular task or domain based on data from one or more data collections. The process includes (iii) presenting a set of one or more well-organized machine learning models (e.g., Model 228), (a) providing at least a portion of the data from one or more data collections to each task-specific orchestration (e.g., user-specific data 1828 shown in Figure 18A) and (b) presenting the user with different user interfaces for communicating with each task-specific orchestration (e.g., user interfaces shown in Figure 11), and (iv) presenting a response object in accordance with receiving a prompt provided by the user in the different user interface, the response object being generated by each task-specific orchestration based on (1) a prompt provided by the user and (2) at least a portion of the data from one or more data collections. For example, the user is presented with a set of task-specific agents, and in response to the user selecting a task-specific agent from the set, the task-specific agent is executed and at least some of the data is provided.

[0302] (C2) In some embodiments of C1, the data from one or more data collections includes at least one live data collection that is updated in real time while the user is using the data. For example, input B, represented by block 1834D in Figure 18D, may be a live data collection of clinical data, as indicated by data type indicator 1836. In some embodiments, the agent builder module is communicatively coupled to two or more document collections and configured to restrict access to each document collection based on user authentication data. For example, the agent builder application 1800 may allow the user to access only certain data sources, such as the live collection of clinical data represented by block 1834D, based on a determination that the user is permitted to access clinical data (e.g., based on authentication credentials provided by the user).

[0303] (C3) In some embodiments of C1 or C2, the user interface includes (i) a first user interface element for conducting chats with each customized agent that does not have document referencing capabilities (for example, by selecting user interface element 1818A shown in Figure 18A for a basic AI chat session), (ii) a second user interface element for conducting interactions with customized agents that have the capability to refer to medical documents (for example, by selecting user interface element 1818B in Figure 18B to initialize a chat with a document), and (iii) a third user interface element for replicating customized agents with which the user has previously interacted using a different user interface (for example, by selecting user interface element 1818C in Figure 18C to replicate an existing agent).

[0304] (C4) In some embodiments of C1 to C3, the response object includes a set of query instructions for accessing a portion of the data from one or more data collections (for example, the user interface shown in Figure 12C includes instructions that link the patient master file table and the molecular master file table in R based on the user entering the prompt, "How many copies of that gene are needed to be considered amplified?").

[0305] (C5) In some embodiments of any of C1 to C4, the method further includes presenting a user interface element corresponding to the respective settings of a configuration file for each task-specific orchestration, the configuration file setting up a working environment for each task-specific orchestration and one or more type-specific configuration objects for each task-specific orchestration. For example, user interface element 1826A includes a selectable user interface element for specifying an environment for instantiating an orchestration (for example, an environment named "Alpha" which may correspond to a particular workload plane of node architecture 6106).

[0306] (C6) In some embodiments of any of C1-C5, the method further comprises presenting a second user interface element adjacent to the user interface, which is different from the set of user interface elements, and which is configured to allow the user to add or modify each user interface element of the set of user interface elements that represent the respective task-specific orchestration. For example, user interface 1830 includes a chat user interface element 1838 for chatting with orchestration 1850, while presenting a modifiable workflow representation 1832 that represents orchestration 1850 which the user can modify while interacting with the chat user interface element 1838.

[0307] (C7) In some embodiments of C1 to C6, (i) the set of user interface elements includes a default user interface element corresponding to a default set of data automatically provided to a task-specific orchestration created by the user, and (ii) the default user interface element corresponds to precision medical data associated with the user (e.g., precision medical data related to the user). For example, the user interfaces shown in Figures 10A to 10C include a user interface element corresponding to a trial identifier, the trial identifier being selected to represent data associated with a particular clinical trial.

[0308] (C8) In some embodiments of C7, precision medical data includes a set of patients associated with the user (e.g., a cohort of patients the user is responsible for).

[0309] (C9) In some embodiments of any of C1 to C8, the selected task-specific orchestration is an orchestration for selecting other available agents to perform a particular task, and the response object provided in response to a prompt includes a workflow representation of another task-specific orchestration for performing the task identified based on the input prompt. For example, the representation 1900 shown in Figures 19A and 19B represents a composite orchestration that allows the user to generate an orchestration based on a prompt, and can optionally present a workflow representation of the orchestration generated after the prompt is received.

[0310] (C10) In some embodiments of any of C1 to C9, different user interfaces include affordances for opening either (i) a cohort builder tool or (ii) a table builder tool. In some embodiments, the method further includes presenting different user interface elements within different user interfaces corresponding to each selected builder tool in response to a user selection of an affordance (for example, the user interface shown in Figure 11B includes a section having a cohort builder tool).

[0311] (C11) In some embodiment of any one of C1 to C10, the method further includes, according to receiving a prompt from a user, (i) generating an embedding corresponding to the prompt, and (ii) comparing the embedding corresponding to the prompt with a number of embeddings in a vector database (e.g., vector database 6240).

[0312] (C12) In some embodiments of C11, the embedding is generated based on one of the following: (i) context obtained by task-specific orchestration, and (ii) user conversation history having task-specific orchestration. For example, Figure 14 illustrates a system that determines one or more embeddings 6240 based on prompts received from client device 102.

[0313] (C13) In some embodiments of C1 to C12, the user interface presented to the user for selecting each task-specific orchestration is hosted within a control plane defined by the control plane's access to a first set of data sources and / or users. In some embodiments, different user interfaces presented to the user for interacting with each task-specific orchestration are hosted within a workload plane defined by the workload plane's access to a second set of data sources and / or users, distinct from the first set.

[0314] (C14) In some embodiments of C13, different user interfaces include user interface elements that contain information about the workload plane. For example, user interface element 1826A includes instructions about the work environment (e.g., the workload plane) configured for the orchestration 1850 to unfold.

[0315] (C15) In some embodiments of C1 to C14, the response object includes one or more affordances for accessing the respective data sources. In some embodiments, in response to a user selection of each of the one or more affordances, a different user interface is presented to the user, and the different user interface includes a respective user interface element (e.g., user interface 500 shown in Figures 5A and 5B) for retrieving data in the respective data source corresponding to the respective affordance.

[0316] (C16) In some embodiments of C15, other different user interfaces include (i) a general-purpose language model and (ii) a user interface for interacting with one of the task-specific orchestrations.

[0317] (C17) In some embodiments of C16, task-specific orchestration is fine-tuned using information about each data source corresponding to each affordance. For example, each task-specific agent may be specifically designed to interact with data associated with the user interface 500, so that the user can provide prompts to the chat user interface while interacting with the user interface 500 and gain insights into the data sources that can be analyzed using the provided filters.

[0318] (C18) In some embodiments of C16 or C17, one or more filters are automatically applied when other different user interfaces are presented based on prompts provided by the user. For example, based on the content of the prompt, the user interface shown in Figure 5A may automatically present a set of pre-selected filters based on the content of the user prompt.

[0319] (C19) In some embodiments of C1 to C18, the selected task-specific orchestration is associated with a data source that includes live collection, which is updated in real time while the user interacts with the task-specific orchestration, and the user is given instructions about additional data added to the live collection as it is determined that additional data has been added to the live collection. For example, while the user is interacting with the chat user interface 1838 in Figure 18F, a clinical sample may be added to input B (represented by block 1834B), and a message indicating that the data has been updated based on the additional data may be presented within the chat user interface.

[0320] (C20) In some embodiments of C1 to C19, the desired cohort is determined based on a prompt provided by the user, and the response object provided in response to the prompt includes the number of subjects in the target dataset corresponding to the desired cohort.

[0321] (D1) In other embodiments, some embodiments include a method for modifying the functionality of a task-specific orchestration. In some embodiments, the method is carried out in a computing system (e.g., platform 100, client device 102, or server system 106). The method (i) presents a graphical representation of the structure of a task-specific orchestration for providing information (e.g., medical information) based on data (e.g., medical data) about the task-specific orchestration (e.g., workflow representation 1832 shown in Figure 18D, representation 1900 shown in Figures 19A and 19B) while the user interface of the task-specific orchestration builder is presented on the display of an electronic device (e.g., user interface 1830 shown in Figures 18D to 18F, which may be described herein as a workflow editor user interface), wherein (a) the task-specific orchestration has received a prompt from the user and is relating to the task-specific orchestration The graphical representation includes, (b) presenting a graphical representation which includes (1) a first user interface element that provides input for modifying the composition of a task-specific orchestration, and (2) a plurality of second user interface elements that provide input for modifying each block for performing each intermediate calculation operation of the task-specific orchestration, and (ii) detecting user input directed to one of the first user interface elements among the plurality of second user interface elements, and (iii) modifying the functionality of each task-specific orchestration in response to the detection of user input.

[0322] (D2) In some embodiments of D1, the method further comprises presenting a different user interface in response to detecting a different user input, the different user interface comprising a set of form user interface elements (e.g., user interface element 1826A and / or user interface element 1826B shown in Figures 18B and 18C), where (i) a first subset of the set of form user interface elements is associated with an orchestration level modification for a task-specific orchestration (e.g., a first set of user interface elements for modifying the orchestration level setting for orchestration 1850), and (ii) a second subset of the set of form user interface elements is associated with a block-level modification for a task-specific orchestration (e.g., a second set of user interface elements 1826B for modifying a block-level configuration 1854 of a block associated with a large language model of the orchestration).

[0323] (D3) In some embodiments of D1 or D2, each of the multiple second user interface elements includes a block-level input which includes one or more of the following: a model type, a specifier indicating the type of output, a maximum output length configured to be provided by the block, a specifier indicating one or more types of input documents that the block can take in, a specifier indicating whether to use conversation history from previous conversations between the user and task-specific orchestrations, and a specifier indicating whether metadata should be provided to the output (for example, a user-configurable setting for a large language model, represented by user interface element 1826B in Figure 18C).

[0324] (D4) In some embodiments of D1 to D3, the task-specific orchestration builder user interface includes one or more third user interface elements that show the data flow between two or more respective block-level user interface elements. For example, a connector shown connecting the output of block 1834D to the input for an attachment of block 1834C representing a large language model.

[0325] (D5) In some embodiments of D4, each of the multiple second user interface elements includes instructions based on the determination that each other third user interface element needs to be connected between each second user interface element and each other second user interface element in order to be operablely integrated into a task-specific orchestration. For example, user interface elements 1834D and 1834E include patterns (which may correspond to different colors) indicating that user interface elements 1834D and 1834E must be connected to other blocks in the representation of the task-specific orchestration 1842 in order that the intermediate calculation operation is performed as part of the operation of the task-specific orchestration 1842.

[0326] (D6) In some embodiments of any of D1 to D5, the method further includes presenting a different graphical representation from the graphical representation, in accordance with detecting user input directed to one of the second user interface elements, the other graphical representation corresponding to a different task-specific orchestration corresponding to the second user interface element. For example, the superagent task-specific orchestration represented by representation 1900 may be alternatively presented as a block in another representation of a different task-specific orchestration.

[0327] (D7) In some embodiments of D1 to D6, a block selection user interface element is presented adjacent to the graphical representation, and each block selection user interface element includes an affordance for instantiating an additional second user interface element within the graphical representation of the task-specific orchestration. For example, the user interface 1830 in Figure 18D includes a block selection user interface element for adding a block to the task-specific orchestration 1850.

[0328] (D8) In some embodiments of D7, the user interface corresponds to a portion of the data plane associated with credentials provided before presenting the user interface, and the method further includes determining which affordances to present in the block selection user interface based on the portion of the data plane associated with the credentials.

[0329] (D9) In some embodiments of D1 to D8, each of the multiple second user interface elements includes a data instruction affordance, where the data instruction affordance indicates that the data source associated with each second user interface element includes live collection that is updated in real time.

[0330] (D10) In some embodiments of D1 to D4, the method further includes instantiating block-level user interface elements in a graphical representation in response to user input directed to agent-level user interface elements in the user interface. For example, the user may select agent-level settings for agent types in orchestration 1850, each of which may include a default set of second user interface elements generated when each task-specific orchestration having its respective agent type is instantiated.

[0331] (D11) In some embodiments of D1 to D10, each of the plurality of second user interface elements includes a data instruction affordance representing one or more of the following: (i) an input which is configured to be received by the respective intermediate computation operation associated with each second user interface element, and / or (ii) an output which is configured to be provided by the respective intermediate computation operation corresponding to each second user interface element.

[0332] (D12) In some embodiments of D11, each data indication affordance includes its respective visual characteristic, and each visual characteristic is based on the respective data type of the respective data indication affordance.

[0333] (D13) In some embodiments of D1 to D12, each of the multiple second user interface elements is configured to provide a list of agents identified as relating to prompts provided for task-specific orchestration (e.g., the agent list user interface element shown in Figure 19A).

[0334] (D14) In some embodiments of D1 to D13, the first user interface element includes affordances to indicate whether a task-specific orchestration can be configured to use or process personal health information (PHI). In some embodiments, the task-specific orchestration is configured to recognize whether the information is PHI (e.g., during operation and / or interaction with the user) and to take appropriate action (e.g., to ensure that security measures are active).

[0335] (D15) In some embodiments of D1 to D14, the first user interface element includes affordances to indicate whether the task-specific orchestration is available to other users of the task-specific orchestration builder. For example, the user interface 1830 shown in Figure 19A includes agent settings that allow the user to specify whether the task-specific orchestration being created is visible to internal members of the organization or is privately viewable only by the user.

[0336] (D16) In some embodiments of D1 to D15, the method further includes presenting a chat user interface element adjacent to a graphical representation in response to user input for interacting with a task-specific orchestration represented by a graphical representation, the chat user interface element being configured to allow the user to interact with the task-specific orchestration. For example, Figure 18F shows a user interface 1830 including a chat user interface 1838, which the user can interact with a task-specific orchestration 1850.

[0337] (D17) In some embodiments of D1 to D16, each of the multiple second user interface elements is associated with a task-specific machine learning model, and each second user interface element includes an information affordance indicating a system prompt configured to be provided to the task-specific machine learning model while task-specific orchestration is being used (for example, the system prompt of user interface element 1834C states, "You are a helpful AI assistant").

[0338] (D18) In some embodiments of D17, each second user interface element includes another information affordance indicating conversation history provided as context to a task-specific machine learning model while task-specific orchestration is being used (for example, user interface element 1834C includes a data instruction affordance for providing conversation history as input to a large language model represented by user interface element 1834C).

[0339] (D19) In some embodiments of D17, each second user interface element includes yet another information affordance indicating the type of large-scale language model, which includes a task-specific machine learning model. For example, user interface element 1834C includes an information affordance indicating that the large-scale language model corresponding to user interface element 1834C uses a gpt-4-turbo model.

[0340] (D20) In some embodiments of D1 to D19, each of the multiple second user interface elements includes an information affordance (e.g., a template indication affordance shown within user interface element 1834D) that indicates a template to be applied to the input text received as input to a set of intermediate computation operations associated with the second user interface element.

[0341] (E1) In other embodiments, some embodiments include a method for deploying a task-specific machine learning model (e.g., Method 2200, Model 228, Agent Module 1602, etc.). In some embodiments, the method is implemented in a computing system (e.g., Platform 100, Client Device 102, or Server System 106).In some embodiments, the method includes (i) receiving a prompt from a user, the prompt being associated with one or more commands and / or one or more tokens; (ii) identifying a first task-specific component (e.g., a machine learning model) in a plurality of task-specific components according to a first command in one or more commands, each task-specific component being (a) associated with at least one node in a plurality of interconnected nodes, and (b) defining conditional logic for performing a particular task of the task-specific component; and (iii) applying some or all of the tokens in the plurality to a first node in at least one node associated with the first task-specific component, the application being (1) sending an access token associated with the first task-specific component to a remote device via a communication network. The process includes (1) communicating, (2) retrieving multiple restricted data from sources other than the first task-specific component via a communication network in accordance with access token authentication, and (3) determining a correlation between a first node and a second node based on an evaluation of one or more restricted data within the multiple restricted data, and includes (1) the second node being interconnected with the first node, (2) generating multiple text data different from a prompt when the correlation between the first node and the second node satisfies the threshold condition of the first node, (3) repeatedly applying some or all of the multiple tokens to the first node when the correlation between the first node and the second node cannot satisfy the threshold condition of the first node, (4) passing some or all of the multiple tokens to the second node, and (5) applying some or all of the multiple tokens to the second node.

[0342] (E2) In some embodiments of E1, one or more commands are determined from the prompt (e.g., derived from the prompt, translated from the prompt, etc.).

[0343] (E3) In some embodiments of E1 or E2, one or more commands include the intent of the prompt (e.g., the query associated with the prompt, the last conversation event information in Figure 18F, etc.).

[0344] (E4) In some embodiment of any of E1 to E3, one or more tokens include 10 to 100,000 tokens (for example, at least 100 tokens, at least 1,000 tokens, at least 10,000 tokens, at least 25,000 tokens, at least 40,000 tokens, at least 50,000 tokens, at least 60,000 tokens, at least 75,000 tokens, at least 90,000 tokens, etc.).

[0345] (E5) In some embodiments of E1 to E4, one or more tokens collectively represent the entire prompt (for example, multiple text data associated with a prompt are translated into one or more tokens in which one or more tokens collectively represent all the information in the multiple text data).

[0346] (E6) In some embodiments of E1 to E4, one or more tokens collectively represent less than all of the prompt (multiple text data associated with a prompt are translated into one or more tokens in which one or more tokens collectively represent less than all of the information in the multiple text data).

[0347] (E7) In some embodiments of any of E1 to E6, one or more tokens include one or more character tokens (e.g., each token represents one or more characters associated with a text string), one or more subword tokens (e.g., each token represents one or more or two or more characters associated with a text string), one or more word tokens (e.g., each token represents a word in a text string), or a combination thereof.

[0348] (E8) In some embodiments of E1 to E7, the conditional logic (e.g., logic 6112) of each task-specific component (e.g., agent module 6102) is defined at least partially by a different user (e.g., client device 102 or system server 106).

[0349] (E9) In some embodiments of any of E1 to E8, the method includes determining a correlation between a first node (e.g., first node 6108-1) and a second node (e.g., second node 6108-2). In some embodiments, determining the correlation includes determining one or more vector embeddings associated with a prompt (e.g., vector database 904, embedding 904, etc.).

[0350] In some embodiments of (E10)E9, each vector embedding within one or more vector embeddings is a predetermined vector embedding (for example, generated from an evaluation of a first prompt received from a second user, generated before the prompt is received).

[0351] (E11) In some embodiments of E1 to E10, determining the correlation between a first node 6208-1 and a second node 6208-2 includes identifying one or more data sources (e.g., data module 240, database 400, knowledge database 404, domain, etc.) associated with the first node 6208-1 and the second node 6208-2.

[0352] (E12) In some embodiments of any E1 to E11, the method further includes applying some or all of the tokens of a plurality of tokens to the first node 6208-1 (in accordance with a determination such as whether a threshold condition is met, recursively, etc.) over at least 10 iterations.

[0353] (E13) In some embodiments of E1 to E12, each node 6108 in a plurality of interconnected nodes 6108 is associated with a corresponding classification in a plurality of classifications (e.g., each domain within a plurality of domains, e.g., less than the entire input space, or a portion of the knowledge database 404).

[0354] (F1) In other embodiments, some embodiments include a method for constructing a task-specific machine learning model (e.g., Method 2300). In some embodiments, the method is implemented in a computing system (e.g., Platform 100, Client Device 102, or Server System 106). The method comprises (i) modifying a machine learning model (e.g., Model 228, Agent Module 6108, or Node 6108, etc.) configured to receive requests (e.g., prompts) from a user and perform a specific task (e.g., a clinical task), and (ii) incorporating a corresponding node architecture (e.g., Node Architecture 6106) that defines conditional logic (e.g., logic 6112) for performing a specific task (e.g., Method 2400) based on the machine learning model, wherein (a) the conditional logic 6112 is executed according to a first sequence of a first set of interconnected nodes 6108 from a plurality of nodes 6108, and (b) the first sequence includes an input node 6108, at least one output node 6108, and an intermediate node 6180 disposed between the input node 6108 and at least one output node 6108, and (iii) a remote device (e.g., Platform 100, Client Device To generate a representation of a corresponding node architecture (e.g., user interface 1800 or user interface 1830) for display in vice 102 or server system 106), wherein the representation includes a plurality of input features, the plurality of input features including (I) a first input feature for configuring conditional logic 6112 of the corresponding node architecture 6106 (e.g., orchestration 1850 or agent-level configuration 1852), and (I) a second input feature for configuring parameters (e.g., parameter 6110-1, parameter 6110-2, ..., or parameter 6110-K) of a corresponding node in a first set of interconnected nodes 6108 (e.g., first node 6108-1), and (iv) receiving a selection of either the first or second input features,The first set of interconnected nodes 6108 includes (v) receiving, defining a second order of a second set of interconnected nodes 6108 from multiple nodes 6108, and updating the conditional logic 6112 of the corresponding node architecture 6106 (e.g., conditional logic formed from the set logic 6112 of the nodes of the node architecture 6106) according to the second order of the second set of interconnected nodes 6106. In some embodiments, the first set of interconnected nodes 6108 includes one or more data source nodes 6108 (e.g., associated with data module 240 or knowledge database 404), one or more machine learning model nodes 6108 (e.g., associated with model 228), and one or more conditional logic nodes (e.g., associated with logic 6112 to evaluate inputs and / or to generate outputs based on inputs).

[0355] (F2) In some embodiments of F1, requests are generated by the user by selecting and arranging graphical user interface elements (e.g., orchestration 1850, agent-level configuration 1852, and / or block-level configuration 1854) within a user interface (e.g., user interface 1800 or user interface 1830) associated with the corresponding node architecture 6106.

[0356] (F3) In some embodiments of F2, the user interface includes an agent builder component within the control plane of the computer system (for example, a first plane configured to manage the transmission of data through the communication network 104).

[0357] (F4) In some embodiments of F1 to F3, the request includes multiple text data (e.g., “What is PFS?” in Figure 8A, the text prompt in Figure 9A, the text prompt in Figure 9B, “What is the reason for Linda Watson canceling the order?” in Figure 9C, the query in Figure 10A, the query in Figure 10B, the query in Figure 10C, yes or no in Figure 11A, or the input text in Figure 11B).

[0358] (F5) In some embodiments of any of F1 to F4, a specific clinical task (e.g., Method 2400) includes (i) generating a summary report of a patient's medical records, (ii) instructing a patient through a care plan, (iii) creating patient care guidelines based on a patient's health profile, (iii) identifying patients who require hospital follow-up, (v) identifying changes in standard treatment for a disease setting, or (vi) evaluating unstructured data associated with a patient to identify a cohort of similar patients.

[0359] (F6) In some embodiments of F1 to F5, the input node 6108 is configured to receive prompts from a user associated with a specific clinical task (e.g., method 2400) (e.g., 6108-1 “Extract Text” in Figure 9A, 6108-1 “Extract Text” in Figure 9B, 6108-1 “Embed API” in Figure 9C, or 1834C in Figure 18D).

[0360] (F7) In some embodiments of any of F1-F6, the input node 6108-1 is configured to receive prompts from users associated with one or more specific clinical tasks (e.g., Method 2400).

[0361] (F8) In some embodiments of F1 to F7, the output node 6108 (e.g., node 6108-5 in Figure 9A, node 6108-10 in Figure 9B, or 6108-7 in Figure 9C) is configured to generate a response to a user request (e.g., output, response in Figure 8A, response in Figure 8B, relevant chunk in Figure 9A, response in Figure 9B, reason for PC status in Figure 9C) based on the respective task-specific machine learning model associated with the output node 6108 (e.g., model 228).

[0362] (F9) In some embodiment of F1 to F8, each machine learning model node 6108 within one or more machine learning model nodes 6108 is configured to retrieve information corresponding to a request using a corresponding domain associated with its respective machine learning model (e.g., a domain of the knowledge database 404).

[0363] (F10) In some embodiments of F1 to F9, each machine learning model node within one or more machine learning model nodes includes one or more parameters (e.g., parameter 6110 in Figure 2B) and one or more functions (e.g., logic 6112 in Figure 2B) for interacting with other nodes 6108 within a plurality of nodes 6108 (e.g., a first logic 6112-1 that passes data such as a first token from a first node 6108-1 to a second node 6108-2 adjacent to the first node 6108-1 in the node architecture 6106 when the conditions are met, and / or a second logic 6112-2 that passes data from a first node 6108-1 to a third node 6108-3 adjacent to the first node 6108-1 and the second node 6108-2 when the conditions are met).

[0364] (F11) In any of the embodiments of F1 to F10, the method further comprises generating a configuration file for the corresponding node architecture 6106, the configuration file setting up the working environment for the corresponding node architecture and one or more type-specific machine learning models 228 associated with the corresponding node architecture 6106.

[0365] (G1) In other embodiments, some embodiments include a method for deploying a task-specific machine learning model (e.g., agent module 6102 and / or model 228). In some embodiments, the method is performed on a computing system (e.g., platform 100, client device 102, or server system 106). The method (i) receives a request from a user (e.g., prompt, “What is PFS?” in Figure 8A, text prompt in Figure 9A, text prompt in Figure 9B, “Linda” in Figure 9C) to invoke machine learning model 228 which is configured to perform a specific task (e.g., a specific clinical task and / or method 2400). What is the reason for canceling Watson's order?, receiving the query in Figure 10A, the query in Figure 10B, the query in Figure 10C, the input text in Figure 11A, the yes or no in Figure 11B, etc., where the request is associated with a particular task and includes one or more commands defined by the user, and (ii) taking in a corresponding node architecture (e.g., node architecture 6106) that defines conditional logic (e.g., set logic 6112 of node 6108) for performing a particular task of the task-specific machine learning model 228 based on the machine learning model 228, where (a) the conditional logic 6118 follows the order of the interconnected set of nodes 6108, and (b) the order follows the input node (e.g., first node 61 08-1) including at least one output node, e.g., node 6108-5 in Figure 9A, node 6108-10 in Figure 9B, or node 6108-7 in Figure 9C, etc., and intermediate node 6108 disposed between the input node and at least one output node, (c) each node of the plurality of interconnected nodes is connected to at least one node in the plurality of interconnected nodes (e.g., by an edge, etc.), and (iii) applying one or more commands to the input node in order of the set of interconnected nodes, the application including determining the correlation between the input node and at least one intermediate node in the set of interconnected nodes (e.g., through the order of the set of interconnected nodes,(iv) examining in detail some or all of one or more commands, applying them, and generating one or more data elements associated with the results of one or more commands (e.g., output, response in Figure 8A, response in Figure 8B, associated chunk in Figure 9A, response in Figure 9B, reason for PC status in Figure 9C) based on the process by which one or more commands progress to the output node through a sequence of interconnected nodes, within at least one output node, and (v) communicating one or more data elements (e.g., via communication network 104 for display in a computer system). In some embodiments, the set of interconnected nodes includes one or more data source nodes, one or more language model nodes, and one or more conditional logic nodes.

[0366] (H1) In other embodiments, some embodiments include a method for performing a clinical task (e.g., Method 2400). In some embodiments, the Method is performed on a computing system (e.g., Platform 100, Client Device 102, or Server System 106). The Method (i) prompts (e.g., “What is PFS?” in Figure 8A, text prompt in Figure 9A, text prompt in Figure 9B, “Linda” in Figure 9C) via a machine learning model (e.g., Model 228) trained to assist in performing a task (e.g., a clinical task trained on one or more domains of a knowledge database 404). (ii) receiving the prompt, such as the query in Figure 10A, the query in Figure 10B, the query in Figure 10C, the input text in Figure 11A, and the yes or no in Figure 11B; (ii) in response to receiving the prompt, the first computing system generates a natural language response (e.g., the output, the response in Figure 8A, the response in Figure 8B, the relevant chunk in Figure 9A, the response in Figure 9B, and / or the reason for the PC status in Figure 9C) based on analysis by the machine learning model 228 of a repository of data (e.g., the first domain, part or all of the knowledge database 404, the vector database 904, or the data module 240) that has been determined to respond to and be related to the prompt; and (iii) providing the natural language response to a second computing system different from the first computing system (e.g., the other side of platform 100, a client device 102, or a server system 106). In some embodiments, the tasks are to (i) generate reports of a patient's medical records, (ii) instruct the patient through a care plan, (iii) create patient care guidelines based on the patient's health profile, (iii) identify patients who require follow-up in a hospital, (v) identify changes in standard treatment for a disease setting, or (vi) evaluate unstructured data associated with a patient to identify a cohort of similar patients.

[0367] (H2) In some embodiments of H1, the task generates a summary report of a patient's medical records, and a machine learning model (e.g., Model 228) is trained using medical records of patients other than the patient (e.g., trained with data associated with a first target population from which the patient is excluded).

[0368] (H3) In some embodiments of H1, the task is to guide a patient through a first care plan, and a machine learning model (e.g., Model 228) is trained using a second care plan different from the first care plan (e.g., trained with a lung cancer care plan when guiding a patient through a breast cancer care plan).

[0369] (H4) In some embodiments of any of H1 to H3, the method further includes, before generation, selecting a repository for data (e.g., a first domain, knowledge database 404, vector database 904, or some or all of data module 240) from among multiple repositories (e.g., system database 400 in Figure 4) based on the identification of domains within multiple domains associated with the data repository.

[0370] In some embodiments of (H5)H4, each repository of data from among multiple repositories is associated with a corresponding domain within multiple domains (e.g., a first domain associated with somantic DNA, a second domain associated with germline DNA, a third domain associated with RNA, a fourth domain associated with DNA methylation, a fifth domain associated with therapeutics, ..., and an Nth domain associated with clinical guidelines in Figure 4).

[0371] (H6) In some embodiments of H1 to H5, the machine learning model 228 is selected from among multiple available machine lear...

Claims

1. A method for deploying a task-specific machine learning model, wherein the method is Receiving a prompt from a user, wherein the prompt is associated with one or more commands and multiple tokens, Identifying a first task-specific machine learning model in a plurality of task-specific machine learning models according to a first command in one or more of the aforementioned commands, wherein each task-specific machine learning model is (i) associated with at least one node in a plurality of interconnected nodes, and (ii) defines conditional logic for performing a specific task of the task-specific machine learning model. Applying some or all of the aforementioned plurality of tokens to a first node in the at least one node associated with the first task-specific machine learning model, wherein the application is Communicating an access token associated with the first task-specific machine learning model to a remote device via a communication network, Through the aforementioned communication network, in accordance with the authentication of the access token, multiple restricted data are taken from sources other than the first task-specific machine learning model, The process includes determining the correlation between the first node and the second node based on the evaluation of one or more restricted data within the plurality of restricted data, The second node is interconnected with the first node, When the correlation between the first node and the second node satisfies the threshold condition of the first node, a plurality of text data different from the prompt are generated. When the correlation between the first node and the second node cannot satisfy the threshold condition of the first node, the process of applying some or all of the multiple tokens to the first node is repeated. Passing some or all of the tokens among the plurality of tokens to the second node, A method comprising applying some or all of the tokens among the plurality of tokens to the second node, thereby deploying the task-specific machine learning model.

2. The method according to claim 1, wherein one or more commands are determined from the prompt.

3. The method according to claim 1, wherein the one or more commands include the intent of the prompt.

4. The method according to claim 1, wherein the plurality of tokens include 10 to 100,000 tokens.

5. The method according to claim 1, wherein the plurality of tokens collectively represent the entirety of the prompt.

6. The method according to claim 1, wherein the plurality of tokens collectively represent less than all of the prompts.

7. The method according to claim 1, wherein the plurality of tokens include one, one or more character tokens, one or more subword tokens, one or more word tokens, or a combination thereof.

8. The method according to claim 1, wherein the conditional logic of each task-specific machine learning model is defined at least partially by different users.

9. The method according to claim 1, wherein determining the correlation between the first node and the second node includes determining one or more vector embeddings associated with the prompt.

10. The method according to claim 9, wherein each vector embedding in the one or more vector embeddings is a predetermined vector embedding.

11. The method according to claim 1, wherein determining the correlation between the first node and the second node includes identifying one or more data sources associated with the first node and the second node.

12. The method according to claim 1, further comprising applying some or all of the plurality of tokens to the first node over at least 10 iterations.

13. The method according to claim 1, wherein each node in the plurality of interconnected nodes is associated with a corresponding classification in the plurality of classifications.

14. It is a method, Obtaining medical data from one or more data collections, In a user interface displayed on a display that electronically communicates with a computing system, the presenting of a set of user interface elements selected from a plurality of user interface elements, wherein each user interface element in the set of user interface elements represents a task-specific orchestration from a plurality of task-specific orchestrations, and each of the task-specific orchestrations includes one or more machine learning models that are fine-tuned for a specific task or domain based on the medical data from one or more data collections. In response to user selections of each user interface element representing the specific orchestration for each task, At least a portion of the medical data from one or more of the aforementioned data collections is provided to the respective task-specific orchestrations, and Presenting the user with different user interfaces for communicating with each task-specific orchestration, A method comprising presenting a response object in accordance with receiving a prompt provided by the user in the different user interfaces, wherein the response object is generated by the respective task-specific orchestration based on (i) the prompt provided by the user and (ii) at least a portion of the medical data from one or more data collections.

15. The method according to claim 14, wherein the medical data from one or more data collections includes at least one live data collection in which the medical data is updated in real time while the user is using the medical data.

16. The aforementioned user interface (i) A first user interface element for conducting chats with each customized agent, which does not include document referencing capabilities, (ii) A second user interface element for conducting an interaction with a customized agent capable of referencing medical documents, (iii) The method of claim 14, further comprising a third user interface element for replicating a customized agent with which the user has previously interacted using the different user interface.

17. The method according to claim 14, wherein the response object includes a set of query instructions for accessing a portion of medical data from one or more data collections.

18. The method according to claim 14, further comprising presenting user interface elements corresponding to the respective settings of the configuration files for each of the task-specific orchestrations, wherein the configuration files define the working environment for each of the task-specific orchestrations and one or more type-specific configuration objects for each of the task-specific orchestrations.

19. The method according to claim 14, further comprising presenting a second user interface element adjacent to the user interface, which is different from the set of user interface elements, the second user interface element being configured to allow a user to add or modify each of the user interface elements of the set of user interface elements that represent the respective task-specific orchestrations.

20. The set of user interface elements includes default user interface elements corresponding to a default set of data that is automatically provided to the task-specific orchestration created by the user, The method according to claim 14, wherein the default user interface element corresponds to precision medical data associated with the user.

21. The method according to claim 20, wherein the precision medical data includes a set of patients associated with the user.

22. The aforementioned selected task-specific orchestration is an orchestration for selecting other available agents to perform a specific task. The method according to claim 14, wherein the response object provided in response to the prompt includes a workflow representation of another task-specific orchestration for performing a task identified based on the input prompt.

23. The different user interfaces include affordances for opening one of (i) a cohort builder tool or (ii) a table builder tool, and the method The method according to claim 14, further comprising presenting another user interface element in the different user interface corresponding to each selected builder tool in response to the user selection of the affordance.

24. Further upon receiving the prompt from the user, To generate an embedding corresponding to the aforementioned prompt, The method according to claim 14, further comprising comparing the embedding corresponding to the prompt with a plurality of embeddings in a vector database.

25. The method according to claim 24, wherein the embedding is generated based on one of (i) a context obtained by the task-specific orchestration and (ii) the user's conversation history having the task-specific orchestration.

26. The user interface presented to the user for selecting the respective task-specific orchestration is hosted within the control plane, which is defined by the control plane's access to a first set of data sources and / or users. The method according to claim 14, wherein the different user interfaces presented to the user for interacting with each of the task-specific orchestrations are hosted in a workload plane defined by the user's access to a second set of data sources and / or the workload plane, which is different from the first set.

27. The method according to claim 26, wherein the different user interfaces include a user interface element that includes information about the workload plane.

28. The response object includes one or more affordances for accessing each data source, and the method The method according to claim 14, further comprising presenting to the user another different user interface in response to the user's selection of each of the one or more affordances, wherein the other different user interface includes a user interface element for retrieving data in each data source corresponding to each of the affordances.

29. The method according to claim 28, wherein the other different user interface includes a user interface for interacting with (i) a general-purpose language model and (ii) a task-specific orchestration.

30. The method according to claim 29, wherein the task-specific orchestration is fine-tuned using information about each of the data sources corresponding to each of the affordances.

31. The method according to claim 28, wherein one or more filters are automatically applied when the other different user interface is presented based on the prompt provided by the user.

32. The selected task-specific orchestration is associated with a data source that includes live collection, which is updated in real time while the user interacts with the task-specific orchestration. The method according to claim 14, wherein, upon determining that additional data has been added to the live collection, instructions are provided to the user regarding the additional data to be added to the live collection.

33. The desired cohort is determined based on the prompt provided by the user. The method according to claim 14, wherein the response object provided in response to the prompt includes the number of subjects in the target dataset corresponding to the desired cohort.

34. A method for constructing a task-specific machine learning model in a computer system, To modify machine learning models configured to perform specific clinical tasks, we receive requests from users and The process involves incorporating a corresponding node architecture that defines conditional logic for performing the specific clinical task using the machine learning model, based on the machine learning model. The conditional logic is executed according to a first order of a first set of interconnected nodes from multiple nodes. The first sequence includes an input node, at least one output node, and an intermediate node disposed between the input node and the at least one output node. The first set of interconnected nodes includes one or more data source nodes, one or more machine learning model nodes, and one or more conditional logical nodes, and is capable of ingestion. To generate a representation of the corresponding node architecture for display on a remote device, wherein the representation is A first input feature for configuring the conditional logic of the corresponding node architecture, Representing a plurality of input features, including a second input feature for configuring the parameters of a corresponding node in a first set of interconnected nodes, Receiving a selection of either the first input feature or the second input feature, wherein the selection of either the first or second input feature defines a second order of a second set of interconnected nodes from the plurality of nodes. A method comprising updating the conditional logic of the corresponding node architecture according to the second order of the second set of interconnected nodes, thereby configuring how the machine learning model performs the particular clinical task.

35. The method according to claim 34, wherein the request is generated by the user by selecting and arranging graphical user interface elements within the user interface associated with the corresponding node architecture.

36. The method according to claim 35, wherein the user interface includes an agent builder component within the control plane of the computer system.

37. The method according to claim 34, wherein the request includes a plurality of text data, each containing one or more text strings entered by the user.

38. The aforementioned specific clinical task, (i) To generate a summary report of the patient's medical records, (ii) Instructing patients through care plans, (iii) Develop patient care guidelines based on the patient's health profile. (iii) Identifying patients who require follow-up observation in the hospital. (v) Identifying changes in standard treatment for disease setting, or (vi) The method according to claim 34, comprising evaluating unstructured data associated with a patient to identify a cohort of similar patients.

39. The method according to claim 34, wherein the input node is configured to receive prompts from a user associated with the particular clinical task.

40. The method according to claim 34, wherein the input node is configured to receive prompts from a user associated with the particular clinical task.

41. The method according to claim 34, wherein the output node is configured to generate a response to the request from the user based on a task-specific machine learning model associated with the output node.

42. The method according to claim 34, wherein each machine learning model node within the one or more machine learning model nodes is configured to obtain information corresponding to the request using the corresponding domain associated with each machine learning model.

43. The method according to claim 34, wherein each machine learning model node in the one or more machine learning model nodes includes one or more parameters and one or more functions for interacting with other nodes in the plurality of nodes.

44. The method of claim 34, further comprising generating a configuration file for the corresponding node architecture, wherein the configuration file sets up a working environment for the corresponding node architecture and one or more type-specific machine learning models associated with the corresponding node architecture.

45. It is a method, In a first computing system that communicates with a machine learning model trained to assist in performing a clinical task, receiving a prompt means that the clinical task is (i) To generate reports of patient medical records, (ii) Instructing patients through care plans, (iii) Develop patient care guidelines based on the patient's health profile. (iii) Identifying patients who require follow-up observation in the hospital. (v) Identifying changes in standard treatment for disease setting, or (vi) Evaluating unstructured data associated with patients to identify cohorts of similar patients, receiving, In response to receiving the prompt, the first computing system generates a natural language response based on the analysis of the repository of data determined to be related to the prompt by the machine learning model, A method comprising providing the natural language response to a second computing system different from the first computing system.

46. The method according to claim 45, wherein the one clinical task is to generate a summary report of the patient's medical records, and the machine learning model is trained using the medical records of patients other than the patient.

47. The method according to claim 45, wherein the one clinical task guides a patient through a first care plan, and the machine learning model is trained using a second care plan different from the first care plan.

48. The method of claim 45, further comprising, prior to generating, selecting a repository for the data from among a plurality of repositories based on the identification of a domain in a plurality of domains associated with the repository for the data.

49. The method according to claim 48, wherein each repository of data from among the multiple repositories is associated with a corresponding domain within the multiple domains.

50. The method according to claim 45, wherein the machine learning model is selected by conditional logic from among a plurality of available machine learning models based on the content of the prompt.

51. The method according to claim 45, wherein generating the report of the patient's medical records includes anonymizing personally identifiable information from the patient's medical records in accordance with one or more rules defined by a task-specific machine learning model.

52. The method according to claim 45, wherein generating the report of the patient's medical record includes determining demographic information associated with the patient.

53. The method according to claim 45, wherein generating the report of the patient's medical records includes determining the patient's past medical condition.

54. The method according to claim 45, wherein generating the report of the patient’s medical records includes determining one or more care plans for the patient.

55. The method according to claim 45, wherein generating the report of the patient’s medical record includes determining one or more therapies administered to the patient.

56. The method according to claim 45, wherein generating the report of the patient’s medical record includes determining an outline of specific care instructions for the patient.

57. The method according to claim 45, wherein instructing the patient through the care plan includes evaluating one or more clinical publications associated with different care plans.

58. The method according to claim 45, wherein guiding the patient through the treatment plan includes performing an assessment of the patient.

59. The method according to claim 58, wherein the assessment includes one or more prompts configured to elicit information from the patient.

60. The method according to claim 58, wherein the assessment includes biometric assessment of the patient.

61. The method according to claim 45, wherein developing the patient care guidelines based on the patient's health profile includes evaluating one or more clinical publications.

62. The method according to claim 45, wherein developing the patient care guidelines based on the patient's health profile includes determining one or more discrepancies between a first therapy and one or more biometric or health parameters associated with the patient's medical record.

63. The method according to claim 45, wherein creating the patient care guidelines based on the patient's health profile includes generating one or more charts specific to the patient.

64. Based on the determination that the prompt requires information from at least two machine learning models, The routing involves routing information between a first machine learning model and a second machine learning model, wherein each of the first and second machine learning models is trained to perform a single clinical task. The method according to claim 45, further comprising generating a natural language response based on information from each of the first and second machine learning models.

65. It is a method, In the user interface of a computing device, receiving a user identifier and prompt related to an identified clinical task, Determine the task-specific set of components and the set of databases accessed by the user identifier, A machine learning model trained to select from the aforementioned set of task-specific components selects a task-specific component from the aforementioned set of task-specific components based on a prompt, and Based on the prompt, the task-specific component is communicatively linked to a database from the set of databases, Providing the prompt to the task-specific component, Receiving a response to the prompt, wherein the response is generated by the task-specific component using information from the database. A method comprising providing the aforementioned response to the user.

66. Receiving a second user identifier and the prompt related to the identified clinical task, Determine a second set of task-specific components and a second set of databases accessed by the user identifier, The machine learning model selects a second task-specific component from a second set of task-specific components based on the prompt, Based on the prompt, the second task-specific component is communicably linked to the second database from the second set of databases, Providing the prompt to the second task-specific component, The method according to claim 65, further comprising receiving a second response to the prompt, wherein the second response is generated by a second task-specific component using information from the second database.

67. The method according to claim 65, wherein the set of task-specific components includes one or more task-specific agent modules.

68. The method according to claim 65, wherein the user identifier includes an authentication token for the user.

69. The method according to claim 65, wherein the set of databases includes one or more databases that store data owned by the user.

70. The method according to claim 65, wherein the machine learning model is a component of the superagent module.

71. The method according to claim 65, wherein the task-specific components include an interconnected node architecture.

72. The method according to claim 65, wherein the task-specific component includes a patient query agent, and the database stores information from medical documents provided by the user.

73. The method according to claim 65, wherein each task-specific component in the set of task-specific components has corresponding individual or group-level permission data, and determining which set of task-specific components the user identifier accesses includes comparing the user identifier with the permission data.

74. The method according to claim 65, wherein the task-specific component includes a care gap agent configured to identify gaps in the patient care plan, and the database stores the user's patient care plan data.

75. A method for selecting from task-specific machine learning models to address a clinical task, wherein the method is Receiving prompts from the user, In accordance with the determination that the prompt requests assistance with a clinical task, Based on the prompt, a machine learning model trained to select from among multiple task-specific machine learning models, each trained to support one of multiple clinical tasks, selects a task-specific machine learning model from among the multiple task-specific machine learning models, To provide the prompt to each of the task-specific machine learning models selected from the plurality of task-specific machine learning models, Receiving a response to the prompt, wherein the response is generated by the respective task-specific machine learning model. A method comprising providing the response to the user in accordance with the determination that the response addresses the clinical task.

76. The method according to claim 75, wherein the prompt includes a patient identifier, the patient's attributes, the patient's test results, the patient's diagnosis, or a combination thereof.

77. The method according to claim 75, wherein the prompt is generated by the user by selecting and arranging graphical user interface elements within the user interfaces associated with the plurality of task-specific machine learning models and / or the machine learning models.

78. The method according to claim 75, wherein the prompt includes a plurality of text data, each containing one or more text strings entered by the user.

79. The aforementioned clinical task, (i) To generate a summary report of the patient's medical records, (ii) Instructing patients through care plans, (iii) Develop patient care guidelines based on the patient's health profile. (iii) Identifying patients who require follow-up observation in the hospital. (v) Identifying changes in standard treatment for disease setting, or (vi) The method of claim 75, comprising evaluating unstructured data associated with a patient to identify a cohort of similar patients.

80. The method according to claim 75, wherein each of the task-specific machine learning models is selected from among the plurality of task-specific machine learning models based on the differences in the key component analysis of the prompt for each of the task-specific machine learning models within the plurality of task-specific machine learning models.

81. Based on the prompt, select at least two task-specific machine learning models from the multiple task-specific machine learning models, To provide some or all of the prompts to each of the at least two task-specific machine learning models selected from the plurality of task-specific machine learning models, The method according to claim 75, further comprising receiving information from each of the at least two task-specific machine learning models, wherein the response corresponds to a combination of the respective information from the at least two task-specific machine learning models.

82. As an initial terminal task-specific machine learning model, the first task-specific machine learning model of the at least two task-specific machine learning models, and As a machine learning model specific to the final terminal task, the second task-specific machine learning model from the at least two task-specific machine learning models is selected. The prompt is provided to the first task-specific machine learning model, Receiving information from the first task-specific machine learning model, The information described above is provided to the second task-specific machine learning model, The method according to claim 81, further comprising receiving the response to the prompt from the second task-specific machine learning model, wherein the response is generated by the second task-specific machine learning model.

83. Determining that the aforementioned prompt requests assistance with a clinical task, The prompt is parsed into one or more commands, thereby forming the intent of the prompt to request assistance with a clinical task. The method of claim 75, further comprising identifying a first domain of a plurality of domains associated with the intent of the prompt.

84. Determining that the aforementioned prompt requests assistance with a clinical task, The aforementioned prompt is applied to a machine learning model to generate a first response that differs from the aforementioned prompt and responds to the aforementioned prompt from the user. Obtaining the first domain of multiple domains in the input space associated with the prompt, Further comprising evaluating the value of the first response, When the value of the first response satisfies the threshold condition, the first response is communicated to the user via the communication network, and When the value of the first response cannot satisfy the threshold condition, Identify a first task-specific machine learning model associated with the first domain, The method according to claim 75, wherein the first response and / or the prompt is applied to the first task-specific machine learning model to generate a second response that is different from the first response and responds to the prompt.

85. The method according to claim 75, wherein each of the task-specific machine learning models is trained on a first domain of a plurality of domains.

86. The method according to claim 75, wherein each of the multiple domains includes at least one task-specific machine learning model trained on the respective domain.

87. The method according to claim 85, wherein the selection of each task-specific machine learning model from the plurality of task-specific machine learning models is based on the identification of the first domain through association with the prompt.

88. Providing the aforementioned prompts to the respective task-specific machine learning models is This includes applying the prompt to a first node in a plurality of interconnected nodes, thereby generating a response that, unlike the prompt, responds to the prompt from the user, The first node is associated with a first domain-specific machine learning model of the plurality of task-specific machine learning models, Each of the aforementioned task-specific machine learning models is (i) associated with at least one node of the aforementioned plurality of interconnected nodes, and (ii) defines conditional logic for performing a specific task. The method according to claim 85, wherein each node of the plurality of interconnected nodes is connected by an edge to at least one node of the plurality of interconnected nodes.

89. The method according to claim 75, wherein selecting the respective task-specific machine learning model comprises generating the task-specific machine learning model having conditional logic configured to respond to the prompt.

90. The method according to claim 75, wherein selecting a machine learning model specific to each task includes identifying a first classification of machine learning models and selecting a machine learning model specific to each task based on its association with the first classification of machine learning models.

91. The method according to claim 75, wherein selecting the respective task-specific machine learning models comprises forming a first sequence for a plurality of interconnected nodes.

92. It is a method, Based on user input, a machine learning model trained to select from multiple task-specific components determines that a received prompt requests assistance for one or more clinical tasks, and selects a set of task-specific components from the multiple task-specific components based on the prompt. Acquiring orchestration data for the set of task-specific components, wherein each task-specific component in the set of task-specific components is configured to support each of the one or more clinical tasks. From the orchestration data, determine at least one data suitability criterion for clinical task data related to one or more clinical tasks, Receiving the aforementioned clinical task data, A method comprising providing a user with a notification indicating that one or more clinical tasks cannot be performed using the clinical task data, in accordance with the determination that the clinical task data does not meet the at least one data suitability criterion.

93. The method according to claim 99, further comprising identifying one or more sets of data interfaces based on the acquired orchestration data, wherein each data interface in the set of data interfaces corresponds to each task-specific component of the set of task-specific components, and the at least one data compatibility criterion is determined based on one or more attributes of the set of data interfaces.

94. The method according to claim 99, further comprising providing the user with another notification indicating that the clinical task data has been validated for one or more clinical tasks in accordance with the determination that the clinical task data meets the at least one data conformity criterion.

95. The aforementioned prompt is received as text input from the user, Identifying the intent of the aforementioned text input, The method according to claim 99, further comprising determining that the prompt requests assistance with one or more clinical tasks based on the identified intent.

96. The method according to claim 99, wherein the clinical task data includes image data, and the at least one data suitability criterion relates to at least one of the size of the image data, the resolution of the image data, and the color spectrum of the image data.

97. The method according to claim 99, wherein the orchestration data includes one or more attributes of the task-specific set of machine learning models, one or more input parameters of the task-specific set of machine learning models, and one or more configuration parameters of the task-specific set of machine learning models.

98. The method according to claim 99, wherein the at least one data conformance criterion includes at least one of a data format requirement, a data type requirement, and a data label requirement.

99. The method according to claim 99, wherein the plurality of task-specific components include a plurality of task-specific machine learning models.

100. The method according to claim 99, wherein the plurality of task-specific components include one or more transformation components, and each of the one or more transformation components is configured to apply a transformation to biological data to produce an output.

101. In accordance with the determination that the clinical task data satisfies at least one data compatibility criterion, By performing one or more clinical tasks through the aforementioned set of task-specific components, one or more task results are obtained. The method according to claim 99, further comprising providing the user with an output showing one or more task results.

102. The method according to claim 108, wherein the output includes a natural language output summarizing the results of one or more tasks.

103. The method according to claim 108, wherein the output is generated by a machine learning component using the one or more task results.

104. The method of claim 108, further comprising obtaining target data about the target, wherein the output indicates how the one or more task results relate to the target.

105. The method according to claim 108, wherein the output is individualized for a specific target.

106. The method according to claim 108, wherein the user input includes a user query, and the output provides an answer to the user query based on one or more task results.

107. The method according to claim 113, wherein the user query relates to a specific target.

108. It is a method, Receiving requests from users regarding the execution of specific tasks, In response to the aforementioned request, generating an agent module for performing the aforementioned specific task, wherein the generation is The process involves selecting a set of agent building blocks from a plurality of available agent building blocks, wherein each of the available agent building blocks has its respective assigned function. To form the agent block, connect the set of agent building blocks, The agent module is instructed to execute the information from the request and provide it to the agent module. In response to providing the agent module information from the aforementioned request, the recipient receives a response from the agent module corresponding to the execution of the specific task, A method comprising providing the aforementioned response to the user.

109. The method according to claim 115, further comprising obtaining information about a plurality of previously generated agent modules, wherein the agent module is generated in accordance with the determination that none of the plurality of previously generated agent modules are configured to perform the particular task.

110. The method according to claim 116, further comprising ceasing to generate the agent module in accordance with the determination that the first agent module of the plurality of previously generated agent modules is configured to perform the particular task.

111. The method according to claim 117, further comprising providing the information from the request to the first agent module of the plurality of previously generated agent modules, in accordance with the determination that the first agent module is configured to perform the particular task.

112. The method according to claim 115, wherein the plurality of available agent building blocks include one or more of a set of data building blocks, a set of operator building blocks, and a set of tool building blocks.

113. The method according to claim 115, further comprising obtaining specification information for the plurality of available agent components, wherein the specification information for each agent component includes the respective assigned function, one or more input data types, and one or more output data types.

114. The method according to claim 115, wherein the agent module is automatically generated and executed without further input from the user.

115. The verification of the agent module is performed in accordance with the determination that the agent module is valid. The method of claim 115, further comprising generating a revised agent module using the invalidity data of the agent module in accordance with the determination that the agent module is invalid.

116. The method according to claim 115, wherein the agent module includes one or more machine learning models.

117. The method according to claim 115, wherein the agent module includes a template building block that is coupled to the output of the agent module and configured to convert information acquired by the agent module into a natural language response.

118. The method according to claim 115, wherein the agent module includes a template building block that is coupled to the input of the agent module and configured to convert the received input into a programming language object.

119. Further includes receiving a user identifier for the aforementioned user, The agent module is generated by Based on the user identifier, identify one or more datasets accessible to the user, The method according to claim 115, comprising connecting the set of agent building blocks to one or more datasets.

120. A non-temporary computer-readable storage medium that, when executed by one or more processors, includes instructions causing one or more processors to perform any of the methods of claims 1 to 126.

121. A computing system, One or more processors, Memory and A computing system comprising one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods of claims 1 to 126.

122. A computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by a computing system, the storage medium includes instructions that cause the computing system to perform any of the methods of claims 1 to 126.

123. A graphical user interface on a computing system comprising a display, memory, and one or more processors for executing one or more programs stored in the memory, wherein the graphical user interface includes a user interface displayed according to any of the methods of claims 1 to 126.

124. An information processing device for use in a computing system, wherein the information processing device is An information processing apparatus comprising means for carrying out any of the methods of claims 1 to 126.

Images