Collaborative artificial intelligence method and system

Abstract

To provide a method and system of audibly broadcasting responses to a user based on user queries about a specific patient molecular report.SOLUTION: The method comprises: receiving an audible query from the user to a microphone coupled to a collaboration device; identifying at least one intent associated with the audible query; identifying at least one data operation associated with the at least one intent; associating each of the at least one data operation with a first set of data presented on the molecular report; executing each of the at least one data operation on a second set of data to generate response data; generating an audible response file associated with the response data; and providing the audible response file for broadcasting via a speaker coupled to the collaboration device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 62 / 871,667, entitled "COLLABORATIVE ARTIFICIAL INTELLIGENCE METHOD AND SYSTEM," filed July 8, 2019, U.S. Provisional Patent Application No. 62 / 855,646, entitled "COLLABORATIVE ARTIFICIAL INTELLIGENCE METHOD AND APPARATUS," filed May 31, 2019, and U.S. Provisional Patent Application No. 62 / 835,339, entitled "COLLABORATIVE ARTIFICIAL INTELLIGENCE METHOD AND APPARATUS," filed April 17, 2019.

[0002] Applications Incorporated by Reference Each of the following US patent applications is incorporated herein by reference in its entirety: (1) U.S. Patent Application No. 16 / 657,804, entitled "DATA BASED CANCER RESEARCH AND TREAMENT SYSTEMS AND METHODS," filed October 18, 2019. (2) U.S. Patent Application No. 16 / 671,165, entitled "USER INTERFACE, SYSTEM, AND METHOD FOR COHORT ANALYSIS," filed December 31, 2019. (3) U.S. Patent Application No. 16 / 732,168, filed December 31, 2019, entitled "A METHOD AND PROCESS FOR PREDICTING AND ANALYZING PATIENT COHORT RESPONSE, PROGRESSION, AND SURVIVAL."

[0003] STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT Not applicable [Background technology]

[0004] The field of the disclosure is systems for accessing and manipulating large, complex data sets in a manner that allows system users to develop new insights and conclusions with minimal user interface friction that impedes access and manipulation.

[0005] The present disclosure describes innovations described in the context of an exemplary healthcare professional collaborating with a patient to diagnose disease states, prescribe treatments, and administer those treatments to improve overall patient health. Additionally, while many different types of healthcare professionals (e.g., doctors, psychologists, physical therapists, nurses, administrators, researchers, insurance professionals, pharmacists, etc.) in many different medical fields (e.g., cancer, Alzheimer's disease, Parkinson's disease, psychiatric disorders, cardiology, immunology, infectious diseases, and diabetes) will benefit from the disclosed innovations, unless otherwise specified, the innovations are described in the context of an exemplary oncologist / researcher (hereinafter "oncologist") and / or cancer medical researcher who collaborates with patients to diagnose the cancer condition (e.g., all physiological, behavioral, medical history, genetic, and treatment effect factors), understand and evaluate existing data and guidelines for their patient and similar patients, prescribe treatments, administer those treatments, and monitor patient outcomes, all to improve overall patient health.

[0006] Many professions require complex thinking, which requires considering many factors when selecting a solution to an encountered situation, hypothesizing new factors and solutions, and testing them to confirm they are effective. For example, an oncologist considering a particular patient's cancer status must optimally consider many different factors when assessing the patient's cancer status and developing and administering an optimized treatment plan. These factors might include the patient's family history, past medical conditions, current diagnosis, the genomic / molecular profile of the patient's inherited DNA and the patient's tumor DNA, current nationally recognized guidelines for treatment standards within that cancer subtype, the patient's condition, available clinical trials relevant to the patient, available pharmaceutical therapies and other potential therapeutic interventions that may be good options for the patient, as well as recently published research on data from similar patients. Furthermore, because cancer and cancer treatment research is rapidly evolving, researchers must continually utilize data, new research, and new treatment guidelines to critically think about new factors and treatments when diagnosing the cancer status and optimizing treatment plans.

[0007] In particular, it is no longer possible for oncologists to stay up-to-date on all new research in the field of cancer treatment. Similarly, it is extremely difficult for oncologists to manually analyze the medical records and outcomes of thousands or even millions of cancer patients each time they wish to make a specific treatment recommendation for a particular patient being treated by that oncologist. First, oncologists often lack access to health information from institutions other than their own. In the United States, implementation of a federal law known as the Health Insurance Portability and Accountability Act of 1996 ("HIPAA") significantly limits one healthcare provider's ability to access another's medical records. Furthermore, healthcare systems face administrative, technical, and financial challenges when making data available to third parties for aggregation with similar data from other healthcare systems. To the extent that medical information from multiple patients seen at multiple providers is aggregated in a single repository, a system and method for structuring that information using a common data dictionary or library of data dictionaries is needed. When multiple institutions are responsible for developing a single, centralized repository, significant discrepancies can arise regarding the structure of one or more data dictionaries, how data is accessed, which individuals or other donors are authorized to access the data, and the amount of data that is accessible. Furthermore, the scope of searchable data can be overwhelming for oncologists who want to perform manual reviews. Every patient has health information that contains hundreds or even thousands of data elements. Including sequencing information, such as from next-generation sequencing, in the health information accessed and analyzed significantly increases the amount of health information that can be analyzed. For example, a single FASTQ or BAM file generated during whole-exome sequencing consumes gigabytes of storage, despite containing sequences from only a patient's exome, which is thought to represent approximately 1–2% of the entire human genome.

[0008] In this regard, an oncologist may have a simple question—"What is the best medication for this particular patient?"—the answer requires a vast amount of health information, analytical software modules to analyze that information, and a hardware framework to enable those modules to run to provide an answer. Almost every query / idea / concept is a work in progress that evolves over time as critical thinking is applied and the relationships of one or more additional relevant factors are recognized and / or better understood. Every query begins as a hypothesis rooted in a set of interrelated raw materials (e.g., data). A hypothesis is typically tested by asking a question related to the hypothesis and determining whether the hypothesis is consistent and enduring when considered in light of the raw materials and the answers to the question. A consistent / enduring hypothesis will depend on ideas (i.e., facts) and additional raw materials to generate the next iteration of the initial idea, not just entirely new ideas.

[0009] When considering a particular cancer condition, oncologists consider known factors (e.g., patient condition, previous treatment, treatment response), develop a hypothesis regarding optimized treatment, consider that hypothesis in light of previous data and previous studies relating similar cancer conditions to treatment response, and may prescribe the hypothesized treatment to the patient if the previous data indicates high efficacy for the treatment hypothesis. If the data indicates low efficacy of the treatment, oncologists reconsider and generate a different hypothesis, continuing the cycle of iterative testing and conclusion until an effective treatment plan is identified. Cancer researchers perform a similar iterative hypothesis, data testing, and conclusion process to derive new cancer research insights.

[0010] Tools have been developed to help oncologists diagnose cancer conditions, select and manage optimal treatments, and explore and examine new cancer condition factors, new cancer conditions (e.g., diagnoses), new treatment factors, new treatments, and new efficacy factors. For example, large cancer databases have been developed and maintained for access and manipulation by oncologists to explore diagnostic and treatment options, as well as new insights and treatment hypotheses. Computers enable access to and manipulation of cancer data and its derivatives.

[0011] Because cancer data tend to be voluminous and multifaceted, many useful representations include a significant amount of detail and specific arrangement of data or data derivatives that are best visually represented. For this reason, oncology and research computer workstations typically include traditional interface devices, such as one or more large flat-panel display screens for presenting data representations, and a keyboard, mouse, or other mechanical input device for entering information, operating interface tools, and presenting many different data representations. Often, the workstation's computer / processor runs an electronic medical record (EMR) or medical research application program (hereinafter "research application") that presents various data representations and on-screen cursor-selectable control icons for selecting various data access and manipulation options.

[0012] While traditional computers and workstations work well as interfaces for data access and manipulation, they have several drawbacks. First, using a computer interface often requires oncologists to click through various interfaces multiple times to find specific information. This is a tedious and time-consuming process that often results in oncologists not achieving the desired results and receiving answers to the questions they are trying to ask.

[0013] Second, it's often difficult to capture hypothetical queries as they arise, leading to ideas not being followed up on in a timely manner or being lost forever. Because queries aren't restricted to a specific time schedule, they often arise at inconvenient times when oncologists aren't logged into their workstations and are using research applications that could be used to capture and test ideas. For example, an oncologist might be at home when they become curious about some aspect of a patient's cancer condition, or some statistics related to one of their patients, or when they first formulate a treatment hypothesis for a particular patient's cancer condition. In this case, if the oncologist's workstation is located in a remote medical facility, the oncologist cannot easily query the database or capture or test their hypothesis.

[0014] Furthermore, even in this case, even if oncologists can use a laptop or other home computer to access research applications from home, the friction associated with using the application often becomes a hindrance. In this regard, accessing the application requires oncologists to retrieve their laptop or physically travel to a stationary computer in their home, boot up the computer's operating system, log on to the computer (e.g., enter a username and password), select and launch the research application, navigate through several application screenshots to the desired database access tool suite, and then enter query or hypothesis-defining information to begin hypothesis testing. This application access friction is often enough to discourage immediate query or hypothesis capture and testing, especially if oncologists simply assume they will remember the query or hypothesis the next time they access a computer interface. As anyone with a lot of ideas knows, ideas are fleeting, and therefore ideas not captured immediately are often lost. More importantly, oncologists typically have limited time to spend on each patient case, requiring immediate resolution of questions and queries while assessing information specific to that patient.

[0015] Third, oftentimes, new queries or hypotheses arise while oncologists are engaged in other activities unrelated to oncology activities. Here, as with many, they simply do not qualify for immediate consideration and testing via traditional research applications. Again, the idea is lost if not captured immediately.

[0016] Fourth, oncology and research data activities often involve a series of sequential questions or requests (hereafter "requests") that focus on increasingly detailed data responses, where intermediate results are not particularly interesting and require the oncologist / researcher to repeatedly enter additional input to define the next level of request. Furthermore, while in many cases a visual representation of the data responses to oncology and research requests is optimal, in other cases visual representations tend to hinder usability and can even be overwhelming. In these cases, while a visual representation is usable, the representation can require significant time and effort to consume the presented information (e.g., reading the results, mentally summarizing the results, etc.). In short, traditional oncology interfaces are often cumbersome to use.

[0017] Furthermore, today, oncologists and other specialists do not have an easy mechanism for querying large, complex databases and receiving answers in real time without interacting with electronic health record systems or other cumbersome software solutions. In particular, there is a need for systems and methods that allow a provider to query a device using his or her voice with questions related to the optimal care of his or her patient, with answers to those questions generated from unique datasets that provide context and new information relevant to the patient, including vast amounts of real-world historical clinical information combined with other forms of medical data, such as molecular data from omics sequencing and imaging data, as well as data derived from such data using analytics to determine the optimal path for that single patient.

[0018] What is needed, therefore, is an intuitive interface to complex databases that allows oncologists, researchers, and other professionals and database users to access and manipulate data in a variety of ways to generate queries, test hypotheses or new ideas, and thereby explore those ideas in the context of various datasets while minimizing the friction of access and manipulation. It would be advantageous for the interface to be always-present, or at least portable, and therefore essentially always available. It would also be advantageous for the system associated with the interface to memorialize user interface interactions, allowing oncologists or researchers to revisit the interactions at a later time, so that they can re-engage with the purpose of continuing their line of questioning or hypothesis testing without losing their previous thinking.

[0019] It would also be advantageous to have a system that captures the thoughts of oncologists for several purposes, such as developing better medical assistance systems, generating automated records and documentation, and providing services such as scheduling appointments, tests and procedures, and preparing prescriptions.

[0020] It would also be advantageous to have an interface available across several different form factors. [Prior art documents] [Patent documents]

[0021] [Patent Document 1] U.S. Patent Application No. 16 / 671,165, filed October 31, 2019 [Patent Document 2] U.S. Patent Application No. 16 / 732,168, filed December 31, 2019 [Patent Document 3] U.S. Patent Application No. 16 / 657,804 filed October 18, 2019 Summary of the Invention [Means for solving the problem]

[0022] It has been recognized that a relatively small, portable voice-activated and audio-responsive interface device (hereinafter, "collaborative device") can be provided that enables oncologists to perform at least initial database access and manipulation activities. In at least some embodiments, the collaboration device includes a processor linked to each of a microphone, a speaker, and a wireless transceiver (e.g., a transmitter and a receiver). The processor executes software for capturing voice signals generated by the oncologist. An automatic speech recognition (ASR) system converts the voice signals into a text file, which is then processed by a natural language processor (NLP) or other artificial intelligence module (e.g., a natural language understanding module, etc.) to generate data operations (e.g., commands to perform some data access or manipulation process, such as querying, filtering, memorializing, clearing previous query and filter results, making notes, etc.).

[0023] In at least some embodiments, the collaboration device is used within a collaboration system that includes a server that maintains and operates an industry-specific data repository. Data operations are received by the collaboration server and used to access and / or manipulate database data, thereby generating a data response. In at least some cases, the data response is returned to the collaboration device as an audio file that is broadcast to the oncologist as a result associated with the original query.

[0024] In some cases, the transcription of the voice signal into a text file is performed by the collaboration device processor, or in other cases, the voice signal is transmitted from the collaboration device to a collaboration server, which performs the transcription into the text file. In other cases, the text file is converted into data operations by the collaboration device processor, or in other cases, the conversion is performed by the collaboration server. In some cases, the collaboration server maintains or has access to an industry-specific database, so that the server acts as an intermediary between the collaboration device and the industry-specific database.

[0025] In at least some embodiments, the collaboration device is a dedicated collaboration device that serves only as an interface to the collaboration server and industry-specific database. In such cases, because the collaboration interface device is always on and can run only a single dedicated application program, the device requires no start-up time and can be activated essentially immediately through a single activation activity performed by the oncologist.

[0026] For example, in some cases, the collaboration device's motion sensors (e.g., accelerometer, gyroscope, etc.) are linked to the processor so that simply lifting the device can cause the processor to activate an application. In other cases, the collaboration device processor may be programmed to "listen" for the phrase "Hey query," and when received, activates to capture the next voice signal utterance, which serves as seed data for generating a text file. In other cases, the processor may be programmed to listen for a different activation phrase, such as the system's brand name, or a combination of the brand name and a command display. For example, if the system's brand name is "One," the activation phrase could be "One" or "Go, One," etc. In still other cases, the collaboration device may simply listen for voice signal utterances that it can recognize as oncology queries and then automatically use the recognized query as seed data for text generation.

[0027] In addition to providing audio responses to data actions, at least in some cases, the system automatically records and stores the data actions (e.g., data defining the actions) and responses as collaboration records for subsequent access. The collaboration record may include the original audio signal and broadcast response, or a text file and a text response corresponding to the data response, or both. Here, the stored collaboration record provides details about the oncologist's search and data action activities that serve to automatically memorialize hypotheses or ideas the oncologist was considering. If the oncologist requests a series of queries, those queries and data responses may be stored as one-line questions that, together, provide more detailed information to characterize the oncologist's initial hypothesis or idea. The system may then allow the oncologist to access the memorialized queries and data responses, re-entering the associated flow state and continuing hypothesis testing and data manipulation using a workstation-type interface or other computing device better suited to presenting more complex data sets and data representations, including a display screen and perhaps an audio device such as a speaker or microphone.

[0028] In addition to simple data search queries, other voice signal data action types are possible. For example, the system might support a filter action where an oncologist's voice signal message defines a subset of an industry-specific database set. For example, an oncologist could say, "Access all medical records of male patients over the age of 45 who have had pancreatic cancer since 1990," and have the system generate an associated subset of relevant data that meets the specified criteria.

[0029] Importantly, some data responses to oncology queries will be "audio-friendly," i.e., the response can be fully understood and comprehended when broadcast as an audio message. In other cases, the data response may not be well-suited for presentation as simply audio output. For example, if the query contains the phrase "Who was the patient you saw during your last office visit last Thursday?", an audio-friendly response might be "Mary Brown." On the other hand, if the query is "List all medications prescribed since 1978 to men over 45 with pancreatic cancer," and the response contains a list of 225 medications, the list would not be audio-friendly because it would take a long time to broadcast each list entry, and comprehension of all the list entries would be questionable at best.

[0030] If the data response is best presented visually, the system may take alternative or additional steps to provide the response in a user-friendly format. The system may simply indicate, as part of the audio response, that the response data is better presented in a visual format and then present an audio response. If there is a large display screen nearby, such as a computer monitor or television (TV), such as a smart TV, the system may pair with that display to present the visual data with or without audio data. The system may simply indicate that an appropriate audio response is not available. In some embodiments, the system may pair with a computing device that includes a display, such as a smartphone, tablet computer, etc.

[0031] Thus, at least some embodiments of the present invention enable intuitive and rapid access to complex data sets essentially anywhere within a wireless communication zone, allowing oncologists to initiate thought processes in real time as they arise. By answering questions as they arise, the system allows oncologists to dig deeper into the data in the moment and continue thought processes throughout the progression of the query. Some embodiments memorialize the oncologist's queries and responses, allowing the oncologist to subsequently revisit the information and continue related queries. If visual and audio responses are available, the system may adapt to provide visual responses when visual capabilities are present, or may simply store the visual responses as part of a collaboration record for subsequent access when the oncologist has access to a workstation, etc.

[0032] In at least some embodiments, the present disclosure includes a method for interacting with a database to access data therein, the method for use with a collaboration device including a speaker, a microphone, and a processor, the method comprising: associating distinct sets of condition-specific intents and supporting information with different clinical report types, the supporting information including at least one intent-specific data operation for each condition-specific intent; receiving a voice query via the microphone for information; identifying a particular patient associated with the query; identifying condition-specific clinical reports associated with the identified patient; upon selecting one of the condition-specific intents, attempting to select one of the condition-specific intents associated with the identified condition-specific clinical report as a match for the query; performing at least one data operation associated with the selected condition-specific intent to generate results; and using the results to form a query response and broadcasting the query response via the speaker.

[0033] In some cases, the method is for use with at least a first database containing information in addition to the clinical report, and the method further comprises obtaining at least a subset of the information in addition to the clinical report in response to a query, and using the results to form the query response comprises using the results and the additionally obtained information to form the query response.

[0034] In some cases, the at least one data operation includes at least one data operation for accessing additional information from a database, and obtaining at least the subset includes obtaining data for each of the at least one data operation for accessing additional information from the database.

[0035] Some embodiments include a method for interacting with a database to access data therein, the method for use with a collaboration device including a speaker, a microphone, and a processor, the method comprising: associating distinct sets of condition-specific intents and supporting information with different clinical report types, the supporting information including at least one intent-specific primary data operation for each condition-specific intent; receiving a voice query via the microphone for information; identifying a particular patient associated with the query; identifying condition-specific clinical reports associated with the identified patient; upon selecting one of the condition-specific intents, attempting to select one of the condition-specific intents associated with the identified condition-specific clinical report as a match for the query; performing the primary data operation associated with the selected condition-specific intent to generate results; performing supplementary data operations on data from a database including data in addition to the clinical report data to generate additional information; and broadcasting the query response via the speaker using the results and the additional information to form a query response.

[0036] Some embodiments include a method for audibly broadcasting a response to a user based on the user's query about a particular patient's molecular report, the method comprising: receiving an audible query from the user to a microphone coupled to a collaboration device; identifying at least one intent associated with the audible query; identifying at least one data action associated with the at least one intent; associating each of the at least one data action with a first set of data presented in the molecular report; performing each of the at least one data action on a second data set to generate response data; generating an audible response file associated with the response data; and providing the audible response file for broadcast via a speaker coupled to the collaboration device.

[0037] In at least some cases, the audible query includes a question about a nucleotide profile associated with the patient. In at least some cases, the nucleotide profile associated with the patient is a cancer profile of the patient. In at least some cases, the nucleotide profile associated with the patient is a germline profile of the patient. In at least some cases, the nucleotide profile is a DNA profile. In at least some cases, the nucleotide profile is an RNA expression profile. In at least some cases, the nucleotide profile is a mutation biomarker.

[0038] In at least some cases, the mutation biomarker is a BRCA biomarker. In at least some cases, the audible query includes a question about treatment. In at least some cases, the audible query includes a question about genes. In at least some cases, the audible query includes a question about clinical data. In at least some cases, the audible query includes a question about next generation sequencing panels. In at least some cases, the audible query includes a question about biomarkers.

[0039] In at least some cases, the audible query includes a question about immune biomarkers. In at least some cases, the audible query includes a question about antibody-based testing. In at least some cases, the audible query includes a question about clinical trials. In at least some cases, the audible query includes a question about organoid assays. In at least some cases, the audible query includes a question about pathology images. In at least some cases, the audible query includes a question about a type of disease. In at least some cases, at least one intent is a biomarker-related intent. In at least some cases, the biomarker is a BRCA biomarker. In at least some cases, at least one intent is a clinical condition-related intent. In at least some cases, at least one intent is a clinical trial-related intent.

[0040] In at least some cases, at least one intent is related to a pharmaceutical. In at least some cases, the pharmaceutical intent is related to a chemotherapy pharmaceutical. In at least some cases, the pharmaceutical intent is related to a PARP inhibitor intent. In at least some cases, at least one intent is related to a gene. In at least some cases, at least one intent is related to an immunology. In at least some cases, at least one intent is related to a knowledge database. In at least some cases, at least one intent is related to a testing method. In at least some cases, at least one intent is related to a gene panel. In at least some cases, at least one intent is related to reporting. In at least some cases, at least one intent is related to an organoid process. In at least some cases, at least one intent is related to imaging.

[0041] In at least some cases, at least one intent is related to a pathogen. In at least some cases, at least one intent is related to a vaccine. In at least some cases, at least one data operation includes an operation for identifying at least one treatment option. In at least some cases, at least one data operation includes an operation for identifying knowledge about a treatment. In at least some cases, at least one data operation includes an operation for identifying knowledge related to at least one pharmaceutical agent (e.g., "Which pharmaceutical agent is associated with high CD40 expression?"). In at least some cases, at least one data operation includes an operation for identifying knowledge related to mutation testing (e.g., "Was Dwayne Holder's sample tested for a KMT2D mutation?"). In at least some cases, at least one data operation includes an operation for identifying knowledge related to the presence of a mutation (e.g., "Does Dwayne Holder have a KMT2C mutation?"). In at least some cases, at least one data operation includes an operation for identifying knowledge related to a tumor characteristic (e.g., "Is Dwayne Holder's tumor likely a BRCA2-driven tumor?"). In at least some cases, at least one data operation includes an operation for identifying knowledge related to testing requirements (e.g., "What percentage of tumors does Tempus require for a TMB result?"). In at least some cases, at least one data operation includes an operation for querying definitional information (e.g., "What is PDL1 expression?"). In at least some cases, at least one data operation includes an operation for querying expert information (e.g., "What is the clinical relevance of PDL1 expression?", "What are the common risks associated with Whipple surgery?"). In at least some cases, at least one data operation includes an operation for identifying information related to recommended treatment (e.g., "Dwayne Holder is in the 88th percentile for PDL1 expression, is he a candidate for immunotherapy?").In at least some cases, the at least one data operation includes an operation for querying information related to the patient (e.g., Dwayne Holder). In at least some cases, the at least one data operation includes an operation for querying information related to patients having one or more clinical characteristics similar to the patient (e.g., "What are the most common adverse events for patients similar to Dwayne Holder?").

[0042] In at least some cases, at least one data operation includes an operation for querying information related to a patient cohort (e.g., "What are the most common adverse events for pancreatic cancer patients?"). In at least some cases, at least one data operation includes an operation for querying information related to clinical trials (e.g., "Which clinical trial is best for Dwayne?").

[0043] In at least some cases, the at least one data operation includes an operation for querying for a feature associated with a genomic mutation. In at least some cases, the feature is loss of heterozygosity. In at least some cases, the feature reflects a cause of the mutation. In at least some cases, the cause is germline. In at least some cases, the cause is somatic. In at least some cases, the feature includes whether the mutation is a tumor driver. In at least some cases, the first set of data comprises a patient name.

[0044] In at least some cases, the first set of data comprises patient age. In at least some cases, the first set of data comprises a next generation sequencing panel. In at least some cases, the first set of data comprises genomic variants. In at least some cases, the first set of data comprises somatic genomic variants. In at least some cases, the first set of data comprises germline genomic variants. In at least some cases, the first set of data comprises clinically actionable genomic variants. In at least some cases, the first set of data comprises loss-of-function variants. In at least some cases, the first set of data comprises gain-of-function variants.

[0045] In at least some cases, the first set of data comprises immunological markers. In at least some cases, the first set of data comprises tumor mutational burden. In at least some cases, the first set of data comprises microsatellite instability status. In at least some cases, the first set of data comprises a diagnosis. In at least some cases, the first set of data comprises a treatment. In at least some cases, the first set of data comprises a treatment approved by the U.S. Food and Drug Administration. In at least some cases, the first set of data comprises pharmaceutical therapy. In at least some cases, the first set of data comprises radiation therapy. In at least some cases, the first set of data comprises chemotherapy. In at least some cases, the first set of data comprises cancer vaccine therapy. In at least some cases, the first set of data comprises oncolytic virus therapy.

[0046] In at least some cases, the first set of data comprises immunotherapy. In at least some cases, the first set of data comprises pembrolizumab therapy. In at least some cases, the first set of data comprises CAR-T therapy. In at least some cases, the first set of data comprises proton beam therapy. In at least some cases, the first set of data comprises ultrasound therapy. In at least some cases, the first set of data comprises surgery. In at least some cases, the first set of data comprises hormone therapy. In at least some cases, the first set of data comprises off-label use. In at least some cases, the first set of data comprises gene editing therapy. In at least some cases, the gene editing therapy may be clustered regularly interspaced short palindromic repeats (CRISPR) therapy.

[0047] At least in some cases, the first set of data comprises an on-label use. At least in some cases, the first set of data comprises a bone marrow transplant event. At least in some cases, the first set of data comprises a cryoablation event. At least in some cases, the first set of data comprises radiofrequency ablation. At least in some cases, the first set of data comprises a monoclonal antibody therapy. At least in some cases, the first set of data comprises an angiogenesis inhibitor. At least in some cases, the first set of data comprises a PARP inhibitor.

[0048] In at least some cases, the first set of data comprises a targeted therapy. In at least some cases, the first set of data comprises an indication for use. In at least some cases, the first set of data comprises a clinical trial. In at least some cases, the first set of data comprises distances to locations where clinical trials will be conducted. In at least some cases, the first set of data comprises variants of unknown significance. In at least some cases, the first set of data comprises mutation effects.

[0049] In at least some cases, the first set of data comprises a variant allele fraction. In at least some cases, the first set of data comprises a low coverage region. In at least some cases, the first set of data comprises a medical history. In at least some cases, the first set of data comprises biopsy results. In at least some cases, the first set of data comprises imaging results. In at least some cases, the first set of data comprises MRI results.

[0050] In at least some cases, the first set of data comprises CT results. In at least some cases, the first set of data comprises a treatment prescription. In at least some cases, the first set of data comprises a treatment administration. In at least some cases, the first set of data comprises a cancer subtype diagnosis. In at least some cases, the first set of data comprises a cancer subtype diagnosis by RNA class. In at least some cases, the first set of data comprises results of a treatment applied to organoids grown from the patient's cells. In at least some cases, the first set of data comprises a tumor quality measure. In at least some cases, the first set of data comprises a tumor quality measure selected from at least one of the set of PD-L1, MMR, tumor-infiltrating lymphocyte count, and tumor ploidy. In at least some cases, the first set of data comprises a tumor quality measure obtained from image analysis of a pathology slide of the patient's tumor. In at least some cases, the first set of data comprises a signaling pathway associated with the patient's tumor.

[0051] At least in some cases, the signaling pathway is the HER pathway. At least in some cases, the signaling pathway is the MAPK pathway. At least in some cases, the signaling pathway is the MDM2-TP53 pathway. At least in some cases, the signaling pathway is the PI3K pathway. At least in some cases, the signaling pathway is the mTOR pathway.

[0052] In at least some cases, the at least one data operation includes an operation for querying treatment options, the first set of data comprising genomic variants, and the associating step comprises coordinating an operation for querying treatment options based on the genomic variants. In at least some cases, the at least one data operation includes an operation for querying medical history data, the first set of data comprising a treatment, and the associating step comprises coordinating an operation for querying medical history data elements based on the treatment. In at least some cases, the medical history data is a medication prescription, the treatment is pembrolizumab, and the associating step comprises coordinating an operation for querying a prescription for pembrolizumab.

[0053] In at least some cases, the second set of data comprises clinical health information. In at least some cases, the second set of data comprises genomic variant information. In at least some cases, the second set of data comprises DNA sequencing information. In at least some cases, the second set of data comprises RNA information. In at least some cases, the second set of data comprises DNA sequencing information from short read sequencing. In at least some cases, the second set of data comprises DNA sequencing information from long read sequencing. In at least some cases, the second set of data comprises RNA transcriptome information. In at least some cases, the second set of data comprises RNA full transcriptome information. In at least some cases, the second set of data is stored in a single data repository. In at least some cases, the second set of data is stored in multiple data repositories.

[0054] In at least some cases, the second set of data comprises clinical health information and genomic variant information. In at least some cases, the second set of data comprises immunological marker information. In at least some cases, the second set of data comprises microsatellite instability immunological marker information. In at least some cases, the second set of data comprises tumor mutation burden immunological marker information. In at least some cases, the second set of data comprises clinical health information comprising one or more of demographic information, diagnostic information, evaluation results, test results, prescribed or administered treatments, and outcome information.

[0055] In at least some cases, the second set of data comprises demographic information comprising one or more of patient age, patient date of birth, sex, race, ethnicity, system of care, comorbidities, and smoking history. In at least some cases, the second set of data comprises diagnostic information comprising one or more of tissue of origin, date of initial diagnosis, histology, histology grade, metastatic diagnosis, date of metastatic diagnosis, one or more sites of metastasis, and stage information. In at least some cases, the second set of data comprises stage information comprising one or more of TNM, ISS, DSS, FAB, RAI, and Binet. In at least some cases, the second set of data comprises assessment information comprising one or more of performance status (including ECOG or Karnofsky status), performance status score, and performance status date.

[0056] In at least some cases, the second set of data comprises laboratory information comprising one or more of lab type (e.g., CBS, CMP, PSA, CEA), lab result, lab unit, date of lab service, date of molecular pathology test, assay type, assay result (e.g., positive, negative, equivocal, mutation, wild type), molecular pathology method (e.g., IHC, FISH, NGS), and molecular pathology provider. In at least some cases, the second set of data comprises treatment information comprising one or more of drug name, drug start date, drug end date, drug dosage, drug units, number of drug cycles, surgical procedure type, surgical procedure date, radiation site, radiation modality, radiation start date, radiation end date, total dose of radiation administered, and total percentage of radiation administered.

[0057] In at least some cases, the second set of data comprises outcome information comprising one or more of: response to treatment (e.g., CR, PR, SD, PD), RECIST score, date of outcome, date of observation, date of progression, date of recurrence, adverse event to treatment, date of presentation of adverse event, grade of adverse event, date of death, date of last follow-up, and condition at last follow-up. In at least some cases, the second set of data comprises information that has been de-identified in accordance with de-identification methods permitted by HIPAA.

[0058] In at least some cases, the second set of data comprises information that has been de-identified in accordance with Safe Harbor de-identification methods permitted by HIPAA. In at least some cases, the second set of data comprises information that has been de-identified in accordance with statistical de-identification methods permitted by HIPAA. In at least some cases, the second set of data comprises clinical health information of patients diagnosed with a cancer condition.

[0059] In at least some cases, the second set of data comprises clinical health information for patients diagnosed with cardiovascular disease. In at least some cases, the second set of data comprises clinical health information for patients diagnosed with a diabetic condition. In at least some cases, the second set of data comprises clinical health information for patients diagnosed with an autoimmune condition. In at least some cases, the second set of data comprises clinical health information for patients diagnosed with a lupus condition.

[0060] In at least some cases, the second set of data comprises clinical health information of patients diagnosed with a psoriasis condition. In at least some cases, the second set of data comprises clinical health information of patients diagnosed with a depression condition. In at least some cases, the second set of data comprises clinical health information of patients diagnosed with a rare disease.

[0061] In at least some embodiments, the present disclosure provides a method for audibly broadcasting a response to a user based on a user query about a specific patient's molecular report. The method can be used in a collaboration device including a processor, a microphone, and a speaker linked to the processor. The method can include storing a plurality of patient's molecular reports in a system database, receiving an audible query from a user via the microphone, identifying at least one intent associated with the audible query, identifying at least one data operation associated with the at least one intent, accessing the specific patient's molecular report, performing at least one of the identified at least one data operation on the first set of response data included in the specific patient's molecular report to generate a first set of response data, using the first set of response data to generate an audible response file, and broadcasting the audible response file via the speaker.

[0062] In at least some cases, the method further includes identifying eligible parameters in the audible query, and identifying the at least one data action includes identifying the at least one data action based on both the identified intent and the eligible parameters.

[0063] In at least some cases, at least one of the eligibility parameters may include patient identity.

[0064] In at least some cases, at least one of the qualification parameters may include a medical condition of the patient.

[0065] In at least some cases, at least one of the qualifying parameters can include a genetic variation.

[0066] In at least some cases, at least one of the eligibility parameters may include a type of treatment.

[0067] In at least some cases, the method further includes identifying qualifying parameters in the molecular report of the particular patient, and identifying the at least one data action includes identifying the at least one data action based on both the identified intent and the qualifying parameters.

[0068] At least in some cases, the method may further include storing a general knowledge database containing non-patient-specific data on the particular topic, and wherein identifying at least one data action associated with the at least one intent includes identifying at least first and second data actions associated with the at least one intent, wherein the first data action is associated with a molecular report of the particular patient and the second data action is associated with the general knowledge database.

[0069] In at least some cases, a second data operation associated with a general knowledge database can be first performed to generate a second data operation result, the second data operation result can be used to define the first data operation, and a second first data operation associated with a molecular report for a particular patient can be performed to generate a first set of response data.

[0070] In at least some cases, a first data operation associated with a molecular report of a particular patient can be first performed to generate a first data operation result, the first data operation result can be used to define a second data operation, and a second data operation associated with a general knowledge database can be second performed to generate a first set of response data.

[0071] In at least some cases, identifying the at least one intent may include determining that the audible query is associated with a particular patient, accessing a molecular report for the particular patient, determining a cancer status for the particular patient from the molecular report, and then selecting an intent from a pool of intents related to the cancer status.

[0072] In at least some cases, the method may further include storing a general knowledge database containing non-patient-specific data on a particular topic, and the method further includes selecting an intent associated with the general knowledge database upon determining that the audible query is not associated with a particular patient.

[0073] In at least some cases, the collaboration device may include a portable wireless device that includes a wireless transceiver.

[0074] In at least some cases, the collaboration device may be a handheld device.

[0075] In at least some cases, the collaboration device may include at least one visual indicator, and the processor is linked to the visual indicator and controllable to change at least some aspects of the appearance of the visual indicator to indicate different states of the collaboration device.

[0076] In at least some cases, the processor can be programmed to monitor the microphone input to identify a "wake up" phrase, and the processor monitors for an audible query after the wake up phrase is detected.

[0077] In at least some cases, the series of audible queries may be received via a microphone, and at least one of the identified data operations may include identifying a subset of data that may be used in a subsequent query to identify an intent associated with the subsequent audio query.

[0078] At least in some cases, the method may further include identifying at least one activity that the collaboration device user may want to perform based on at least one audible query received via the microphone and associated data in the system database, and initiating the at least one activity.

[0079] In at least some cases, initiating the at least one activity may include generating a second audible response file, broadcasting the second audible response file to a user seeking verification that the at least one activity should be performed, monitoring a microphone for a positive response, and initiating the at least one activity upon receiving the positive response.

[0080] In at least some cases, at least one activity may include periodically capturing health information from electronic health records contained in a system database.

[0081] In at least some cases, at least one activity may include checking the status of an existing clinical or test order.

[0082] In at least some cases, at least one activity may include ordering a new clinical or laboratory order.

[0083] At least in some cases, the collaboration device may be a smartphone, a tablet computer, a laptop computer, a desktop computer, or an Amazon Echo.

[0084] In at least some cases, initiating the at least one activity may include automatically initiating the at least one activity without initiating input from a user.

[0085] At least in some cases, the method may further include storing and maintaining a general cancer knowledge database; persistently updating the molecular report of the particular patient; automatically identifying at least one intent and associated data operation related to the general cancer knowledge database based on the molecular report data of the particular patient; persistently performing the associated data operation on the general cancer knowledge database to generate a new set of response data that has not previously been generated; upon generating the new set of response data, using the new set of response data to generate another audible response file; and broadcasting the other audible response file via a speaker.

[0086] At least in some cases, the method may be used in conjunction with an electronic health record system that maintains health records associated with multiple patients, including the particular patient, and the method further includes identifying at least another data action associated with the at least one intent and performing the other data action on the particular patient's health record to generate additional response data.

[0087] In at least some cases, using the first set of response data to generate the audible response file may include using the response data and the additional response data to generate the audible response file.

[0088] In at least some embodiments, the present disclosure provides a method for use in a collaboration device that includes a processor, a microphone, and a speaker linked to the processor to audibly broadcast a response to a user based on a user query about a molecular report for a particular patient. The method includes the steps of storing individual molecular reports for each of a plurality of patients in a system database, storing a general cancer knowledge database containing non-patient-specific data on the topic of cancer, receiving an audible query from a user via a microphone, identifying at least one intent associated with the audible query, identifying at least a first data operation associated with the at least one intent and the molecular report of the particular patient, identifying at least a second data operation associated with the at least one intent and the general cancer knowledge database, accessing the molecular report of the particular patient and the general cancer knowledge database, performing at least a first data operation on the first data set included in the molecular report of the particular patient to generate a first response data set, performing at least a second data operation on the general cancer knowledge database to generate a second response data set, using at least one of the first and second sets of response data to generate an audible response file, and broadcasting the audible response file via a speaker.

[0089] In at least some embodiments, the present disclosure provides a method for use in a collaboration device including a processor, a microphone, and a speaker linked to the processor for audibly broadcasting a response to a user based on a user query about a specific patient's molecular report, the method including: storing a plurality of patients' molecular reports in a system database; receiving an audible query from a user via the microphone; determining that the audible query is associated with the specific patient; accessing the specific patient's molecular report; determining the specific patient's cancer status from the molecular report; identifying at least one intent from a pool of intents related to the specific patient's cancer status and the audible query; identifying at least one data operation associated with the at least one intent; performing at least one of the identified at least one data operation on the first set of data included in the specific patient's molecular report to generate a first set of response data; using the first set of response data to generate an audible response file; and broadcasting the audible response file via the speaker.

[0090] In at least some embodiments, the present disclosure provides a method for use in a collaboration device including a processor, a microphone, and a speaker linked to the processor, for audibly broadcasting a response to a user based on a user query about a patient. The method includes storing health records of a plurality of patients in a system database and storing a general cancer knowledge database, receiving an audible query from a user via the microphone, identifying a particular patient associated with the audible query, accessing the health record of the particular patient, identifying cancer-related data in the health record of the particular patient, identifying at least one intent associated with the identified cancer-related data, identifying at least one data operation associated with the at least one intent, performing the at least one data operation on the general cancer knowledge database to generate a first set of response data, using the first set of response data to generate an audible response file, and broadcasting the audible response file via the speaker. [Brief explanation of the drawings]

[0091] [Figure 1] 1 is a schematic diagram illustrating a collaboration system consistent with at least some aspects of the present disclosure, including a portable wireless collaboration device. [Figure 2] FIG. 2 is a schematic diagram illustrating components of the exemplary collaboration device shown in FIG. [Figure 3] FIG. 10 is a schematic diagram of a second exemplary collaboration device. [Figure 4] FIG. 4 is a schematic diagram illustrating components of the second exemplary collaboration device shown in FIG. 3. [Figure 5] 1 is a flowchart illustrating a collaboration process consistent with at least some aspects of the present disclosure. [Figure 6]1 is a schematic diagram illustrating a collaboration device user interacting with the system of claim 1. FIG. [Figure 7] FIG. 1 is a schematic diagram illustrating a workstation that can be used to access stored collaboration session data. [Figure 8] FIG. 8 is a screenshot similar to FIG. 7, but showing another screenshot. [Figure 9] FIG. 1 is a schematic diagram illustrating a portable audible collaboration device being used in combination with a workstation including a display. [Figure 10] FIG. 9 is a schematic diagram showing another screenshot similar to the diagram of FIG. 8. [Figure 11] FIG. 10 is a schematic diagram illustrating a second collaboration system consistent with at least some aspects of the present disclosure, in which a portable collaboration device is running an AI application to generate seed data for data operations and converts data responses into audio response files that are broadcast via the collaboration device. [Figure 12] FIG. 10 is a schematic diagram illustrating a third collaboration system consistent with at least some aspects of the present disclosure. [Figure 13] 1 is a schematic diagram illustrating several collaborating devices that can communicate with each other and / or with at least one of a first transceiver and a second transceiver using mesh networking. [Figure 14] 10A-10C illustrate two additional collaboration device configurations, including a cube-shaped configuration and a tablet-type configuration. [Figure 15] FIG. 1 is a schematic diagram illustrating a workstation including various types of input / output collaboration devices. [Figure 16] FIG. 1 illustrates a headset that may operate as yet another type of input / output audio interface consistent with at least some aspects of the present disclosure. [Figure 17A]FIG. 1 is a schematic diagram showing page 1 of the Pancreas Clinical Report, which can be printed in hard copy or accessed electronically via a workstation, pad, smartphone device, etc. [Figure 17B] FIG. 1 is a schematic diagram showing page 2 of the Pancreas Clinical Report, which can be printed in hard copy or accessed electronically via a workstation, pad, smartphone device, etc. [Figure 17C] FIG. 1 is a schematic diagram showing page 3 of the Pancreas Clinical Report, which can be printed in hard copy or accessed electronically via a workstation, pad, smartphone device, etc. [Figure 18] 6 is a flowchart similar to the chart shown in FIG. 5, but in which condition-specific clinical records and associated intents are used to drive the query process. [Figure 19] FIG. 10 illustrates an audio response process consistent with at least some aspects of the present disclosure. [Figure 20] FIG. 1 illustrates a system database, consistent with at least some aspects of the present disclosure. [Figure 21] 10A-10C illustrate screenshots used by a system administrator to specify system intents, intent parameters, and answer formats for provider panel types, consistent with at least some aspects of the present disclosure. [Figure 22] FIG. 22 is similar to FIG. 21 but includes a screenshot for specifying gene-specific system information. [Figure 23] FIG. 23 is similar to FIG. 22 but includes a screenshot for specifying provider methods. [Figure 24] FIG. 10 is a schematic diagram of an exemplary fourth exemplary system including a mobile device. [Figure 25] FIG. 1 illustrates a screenshot of a mobile application. [Figure 26] FIG. 26 shows a second screenshot of the mobile application in FIG. 25. [Figure 27] FIG. 26 shows a third screenshot of the mobile application in FIG. 25. [Figure 28] FIG. 10 is a schematic diagram of a fifth exemplary collaboration system. [Figure 29] 1 is a flowchart of a process for generating supplemental content for a physician based on molecular reports associated with a particular patient. [Figure 30] 1 is a flowchart of a process for generating non-patient-specific supplemental content for a physician. [Figure 31] 1 is a flowchart of a process that may be used for onboarding oncologists. [Figure 32] 1 is a screenshot for use by a system administrator to visually specify system intents, intent parameters, and answer formats for a provider panel type, consistent with at least some aspects of the present disclosure. [Figure 33] FIG. 1 is a schematic diagram of an intent extraction architecture. [Figure 34] FIG. 1 is a schematic diagram of a question and answer workflow. [Figure 35] FIG. 1 is a schematic diagram of an exemplary conversation workflow. [Figure 36] 1 is a flowchart of a process for providing an audible response to an oncologist using at least one microservice and / or engine, consistent with at least some aspects of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0092] Various aspects of the present disclosure will now be described with reference to the drawings, wherein like reference numerals correspond to like elements throughout the several views. It should be understood, however, that the following drawings and the related detailed description are not intended to limit the claimed subject matter to the particular forms disclosed. Rather, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the claimed subject matter.

[0093] In the following detailed description, reference will be made to specific embodiments in which the present disclosure may be practiced, with reference to the accompanying drawings, which form a part hereof and which are shown by way of illustration. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure. It should be understood, however, that the detailed description and specific examples, while indicating examples of embodiments of the present disclosure, are given by way of illustration only, and not by way of limitation. It will be apparent to those skilled in the art from this disclosure that various substitutions, modifications, additional rearrangements, or combinations thereof may be made within the scope of the present disclosure.

[0094] In accordance with common practice, various features illustrated in the drawings may not be drawn to scale. The figures presented herein are not intended to be actual drawings of any particular method, device, or system, but merely idealized representations employed to describe various embodiments of the present disclosure. Thus, dimensions of various features may be arbitrarily expanded or reduced for clarity. Furthermore, some of the drawings may be simplified for clarity. Thus, the drawings may not depict all components of a given apparatus (e.g., device) or method. Furthermore, like reference numerals may be used to denote like features throughout this specification and the drawings.

[0095] The information and signals described herein may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof. Some figures may show signals as single signals for clarity of presentation and explanation. It will be understood by those skilled in the art that a signal may represent a bus of signals, and that the bus may have various bit widths, and that the present disclosure may be implemented with any number of data signals, including a single data signal.

[0096] The various illustrative logical blocks, modules, circuits, and algorithmic acts described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, the various illustrative components, blocks, modules, circuits, and acts have been described generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments described herein.

[0097] Further, it should be noted that the embodiments may be described in terms of a process that is depicted as a flowchart, a flow diagram, a structure diagram, or a block diagram. While a flowchart may describe operational acts as a sequential process, many of these acts may be performed in a different order, in parallel, or substantially simultaneously. Additionally, the order of acts may be rearranged. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, or the like. Furthermore, the methods disclosed herein may be implemented in hardware, software, or both. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another.

[0098] Reference to elements herein using designations such as "first," "second," etc. should be understood not to limit the quantity or order of those elements unless such limitation is expressly stated. Rather, these designations may be used herein as a convenient method of distinguishing between two or more elements or instances of an element. Thus, reference to a first and a second element does not imply that only two elements may be used therein, or that the first element must precede the second element in some way. Also, unless otherwise specified, a set of elements may comprise one or more elements.

[0099] As used herein, the terms "component," "system," and the like are intended to refer to a computer-related entity that is either hardware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to being, a processor, an object, an executable, a thread of execution, a program, and / or a process running on a computer. Illustratively, both an application running on a computer and the computer can be a component. One or more components may reside within a process and / or thread of execution, and a component may be localized on one computer and / or distributed between two or more computers or processors.

[0100] The word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects or designs.

[0101] Furthermore, the disclosed subject matter may be implemented as a system, method, apparatus, or article of manufacture using standard programming and / or engineering techniques to generate software, firmware, hardware, or any combination thereof, for controlling a computer or processor-based device to implement aspects described herein. As used herein, the term “article of manufacture” (or alternatively, “computer program product”) is intended to encompass a computer program accessible from any computer-readable device, carrier, or medium. For example, computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., cards, sticks). Furthermore, it should be understood that carrier waves can be used to carry computer-readable electronic data, such as those used in sending and receiving email or accessing networks such as the Internet or a local area network (LAN). Of course, those skilled in the art will recognize that many modifications may be made to this configuration without departing from the scope or spirit of the claimed subject matter.

[0102] The term "genetic analyzer" is used herein to mean a device, system, and / or method for determining the characteristics (including the sequence) of nucleic acid molecules (including DNA, RNA, etc.) present in a biological specimen (including a tumor, biopsy, tumor organoid, blood sample, saliva sample, or other tissue or bodily fluid).

[0103] The term "genetic profile" is used herein to refer to the combination of one or more variants, RNA transcriptome, or other informative genetic characteristics determined for a patient from next-generation sequencing, which is sometimes commonly referred to as "massively parallel sequencing."

[0104] The term "gene sequence" is used herein to mean a record of the series of nucleotides present in a patient's RNA or DNA, as determined from sequencing the patient's tissue or body fluids.

[0105] The term "variant" is used herein to mean a difference in a gene sequence or gene profile when compared to a reference gene sequence or expected gene profile.

[0106] The term "expression level" is used herein to mean the number of copies of an RNA or protein molecule produced by a gene or other genetic locus, which may be defined by chromosomal location or other genetic mapping indicator.

[0107] The term "gene product" is used herein to mean a molecule (including a protein or RNA molecule) produced by manipulation (including transcription) of a gene or other genetic locus, which may be defined by chromosomal location or other genetic mapping indicator.

[0108] Referring now to the drawings, in which like reference numerals correspond to like elements throughout the several views, and more particularly to FIG. 1 , the present disclosure will be described in the context of an exemplary collaboration system 10 consistent with at least some aspects of the present disclosure. System 10 includes a collaboration server 12, an artificial intelligence (AI) server 14, a user interface collaboration device 20, and a service provider database 18. Referring again to FIG. 1 , in the illustrated embodiment, AI server 14 is shown as separate from collaboration server 12. Nevertheless, it should be understood that in at least some embodiments, the functionality of the two servers may be performed via a single server. Similarly, while exemplary system 10 is described herein as having certain process steps or functions performed by server 12 and others performed by server 14, in other cases, the division of functions and steps between the two servers 12 and 14 may differ. Furthermore, in at least some embodiments, some of the processes performed by servers 12 and 14 may be performed by a processor located within collaboration device 20. For example, at least in some cases, some or most of the processes related to speech recognition, intent matching, parameter extraction, and audio response generation performed by AI server 14 may be performed by collaboration device 20. By performing at least some of the processes performed by servers 12 and 14 on collaboration device 20, the latency for outputting generated responses can be reduced by up to two seconds, as described in further detail below.

[0109] The collaboration server 12 is linked to a wireless transceiver (e.g., a transmitter and receiver) 16, enabling wireless two-way communication between the collaboration device 20 and the collaboration server 12. The transceiver 16 may be any type of wireless transceiver, including, for example, a cellular telephone transceiver, a Wi-Fi transceiver, a Bluetooth transceiver, a combination of different types of transceivers (e.g., including Bluetooth and cellular), etc. The server 12 executes software applications or modules to perform the various processes and functions described throughout this specification. In particular, the server 12 executes a collaboration application 60 that includes, among other things, a visual response module 62 and a data behavior module 64. Server 12 receives user voice queries (hereinafter “voice messages”) 59 captured by device 20, collaborates with AI server 14 to identify the meaning of the voice messages (e.g., intent and key parameters), performs data operations on data in database 18 that match the voice messages to generate data responses, collaborates with AI server 14 to generate audio response files based on the data responses and, at least in some cases, visual response files, and sends response files 73, 77 back to collaboration device 20. Device 20 then broadcasts the audio responses to the user (66) and, if a visual response is suitable for presentation via device 20, generates a visual response in some manner (e.g., presenting content on display 48 of device 20, illuminating signal lights 50, etc.). Display 48 and / or signal lights 50 may be considered visual indicators.

[0110] 2, collaboration device 20 includes an external housing 22, a device processor 30, a battery 32 or other power source, device memory 34, a wireless transceiver 36, one or more microphones 38, and one or more speakers 44 or audio output devices, as well as several components or processes that can be used to activate device 20 to begin user collaboration activities. External housing 22 includes an outer surface that forms a sphere in the illustrated example, with the diameter of the sphere selected so that device 20 can be easily held in the hand by an oncologist. For example, the diameter of device 20 will most often be between three-quarters and five inches, and in particularly advantageous embodiments, the diameter will be between one and a quarter inch and two inches.

[0111] In other cases, the outer housing includes an exterior surface that forms a cube or other three-dimensional rectangular prism. In such cases, in particularly advantageous embodiments, the maximum dimension of the three-dimensional shape (height, width, depth) is between 1 1 / 4 inches and 2 inches.

[0112] System 10 may be implemented in other ways. For example, collaboration device 20 may be a smartphone, tablet, laptop, desktop, or other computing device such as an Apple iPhone®, a smartphone running the Android® operating system, or an Amazon Echo. Some of the processes performed by servers 12 and 14 may be performed through the use of an app or another program running on a processor located within collaboration device 20.

[0113] The exterior surface may be formed by several different components from several different materials, including opaque materials for some of the surfaces and transparent or translucent materials for other portions that need to pass light from indicator lights mounted within the housing. The exterior surface of the housing may form apertures for speakers and microphones, a charging port opening (not shown), and other apertures or openings for different purposes. The housing forms an internal cavity within which most of the other device components are mounted. While device 20 may include a single speaker and a single microphone, in an optimized assembly, device 20 includes multiple speakers and microphones arranged around the housing assembly to capture the oncologist's audio signals from all directions.

[0114] There are many different hardware configurations that may be used to provide the collaboration device processor 30. One particularly useful processor for the purposes of the present device 30 is the Qualcomm QCS405 SoC (system-on-chip), which supports many different types of connectivity, including Bluetooth, ZigBee, 802.11ac, 802.11 ax-ready, USB-C 2.0 / 3.0, and others. This solution includes an on-device AI engine that enables execution of AI algorithms on the device, so that, at least in some cases, the AI ​​functions described herein with respect to the server 14 may be performed by the processor 30. This SoC supports up to four microphones and supports high-performance keyword detection. The processor 30 is linked to each of a battery 32, memory 34, a transceiver 36, a microphone 38, and a speaker 44. In some embodiments, the battery 32 can be charged using a charging dock (not shown).

[0115] While device 20 is activated and remains active, microphone 38 captures user voice messages 57 that are provided to processor 30. Processor 30 transmits the voice messages (59) via transceiver 36 to collaboration server 12 (see again FIG. 1 ). Audio response files are received (81) by transceiver 36 of device 20, and processor 30 broadcasts those response files via speaker 44. Although not shown, it is contemplated that device 20 may also include some type of haptic signaling component (e.g., a vibrator, etc.) to indicate one or more device states.

[0116] Device 20, and more specifically, memory 34, may include acoustic processes, light control processes, security processes, connectivity processes, and other suitable purposes. These processes may be stored as firmware in a portion of memory 34 that is non-volatile and, in some embodiments, read-only. The firmware may include an acoustic process that recognizes a library of wake words or phrases spoken by the oncologist. For example, the library may include the phrases "Tempus ONE" or "Hey, One." By storing key phrases repeatedly used by the oncologist directly in memory 34, latency in processing commands spoken by the oncologist may be reduced. The acoustic processes may also include a silence detection process, a fallback audio response playback process that audibly notifies the oncologist of errors or timeouts that occur during data transmission (e.g., TCP packets or HTTP messages), a speaker protection algorithm, a digital signal processing (DSP) algorithm, and / or other suitable processes related to acoustics.

[0117] The firmware may include a conversation flow process for determining whether a follow-up question requires the use of a wake word phrase, such as "Tempus ONE." For example, an initial question, "What were Dwayne Holder's results?" may be followed by "And how old was he?" without the need to use the "Tempus ONE" wake word phrase or re-specify the patient's name. The question, "And how old was he?" may not be relevant unless the person in question has already been identified, in which case a follow-up question can be asked. The firmware may include a battery status algorithm for determining the charge level and / or state of charge of the battery 32. The firmware may include connection and security processes for storing and / or maintaining secure element cryptographic keys, storing device identifiers, storing valid networks (e.g., Wi-Fi networks), and other suitable processes. In some embodiments, the firmware may be updated over the air.

[0118] The firmware may also include a lighting control process for controlling the indicator light 50. The lighting control process may change the color and / or brightness of the indicator light 50 and pulse the indicator light 50 on and off.

[0119] Firmware processes can be used to control indicator lights 50 and / or speaker 44 based on the state of collaboration device 20. Some states are initiated by the oncologist. The oncologist can actuate one or more of input buttons 52, pronounce commands, and / or move collaboration device 20 (e.g., by placing collaboration device 20 in a charging dock). Exemplary light and speaker controls based on the state of collaboration device 20 are included in Table 1 below. Some oncologist interactions include "app" functionality, which is described in more detail below.

[0120] [Table 1A]

[0121] [Table 1B]

[0122] In at least some cases, device 20 may be activated by a specially voiced voice command. To this end, processor 30 may always be on, monitoring for a special trigger activation command, such as "Hey, query." Once an activation command is received, processor 30 may be activated to participate in the user collaboration session. Here, processor 30 may confirm the activation command by transmitting a response, such as "Hello, how can I help you?" or a tone or other audio indication, and then enter a "listening" state to capture a subsequent user voice message. Once the subsequent voice message is captured, the collaboration session may proceed as described above.

[0123] In addition to or instead of being activated by a spoken activation command, device 20 may be activated by selecting a device activation button or touch sensor, such as when device 20 is picked up or otherwise moved. To this end, see optional input button 52 and motion and orientation sensors 40 and 42 in FIG. 2. The motion sensor may include an accelerometer, a gyroscope, both an accelerometer and a gyroscope, or some other type of motion sensor device.

[0124] In addition to being able to provide audio responses to a user's queries, at least in some cases, device 20 is equipped to provide some kind of visual response. For example, in a simple case, device 20 may include one or more indicator lights 50, which may activate or control LEDs or other light sources to change color to complicate different device 20 states. For example, at least in some cases, when device 20 is inactive and waiting to be activated, indicator light 50 may be off or dim green. Here, while device 20 is activated and waiting for or listening to an audio message, light 50 may be activated bright green to indicate "go." When the user is speaking and an audio message is being captured by device 20, light 50 may be activated teal to indicate an audio message capture state. When the query audio signal ends, light 50 may illuminate yellow to indicate a "thinking" or query processing state. When the audio response is being broadcast to the user, the light 50 may illuminate orange to indicate an output status, and once the audio response is complete, the light 50 may illuminate bright green again to indicate that the device is again waiting or listening for the next voice message to be spoken by the user.

[0125] In at least some cases, whenever device 20 is activated and waiting for a new or next voice message, device 20 may be programmed to wait in an active state for a threshold period (e.g., 30 seconds) and then assume an inactive state waiting to be reactivated via another activation utterance or other user input. In other cases, once device 20 is activated, it may remain activated for a longer period (e.g., 10 minutes) and enter the deactivated listening state only if the user utters a deactivation phrase (e.g., "end session," "end query," or "Hey, query" followed by "end session") or otherwise actively deactivates device 20 (e.g., by selecting deactivation input button 52).

[0126] Still referring to FIG. 2 , in some cases, device 20 may include one or more flat, curved, or other contoured display screens 48 for presenting visual responses to user queries when the visual responses are suitable for consumption via a relatively small display screen. Here, for example, a short answer to a user query may be presented as text via display 48. As another example, a summary phrase associated with a data response including data that cannot be easily presented via a small display screen may be generated and presented via display 48. Other text phrases or graphics are contemplated for other purposes. For example, if the visual response is presented via some other display device (e.g., a display device paired or otherwise associated with collaboration device 20), a text message may be presented via display 48 indicating that additional information or a visual response is being presented via the associated display. As another example, display 48 may be controlled to illuminate a particular color to indicate a status, as described above with respect to light device 50, or may present only the answer to the query in text format. Referring again to FIG. 1 , AI server 14 executes software application programs and modules that perform various functions consistent with at least some aspects of the present disclosure. In at least some cases, the AI ​​server 14 includes an automatic speech recognition (ASR) module 70, an intent matching module 72, a parameter extraction module 74, and an audio response module 76.

[0127] The ASR module 70 receives user voice messages from the collaboration application 60 (61) and automatically converts the voice signal into text corresponding to the user-spoken voice message, essentially in real time. Thus, if the oncologist's voice signal message is, "How many male patients over 45 years old have pancreatic cancer?" or "What type of treatment should be prescribed for this patient?", the ASR module 70 generates matching text using speech recognition software. Speech recognition applications are well known in the art and include Nuance's Dragon software, Google Voice (voice) by Google, and IBM's Watson. In some cases, the recognition application supports industry-specific term / phrase dictionaries in which specific terms and phrases used within an industry are defined and can be recognized. In some cases, user-specific dictionaries are also supported for terms or phrases routinely used by specific oncologists. In either case, new terms and phrases can be added to the industry and user dictionaries. The text file is provided to the intent matching module 72.

[0128] The intent matching module 72 includes a natural language processor (NLP) programmed to determine the intent of the user's voice signal message. Here, for example, the intent may be to identify a data subset in database 18. As another example, the intent associated with the phrase "How many male patients over 45 years old have pancreatic cancer?" may be to identify the number of patients. As another example, the intent associated with the phrase "What type of treatment should be prescribed for patient John Doe?" may be to identify the treatment that the system determines will maximize patient John Doe's quality of life. Literally thousands of other intents may be recognized by the matching module 72. Intents are described in more detail below.

[0129] 1, parameter extraction module 74 extracts key parameters from user-uttered voice messages. For example, parameters extracted from the phrase "How many male patients over the age of 45 have pancreatic cancer?" may include "pancreas," "male," and "45 years old." For each user voice message, AI server 14 returns (63) (i) the associated text file, (ii) the matching intent, and (iii) the extracted parameters to collaboration server 12, more specifically, data behavior module 64.

[0130] The data action module 64 accesses the database 18 and creates a collaboration record on the database to memorialize the collaboration session (65). The text file received from the server 14 is stored in the database 18 along with the date and time, oncologist identification information, etc. The data action module 64 converts the intent and extracted parameters into data actions and then performs the actions 65 on the data in the database 18. For example, for the voice message, "How many male patients over 45 years old have pancreatic cancer?", the action module 64 constructs a database query to retrieve the number of male patients over 45 years old (e.g., intent) who have pancreatic cancer (e.g., extracted parameters). The data action results in a data response including the number of male patients over 45 years old who have pancreatic cancer.

[0131] As another example, for the voice message, "What type of medication should I prescribe for John Doe?", the operational module 64 constructs a database query to search for medications (e.g., intent) for a cohort of patients who are clinically similar to patient John Doe and for whom such medications have produced optimal outcomes for the cohort. Determining whether a cohort of patients is clinically similar may be accomplished by querying the database 18 for patients with certain factors identical and / or similar to those of John Doe, such as age, cancer stage, previous treatments, mutations, RNA expression, etc. As a simple example, if John Doe has a PTEN genomic mutation, the database 18 may select all patients who also have a PTEN genomic mutation for inclusion in the cohort. As another example, if John Doe has metastatic prostate cancer but has become unresponsive to first-line androgen-suppressive therapy, the database 18 may select all metastatic prostate cancer patients who have become unresponsive to first-line androgen-suppressive therapy for inclusion in the cohort.

[0132] As another example, for the voice message "What is Jane Smith's expected progression-free survival if she is prescribed Keytruda?", the operation module 64 constructs a database query to search for patients clinically similar to Jane Smith, selects a cohort of those patients who were prescribed Keytruda, analyzes the progression-free survival of the selected cohort of patients, and returns the mean progression-free survival from the selected cohort.

[0133] As indicated above, the physician's voice message may be related to a question about a specific individual. The operations module 64 may further be configured to access a patient data repository to identify the patient's clinical, genomic, or other health information. The patient data repository can take many forms and may include an electronic health record, a health information exchange platform, a patient data warehouse, a research database, etc. The patient data repository may include data stored in a structured format, such as a relational database, a JSON file, or other data storage configurations known in the art. The operations module 64 may communicate with the patient data repository in various ways, such as through data integration, may use various technologies, or may rely on various frameworks, such as Fast Healthcare Interoperability Resources (FHIR). The patient data repository may be owned, operated, and / or managed by the physician, the physician's employer, a hospital, a physician's office, a clinical laboratory, a contract research organization, or another entity associated with the delivery of health care. The patient data repository may include all of the patient's health information or a subset of the patient's health information. For example, a patient data repository may include structured data including patient demographic information (e.g., age, sex), a clinical description of the patient's cancer (e.g., stage such as "stage 4" and subtype such as "pancreatic cancer"), a genomic description of the patient and / or the patient's cancer (e.g., list of nucleotides in specific introns or exons, somatic variants such as "BRAF mutations," mutant allele frequencies, immunological markers such as microsatellite instability and tumor mutation burden, RNA over- or under-expression, a list of pathways affected by the variants found, etc.), an imaging description of the patient's cancer (e.g., features derived from radiology or pathology images), an organoid origin description of the patient's cancer (e.g., a list of treatments that have been effective in reducing or destroying organoid cells derived from the patient's tumor), and a list of previous and current pharmaceutical therapies, treatments, surgeries, procedures, or other treatments.

[0134] The operational module 64 may use various methods to identify how a particular patient being queried is clinically similar to other individuals whose data are stored in the database 18. An example of determining clinical similarity is described in U.S. patent application Ser. No. 16 / 671,165, filed October 31, 2019, the contents of which are incorporated by reference in their entirety for all purposes. Another example of determining clinical similarity is described in U.S. patent application Ser. No. 16 / 732,168, filed December 31, 2019, the contents of which are incorporated by reference in their entirety for all purposes.

[0135] Determining which medications provided the optimal outcome for a cohort of identified individuals may be determined by comparing outcome information stored in database 18 for those individuals with the medications prescribed or administered, dividing the cohort into subcohorts, analyzing outcome measures such as progression-free survival, overall survival, and survival for each subcohort, and returning one or more measures indicative of the optimal outcome.

[0136] In another example, data actions module 64 may select a first treatment from a list of treatments, examine information from all patients in database 18 who were provided the first treatment, divide the patient group into a first cohort of patients with a positive outcome and a second cohort of patients without a positive outcome, compare health characteristics (e.g., clinical, genomic, and / or imaging) of the queried patient with the health characteristics of the first cohort, compare the health characteristics of the queried patient with the health characteristics of the second cohort, and determine whether the characteristics of the queried patient are closer to the characteristics of the first cohort or the second cohort. If the characteristics of the queried patient are more clinically similar to the first cohort, data actions module 64 may prepare a data response indicating the first treatment. If the characteristics of the queried patient are more clinically similar to the second cohort, data actions module 64 may not prepare a data response indicating the first treatment. The data operations module 64 may then select a second, third, fourth, etc. treatment from the list of treatments and repeat the above process for each selected treatment, or continue until all treatments in the treatment list have been explored. Various algorithmic approaches using mathematical or statistical methods known in the art may be used on the associated health characteristics to determine whether the queried patient characteristics are clinically similar to the first or second cohort, including mean, median, principal component analysis, etc.

[0137] In another example, the data operations module 64 may select all or a subset of records from a patient in the database 18. From these records, the module 64 may then select records from a first cohort of patients with similar genomic biomarkers to the queried patient. The module 64 may then filter the first cohort for patients prescribed a first treatment from the list of treatments. The module 64 may then examine the outcomes of the patients in the first cohort, subdivide the first cohort into two or more subcohorts based on the outcomes, and divide patients with similar outcomes into the same subcohort. Each subcohort may be further divided into additional subcohorts like the first cohort, and so on, until there are no significant outcome differences within each subcohort. At this point in the method, there may be dozens or more subcohorts. The data operations module 64 may then compare the health characteristics of the queried patient with those in each subcohort to identify the subcohort that is most clinically similar to the patient's health characteristics. The data operations module 64 may then select a second, third, fourth, etc. treatment from the list of treatments and repeat the above process for each selected treatment, continuing until all treatments in the list of treatments have been explored.

[0138] Data behavior module 64 returns the data response to AI server 14 (67), and more specifically, to audio response module 76, which uses the data to generate an audio response file. For example, if 576 male patients over the age of 45 had pancreatic cancer in the retrieved dataset, response module 76 might generate the phrase, "576 male patients over the age of 45 had pancreatic cancer." The audio response file is transmitted (71) to collaboration application 60, which stores the response file as well as its text representation in a collaboration record in database 18 for later access. Collaboration application 60 also transmits (73) the audio response file via transceiver 16 to collaboration device 20, which then broadcasts the audio file to users.

[0139] The AI ​​server 14 may be provided through many different software application programs. One particularly useful suite of software modules that may provide AI capabilities is the Qualcomm Smart Audio 400 Platform development kit, which can be used with the Qualcomm SoC processors mentioned above. Another useful suite is the Dialogflow program developed and maintained by Google. Dialogflow is an end-to-end, build-once-deploy-everywhere suite for creating conversational interfaces for websites and mobile applications. System administrators can use the Dialogflow interface to define a set of intents, training phrases, parameters, and responses to the intents. An intent is a general user intent (e.g., a user's desire) to access or manipulate database data in a particular way. For example, one intent may be to generate a database data subset (e.g., patients that meet qualifying query parameters). As another example, another intent may be to return a number (e.g., the number of patients that meet qualifying parameters). Other intents may be welcome intents (e.g., when a user first activates device 20), adverse sequence intents (e.g., returning a list of adverse sequences or at least an indication thereof to a treatment regimen), medication intents (e.g., returning a list or indication of previous medications), schedule event intents (e.g., scheduling an appointment, test, procedure, etc.), etc. A typical system is expected to include hundreds, and possibly thousands, of intents.

[0140] For each intent, the administrator provides a relatively small set of seeds or training phrases that are used to train the intent-matching module to recognize the intent associated with the received voice message. The training phrases include phrases that a user might say if the goal or purpose associated with the utterance matches the associated intent. For example, if there is an intent to return a large number of patients who meet qualifying parameters (e.g., age, disease, condition, cancer gene, mutation, residence, stage, treatment, medical YYY side effect, outcome, etc.), some typical training phrases might be "How many patients have pancreatic cancer?", "How many stage 3 breast cancer patients in Chicago are HER2 positive?", "How many patients have experienced side effects while taking XXX medication?", "How many ovarian cancer patients have developed p85 PIK3CA mutations in the past 48 months?", "What percentage of basal cell carcinoma patients have undergone cryosurgery in the past 18 months?", and "How many smokers also have lung cancer?" Dialogflow also supports follow-up intents, which may be sequentially associated with other intents, more specifically, second or subsequent intents recognized in a series of questions after a first intent has been identified. For example, a first phrase, "How many patients with ovarian cancer have had p85 PIK3CA mutations in the past 48 months?" might be followed by a second phrase, "How many of these patients have been seen in the past 12 months?" As another example, for the purpose of returning suggested treatments for a particular patient, some exemplary training phrases might be, "What type of medication should John Doe be prescribed?", "What type of immunotherapy should this patient receive?", or "What is Jane Smith's expected progression-free survival if we prescribe Keytruda?"

[0141] Once a small set of training or seed phrases is provided by an administrator, a machine learning module (e.g., an AI engine) uses those phrases to automatically train and generate many other similar phrases that can be associated with an intent. This automated training process, in which many similar queries are generated and associated with a particular intent, is called "fanning," and the newly generated queries are called "fan queries." The machine learning module stores the complete set of training phrases and derived phrases (hereinafter "intent phrases") for use during collaboration sessions. When a user subsequently uses the system to utter a phrase that is similar to, but not an exact match of, one of the intent phrases, the intent matching module recognizes the user's intent despite the inexact match and responds accordingly. Furthermore, if an utterance is similar to, but not exactly the same as, one of the intent phrases, the system automatically saves the utterance as an additional intent phrase associated with the intent and may train additional other intent phrases based on it, so that the intent matching module becomes more intelligent over time.

[0142] In most cases, a system user's intent alone is insufficient to identify the specific information the user is seeking or how to respond, requiring the user to vocalize or provide additional query parameters. Dialogflow allows an administrator to specify a set of parameter types to extract from received voice messages. For example, some parameters may include date, time, age, illness, condition, medication, treatment, procedure, physical condition, mental condition, etc. For each parameter type, the administrator specifies example parameter phrases or data combinations (hereinafter "parameter phrases") that a system user may vocalize to indicate the parameter. The machine learning module also uses the administrator-specified parameter phrases to train a larger set of parameter phrases that can be used to recognize instances of the parameter. During a collaboration session in which a user query is received, after module 72 identifies the intent, extraction module 74 uses the parameter phrases to extract parameter values ​​from the user's voice message. The intent and extracted parameters together provide the raw material needed by data actions module 64 to formulate data actions to perform on data in database 18 (see again Figure 1).

[0143] Dialogflow allows administrators to tag some parameters as needed and define feedback prompts that will be presented to users if a received voice message does not contain the required parameters. So, for example, if a particular intent requires a date and the query associated with that intent does not contain a date parameter, the system may automatically present the user with a feedback prompt requesting the date (e.g., "What date range are you interested in?").

[0144] Dialogflow also guides administrators in defining intent responses. Intent responses typically include text responses that specify one or more phrases, data responses, or formatted combinations of text and data that can be used to respond to a user's query. For example, if the intent is to return the number of patients who meet eligible parameters, the response phrase might be "The number of patients who have _" with spaces representing data fields to be filled with parameters from the voice message, data from a database, data derived from a database, or options specified in combination with the response phrase.

[0145] Hereinafter, the intent and all information associated with a particular intent specified by the system (e.g., parameters, fun queries, data actions, and answer phrases) may be referred to as the intent and supporting information to simplify this description.

[0146] At least in some cases, module 72 may identify at least one intent in the query. At least in some cases, the query may be an audible query. At least in some cases, the at least one intent may be an intent related to a clinical trial. At least in some cases, the at least one intent may be related to a pharmaceutical. At least in some cases, if the intent is related to a pharmaceutical, the intent may be referred to as a pharmaceutical intent. At least in some cases, the pharmaceutical intent may be related to a pharmaceutical, such as chemotherapy. At least in some cases, the pharmaceutical intent may be related to a PARP inhibitor intent. At least in some cases, the at least one intent may be related to a gene. At least in some cases, the at least one intent may be related to immunology. At least in some cases, the at least one intent may be related to a knowledge database. At least in some cases, the at least one intent may be related to a testing methodology. At least in some cases, the at least one intent may be related to a gene panel. At least in some cases, the at least one intent may be related to reporting. At least in some cases, the at least one intent may be related to an organoid process. At least in some cases, at least one intent may be related to imaging. At least in some cases, at least one intent may be related to pathogens. At least in some cases, at least one intent may be related to vaccines.

[0147] 1, the response module 76 uses the response phrases to generate responses, and more specifically, to generate audio response files that are sent back to the collaboration server 12. Again, it is contemplated that a typical system may include hundreds or thousands of response phrases, at least one response phrase format or structure for each intent supported by the system.

[0148] In the illustrated exemplary system 10, the AI ​​server 14 does not control the database 18 and therefore transmits the intent and extracted parameters back to the collaboration server 12, which executes the data behavior module 64. In this case, it is conceivable that many data responses may not be able to be presented to the user in an easily digestible audio response file. For example, in some cases, the data response may include a graphical display of comparative cancer data that cannot be easily explained audibly in an auditory-friendly manner. In these cases, after the data behavior module 64 receives the data response from the database 18, the module 64 may pass the data to the visual response module 62, which generates an appropriate visual response to the user's query, which transmits the visual response to the device 20 via the transceiver 16 for presentation.

[0149] In at least some cases, a summary audio response may be formulated by system 10 and broadcast via device 20 as needed. For example, in some cases, the data response may simply comprise a list-type subset of database data to form the basis for additional searching and data manipulation. For example, a sub-dataset may include data on all male cancer patients who have had adverse reactions to taking a medication since 1998. This sub-dataset may serve as data for a subsequent query that restricts the cancer type to pancreatic, or the treatment to treatment XXX, or any other more detailed parameter combination. In these cases where the database subset is limited, an appropriate audio response file may include a summary response such as, for example, "A subset of data on all male cancer patients who have had adverse reactions to taking a medication has been identified" (see 66 in Figure 1). This response phrase is specified via Dialogflow or other conversation definition software application.

[0150] It is believed that, at least in some cases, the system may be unable to associate an oncologist's voice query (i.e., an audible query) with an intent or system-supported parameter with a high level of confidence. In some cases, AI server 14 may be assigned a confidence factor for each intent and extracted parameter and may be programmed to return one or more probing queries to the oncologist if the confidence factor for the intent or parameter value falls below some threshold level. In some cases, to help direct the oncologist to system-supported queries, the probing feedback queries may be tailored or customized to known structures or data content in database 18 or to intents and parameters supported by AI server 14.

[0151] If the intent and / or extracted parameters are not supported by the AI ​​server or other system process, system 10 may generate a record of the unsupported query for review by an administrator and subsequent access by the oncologist. In such cases, the system may present the unsupported query and related information to an administrator during a system maintenance session to allow the administrator to determine whether a new intent and / or parameters need to be specified in Dialogflow or via some other query flow application. If the administrator specifies new intent and / or parameters, the system may provide a data response to the query and update the collaboration record containing the unsupported query to indicate that the query is now supported, and the oncologist may be notified via email, text, or other method that the query is supported during a subsequent collaboration session.

[0152] In some cases, database 18 may include an electronic health record database from a hospital or hospital system. In other cases, database 18 may include an electronic data warehouse with data extracted from EHRs, transformed, and loaded into a multidimensional data format. In other cases, database 18 may include data collected from multiple hospitals, clinics, health systems, and other providers throughout the United States and / or internationally. The data in database 18 may include clinical data elements reflecting the health status of multiple patients over time. Clinical data elements may include, but are not limited to, demographics, age / DOB, sex, race / ethnicity, facility, relevant comorbidities, smoking history, diagnosis, site (tissue of origin), date of initial diagnosis, histology, histology grade, metastatic diagnosis, date of metastatic diagnosis, site of metastasis, stage (e.g., TNM, ISS, DSS, FAB, RAI, Binet), evaluation, lab and molecular pathology, lab type (e.g., CBS, CMP, PSA, CEA), lab result and unit, lab date, performance status (e.g., ECOG, Karnofsky), performance status score, performance status date, molecular pathology test date, gene / biomarker / assay, gene / biomarker / assay result (e.g., positive, negative), may include subtype-specific data elements (e.g., gender, ambiguous, mutation, wild type), molecular pathology method (e.g., IHC, FISH, NGS), molecular pathology provider, additional subtype-specific data elements (e.g., prostate PSA), treatment, drug name, drug start date, drug end date, drug dose and units, number of drug cycles, type of surgical procedure, date of surgical procedure, radiation site, radiation modality, radiation start date, radiation end date, total dose of radiation administered, and total percentage of radiation administered, outcome, response to treatment (e.g., CR, PR, SD, PD), RECIST, outcome / observation date, date of progression, date of recurrence, adverse event to treatment, date of presentation of adverse event, grade of adverse event, date of death, date of last follow-up, and disease status at last follow-up.The information in database 18 may have data in a structured format, for example, through the use of a data dictionary or metadata repository, which is a repository of information about information such as meaning, relationships to other data, origin, usage, and format. The information in database 18 may be in the form of original medical records such as pathology reports, progression records, DICOM images, medication lists, etc.

[0153] Database 18 may further include other health data associated with each patient, such as next-generation sequencing (NGS) information generated from the patient's blood, saliva, or other normal specimens; NGS information generated from the patient's tumor specimen; imaging information, such as radiology images, pathology images, or extracted features thereof; metabolic information, epigenetic analysis, proteomics information, and other omics information. Examples of NGS information may include DNA sequencing information and RNA sequencing information. Examples of imaging information may include planning CT scans, contours, radiation plans, dose distributions, and radiation therapy images, such as cone-beam CT scans, radiology scans, CT scans, and PET scans. Information in database 18 may include disease state information at the time of diagnosis (e.g., cancer diagnosis), longitudinal patient information, such as 6 months post-diagnosis, 1 year post-diagnosis, 18 months post-diagnosis, 2 years post-diagnosis, 30 months post-diagnosis, 3 years post-diagnosis, 42 months post-diagnosis, and 4 years post-diagnosis. Information in database 18 may include protected health information. Information in database 18 may include de-identified information.For example, information in Database 18 includes (1) patient name, (2) address, city, county, neighborhood, zip code, and their equivalent geographic codes, but according to data currently published by the Census Bureau, (a) all geographic subdivisions smaller than a state, excluding the first three digits of the zip code, if the geographic unit formed by combining all zip codes with the same first three digits contains 20,000 people or more, and (b) all such geographic units containing 20,000 people or less have the first three digits of the zip code changed to 000, (3) all elements (excluding year) of dates directly related to the individual, including date of birth, date of enrollment, date of discharge, date of death, and all ages 89 and older, and such ages and elements are aggregated into a single category for ages 90 and older. The database 18 may be in a structured format that does not include, except that it may include, all elements of a date (including year) that indicate such age, (4) telephone numbers, (5) vehicle ID and serial numbers, including license plate numbers, (6) fax numbers, (7) device identifiers and serial numbers, (8) email addresses, (9) web universal resource locators (URLs), (10) social security numbers, (11) internet protocol (IP) addresses, (12) medical record numbers, (13) biometric identifiers, including fingerprints and voiceprints, (14) health insurance beneficiary numbers, (15) full-face photographs and any equivalent images, (16) account numbers, (17) certificate / license numbers, and (18) any other unique identifying numbers, features, or codes. The number of records of information in database 18 may reflect information from 10, 100, 1,000, 10,000, 100,000, 1,000,000, 10,000,000, or more patients. Other examples of the type of information in database 18 are described in U.S. patent application Ser. No. 16 / 657,804, filed October 18, 2019, the contents of which are incorporated herein by reference in their entirety for all purposes.

[0154] Collaboration device 20 can reduce the amount of personal health information that may currently be included in emails sent to oncologists. Collaboration device 20 can delete the generated response (visual or audio) from memory 34 after the response is output through speaker 44. Thus, personal health information may be eliminated from memory external to database 18. Collaboration device 20 can be configured to recognize a “can you repeat that?” command, which causes collaboration device 20 to re-query the last question asked by the oncologist (e.g., from collaboration server 12), play the response again, and delete it from memory 34.

[0155] 2 and 3, an embodiment of an exemplary second collaboration device 20a is shown. The collaboration device 20a can include a second external housing 22a including seven substantially flat sides. The collaboration device can include a second light device 50a, one or more second microphones 38a that can be positioned near a small circular opening in the second external housing 22a, and a second speaker 44a that can be positioned near a substantially oval opening in the second external housing 22a. In some embodiments, the second external housing 22a can be made of metal and may include stainless steel and / or anodized steel. The second collaboration device 20a can be approximately 1.5 inches wide, 1.5 inches high, and 1.5 inches deep. In some embodiments, the second speaker 44a can be headphones coupled to the second collaboration device wirelessly (e.g., via Bluetooth) or via a wired connection (e.g., a 3.5 mm jack audio cable).

[0156] 2, 3, and 4, the second collaboration device 20a, similar to the collaboration device 20 described above, may include a processor 30 linked to a battery 32, a memory 34, a transceiver 36, input buttons 52, a display screen 48, and each of the sensors 40, 42. The processor 30 may also be linked to a second light device 50a, a second microphone 38a, and a second speaker 44a. The transceiver 36 may be configured to communicate using a 5G cellular network protocol.

[0157] The second collaboration device 20a may include a touch interface 23 capable of receiving input from the oncologist. The touch interface 23 may be linked to a processor 30. The touch interface 23 may include a second external housing 22a and a sensor (not shown), such as a force sensor, coupled to the second external housing 22a. The sensor may sense deflection of the second external housing 22a and output a corresponding signal to the processor 30. In some embodiments, the sensor may include a state-of-the-art force sensor film. The second external housing 22a may be marked (e.g., engraved) in appropriate locations to identify different sensing areas that may correspond to different virtual buttons (e.g., "power," "OK," "mute," etc.) for the oncologist.

[0158] In some embodiments, touch interface 23 may include one or more touch sensors positioned to receive input from the oncologist. The one or more touch sensors may include one or more capacitive touch sensors configured to output signals to processor 30. In some embodiments, the touch sensors may be positioned below one or more display screens 48. Processor 30 may provide input (e.g., by displaying instructions and / or prompts on one or more display screens 48 and / or by issuing audible instructions at speaker 44a) to prompt the user to receive a signal associated with the oncologist from the one or more touch sensors. Processor 30 may authenticate the oncologist based on the signal by determining whether the signal matches a predetermined fingerprint profile associated with the oncologist. Processor 30 may determine a selection (e.g., menu options, power on / off commands, mute commands, etc.) based on the signal.

[0159] The second collaboration device 20a may also include a power interface module 33 coupled to the battery 32 to power the battery 32. The power interface module 33 may include any suitable hardware for regulating the power supplied to the battery. The power interface module 33 may include a hardwired interface (not shown) for connecting to a complementary interface coupled to an external power source. The hardwired interface may include copper or gold contact pins and may be magnetic. Alternatively, the power interface module 33 may include one or more transformers (not shown) for wirelessly receiving power. The one or more transformers may include a pot-core transformer configured to receive power transmitted at approximately 300 MHz, which is higher than other wireless charging systems that use standards such as the Qi wireless power transmission standard. Certain wireless charging systems may use coils to implement wireless charging. These wireless charging systems, which may be compatible with the Qi standard, may not be usable with the second collaboration device 20a due to the small size of the collaboration device 20a. In embodiments in which the second external housing 22a is made of metal, care must be taken to ensure that wireless charging does not result in heating of the second external housing 22a. A pot core transformer may funnel the transferred energy into the battery 32 to prevent energy from being dissipated in the second external housing 22a rather than in the coil transformer.

[0160] The second collaboration device 20a may include a mesh networking transceiver 37 linked to the processor 30. The mesh networking transceiver 37 may be a Wi-Fi transceiver, a Z-Wave transceiver, a Zigbee transceiver, or a combination of different types of transceivers (e.g., including Z-Wave and Zigbee). In particular, the mesh networking transceiver 37 may communicate at frequencies other than one or more of the frequencies used by the transceiver 36 to communicate with the transceiver 16, as shown in FIG.

[0161] Although transceiver 36 may be used to communicate with transceivers included in other collaboration devices, mesh networking transceiver 37 may reduce potential transmission traffic on communication frequencies used by transceiver 36 to communicate with transceiver 16. For example, transceiver 36 may be used to communicate with transceiver 16 at a 2.4 GHz frequency (e.g., using a Wi-Fi or Bluetooth protocol), and mesh networking transceiver 37 may be used to communicate with another mesh networking transceiver 37 in another collaboration device 20a at a 900 MHz frequency (e.g., using a Z-Wave protocol).

[0162] The transceiver 36 and / or the mesh networking transceiver 37 may be configured to transmit and receive information using an ultra-wideband (UWB) protocol, which may be useful for detecting the real-time location of the second collaboration device 20a and / or for tracking how an oncologist is operating the second collaboration device 20a.

[0163] The second collaboration device 20a may include a secure element 35 linked to the processor 30. The secure element 35 may perform authentication and encryption tasks, such as key storage. The secure element may include an ATECC608A microchip from Microchip Technology Inc.

[0164] It is understood that at least some of the components included in the second collaboration device 20a, such as the touch interface 23, the second optical device 50a, the second microphone 38a and second speaker 44a, the mesh networking transceiver 37, the secure element 35, and the power interface module 33, may be included in the collaboration device 20 and may be linked to the processor 30 and / or the battery 32. In some embodiments, the processor 30 can be an SoC, such as the Qualcomm QCS405 SoC mentioned above. The SoC can be used to implement edge computing and machine learning processes locally in the second collaboration device.

[0165] Referring now to Figure 5, a process 100 for facilitating a collaboration session that may be implemented via the systems of Figures 1 and 2 is shown, consistent with at least some aspects of the present disclosure. Process 100 will first be described in the context of a system in which the only interface device used by oncologists is collaboration device 20 (e.g., the system does not include a supplemental or additional large display screen or other light-emitting surface for presenting additional visual data response representations to the user). In this type of system, the portions of process 100 enclosed in dashed lines are not present.

[0166] 1, 2, and 5, at process block 102, an industry-specific data set is stored and maintained in database 18. At block 104, intent matching module 72, parameter extraction module 74, and audio response module 76 are each trained using Dialog flow or some other conversation definition application as described above. Further, at block 104, visual response module 62 is programmed to receive data responses from module 64, where the responses provide seed data for constructing a graphic or other visual representation of the response information.

[0167] 1 and 5, in a system including only an interface device 20, control passes from block 104 to block 106, where the collaboration device 20 monitors activation (e.g., voice activation, movement, activation button selection, etc.). When the collaboration device 20 is activated in block 108, control passes to block 112, where an audio signal is captured by the device 20 and the audio signal is transmitted to the collaboration server 12 (57). In block 114, the captured audio signal is transmitted to the AI ​​server 14 (61), where an ASR module 70 transcribes the audio signal into text, an intent matching module 72 examines the text file to determine the oncologist's intent, and a parameter extraction module 74 extracts key parameter values ​​from the transcribed text. The text file, intent, and extracted parameters are returned to the collaboration server 12, and more specifically, to the data operations module 64 (63).

[0168] In block 116, the data action module 64 instantiates a new collaboration record on the database 18 and stores the text file in the collaboration record (65). The action module 64 also uses the intent and extracted parameters and associated values ​​to construct a data action in block 118, which is executed in block 120 resulting in a data response. In process block 124, the action module 64 provides the data response to the AI ​​audio response module 76 (69), which generates an audio response file. The audio response file is sent back to the collaboration application (71) and sent to the collaboration device 20 in process block 126 (73, 81). The audio response file and associated text are stored in block 126 as part of the collaboration record. The audio response file is broadcast over the speaker 44 of the device 20 for the oncologist to hear in block 128 (66), after which control returns to block 106 and the process continues to cycle indefinitely.

[0169] If the collaboration session persists over multiple rounds of oncologist queries and system responses, the oncologist's voice messages and associated text and response files, and each associated text, are stored in a collaboration record, and a series of preceding and following voice and response messages are captured for subsequent access and review.

[0170] In at least some embodiments, the system supports audio file broadcast functionality as well as process status or state information, as well as visual output functionality that provides at least some level of response data in response to user queries. For example, in FIG. 1 , once the oncologist's speech signal is captured by device 20 and AI server 14 generates transcribed text, server 12 may transmit that text file to device 20 for presentation in real time via display 48 as a feedback mechanism to allow the oncologist to confirm that the query was accurately perceived. Here, in some cases, the feedback text may persist (e.g., lasting a few seconds in most scenarios) or may persist for a set period (e.g., 5-7 seconds) before being replaced with a visual data response as needed. In other cases, the feedback text may only be replaced via the next feedback text phrase, allowing the oncologist more time to assess the accuracy of the perceived speech.

[0171] As another example, and still referring to FIG. 1 , where the data response is suited to visual representation or even best presented visually via the device display 48, the data response, or portions thereof, may be provided to the visual response module 62, as shown at 63. In these cases, module 62 uses the data response to create a visual response file (see 77 and 81) that is transmitted to the device 20 to drive the display 48. In some cases, the presented visual response may include a textual representation of an audio response file. In other cases, the visual response may include a reminder, alert, notification, or any other type of user instruction. When a visual file is generated and presented to the user, the collaboration server 12 may store all visual representations as part of the ongoing collaboration record for subsequent access.

[0172] Referring now to FIG. 6 , an exemplary collaboration conversation between oncologist 150 and collaboration device 20 is shown, with the oncologist's voice message shown in left column 160 and the interleaved audio response broadcast by device 20 shown in right column 162. When device 20 is activated, it responds with the phrase “How can I help you?” to prompt oncologist 150 to utter a first substantive query of database 18. Oncologist 150 responds with a first query of “Select patients with pancreatic cancer.” Here, consistent with the above description, AI server 14 ( FIG. 1 ) identifies the intent and query parameters used to construct a data operation that generates a data response and ultimately the audio response “Patients with pancreatic cancer cohort identified.” Oncologist 150 then utters a second query of “Limit cohort to males,” causing the system to construct and execute another data operation to generate another audible response. This back-and-forth "conversation" continues until Oncologist 150 ends the session.

[0173] If the collaboration application 60 stores collaboration records in the database 18, the system allows oncologists to access those records to update their memory or initiate more detailed queries, supported by additional output affordances, such as a large workstation display screen. To this end, see FIG. 7 , which illustrates the input and output devices at a workstation, including a large flat-panel display screen 170, a keyboard 172, and a mouse input device 174. The mouse 174 controls an on-screen pointing icon 176 for selecting virtual icons and tools on the screen, as is well known in interface technology. The screenshot on the display 170 shows a collaborator window 180 containing a list of oncologist-system collaborations for a particular oncologist, selectable to access the complete collaboration record. The list includes two columns, including a date column 182 indicating the date of the corresponding collaboration session, and a collaboration column 184 containing the first query corresponding to each collaboration represented in the list. The first entry in column 184 corresponds to the collaboration session shown in FIG. 6 and is indicated as selected via icon 176, highlighted to indicate selection.

[0174] When the first entry in column 184 is selected, screenshot 190 shown in Figure 8 may be presented, including the complete collaboration record in text with the oncologist query in first column 192 and the audio system response represented as text in second column 194. The example in Figure 8 corresponds to the conversation in Figure 6. Here, although the conversation is presented as text, it is contemplated that the oncologist may play an audio recording of the conversation as a memory aid, and to that end, a selectable "play" icon 196 is provided to play the collaboration audio.

[0175] While collaboration device 20 is advantageous due to its relatively small size and portability, in at least some cases, data response presentation may be more appropriate via visual representation than audio, or audio representation may be best supplemented via visual representation on a larger scale than provided by device display 20. To this end, it is contemplated that portable collaboration device 20 may be supplemented as an output device via a nearby large flat-panel display screen when a larger visual representation of the response data is optimal. Referring now to FIG. 9, an input / output configuration 200 is shown that may serve as an alternative to collaboration device 20 in FIG. 1. In FIG. 9, the input / output configuration includes portable collaboration device 20, a nearby large flat-panel display screen 202, and an input keyboard 204 and mouse device 206, respectively.

[0176] Still referring to FIG. 9 , in at least some cases, device 20 may be programmed to wirelessly “pair” with any Bluetooth or other wireless protocol-enabled display screen in the general vicinity of device 20 when some pairing event occurs. Here, a pairing event may simply include any time device 20 is in proximity to a pairable display 202, regardless of whether device 20 is activated to listen for a user's audio signals. In other cases, device 20 may pair with a display only when device 20 becomes active (e.g., a pairing event would be activation of device 20). In still other cases, pairing may occur only when device 20 receives a video response file requiring large display 202 for content presentation (e.g., a pairing event would be receipt of a video file containing data optimally presented on a large display screen).

[0177] Regardless of the pairing event, pairing may occur automatically upon the occurrence of the event or may require some affirmative action by the user to pair. For example, the affirmative action may include device 20 broadcasting a voice query to the user requesting permission to pair with display 202, and the user uttering a “yes” response in return.

[0178] When device 20 is paired with display 202, an application program executed by the display processor may take over the entire display desktop image and present a large-scale collaboration interface across the entire display screen. Alternatively, the application may open a collaborator window 210, such as that shown in Figure 9, to present a visual response file. In Figure 9, an exemplary visual response representation is shown at 212.

[0179] In at least some cases, collaborator window 210 or desktop image may be automatically presented via display 202 when a pairing event occurs. In other cases, even if device 20 is paired with display 202, collaborator window 210 may not be provided until some secondary triggering event occurs, such as, for example, device 20 is activated or a visual response file is received that is displayed on display 202. In still other cases, window 210 may be presented only after a user takes an affirmative action to pair device 20 with display 202.

[0180] In at least some embodiments, even if device 20 is paired with display 202, the response file may sometimes be presented to the user only via device 20. For example, often collaboration server 12 generates only an audio response file, in which case the audio file is broadcast only via device 20 without a visual representation on display 202. Here, some user queries may result in a response only via device 20, other queries may result in a response only via display 202, and still other queries may result in a combined response via each of device 20 and display 202.

[0181] As noted above, in at least some embodiments, all collaboration system communication with display 202 may be through device 20, such that server 12 does not communicate directly with display 202. In other cases, it is contemplated that display 202 has its own Internet of Things (IoT) address, and thus server 12 can communicate visual response files directly to display 202. In this case, pairing requires a location-based association of device 20 and display 202, with that association information stored in a database by server 12 so that audio and visual response file transmissions to device 20 and display 202 can be coordinated.

[0182] It is contemplated that, at least in some cases, if a visual response file is presented on the paired large display 202, the tailored visual response may be presented via the collaboration device display 48 directing the oncologist to the larger display 202. Similarly, the audio broadcast by the device 20 may direct the oncologist to the larger display 202 and may include some sort of summary message related to the visual representation on the large display 202. In FIG. 7, the illustrated audio broadcast 220 summarizes the visual content on the large display 202, and the device display 48 instructs the oncologist to refer to the larger paired display 202 for more detailed information.

[0183] In still other cases, when the portable collaboration device 20 is far from the large display and cannot be paired, and the response file is best presented via the large display, the system may notify the oncologist that a better response can be obtained by a paired device 20 with an auxiliary large display. Here, the notification may be presented via the device display 48 or audibly via the speaker 44. The notification may be in addition to broadcasting an audio response file that includes the omitted response data.

[0184] When system 10 presents visual data via display screen 202 during a collaboration session, in at least some embodiments, all presented visual files are stored in a collaboration record for subsequent access. To this end, see, for example, FIG. 8, in which third record column 196 includes visual response data 198 corresponding to each of the audio responses in column 194. Here, each visual response is accessible to view information visually presented during the associated collaboration session. FIG. 10 shows one of the visual response icons selected, which opens a sub-window 230 displaying visual content presented during the previous session.

[0185] It is contemplated that, at least in some cases, system 10 will generate data responses suitable for generating both audio and visual response files that are stored in a collaboration record without presenting any visual information during collaboration, where all communication during a collaboration session occurs via device 20, despite the generation of useful visual response files. The visual information may then be subsequently accessed via interfaces similar to those shown in FIGS. 8 and 10.

[0186] 11 , a second exemplary system 300 consistent with at least some aspects of the present disclosure is shown. Here, unlike the system of FIG. 1 in which AI processes are executed by a separate AI server 14, AI processes are executed by a portable collaboration device 20 that passes information to the collaboration server 12 for performance or execution of data operations. As shown, an ASR module 70, an intent matching module 72, and a parameter extraction module 74 are all each included in the device 20. An oncologist's speech signal captured by the device 20 is provided to the ASR module 70, which generates a test that is provided to the intent matching module 72 (310). Module 72 identifies the oncologist's intent, and then module 74 extracts parameters from the speech signal, with the text, intent, and extracted parameters each being wirelessly transmitted to the collaboration server 12 via the transceiver 16 (302). The server 12 operates in the same manner as described above to create and build a collaboration record based on the oncologist's voice message and the system response, and to use the intents and parameters to formulate data operations to be performed on the database 18 to generate the data necessary to respond to the oncologist's query. The data response is sent (304) back to the device 20, where the voice response module 76 generates an audio file to drive the speaker 44 and present the audio response.

[0187] 12, a third exemplary system 320 consistent with at least some aspects of the present disclosure is illustrated. Similar to the second exemplary system 300 illustrated in FIG. 11, the AI ​​processes performed by the independent AI server 14 are executed by the portable collaboration device 20. Unlike the second exemplary system 300, the processes performed by the independent collaboration server 12 are executed by the portable collaboration device 20. The collaboration device 20 is linked to the database 18 to send data operations 322 to the database 18 and receive data responses 324 from the database 18. The collaboration device 20, and more specifically, the audio response module 76, then generates an audio file to drive the speaker 44 and present the audio response as described above.

[0188] 9 and 12 , by implementing at least a portion of the processes performed by AI provider server 14 and collaboration server 12 locally on collaboration device 20, the latency for generating an audio response can be reduced by up to two seconds. In some embodiments, at least a portion of modules 62, 64, 70, 72, 74, 76 and / or collaboration application 60 can be stored on collaboration server 12 or AI provider server 14, periodically updated, and pushed to collaboration device 20. In other words, collaboration server 12 or AI provider server 14 can store the latest versions of modules 62, 64, 70, 72, 74, 76 and / or collaboration application 60 and periodically (e.g., once a day or once a week) update the processes stored on collaboration device 20 to include the latest modules 62, 64, 70, 72, 74, 76 and / or collaboration application 60 processes stored on collaboration server 12 or AI provider server 14. In this way, the process performed by collaboration device 20 can be continually updated, reducing the latency of generating audio responses based on input from the oncologist.

[0189] Referring to FIG. 13, several collaboration devices 20b-e can communicate with each other and / or with at least one of the first transceiver 16a and the second transceiver 16b using mesh networking techniques. The plurality of collaboration devices 20b-e can include a third exemplary collaboration device 20b, a fourth exemplary collaboration device 20c, a fifth exemplary collaboration device 20d, and a sixth exemplary collaboration device 20e. Each of the collaboration devices 20b-e can include at least some of the components of the collaboration device 20 or the second collaboration device 20a described above. In some embodiments, each of the collaboration devices 20b-e can be the collaboration device 20 or the second collaboration device 20a. While four collaboration devices 20b-e are shown, it is understood that more than four collaboration devices can be used. Each of the first transceiver 16a and the second transceiver 16b can be the transceiver 16 described above.

[0190] Each of the multiple collaboration devices 20b-e may include a corresponding transceiver 36b-e, each of which may be substantially similar to the transceiver 36 described above. Each of the multiple collaboration devices 20b-e may be linked to the first transceiver 16a and / or the second transceiver 16b to transmit voice and message signals to the first transceiver 16a and / or the second transceiver 16b using the corresponding transceiver 36b-e included in one of the collaboration devices 20b-e. For example, the fourth collaboration device 20c may be linked to the second transceiver 16b, the fifth collaboration device 20d may be linked to the first transceiver 16a and the second transceiver 16b, and the sixth collaboration device 20e may be linked to the second transceiver 16b.

[0191] The first transceiver 16a and the second transceiver 16b can be linked to the collaboration server 12 to transmit voice signal messages to the collaboration server 12 and to receive visual and audio response files transmitted from the collaboration server 12 as described above. The collaboration server 12 can be linked to the AI ​​provider server 14 to transmit voice signal messages and data responses to the AI ​​provider server 14 and to receive text files associated with the voice signal messages, matching intents, and extracted parameters, as well as audio response files associated with the transmitted data responses from the AI ​​provider server 14. The collaboration server 12 can be linked to the database 18 to create collaboration records and perform data operations within the database 18, as well as to receive transmitted data responses from the database 18.

[0192] Each of the plurality of collaboration devices 20b-e can communicate directly with at least one other collaboration device 20b-e to form a mesh network. The plurality of collaboration devices 20b-e can communicate with each other using a communication protocol supported by a corresponding transceiver 36b-e, such as a Wi-Fi protocol, an UWB protocol, and / or a Zigbee protocol. Each of the plurality of collaboration devices 20b-e can also include a corresponding mesh networking transceiver 37b-e, each of which can be substantially the same as the mesh networking transceiver 37 described above.

[0193] Multiple collaboration devices 20b-e can communicate directly with each other using corresponding mesh networking transceivers 37b-e, thereby reducing transmission traffic on the communication frequencies used by corresponding transceivers 36b-e. Direct connections between collaboration devices 20b-e can be useful when one of the collaboration devices 20b-e cannot communicate with either the first transceiver 16a or the second transceiver 16b. For example, if the third collaboration device 20b cannot communicate with the transceivers 16a-b, the third collaboration device 20b can route communications through the fifth collaboration device 20d linked to the first transceiver 16a to transmit voice signal messages and receive audio and / or visual response files transmitted as described herein. Thus, all of the collaboration devices 20b-e can be linked to the collaboration server 12.

[0194] The location of each of the multiple collaboration devices 20b-e can be determined to potentially prevent loss or theft of the multiple collaboration devices 20b-e. A monitoring process, which may be included in a server in communication with the first transceiver 16a and the second transceiver 16b (e.g., the collaboration server 12), can monitor the locations of the collaboration devices 20b-e. The monitoring process can cause heartbeat messages to be transmitted from the transceivers 16a-b to the collaboration devices 20b-e and can receive heartbeat messages transmitted from the collaboration devices 20b-e to the transceivers 16a-b. The monitoring process can then determine the location of each of the collaboration devices 20b-e based on the heartbeat messages.

[0195] In some embodiments, each of the multiple collaboration devices 20b-e can transmit a heartbeat message to the first transceiver 16a and the second transceiver 16b at a predetermined interval (e.g., every 10 minutes), which may then retransmit the heartbeat message to another device, such as the collaboration server 12. In this manner, other devices and / or processes, such as the collaboration server 12, can track and / or triangulate the location of a given collaboration device (e.g., the fifth collaboration device 20d). In some embodiments, the first transceiver 16a and the second transceiver 16b can be associated with the MAC addresses of wireless access points, which may be associated with GPS coordinates. A monitoring process can then estimate the locations of the collaboration devices 20b-e based on the GPS coordinates (indirectly) associated with the transceivers 16a-b. The monitoring process can determine that a given collaboration device (e.g., third collaboration device 20b) has transmitted heartbeat messages to both transceivers 16a-b and estimate the location of the given collaboration device based on the GPS locations associated with the transceivers 16a-b. The GPS coordinates and / or MAC addresses can be stored in collaboration server 12.

[0196] If a heartbeat message transmitted by one of the plurality of collaboration devices 20b-e is not received by both the first transceiver 16a and the second transceiver 16b, the monitoring process may determine that the device is lost and notify a system administrator and / or the monitoring process that the device is lost. The first transceiver 16a and the second transceiver 16b may also transmit heartbeat messages to the plurality of collaboration devices 20b-e to verify that the device has not been potentially lost or stolen. In some embodiments, if one of the plurality of collaboration devices 20b-e, such as the fifth collaboration device 20d, does not receive a heartbeat message from the transceivers 16a-b within a predetermined interval, the fifth collaboration device 20d may enter a restricted mode that limits the processes that can be executed by the fifth collaboration device 20d and / or lock the fifth collaboration device 20d to help prevent potential tampering with sensitive data.

[0197] Alternatively, or in addition to using transceivers 16a-b to track the locations of multiple collaboration devices 20b-e, the collaboration devices 20b-e themselves can be used to track each other. More specifically, one or more collaboration devices 20b-e can track another one of the collaboration devices 20b-e using one or more of the direct connections between the collaboration devices 20b-e. One of the collaboration devices 20b-e may communicate directly with another one of the collaboration devices 20b-e using the UWB protocol. For example, the third collaboration device 20b can be linked to the fifth collaboration device 20d and the sixth collaboration device 20e. The third collaboration device 20b can send heartbeat messages to the fifth collaboration device 20d and the sixth collaboration device 20e, and in response, the fifth collaboration device 20d and the sixth collaboration device 20e can send heartbeat messages back to the third collaboration device 20b. If the third collaboration device 20b does not receive heartbeat messages returned from the fifth collaboration device 20d and the sixth collaboration device 20e, the third collaboration device 20b may enter a restricted mode that limits the processes that can be performed by the third collaboration device 20b and / or may lock the third collaboration device 20b to help prevent potential tampering of sensitive data. Additionally, the fifth collaboration device 20d and / or the sixth collaboration device 20e may send a notification to at least one of the transceivers 16a-b that the third collaboration device 20b has potentially been lost or stolen. The transceivers 16a-b may transmit the notification to the collaboration server 12 for further processing.

[0198] In some embodiments, at least some of the processes stored and executed on AI server 14 and / or collaboration server 12 may be stored locally on collaboration devices 20b-e. In these embodiments, the processes stored and executed on AI server 14 and collaboration server 12 may be updated, for example, continuously by an external program or internally by collaboration server 12 and / or AI server 14. As processes are updated, collaboration server 12 and / or AI server 14 may update the corresponding processes stored on collaboration devices 20b-e. Processes that may include speech recognition, intent prediction, analysis, and / or routing processes may be updated based on data generated by collaboration devices 20b-e.

[0199] For example, the third collaboration device 20b and the fourth collaboration device 20c may receive speech signal messages with different phrases (e.g., phrases with different word choices) corresponding to the same intent. The AI ​​server 14 can then learn that the different phrases match the same intent and update its associated module (e.g., the intent matching module 72 shown in FIG. 1) accordingly. Some of the collaboration devices 20b-e may be located within the same institution (e.g., the third collaboration device 20b, the fifth collaboration device 20d, and the sixth collaboration device 20e), while others may be located at another institution (e.g., the fourth collaboration device 20c). In this way, the AI ​​server 14 and / or the collaboration server 12 can be updated based on feedback from multiple oncologists from multiple institutions.

[0200] Additionally or alternatively, processes stored and executed on AI server 14 and / or collaboration server 12 can be updated based on external processes. For example, an administrator can add intents to AI provider server 14. The AI ​​provider server can then upload the updated processes to collaboration devices 20b-e.

[0201] After an audible collaboration session, it is often difficult to return to the same dialog flow at a later time because it is difficult to remember the previous and subsequent communications that comprise the dialog. For this reason, at least in some cases, the system allows the user to reinsert themselves into the flow using a display screen such as that shown in FIG. 8. Accordingly, in FIG. 8, a "Continue" button 197 is presented that can be selected to place the entire system 10 in the state that existed at the end of the session. Here, "state" means that all context associated with lines of questioning (e.g., subsets of data, qualifying parameters, etc.) is restored at the end of the session, allowing the oncologist to resume where they left off, if desired.

[0202] One common problem faced by oncologists and physicians is the need to enter notes into patient records each time they see and treat a patient. At least some studies have shown that the typical oncologist spends more than 1.5 hours each day memorializing events and thoughts in patient notes. Some oncologists record or document notes during patient visits, while others wait until they have a break or are "off work" to write notes. When oncologists write notes alongside patients, the physician's attention is divided between the notes and the patient, which is not ideal. When oncologists write notes following a patient visit, thoughts, observations, and findings are often misremembered or captured with fewer details.

[0203] To address this issue, at least in some cases, the portable collaboration device 20 may be programmed to “listen” to oncologist-patient care episodes and record at least a portion of the oncologist and patient interactions essentially in real time as a “raw transcription.” Furthermore, the system processor may be programmed to process the raw transcription data through OCR and NLP algorithms to identify words, phrases, and other content in the captured raw audio signal. It is contemplated that, at least in some cases, the processor may be trained using Dialogflow or other AI software programs to recognize the oncologist's intent from the captured words and phrases and the various parameters necessary to instantiate different types of structured notes, records, or other documents that match one or more of the oncologist's intents. It is also contemplated that the processor may take other patient visit context into account when identifying the oncologist's intent, as well as when identifying key parameters of a particular structured note, record, or document.

[0204] For example, while speaking with a patient with pancreatic cancer, the processor may use the oncologist's appointment schedule to automatically identify the patient and access the patient's medical records to be used as context for audio messages captured during the patient's visit. As the oncologist and patient speak, the processor may be programmed to recognize the oncologist's voice and the patient's voice, where over time the processor trains on the oncologist's voice and becomes able to recognize the oncologist's voice based on tone, pitch, voice quality, etc., and is programmed to assume that other audio signals that do not match the oncologist's are those of the patient.

[0205] In at least some cases, the oncologist can use an intent to indicate the type of structured note the system should generate. For example, in a simple case, the system may be programmed to generate five different structured note types, each containing a different subset of 15 different parameters. Here, during Dialogflow training, the administrator may provide five different phrases for each of the five different note types, with each phrase associated with an intent that generates the associated note type. The processor trains five phrases for each note type and comes up with many other phrases to associate with the note type's intent. Further, during training, the 15 parameter subsets for each note type are specified. Further, structured note types are created and stored in a structured note database for use in instantiating specific instances of the note types for specific patient visits. Furthermore, as with the Dialogflow system described above, feedback queries for at least the required parameters may be created and saved.

[0206] During a patient visit with an oncologist, if the oncologist wants the system to generate a particular note type, the oncologist can simply activate device 20 by uttering "Go, one" and then a phrase such as "Create an instance of the first note type." The processor recognizes the intent to create an instance of the first note type and then listens during the dialogue to select the parameters needed to instantiate the note type instance. In at least some cases, if the system cannot identify some parameters needed for the note instance, device 20 may be programmed to query the oncologist for the missing parameters. Feedback queries may be generated during the patient visit, immediately after the visit when facts and information about the visit are fresh in the oncologist's mind, or at other scheduled times such as breaks, scheduled office hours, etc.

[0207] In other cases, instead of having the physician vocalize the specific note type to create, the system may listen to the oncologist-patient dialogue and identify the oncologist's intent from the ongoing dialogue without a specific request.

[0208] Raw transcriptions, notes, records, or other documents generated by the system during or associated with a patient visit may be stored in the patient's EMR or any other suitable database. The AI ​​can learn over time from the oncologist's utterances and become smarter as described above. Additionally, structured notes may be presented to the oncologist for review before or after storage, allowing the oncologist to review the information in the structured record. If the oncologist changes information captured by the system, any changes may be returned to the system processor and used to further train the processor AI to more effectively capture future intents and / or parameters.

[0209] Another document type that the system may automatically generate, at least in some cases, is a billing document. Here, too, the system processor may "listen" to what the oncologist is saying during the patient visit and identify intent that affects billing. At that point, the processor may begin listening for other parameters to instantiate a complete billing record or document. In some cases, the billing record may be automatically sent to the billing system and presented in some way to the oncologist to verify the accuracy of the billing record before transfer.

[0210] In yet other cases, another document type the system may automatically generate while listening to an oncologist is a schedule appointment. Again, the processor may recognize the oncologist's intent to schedule an appointment from many different utterances and simply listen for other parameters needed to instantiate the complete event scheduling action.

[0211] In a particularly advantageous system, the processor may be programmed to listen to the oncologist and automatically identify several simultaneous intents for generating several different types of notes, records, or documents, and may monitor the oncologist's utterances to identify all the parameters required for each of the simultaneous intents. For example, if the processor determines that a billable activity or event has occurred and that the oncologist wants to simultaneously generate a structured patient visit note, each of the structured bill and structured note requires a distinct subset of 15 parameters, the processor may listen to the oncologist's utterances for all the parameters to instantiate each of the bill record and patient visit note. Again, if the system is unable to capture the required parameters, the processor may generate and broadcast or present (e.g., visually on a display) a query to the oncologist to input the required information at the appropriate time.

[0212] In some cases, an oncologist may indicate automatic document preferences for each patient visit. The system could then automatically assume the intent associated with each preferred document type and simply listen to the oncologist-patient dialogue to identify the parameters required to instantiate each instance of the preferred document type for each patient visit. Thus, for example, an oncologist may want the system to generate structured patient visit notes and structured billing records and tee up next-visit scheduling options for each patient the oncologist attends. Here, at the start of each scheduled patient visit session, the system immediately identifies three intents: a patient visit note intent, a billing record intent, and a scheduling activity intent. The system accesses the structured record for each intent and proceeds to capture all parameters required for the intent. In the case of the scheduling activity intent, the system may identify a specific activity to be scheduled based on the captured parameters and present the oncologist and patient with one or more scheduling options for the specific activity at an appropriate time (e.g., the last five minutes of the scheduled patient visit). Here, the oncologist and patient may agree to decline scheduling any proposed activity or proposed time for the activity.

[0213] In still other cases, after the system processor identifies an intent based on the oncologist-patient interaction, the processor may be programmed to broadcast a query confirming the intent. For example, if the system identifies an intent to generate a patient visit note, the processor may be programmed to broadcast the query, "Do you want a patient visit note generated for this visit?" where an affirmative response causes the processor to identify a structured note format and proceed to collect note format parameters to instantiate the note.

[0214] In at least some embodiments, collaboration device 20 may listen to all utterances by oncologists, and many oncologists may use device 20 to capture their utterances and live audio messages. For example, the system may capture all utterances by oncologists during patient visits, tumor board meetings, office hours, and other situations in which oncologists discuss all aspects of cancer treatment. Here, a system processor or server may be programmed to recognize all utterances by the associated oncologist and distinguish them from the utterances of others (e.g., patients, other healthcare professionals, other researchers, etc.). The processor may store all or at least a subset of the oncologist's live audio messages / utterances and process them to identify text, words, phrases, context, and ultimately the oncologist's impressions. For example, one impression may be that for a pancreatic cancer patient who initially responded well to drug AAA but for whom the drug is no longer effective, drug BBB should be used as the next line of attack.

[0215] In some cases, the system identifies and automatically uses recognized impressions, while in other cases, the system is programmed to immediately present recognized impressions to the oncologist, allowing the oncologist to confirm or reject the impression. Rejected impressions may be discarded or recorded to memorialize the rejection, which itself is useful for future analysis as it is an indicator of a typical oncologist's impression. Confirmed impressions are stored in a system database for later use. In other cases, impressions may only be periodically presented to the oncologist for confirmation or rejection.

[0216] The oncologist's impressions may be used as seed data for the AI ​​machine learning algorithm, so that over time the algorithm learns from the impressions and inputs new data representing the oncologist's thoughts into the database. The system may be programmed to associate different intents with different thoughts, and then, when an oncologist's voice utterance is received, it associates the utterance with the intent, identifies parameters associated with the intent, and then retrieves the oncologist's previous impressions or thoughts and provides a response consistent with the previous thoughts or impressions.

[0217] In at least some cases, where the system collects impressions from many different oncologists, the system may combine impressions and thoughts from multiple oncologists so that every oncologist using the system has access to responses informed by at least a subset of the impressions and thoughts from the entire group. Now, as the database of impressions evolves, when an oncologist poses a question to her collaboration device 20, the system may again identify the intent as well as the parameters necessary to search the database for an answer and identify one or more impressions of interest to answer the question.

[0218] In at least some cases, the system could automatically track the effectiveness of cancer or other treatments to be used as a quality metric associated with oncology impressions, where effective treatments are assigned a high confidence or other type of factor, while less effective treatments are assigned based on the relative effectiveness of other treatments for comparable cancer conditions. When an oncologist queries the system, the system then identifies the intent and necessary parameters for generating a structured data query, returning information relevant to only the most effective impressions.

[0219] In still other cases, the system may rank certain oncologists based on one or more factors and then present query responses based on or representative of the impressions of only the "top" oncologists. For example, oncologists may be ranked based on peer reputation, on patient treatment effectiveness on a risk-adjusted basis, or using other methods (e.g., a combination of factors with different weightings). Here, the response is limited to data relevant to only the top oncologists.

[0220] In still other cases, it is contemplated that a query may be limited to data and impressions of only a specific oncologist. For example, a first oncologist may desire the impressions of a second specific oncologist regarding a particular cancer condition. Here, the first oncologist may limit the query to the second oncologist by a specific name. For example, when a first oncologist is collaborating with device 20 to access information related to a first patient, the first oncologist may simply utter, "What would Sue White say?" In this case, the processor capturing the query recognizes the intent of another oncologist's impressions, identifies Sue White as a defining parameter, and then accesses impressions associated with Sue White and other contextual parameters previously captured and recognized by the system during the previous dialogue (e.g., patient name, cancer condition factors, etc.). The response broadcast or presented to the first oncologist is limited to data and information associated with Sue White.

[0221] In many cases, especially as the system learns during use, it may make mistakes and return information that is not what was requested. In some cases, it may be clear from the response that the query identified by the system is not what the oncologist intended, but in other cases, the incorrect response may not be recognizable from the response. If a response is recognized as incorrect, reflecting an incorrectly identified query, one problem is that the oncologist must repeat the query with a better pronunciation. At least in some cases, if the oncologist declines to respond, the system may automatically attempt to identify a different query and a different appropriate response that the oncologist intended. For example, if, upon hearing a response, the oncologist utters "No" or some other denial phrase, the system recognizes that response, creates a different query based on the intent and parameters, and then issues a different response.

[0222] In some cases, in addition to recognizing an incorrect response, the response can be used to understand the error in the query identified by the system that led to the incorrect response. For example, if an oncologist requests a cancer condition characteristic of Tom Green and the system responds with, "Tom Brown's characteristics are XXX," that answer can be used to identify that the perceived question was incorrect. In this case, to avoid the oncologist having to re-speak the entire query, the system may be programmed to enable partial queries in which the intent and parameters associated with the previous incorrectly perceived query are used along with additional information in the partial query to recognize a different data action to be performed. Thus, in the above example, the oncologist may respond, "No, I meant Tom Green." Now, to access Tom Green's characteristics, the system uses the previous query information, including the intent (e.g., the desired characteristics), as well as the new parameter "Tom Green." The idea here is that the system maintains context during the dialogue so that the oncologist does not have to continually re-speak complex queries that were incorrectly perceived by the system, but instead simply provides a subset of information in the next query selected to resolve the misunderstanding.

[0223] At least in some cases, as indicated above, the answer to a query may not include a clear indication that the query was perceived incorrectly by the system. In some cases, the system may be programmed to provide a confirmation broadcast or other message to the oncologist for each uttered query, or at least a subset thereof, thus allowing the oncologist to confirm or reject the perceived query. Confirmation results in a data action, while rejection causes the system to identify a different query or request a rephrasing of the query. In still other cases, the oncologist may be able to ask the system to broadcast the question (e.g., data action) that the system perceived for confirmation.

[0224] While the present invention may be susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and are described in detail herein. However, it should be understood that the invention is not intended to be limited to the specific forms disclosed. For example, while a spherical collaboration device is described above, the portable device can take many different forms. For example, with reference to FIG. 14, the second exemplary collaboration device 20a may include a cube-shaped device including one or more emissive exterior surfaces for providing visual content. As another example, the third collaboration device may include a tablet-type device 20b or any other portable device equipped with components suitable for performing the above-described functions.

[0225] In still other cases, the portable collaboration device may be one interface device within a larger interface ecosystem that includes other interface devices with the ability for oncologists to seamlessly move between system interface devices during a collaboration session. For example, the ecosystem may include other interface devices, particularly one or more fixed interface devices with enhanced interface affordances, such as a better microphone, larger speaker components, etc. In this regard, reference is made, for example, to FIG. 15 , which illustrates another exemplary interface device 20c that is substantially larger than interface device 20 and is provided for stationary use in a workstation 350. The exemplary interface device 20c includes a larger housing structure that forms a cavity for receiving various components, as described above with respect to FIG. 2 . Here, the speaker is larger and likely of higher quality than the speaker in device 20. In this case, device 20c is intended for use on a counter, work surface, conference table, or similar location.

[0226] In at least some exemplary contemplated systems, devices 20 and 20c may work in conjunction with one another, where a collaboration session can be handed off from one of devices 20 to the other 20c to optimize for a given situation. For example, if an oncologist is roaming while collaborating via device 20 and enters a space (e.g., arrives at a workstation) that includes a more affordable, fixed device 20c, devices 20 and 20c may recognize each other and communicate wirelessly to coordinate the transfer of the collaboration session from device 20 to device 20c. Here, the collaboration session continues even though the fixed device 20c is being used. Similarly, if an oncologist uses device 20c to collaborate and then gets up and leaves the station, the collaboration session may be switched to device 20c automatically, or with user request or confirmation, so that collaboration can continue.

[0227] In still other cases, headphones, smart glasses with speakers and microphones, etc. may be used as collaboration devices in the disclosed system. In this regard, see the exemplary headphone assembly 370 in FIG. 16, which includes an ear speaker 372 and a built-in microphone 374.

[0228] Although described in the context of a dedicated collaboration device, aspects of the invention may also be implemented using any type of computer interface device, whether dedicated or not, that has a microphone and speaker to enable conversation between the user and the system. For example, a user's laptop computer may be used as a collaboration device running a collaboration program, an existing voice-activated smart speaker may be used as a collaboration device, etc.

[0229] Technology, or new technology-based tools, are great when they work well for their intended purpose, but when the technology or tool does not function as expected by the user, the user quickly becomes frustrated and often simply dismisses the technology or tool, going back to resources to complete various tasks. This tendency to quickly dismiss imperfect new technology is exacerbated when users are very busy and therefore time-constrained. Because oncologists tend to be very busy people, they typically have little tolerance for ineffective or inefficient technology and tools.

[0230] One problem with dialog systems such as those described herein is that systems that support only a subset of the queries an oncologist might pose are often unable to identify the precise intent of a received query. In response, the system then generates an answer for the wrong intent, or simply indicates that the system does not currently have an answer to the posed query. These types of incomplete answers cause frustration and often ultimately lead oncologists to dismiss these types of collaboration systems altogether.

[0231] In at least some embodiments, for a given dataset or record type, an essentially rich set of intents / parameters, associated database queries, and responses may be defined using Dialogflow or some other dialog specification software, enabling the system to effectively answer almost any query posed related to the dataset. As new datasets, databases, and record types are linked to the system, additional intents and associated information may be specified for those datasets, databases, and record types. For example, at least in some cases, the system may be programmed to support hundreds of thousands of different intents, including literally predictable intents that may be intended by an oncologist. A team of system administrators / programmers works behind the scenes to identify additional possible intents and supplement the system with new intents / parameters, associated database queries, and responses. Additional intents may be based on existing datasets and record types and / or developed in response to new data types, new information, and / or new oncological insights that evolve over time.

[0232] When a system supports a huge number of different intents (e.g., tens or hundreds of thousands), distinguishing one intent from another is complicated because, naturally, the larger the number of supported intents, the more difficult it becomes to recognize the difference between any given intent and a set of similar but distinct intents. The task of accurately identifying intents is exacerbated in Dialogflow-type systems where the AI ​​engine uses a query "fanning" process to generate literally hundreds or thousands of similar queries during system training and associate them with a particular intent, thus increasing the likelihood of overlapping fan queries for two or more different intents.

[0233] At least some embodiments of the disclosed system use one or any combination of several techniques to recognize intended intents from intents supported by other systems. A first technique relies on the system operating during a collaboration session to distinguish between different “dialog paths” occurring during the session, with information related to a particular dialog path being used to inform subsequent intents during the same dialog path. For example, if a physician asks device 20, “What are the results of my patient, Dwayne Holder’s, sequencing reports?” and then later asks, “What are the best clinical trial options?” the system determines these questions are on a dialog path and answers the clinical trial question based on the clinical trial recommendations provided in Dwayne Holder’s clinical reports (e.g., the system recommends clinical trials for the sequencing reports, and the system has access to all data in each of those reports). In at least some embodiments, only one dialog path is actively followed at a time. Nevertheless, in some cases, the system maintains a memory cache of past dialog paths to help oncologists inform future questions and answers.

[0234] A second technique for recognizing intended intent in a system that supports a vast number of intents involves the system creating "entities" around key concepts relevant to the oncologist's query and the associated system response. For example, medications, medication regimens, clinical trials, patient names, pharmaceutical companies, mutations, variants, adverse events, drug alerts, biomarkers, cancer types, etc. are all examples of entities supported by an exemplary system. While a small number of entities are identified herein, it should be understood that a typical system may support hundreds of different entities.

[0235] In at least some cases, the system may be programmed to connect entities in a query or entities identified in a query path to form entity sets that can be used to narrow down the list of potential best answers to a particular query. For example, if a query path is associated with patient Dwayne Holder and medication XXX, those patient and medication entities may form a set that limits the intent most likely to be associated with a subsequent query. The system may also be programmed to leverage entities to evaluate whether the physician's question is still part of the same dialog path or whether the new question relates to a new topic that will be associated with a new dialog path.

[0236] A third technique for recognizing intended intent in a system that supports a vast number of intents is commonly referred to as "personalization." The idea here is that many particular oncologists routinely follow similar dialog paths and utter similar queries with persistent syntax and word choice, so once the system identifies a particular oncologist's persistent query characteristics and accurately associates them with a particular intent, subsequent queries with similar characteristics can be associated with the same intent, even if they are modified by a different set of query parameters.

[0237] In at least some cases, the system builds a real-time profile of each oncologist or other system user based on the oncologist's past query characteristics (e.g., word choice, syntax, etc.), the query paths taken, previous system-provided responses to those queries, the oncologist's response to the responses (e.g., whether the oncologist's response indicates that the system responded and therefore the perceived intent was correct), and overall system usage. For example, when an oncologist logs into the system, the system may automatically link to a list of patients the oncologist has sent to a sequencing service provider, the results present in those patients' sequencing reports, and the primary treatments and clinical trials recommended for those particular patients. These linked lists support the decision-making process the system utilizes to determine the question the oncologist intends to ask (e.g., the oncologist's intent). For example, if an oncologist logs in and recently saw a patient named Dwayne Holder, even if the system receives distorted audio and converts it to text that sounds like, "How are the results for my patient, Lane Bolda?", the system may be programmed to recognize that the oncologist recently saw Dwayne Holder, who has a name similar to Lane Bolda, and proceed to generate an answer based on that recognition.

[0238] In a particularly advantageous system, all three of the above techniques are used in series, in parallel, or in combination to recognize the intent of an oncologist's query. Thus, for example, the system may use entities to narrow the oncologist's intent when uttering a particular query, may further narrow down the possible intents based on the current query path, and may then select the most likely intent based on personalization features associated with the speaking oncologist.

[0239] At least in some cases, the system may provide tools during system training sessions to avoid subsequent intent confusion. For example, assume that the system is already programmed to support 100,000 different intents when an administrator specifies the 100,001st intent and three associated seed or training queries to drive the AI ​​engine query fanning process. During the fanning process, the system processor may be programmed to compare the fan query for the 100,001st intent with other queries associated with other intents to identify duplicate or substantially identical queries. At least in some cases, the system may be programmed to automatically avoid cases where fan queries for two or more intents are identical or substantially identical.

[0240] In other cases, if the system recognizes that the first and second queries associated with the first and second intents are substantially identical, the system presents a warning to the administrator, allowing the administrator to evaluate the situation and how to handle the confusing situation. In some cases, substantially identical fan queries may mean that the system already supports the newly specified intent, in which case the administrator may simply refrain from enabling the new intent. In other cases, the administrator may select one of the previous and new intents to be associated with the query in question; in other cases, the administrator may allow the fan query to be associated with two intents. In still other cases, the administrator considering the two intents may determine that additional information is needed to distinguish between the previous and new intents, or both, and may further specify factors to consider in identifying one or both of those intents.

[0241] In operation when an oncologist utters a query, if the query is associated with two intents, the system identifies both intents and generates a response query that is broadcast to the oncologist so that the oncologist can consider which intent is meant. In other cases, both intents may match the oncologist's voice query, and thus answers to both queries may be generated and broadcast sequentially to the oncologist for consideration.

[0242] While the goal of a collaboration system is to process any question that can be answered using data in a system dataset or database, at least in some cases, despite the intent recognition techniques described above, the system may simply be unable to clearly identify one intent and / or required parameters associated with the intent from among the many intents supported by the system. For example, it is conceivable that in some cases, the system may be unable to identify the intent associated with a query or may identify two or more intents associated with a query. In these cases, the system may be programmed to facilitate a triage process focusing on the specific intent of the query. In this regard, at least in some cases, the system may be programmed to generate and broadcast a response query to the oncologist, indicating that the system was unable to determine the user's intent and requesting the oncologist to restate the query.

[0243] In other cases where the system identifies two or more intents that may be associated with a query, the system may broadcast a query to the oncologist, such as "Did you mean _?", with the blanks filled in with parameters possibly associated with the first intent collected from the first query. The system may ask about a second or other intent if the oncologist indicates that the first intent was not what was intended.

[0244] If the system cannot recognize a specific intent from a query or follow-up response from an oncologist, the system may automatically broadcast a message to the oncologist indicating that the system was unable to understand the query and that a system administrator should review the query and intent so that the system can be trained to handle the oncologist's query. Queries that cannot be associated with a specific intent are then presented to an administrator, who can review the query in context (e.g., within a dialog path) and either associate the query with a specific system-supported intent or specify a new intent and associated (e.g., required and optional) parameters to be associated with the query. Here, if a new intent is specified, the administrator may specify a small set of additional seed queries for the intent, and the system AI engine may facilitate a fanning process that again generates hundreds of additional queries to associate with the new intent. The administrator then specifies one or more data operations for the new intent, as well as an audible response file for generating an audible response for the intent. As new intents, parameters, data behaviors, and response files are published to the system for use, emails or other notifications are automatically generated and sent to oncologists who raise initially unrecognized queries and, in some cases, appropriate answers to those queries.

[0245] When the system associates a perceived query with an intent supported by a single system and then performs a data operation to access the data necessary to create an audible answer, in at least some cases, the databases and / or records searched will not produce results from which to derive an answer. For example, if an oncologist utters a query about a particular patient by name and no information about that patient exists in the system database, the data operation will not return data to respond to the query. In this case, the system may be programmed to broadcast a message indicating that "there is no data in the system for the patient you identified."

[0246] In other cases, in addition to generating data directly responsive to a query, the system may generate additional data (hereinafter "supplemental data") to supplement the response data. The supplemental data can take essentially any type of form that can be supported by data in the system database, and may include, for example, qualifying statements or phrases that apply to associated directly applicable response phrases, additional data of interest, clinical trials that may be relevant to the query, conclusions based on the data, and data that support the answer statement.

[0247] It is contemplated that supplemental data can be driven by conditional or supplemental data operations, or operations triggered by the results of primary data operations, and associated answer phrases and sentences. For example, a primary data operation that generates data directly responsive to a first query intent may be associated with the first intent, and data from that operation may be used to create direct-response answer phrases that directly respond to an oncologist's query paired with the first intent. Additionally, a secondary or supplemental data operation may also be associated with the first intent and generate data results used to formulate certain types of supplemental answer phrases (e.g., qualifying statements, additional data of interest beyond the data directly associated with the initial query, clinical trials of interest, conclusions, and supporting data, etc.), which do not directly respond to the first query but add additional information of interest to the direct-response answer phrases. Here, when a primary data operation generates results, those results may be used to generate direct-response phrases that respond to the query. Similarly, when a supplemental data operation associated with the first intent generates results, those results may be used to generate second or supplemental answer phrases. In this case, the direct response supplemental phrases may be broadcast in sequence for the oncologist to hear.

[0248] In the above cases, if only the primary data operation generates a result and an associated direct response answer phrase (e.g., if the supplemental data operation cannot generate data that can be used to generate a supplemental response phrase), the system generates only the direct response phrase. Thus, in these cases, the system response to the query may include either only the direct response phrase or a sequence that includes the direct response phrase followed by the supplemental phrase.

[0249] In some cases, three, four, five, or more supplemental data operations and answer phrases may be associated with a single intent in the system. Here, once the intent is identified, all data operations (e.g., primary and each supplemental) may be executed in an attempt to generate a result that can be used to generate and broadcast a sufficient system response. If only a subset of the supplemental data operations generate results, only the phrases associated with those results are generated and broadcast sequentially. Thus, for example, if primary and first through fifth supplemental data operations are associated with an intent, and the data operations generate results for the primary, second, and fifth supplemental operations, the answer will include three consecutive answer phrases: the first for the primary operation result, and the second and third for the second and fifth supplemental operation results.

[0250] A supplemental qualifying statement may be based on the inability to effectively provide a complete answer to a query. For example, if a primary data operation returns 50 different effective medications for a particular cancer condition, instead of audibly broadcasting all 50 medications, the system may identify the three most effective medications and broadcast them as options along with a qualifying statement such as, "There are 47 other effective medications, and if you say email me the complete list of medications, I can send you the complete list now."

[0251] Another type of supplemental qualified statement may be generated by a supplemental data action that evaluates the weight of evidence supporting the result of a primary data action. For example, if a direct query answer indicates that "There is evidence that at least some patients with the cancer condition will respond positively to treatment YYY," whereas only two previous patients with a particular cancer condition responded positively to treatment YYY, a supplemental response might be "However, please note that only two patients responded positively to treatment YYY." In this case, the supplemental data action identifies the number of patients who respond positively and compares it to a statistically significant number associated with a higher level of confidence; if that number is less than the statistically significant number, the action generates the supplemental response as a qualified statement. As another example, if the response of a primary data action is "Chemotherapy is recommended for pancreatic cancer in the adjuvant setting," a qualified supplemental phrase might be "However, the role of radiation is still being explored in clinical studies." This supplemental phrase is generated based on the results of a supplemental data action associated with the query intent.

[0252] Other types of qualifying statements are possible.

[0253] The additional data of interest can be any data, a subset of data, a compilation of data, or a derivative of the system data. For example, if an oncologist asks about the symptom status of a particular patient, the additional data may include additional typical symptom statuses given the particular patient's current cancer status.

[0254] The supplemental response may include more information related to the clinical trials identified in response to the primary data action. For example, here, the phrase directly responding to the query might be, "There are two clinical trials that may be of interest to Dwayne Holder," and the supplemental response might be, "The first clinical trial is 23 miles from your office, and the second clinical trial is 35 miles from your office." Many other supplemental data actions related to clinical trials are considered.

[0255] In at least some cases, at least some databases include specialized clinical reports or other report types developed for a specific purpose, in which data is collected from EMRs and other system databases and used to instantiate specific instances of reports for particular patients and cancer conditions. Here, in at least some cases, the instantiated reports are generated and stored in a persistent format (e.g., dated and immutable), while in other cases, the instantiated reports are stored but dynamic, such that the system periodically updates the reports as the patient's cancer condition progresses over time. When reports are stored in a persistent format, multiple instances of the report are persistently stored, so that a historical record of the report can be created over time. When reports are stored dynamically, historical values ​​of report fields can be stored to enable later generation of time-based instances of the report that reflect the report information at any point during the patient's treatment.

[0256] One advantage of using a particular type of fully formatted clinical report (e.g., pancreatic cancer, breast cancer, melanoma, etc.) is that oncologists who routinely use instantiated instances of a particular report type will immediately understand the type of information available in the report as well as where the information resides in the report. As the familiarity of the report matures, when specific information related to a particular patient's cancer status is sought, the oncologist will know whether that information is in the patient's clinical report and, upon accessing the report, where the specific information is located.

[0257] Another benefit associated with clinical reports is that the reports serve as summaries of EMR data and can include the results of additional complex data operations on the EMR data, eliminating the need for oncologists to manually recreate or process these operations. Thus, reports can include not only clinically relevant EMR data, but also data and other information derived from the raw EMR data. Collaboration device 20 may also provide oncologists with information not available in clinical reports, such as Tempus Insights and actionable mutations.

[0258] Referring now to Figures 17A-17C, three pages of an exemplary clinical report are shown related to patient Dwayne Holder, who is suffering from pancreatic cancer. The report contains all the important clinical information relevant to the patient's cancer status, including clearly marked report sections for genomic variants, immunotherapy markers, FDA-approved treatments and current diagnoses, FDA-approved treatments and other indications, current clinical trials, variants of unknown significance, low-coverage regions, clinically actionable somatic variant details, germline variant details, medical history, and oncologist notes (see the bottom left field in Figure 17A). Here, the report format is simple and clearly defined, allowing oncologists to quickly find the specific information of interest.

[0259] In view of the present disclosure, using formatted clinical reports as the primary data source for driving a voice-based collaboration system facilitates the development of a rich set of intents and the tasks associated with supporting information for those records. In this regard, please refer again to Figures 17A-17C. While a significant amount of clinically relevant patient information is presented in the reports, the amount of information is limited in order for oncologists to quickly become familiar with the report format and available data. Because oncologists know the patient's general cancer condition (e.g., pancreatic, breast, etc.) as well as the report format and report data type for that condition, they will naturally tend to limit system queries to those calculated to be answerable via the report type information. Because the report data (while containing all clinically relevant data) is limited to a specific set of the patient's medical record data, the number of intents required to support anticipated queries is quite limited. For example, the number of intents required to fully support anticipated queries for the Figure 17A-17C reports may be approximately a few thousand, rather than the 100,000 or more required for a full EMR.

[0260] Another benefit associated with using formatted clinical reports as the primary data source to drive a voice-based collaboration system is that the limited number of intents required to fully support anticipated queries makes it much easier for the collaboration system to uniquely distinguish the intended intent from all other supported intents. Thus, for example, if only 5,000 intents are required to fully handle all anticipated queries regarding pancreatic clinical record information, accurate intent recognition is more likely than if 100,000 intents must be supported.

[0261] Another benefit associated with using formatted clinical reports as the primary data source to drive a voice-based collaboration system is that the system can leverage complex data calculations already supported by the overall EMR system that generate important information in the clinical report. Thus, in the context of pancreatic cancer, the example reports in Figures 17A-17C already contain all clinically important data, including the results of complex data operations, so the collaboration system does not need to independently derive the required data and other information.

[0262] In some cases, near the beginning of a collaboration session, when the collaboration system identifies a particular patient, the system identifies the patient's cancer condition and condition-specific clinical medical record and automatically loads for consideration a subset of intents associated with the patient's cancer condition (e.g., "condition-related intents"). In some cases, the condition-related intents may be the only intents considered by the system unless the oncologist instructs otherwise. In other cases, the condition-related intents may be prioritized (e.g., considered the first option or a more weighted option) over other, more general EMR-related intents, such that if the first and second intents in the pool of condition-related intents and more general intents are identified as possible intended intents, the system automatically selects the condition-related intent over the more general intents.

[0263] In at least some embodiments, data actions associated with condition-related intents are limited to the associated clinical record. Thus, for example, referring again to Figures 17A-17C, if Dwayne Holder is identified as a pancreatic cancer patient and a query intent is identified, then data actions in these cases are limited to the data and information presented in the record.

[0264] In other cases, data operations associated with condition-related intents may include any operations related to any EMR or other database data accessible by the system processor, in addition to direct operations on the data and information in the clinical reports Figures 17A-17C.

[0265] In still other cases, the cancer condition-specific intent may be treated as the preferred intent, and other more general dataset intents may be considered only if the system is unable to identify a condition-specific intent that matches the received query. Here, at least in some cases, even if a condition-specific intent is identified, the system may generate a confidence factor associated with the intent, and if the confidence factor is below a certain threshold level, other more general system intents may be considered as candidates that match the particular query.

[0266] Referring now to FIG. 18 , a process 400 similar to the process described above with respect to FIG. 5 is shown, except that the collaboration system automatically limits the intent to a particular cancer condition if a clinical report of that condition is available for the particular patient. While process 400 is similar to the process of FIG. 5 , to simplify this description, some of the process steps of FIG. 5 have been removed from process 400. For example, FIG. 18 does not include, among other things, steps for providing visual responses to oncology questions. Nevertheless, it should be understood that, in at least some embodiments of the present disclosure, any of the additional steps shown in FIG. 5 can be added to process 400 of FIG. 18 .

[0267] 18 , in initial process step 402, an EMR or other system stores and maintains clinical reports for a particular patient and a particular cancer condition (e.g., pancreatic, breast, etc.). In block 404, an administrator uses the example cancer condition-specific clinical reports for each cancer condition to train an essentially complete set of condition-specific intents and other supporting information (e.g., parameters, data actions, and response files or phrases).

[0268] After system training, the system monitors for collaboration device activation in block 406. Once the collaboration device is activated, the system monitors voice signals and collects voice signal queries uttered by the oncologist in decision block 408. Any received utterances are transcribed into text and stored in a text file in process block 412.

[0269] Still referring to FIG. 18 , at decision block 414, the system processor monitors the utterance for information identifying a particular patient. If the oncologist does not identify a particular patient, system control may pass to a process more similar to that shown in FIG. 5 to identify a more general query intent based on a larger data set. If a patient is identified by the oncologist at block 414, control passes to process block 416, where the patient's cancer status is identified in the system database. At block 418, the system determines whether there are condition-specific clinical records stored in the system database for the user. If there are no condition-specific clinical records for the patient, control may also pass to the process shown in FIG. 5 to identify a more general query intent based on a larger data set.

[0270] 18 , if the patient's condition-specific clinical record exists, control passes to block 420, where the system restricts the pool of intents to match the query to condition-related intents (e.g., intents that are specifically associated with the patient's condition-specific clinical record type). Again, in some cases, the restriction simply means that a weighting factor is applied to the intents, making it more likely that the system will select the condition-specific intent over a more general system intent. In other cases, the restriction means that the system will only consider general intents until the oncologist performs some action that causes the system to identify a condition-specific intent.

[0271] Particularly advantageously, once a patient's general cancer status (e.g., pancreatic, breast, etc.) is determined, the system strictly restricts the intent pool (e.g., not considering other intents during the query path or collaboration session) to match queries to the status-specific clinical report set.

[0272] Subsequently, at block 422, the processor compares the received query to the limited intent set to identify the intent, and then extracts intent-related parameters from the query. At process block 424, the system uses the intent and extracted parameters to define one or more data operations (e.g., primary, or primary and supplemental as described above) to be performed on the clinical report data and, at least in some cases, other accessible data sets. At block 426, the data operations are performed to generate information that can be used to respond to the query. At block 428, the response file associated with the intent and data operation is used to create an audio response file, which is transmitted to the collaboration device and broadcast to the oncologist at block 430.

[0273] At least in some cases, the system may support an email feature that allows oncologists to request email copies of different clinical record datasets or other system datasets during a collaboration session. For example, after the system broadcasts information related to a clinical trial that may not be of interest to a particular patient, the oncologist may say, "Send me information related to the trial." Here, the system recognizes the oncologist's intent to obtain an email containing trial information for the trial in question, performs a data operation to access the trial information, and then transmits the information to the oncologist's email address. Furthermore, once the trial information is transmitted via email, the system may generate and broadcast a response to the oncologist indicating that the trial information was sent via email. In other cases, it is contemplated that data and information may be transmitted to the oncologist via other communication systems (e.g., as a text link, via hard copy regular mail, etc.). A more complex email-related dialog path may include the following query: "Therapy Company" represents the name of one or more companies providing the therapy, and "Therapy" represents the name of one or more therapies. Dwayne Holder sequencing results. Does my patient have high TMB? Are they good candidates for immunotherapy? What immunotherapy drugs are currently approved? Who manufactures Therapy? What are the main adverse events of treatment? Please email me the Therapy medication label. Who manufactures Therapy? What is Therapy Company's patient financial assistance phone number? Please email me the Therapy Company Compassionate Use Agreement. Please email me the Tempus insurance reimbursement letter that my patient, Dwayne Holder, has data justifying their off-label use of Therapy.

[0274] In this example, the oncologist may issue several email requests, each of which will deliver a different set of information to the oncologist's email account.

[0275] In at least some cases, when the system receives a query via the collaboration device, data operations are performed on data from two or more different types of datasets. The first type may include a specific patient's genomic dataset, which includes details of the patient's molecular report. The second data type includes data residing in a general knowledge database (KDB), which includes non-patient-specific information about a particular topic based on accepted industry standards (e.g., the effectiveness of a particular drug in treating a particular cancer condition, clinical trial information, drug class-mutation interactions, genes, etc.) or empirical information derived by the service provider, as well as information about the service provider's system capabilities (e.g., information about specific tests and activities performed by the provider, test requirements, etc.). To this end, see the exemplary system database 500 shown in FIG. 20 , which includes a molecular report genomic dataset, a clinical dataset 502, and a non-patient-specific knowledge database (KDB) 504. Arranging the data operations in this manner provides the advantages of the universe of possible intents and data operations that can be associated with any query, as described above, and associated with such an arrangement's results.

[0276] Still referring to FIG. 20 , dataset 502 includes, among other data, genomic, transcriptomic, epigenomic, microbiome, clinical, stored alterations proteomic, proteomic, organoid, image and cohort, and trend datasets described in detail in other patent applications. The KDB, as shown, has separate sub-databases related to specific information types, including provider panels 506 (e.g., information related to gene panels supported by the service provider operating the system), drug classes (e.g., drug class-specific information (e.g., whether a particular class of drug acts on pancreatic cancer or which drugs are considered to be in a particular drug class)), specific genes 508, immune results (e.g., information related to treatments based on the results of a particular immune biomarker), specific drugs, drug class-mutation interactions, mutation-drug interactions, provider methods (e.g., questions regarding the processes performed by the service provider), clinical These subdatabases include clinical conditions such as tests, general immunology, and clinical diseases; terminology sheets (e.g., definitions of industry-specific terms); provider coverage (e.g., information about a provider's tests and results); provider samples (e.g., information about the types of samples a provider can process); knowledge (e.g., scripted questions and answers for various frequently asked questions not categorized in other subdatabases); radiation (e.g., information about appropriate radiation treatments given specific cancer conditions); NCCN guidelines (e.g., national guidelines related to cancer condition classifications, accepted treatments, etc.); and clinical trial questions and answers (e.g., information related to clinical trial locations and administrators). Organizing the KDB into subdatabases facilitates management of the databases as the information in the KDB evolves over time, and also allows for the addition of new subdatabases related to other defined information types.

[0277] To identify the genomic dataset associated with a particular patient's molecular report, the system identifies data operations associated with the query and then associates at least one of those operations with the patient's genomic dataset displayed in the molecular report before performing at least one data operation on the set.

[0278] In at least some cases, the results of a data operation on a patient's molecular report data inform other data operations to be performed on the KDB or inform other operations of the results of operations on the KDB to be performed on the patient's molecular report data. For example, if an oncologist queries, "What are the treatment implications of Dwayne Holder's CDKN2A mutation?" the system may associate that query with an intent. The intent may be associated with two data operations, including a first operation to search a general KDB for appropriate treatments for CDKN2A mutations and a second operation to determine whether the patient is already treated with one or more of the appropriate treatments. In this case, the results of the KDB data operation inform the molecular report data operation. As another example, if an oncologist queries, "Did Dwayne Holder lose heterozygosity for his BRCA2 mutation?", the system again identifies two data operations, this time including a first operation against the genomic dataset associated with Dwayne Holder's molecular report to return the patient's loss of heterozygosity (LOH) value, and a second operation to perform against the KDB to determine whether the patient's mutation and LOH value pairing are known to be tumor drivers. In this case, the results of the molecular report data operation inform the KDB data operation.

[0279] Below, first and second exemplary processes associated with processing the queries "What are the therapeutic implications of Dwayne Holder's CDKN2A mutation?" and "Did Dwayne Holder lose heterozygosity for his BRCA2 mutation?" are respectively described. To simplify this description, the first and second processes will be referred to as the first and second examples, respectively, unless otherwise noted.

[0280] Referring now to FIG. 19 , a process 450 is shown for associating data operations with a genomic dataset represented in a patient's molecular report prior to performing those operations on the dataset, consistent with at least some aspects of the present disclosure. At process block 452, collaboration device 20 (see again FIG. 1 ) receives an audible query from an oncologist via the device microphone related to information displayed in a particular patient's molecular report, which may be stored in a system database. In some embodiments, process 450 may store the particular patient's molecular report and / or molecular reports of other patients in the system database. In this manner, process 450 may store molecular reports for multiple patients. In some embodiments, process 450 may identify a particular patient, as described in connection with FIG. 18 . At least in some cases, the audible query may include a question regarding a nucleotide profile associated with the patient. The nucleotide profile associated with the patient can be the patient's cancer profile. The nucleotide profile associated with the patient can be the patient's germline profile. The nucleotide profile can be a DNA profile. The nucleotide profile associated with the patient can be an RNA expression profile. The nucleotide profile associated with the patient can be a mutation biomarker. The nucleotide profile associated with the patient can be a BRCA biomarker. At least in some cases, the audible query can include a question regarding treatment. At least in some cases, the audible query can include a question regarding genes. At least in some cases, the audible query can include a question regarding clinical data. The clinical data may include at least one of the clinical data elements described above. At least in some cases, the audible query can include a question regarding next generation sequencing panels. At least in some cases, the audible query can include a question regarding biomarkers. At least in some cases, the audible query can include a question regarding immune biomarkers.In at least some cases, the audible query can include a question regarding an antibody-based test. In at least some cases, the antibody-based test can be a blood sample-based antibody test. In at least some cases, the audible query can include a question regarding a clinical trial. In at least some cases, the audible query can include a question regarding an organoid assay. In at least some cases, the audible query can include a question regarding a pathology image. The pathology image can be a slide image, for example, an image generated using whole slide imaging (WSI). In at least some cases, the audible query can include a question regarding a type of disease.

[0281] In some embodiments, at block 452, process 450 can identify at least one qualifying parameter in the audible query. In some cases, the at least one qualifying parameter can include patient identity, patient condition, genetic mutation, and / or type of treatment. In some embodiments, process 450 can identify qualifying parameters in a molecular report of the first patient.

[0282] At block 454, the system identifies at least one intent associated with the audible query. Here, block 454 involves identifying general intents as well as contextual parameters within the query so that specific intents can be formulated. For example, for the first example query, "What are the therapeutic implications of the CDKN2A mutation in Dwayne Holder?", the identified general intent may be "What are the therapeutic implications based on the patient's genetic mutation?", the specific query parameters may include "CDKN2A" and "Dwayne Holder," and the underlined gene and patient fields in the general query are populated with "CDKN2A" and "Dwayne Holder" to generate the specific query intent.

[0283] For the second example query, "Did Dwayne Holder lose heterozygosity for his BRCA2 mutation?", the identified general intent may be "Did the patient experience a genetic trait with a gene mutation?", and the underlined patient, gene mutation, and gene fields in the general query are populated with "Dwayne Holder," "heterozygosity," and "BRCA2," respectively, to generate the specific query intent.

[0284] At least one intent can be associated with the audible query. At least one intent can be a clinical trial-related intent. At least one intent can be pharmaceutical-related. At least one intent can be pharmaceutical-related, where the intent is pharmaceutical. At least one intent can be a pharmaceutical intent. At least one pharmaceutical intent can be a chemotherapy-related pharmaceutical. At least one pharmaceutical intent can be a PARP inhibitor-related intent. At least one intent can be genetic. At least one intent can be immunology-related. At least one intent can be knowledge database-related. At least one intent can be testing methodology-related. At least one intent can be gene panel-related. At least one intent can be reporting-related. At least one intent can be organoid process-related. At least one intent can be imaging-related. At least in some cases, at least one intent may be related to a pathogen. In some embodiments, the pathogen may be a pathogenic mutation. At least in some cases, at least one intent may be related to a vaccine.

[0285] The at least one intent can be associated with at least one activity. The at least one activity can include periodically capturing health information from an electronic health record included in the knowledge database. The at least one activity can include checking the status of an existing clinical or test order. The at least one activity can include ordering a new clinical or test. The at least one activity can include automatically initiating the at least one activity without initiating input from an oncologist. The at least one activity can include uploading a patient's EHR to the knowledge database.

[0286] 19 , once a particular intent is identified, in block 456 the system identifies at least one data action associated with the particular intent, where a database associates data actions with intents. For example, in some cases, one or more data actions may be correlated with each particular intent. In other cases, at least some data actions may depend on the results of other data actions (e.g., a second action is executed only if the result of a first action is within a particular range of values).

[0287] In some embodiments, at block 456, the process 450 may identify at least one data action based on both the identified intent and the at least one qualifying parameter.

[0288] For the first example, for the specific intent, "What are the implications for treatment based on Dwayne Holder's CDKN2A mutation?", exemplary data operations may include: (1) for the CDKN2A mutation, searching for an appropriate treatment in the treatment KDB; and (2) for the appropriate treatment, searching the treatment history portion of the patient's molecular report genomic dataset to identify whether the patient is already receiving the appropriate treatment. Similarly, for the second example, for the specific intent, "Did Dwayne Holder experience loss of heterozygosity due to a BRCA2 mutation?", exemplary data operations may include: (1) searching the LOH value in the patient's molecular report genomic dataset and whether the mutation is germline or somatic; and (2) optionally searching a KDB (e.g., KDB 504) based on the LOH value to determine whether the LOH value and the mutation are known to be tumor drivers.

[0289] At least in some cases, the at least one data operation can include an operation for identifying at least one treatment option. At least in some cases, the at least one data operation can include an operation for identifying knowledge about a treatment. At least in some cases, the at least one data operation can include an operation for identifying knowledge related to at least one medication. For example, the knowledge can be knowing which medication, if any, is associated with high CD40 expression. At least in some cases, the at least one data operation can include an operation for identifying knowledge related to mutation testing. For example, the knowledge can be whether Dwayne Holder's sample was tested for a KMT2D mutation. At least in some cases, the at least one data operation can include an operation for identifying knowledge related to the presence of a mutation. For example, the knowledge can be whether Dwayne Holder has a KMT2C mutation. At least in some cases, the at least one data operation can include an operation for identifying knowledge related to a tumor characteristic. For example, the knowledge can be whether Dwayne Holder's tumor is a BRCA2-driven tumor. At least in some cases, the at least one data operation can include an operation for identifying knowledge related to testing requirements. For example, the knowledge may be the percentage of tumors that Tempus requires for a TMB result. At least in some cases, the at least one data operation may include an operation for querying definition information. For example, the definition information may be a definition of PDL1 expression. At least in some cases, the at least one data operation may include an operation for querying expert information. For example, the expert information may include the clinical relevance of PDL1 expression or what the general risks associated with Whipple surgery are. At least in some cases, the at least one data operation may include an operation for identifying information related to a recommended treatment.For example, the information may be, "If Dwayne Holder is at the 88th percentile for PDL1 expression, is he a candidate for immunotherapy?" In at least some cases, the at least one data operation may include an operation for querying information related to the patient. In at least some cases, the at least one data operation includes an operation for querying information related to patients having one or more clinical characteristics similar to the patient. For example, the information may be, "What are the most common adverse events for patients similar to Dwayne Holder?" In at least some cases, the at least one data operation may include an operation for querying information related to a patient cohort. For example, the information may be, "What are the most common adverse events for patients with pancreatic cancer?" At least in some cases, the at least one data operation can include an operation for querying for information related to clinical trials. For example, the information can be which clinical trial is best for Dwayne Holder. At least in some cases, the at least one data operation can include an operation for querying for a feature related to a genomic mutation. At least in some cases, the feature can be loss of heterozygosity. At least in some cases, the feature can reflect a cause of the mutation. At least in some cases, the cause can be germline. At least in some cases, the cause can be somatic. At least in some cases, the feature can include whether the mutation is a tumor driver.

[0290] 19, in block 458, the system associates each of the at least one data operation with a first dataset (i.e., a first set of data) presented in the molecular report for the particular patient. In a first example, the system associates each of the data operations with CDKN2A, which is displayed in the molecular report as shown in FIG. 17A. In a second example, the system associates the first data operation with BRCA2 and Dwayne Holder in the molecular report genomic dataset. In some embodiments, the system can access the molecular report for the particular patient in block 458.

[0291] In at least some cases, the first set of data may be a gene editing therapy previously studied and / or documented by a reliable information source. In at least some cases, the gene editing therapy may be a clustered regularly interspaced short palindromic repeat (CRISPR) therapy. In at least some cases, the first set of data may include a patient name. In at least some cases, the first set of data may include a patient age. In at least some cases, the first set of data may include a next-generation sequencing panel. In at least some cases, the first set of data may include a genomic variant. In at least some cases, the first set of data may include a somatic genomic variant. In at least some cases, the first set of data may include a germline genomic variant. In at least some cases, the first set of data may include a clinically actionable genomic variant. In at least some cases, the first set of data may include a loss-of-function variant. In at least some cases, the first set of data may include a gain-of-function variant. In at least some cases, the first set of data may include an immunological marker. In at least some cases, the first set of data may include a tumor mutation burden. In at least some cases, the first set of data may include a microsatellite instability status. At least in some cases, the first set of data can include a diagnosis. At least in some cases, the first set of data can include a treatment. At least in some cases, the first set of data can include a treatment approved by the U.S. Food and Drug Administration. At least in some cases, the first set of data can include a pharmaceutical therapy. At least in some cases, the first set of data can include radiation therapy. At least in some cases, the first set of data can include chemotherapy. At least in some cases, the first set of data can include a cancer vaccine therapy. At least in some cases, the first set of data can include an oncolytic virus therapy.In at least some cases, the first set of data can include immunotherapy. In at least some cases, the first set of data can include pembrolizumab therapy. In at least some cases, the first set of data can include CAR-T therapy. In at least some cases, the first set of data can include proton therapy. In at least some cases, the first set of data can include ultrasound therapy. In at least some cases, the first set of data can include surgery. In at least some cases, the first set of data can include hormone therapy. In at least some cases, the first set of data can include off-label use. In some embodiments, the off-label use can include pharmaceutical therapy. In at least some cases, the first set of data can include on-label use. In at least some cases, the first set of data can include bone marrow transplant events. In at least some cases, the first set of data can include cryoablation events. In at least some cases, the first set of data can include radiofrequency ablation. In at least some cases, the first set of data can include monoclonal antibody therapy. In at least some cases, the first set of data can include an angiogenesis inhibitor. In at least some cases, the first set of data can include a PARP inhibitor. In at least some cases, the first set of data can include targeted therapy. In some embodiments, the targeted therapy may be a molecular targeted therapy. At least in some cases, the first set of data may include an indication of use. In some embodiments, the indication of use may be an indication of use of a pharmaceutical agent in treating a condition, such as a disease. At least in some cases, the first set of data may include a clinical trial. At least in some cases, the first set of data may include distances to locations where clinical trials are conducted. At least in some cases, the first set of data may include variants of unknown significance.In some embodiments, variants may be classified as pathogenic, likely pathogenic, variants of unknown significance, likely benign, or benign. At least in some cases, the first set of data may include a mutation effect. In some embodiments, the mutation effect may be positive (e.g., associated with a decreased risk of heart disease), negative (e.g., associated with an increased risk of heart disease), or neutral (e.g., associated with no significant change in risk of heart disease). At least in some cases, the first set of data may include a mutant allele fraction. In some embodiments, the mutant allele fraction may be a ratio of mutant reads for a given mutation. At least in some cases, the first set of data may include low coverage regions. At least in some cases, the first set of data may include a medical history. At least in some cases, the first set of data may include biopsy results. In some embodiments, the biopsy results may include a grade of aggressiveness of the cancer. For example, the grade may range from 1 to 4, with 1 indicating the least aggressive cancer and 4 indicating the most aggressive cancer. At least in some cases, the first set of data may include imaging results. In at least some cases, the first set of data may include MRI results. In at least some cases, the first set of data may include CT results. In at least some cases, the first set of data may include a treatment prescription. In at least some cases, the first set of data may include a treatment administration. In at least some cases, the first set of data may include a cancer subtype diagnosis. In at least some cases, the first set of data may include a cancer subtype diagnosis by RNA class. In at least some cases, the first set of data may include the results of a treatment applied to organoids grown from the patient's cells. In at least some cases, the first set of data may include a tumor quality measurement.In at least some cases, the first set of data may include a tumor quality measure selected from at least one of the set of PD-L1, MMR, tumor-infiltrating lymphocyte count, and tumor ploidy. In at least some cases, the first set of data may include a tumor quality measure obtained from image analysis of pathology slides of the patient's tumor. In at least some cases, the first set of data may include a signaling pathway associated with the patient's tumor. In at least some cases, the signaling pathway may be the HER pathway. In at least some cases, the signaling pathway may be the MAPK pathway. In at least some cases, the signaling pathway may be the MDM2-TP53 pathway. In at least some cases, the signaling pathway may be the PI3K pathway. In at least some cases, the signaling pathway may be the mTOR pathway.

[0292] In at least some cases, the at least one data operation can include an operation for querying treatment options, the first set of data can include genomic variants, and the associating step (i.e., block 458) can include coordinating the operation for querying treatment options based on the genomic variants. In at least some cases, the at least one data operation can include an operation for querying medical history data, the first set of data can include a treatment, and the associating step (i.e., block 458) can include coordinating the operation for querying medical history data elements based on the treatment. In at least some cases, the medical history data can be a medication prescription, the treatment can be pembrolizumab, and the associating step can include coordinating the operation for querying a prescription for pembrolizumab.

[0293] Next, in block 460, the system performs each of the data operations on the second dataset to generate response data. In a first example, a first data operation in the KDB (e.g., the second dataset) identifies palbociclib as an appropriate treatment for the patient's CDKN2A mutation, and a second data operation in the molecular report genomic dataset (e.g., another second data operation) indicates that Dwayne Holder has already been treated with palbociclib. In a second example, response data from the first data operation in Dwayne Holder's molecular report genomic dataset (e.g., the second dataset) indicates that he does not have a pathogenic somatic BRCA2 mutation, but also that he has a pathogenic germline BRCA2 mutation and associated loss of heterozygosity (LOH) (see the BRCA2 section of the molecular report shown at the bottom of Figure 17B, which shows the LOH). In the second example, the first data operation results (e.g., the presence of a germline BRCA2 mutation and somatic LOH) are used to drive a second data operation, and the response data indicates that the tumor is a BRCA2-driven tumor.

[0294] In at least some cases, the second set of data can include clinical health information. In at least some cases, the second set of data can include genomic variant information. In at least some cases, the second set of data can include DNA sequencing information. In at least some cases, the second set of data can include RNA information. In at least some cases, the second set of data can include DNA sequencing information from short-read sequencing. In at least some cases, the second set of data can include DNA sequencing information from long-read sequencing. In at least some cases, the second set of data can include RNA transcriptome information. In at least some cases, the second set of data can include RNA full transcriptome information. In at least some cases, the second set of data can be stored in a single data repository. In at least some cases, the second set of data can be stored in multiple data repositories. In at least some cases, the second set of data can include clinical health information and genomic variant information. In at least some cases, the second set of data can include immunological marker information. In at least some cases, the second set of data can include microsatellite instability immunological marker information. In at least some cases, the second set of data may include tumor mutation burden immunological marker information. In at least some cases, the second set of data may include clinical health information including one or more of demographic information, diagnostic information, evaluation results, test results, prescribed or administered treatments, and outcome information. In at least some cases, the second set of data may include demographic information comprising one or more of patient age, patient date of birth, sex, race, ethnicity, system of care, comorbidities, and smoking history. In at least some cases, the second set of data may include diagnostic information including one or more of tissue of origin, date of initial diagnosis, histology, histology grade, metastatic diagnosis, date of metastatic diagnosis, one or more sites of metastasis, and stage information.In at least some cases, the second set of data can include stage information including one or more of TNM, ISS, DSS, FAB, RAI, and Binet. In some embodiments, the stage information may be referred to as "cancer stage information." In at least some cases, the second set of data can include assessment information including one or more of a performance status including at least an ECOG status or a Karnofsky status, a performance status score, and a performance status date. In at least some cases, the second set of data can include laboratory information comprising one or more of a lab type (e.g., CBS, CMP, PSA, CEA), a lab result, a lab unit, a date of lab service, a date of molecular pathology testing, an assay type, an assay result (e.g., positive, negative, equivocal, mutation, wild-type), a molecular pathology method (e.g., IHC, FISH, NGS), and a molecular pathology provider. In at least some cases, the second set of data may include treatment information including one or more of: drug name, drug start date, drug end date, drug dose, drug unit, number of drug cycles, type of surgical procedure, date of surgical procedure, radiation site, radiation modality, radiation start date, radiation end date, total dose of radiation administered, and total percentage of radiation administered. In at least some cases, the second set of data may include outcome information including one or more of: response to treatment (e.g., CR, PR, SD, PD), RECIST score, outcome date, observation date, date of progression, date of recurrence, adverse event to treatment, date of presentation of adverse event, grade of adverse event, date of death, date of last follow-up, and disease status at last follow-up. In at least some cases, the second set of data may include information de-identified in accordance with de-identification methods permitted by HIPAA. In at least some cases, the second set of data may include information de-identified in accordance with Safe Harbor de-identification methods permitted by HIPAA. In at least some cases, the second set of data may include information that has been de-identified in accordance with statistical de-identification methods permitted by HIPAA.In at least some cases, the second set of data may include clinical health information of patients diagnosed with a cancer condition. In at least some cases, the second set of data may include clinical health information of patients diagnosed with a cardiovascular disease. In at least some cases, the second set of data may include clinical health information of patients diagnosed with a diabetic condition. In at least some cases, the second set of data may include clinical health information of patients diagnosed with an autoimmune condition. In at least some cases, the second set of data may include clinical health information of patients diagnosed with a lupus condition. In at least some cases, the second set of data may include clinical health information of patients diagnosed with a psoriasis condition. In at least some cases, the second set of data may include clinical health information of patients diagnosed with a depression condition. In at least some cases, the second set of data may include clinical health information of patients diagnosed with a rare disease.

[0295] 19 , in block 462, the system creates an appropriate audio response file, and in block 464, the response file is used to broadcast an audible response to the oncologist. In a first example, the system may generate a response that reads, "The provider has recommended palbociclib, a CDK4 / 6 inhibitor, based on Dwayne Holder's CDKN2A mutation. However, since he already received this medication from September 20, 2017, to January 6, 2018, we recommend that you consider targeting one of his other clinically actionable mutations." In a second example, the system may generate a response that reads, "Dwayne Holder's results show a pathogenic germline BRCA2 mutation combined with somatic loss of heterozygosity, indicating that this may be a BRCA2-driven tumor."

[0296] It is recognized that many different query intents may take on a similar format, with the differences between particular intents defined by specific parameters. Similarly, many system responses to different queries may have a similar format, with the differences between particular responses defined by the results produced by specific parameters in the query and / or data operations. For these reasons, in at least some embodiments, a dedicated user interface was developed to ease the burden on system administrators associated with specifying all possible system intents, contextual query parameters, data operations, and audio response files, as well as to manage that information as knowledge evolves over time. The interface generates sub-databases (see sub-databases in FIG. 20) that form the KDB shown in FIG. 20.

[0297] See FIG. 21, which schematically illustrates an exemplary user interface screenshot 520 corresponding to the provider panel sub-database 506 shown in FIG. 20. In addition to presenting the provider panel data set, the screenshot includes individually selectable icons for each sub-database type in FIG. 20, so that an administrator can access any of those sub-databases via a screenshot similar to that shown in FIG. 21. Screenshot 520 includes a spreadsheet-type arrangement of row and column information cells used by the system to process queries and generate responses, as well as interface tools for scrolling up and down and left and right to access additional sub-database information. Although not shown, the exemplary interface also includes a keyboard, mouse device, and / or other input device for interacting with the interface (e.g., scrolling, modifying information, adding or deleting information, etc.).

[0298] Still referring to FIG. 21 , screenshot 520 includes query intents 522A-ZZZ arranged in the first row of cells, with a separate intent in the top cell of each column within the first row. Intents often take the form of predefined queries to which a received query can be associated. The exemplary intent A shown is, "Does provider $panel come with clinical data structuring?" where the "$panel" expression is a parameter collected from a query received from an oncologist. While only a few intents are shown, it should be understood that hundreds or more intents may be expressed and accessed through the interface. The $panel expression is referred to as a parameter field; the system supports many parameter types with different parameter fields, and any intent may include two or more different parameter fields.

[0299] Still referring to FIG. 21, the parameters that may be entered into the $panel parameter field in an intent are listed in cells located in the left column 524 of screenshot 520 and include xT, xE, and xF, and may include many other panel types. Thus, depending on the query received (e.g., does the query refer to an xT panel?), any of xT, xE, xF, etc. may be entered into the $panel field in intent A to define a panel-specific intent.

[0300] In the screenshot, an answer section 526 provides an answer for each intent and parameter combination. Generally, the answer section includes a separate cell for each parameter row and intent column, and a separate scripted answer may be provided in each answer cell for each intent and parameter combination. For example, for intent C and an xT panel, the answer in the associated answer cell 530 is "Yes, the xT panel contains the matching regular sequencing."

[0301] If a general answer format is applicable to each parameter in column 524, an answer format may be provided if a specific parameter is to be used to fill in the parameter field in the answer format. For this purpose, see the answer formats in field 532 that require panel parameters in field $panel. Here, during operation, the system retrieves the appropriate panel parameters from column 524 and populates field $panel as needed. Although not shown in FIG. 21 , negative answer row 536 is also provided, which may include negative answer formats for one or each of the intents listed in row 522.

[0302] Still referring to FIG. 21 , an administrator can modify any intent, add an intent, delete an intent, modify parameters in column 524, add parameters, delete parameters, and / or modify answers by simply selecting the instance of the information to be modified and then entering the various information into the associated cells. In this manner, intents and answers of similar formats with various parameters can be quickly specified and managed without overall effort. For example, in FIG. 21 , assume that an interface specifies 200 different intents and that an administrator wants to add a new panel to the parameter options. Now, the administrator can associate all of the intents in row 522 with the new panel name simply by selecting another cell in the parameter column and naming the new panel. Furthermore, when a new panel is added to the panel column, for each answer format (e.g., see 532 again) that remains valid for the new panel, that answer format is automatically applied to the new panel.

[0303] 22, a second administrator interface screenshot 550 is shown having a similar format to the provider panel screenshot of FIG. 21 and therefore including an intent row 552, an answer section 554, and a parameters section 556. Each example intent includes a parameter field $Gene that is filled with one of the parameters from the parameters column 556 that form part of the received query.

[0304] In FIG. 22, answer section 554 differs from FIG. 21 because an "answer value" is provided for each answer cell (e.g., a cell corresponding to a particular intent column and parameter row combination) that is used in at least one, and possibly two, different ways. First, the answer in the answer cell corresponding to a particular intent and parameter combination can be used to select either answer format 551 or negative answer format 553. To this end, each of the answer formats and negative answer formats for each format includes a rule and a response format, where the rule is applied based on the answer cell value. Thus, for example, for the answer format in cell 560, the rule is "IF TRUE" (e.g., if a TRUE value is in the answer cell), then apply the associated answer format. Similarly, for the negative answer format in cell 562, the rule is "IF FALSE" (e.g., if a FALSE value is in the answer cell), then apply the associated negative answer format. Thus, for example, answer cell 570 contains gene ABCB1 and intent A value TRUE, so the answer format in cell 560 is applied and the response file contains the phrase "Yes, the provider sequences ABCB1." Similarly, answer cell 572 contains gene ABCB4 and intent A value false, so the negative answer format in cell 562 is applied and the response file contains the phrase "No, the provider does not sequence ABCB4."

[0305] Second, at least in some cases, the value of the answer cell can also be used to populate one or more fields in an answer or negative answer format. To this end, see, for example, the answer format in cell 576, which, in addition to including a $Gene field, also includes an $AV (e.g., answer value) field. Here, if an answer format rule is met (e.g., IF AV, the answer cell has an answer value) such that answer format 576 is used to generate a response file, in addition to populating the $Gene field with one of the genes in column 556, the $AV field is populated with the value from the associated answer cell below it. For example, for gene ABCB1, answer cell 578 contains the value 1%, so when intent C is applied and qualified by gene parameter ABCB1, the answer format rule in cell 576 is satisfied, and the response tile contains the phrase "The donor reports that 1% of pancreatic cancer patients have a pathogenic mutation in ABCB1." In negative answer cell 580, the rule is that if the answer cell below it is blank, that cell format is used to generate a response file.

[0306] Although there are two answer format rows shown in each of Figures 21 and 10 (e.g., an answer format row and a negative answer format row), in other cases there may be three or more answer formats that change based on the value of the particular answer field below them to support more complex answer generation schemes.

[0307] Again, as with the data presented in Figure 21, the data in Figure 22 shows only a small subset of the gene data that can be accessed left and right and up and down by scrolling up and down through the parameters and intents. For example, the genes in parameter column 556 may include an entire gene panel (e.g., hundreds of genes), and the intents in row 552 may include hundreds or thousands of intents.

[0308] FIG. 23 shows another management screenshot 600, similar to the screenshots in FIGS. 21 and 22, corresponding to a provider method data set. The spreadsheet representation in FIG. 23 is similar to the representations in FIGS. 21 and 22, including an intent row 602, an answer format section 610, and a parameter column 604. One difference in FIG. 23 is that the first intent, A, includes two parameter fields, and the parameter section includes first and second parameter rows, one for each of the parameter fields in intent A. More specifically, the parameter section includes a first column listing tests and a second column listing test methods for populating the associated $test and $testmethod fields in the intent statement. Furthermore, in at least some cases, answer formats, such as the negative answer format shown in cell 606, include more than one parameter or value field. The operation here is similar to that described above, but uses two parameters to instantiate a particular intent and final response file.

[0309] Referring again to Figure 20, interface screenshots similar to those described in Figures 21-23 are included in a system for specifying the intent, parameters, and answer format for each information type associated with the sub-database shown. Some screenshots include specific scripted answers for specific intents, while others rely on answer formats, one or all formatting rules, and entering intent parameters and / or database values ​​into answer fields that appear in answer cells as described above. Other combinations of screenshots and tools are possible.

[0310] At least in some cases, the system may allow an oncologist to request visual access to query answers and / or related information (e.g., related documentation (e.g., clinical trial information, drug label warnings, etc.)). For example, the oncologist may say, "Make the answer available on the system web platform," and the system may make the latest broadcast answer available via a nearby or dedicated computer display screen for the oncologist. At least in some cases, the system is contemplated to allow an oncologist or other user to provide a query via a typed question instead of an audible query. For example, rather than speaking a question, the oncologist may type the query into a mobile phone or other computing device, and the query may be processed as described herein.

[0311] 24 , a fourth exemplary system 650 is shown that includes a mobile device 652. The fourth exemplary system may include a collaboration device 20, a collaboration server 12, an AI provider server 14, and a database 18. The collaboration device 20, the collaboration server 12, the AI ​​provider server 14, and the database 18 may be linked together as described above in conjunction with FIG. 1. The mobile device 652 may be used in combination with the collaboration device 20 to authenticate and / or onboard user credentials and to perform at least some of the functions of the collaboration device 20 (e.g., process queries regarding patients), among other suitable uses.

[0312] The mobile device 652 can be a smartphone, a tablet, or another suitable mobile computing device. The mobile device 652 can include a camera 653, a speaker 654, a fingerprint sensor 656, and input buttons 658, as well as a touchscreen 660. Similar to the collaboration device 20, the mobile device 652 can transmit (666) voice signal messages to the transceiver 16 for processing by the collaboration server 12 and / or the AI ​​provider server 14. The mobile device 652 can also receive (662) a visual response file and / or receive (664) an audio response file generated based on the voice message signal from the transceiver 16.

[0313] Additionally, the mobile device 652 can transmit authentication information to the transceiver 16 to unlock the collaboration device 20 (668). The collaboration device 20 may be configured to request authentication from the oncologist at predetermined times (e.g., every 30 minutes, every hour, etc.) or when the oncologist moves the collaboration device 20. For example, the collaboration device 20 can detect that it has been moved if it loses contact with the transceiver 16. The oncologist may move the collaboration device to another room in the same building (e.g., a hospital) or to another building entirely (e.g., another hospital, a home office, etc.). It is understood that the collaboration device 20 is mobile and can be moved and used in various locations with appropriate connectivity (e.g., wireless internet).

[0314] The mobile device 652 may install a mobile device application (not shown) that can determine which authentication credentials are needed at a particular time and, if applicable, display a notification about the needed authentication credentials on the touchscreen 660. For example, if the mobile device application determines that 30 minutes have passed since the last authentication, the mobile device application may output a notification that the oncologist must re-authenticate before using the full functionality of the collaboration device 20 (e.g., querying the collaboration device 20 for a particular patient). In some embodiments, the mobile device application may not output a notification, and the collaboration device 20 may prompt the oncologist to re-authenticate when the oncologist attempts to query the collaboration device.

[0315] The mobile device 652 can provide multiple forms of authentication information to the transceiver (668). The authentication information can include a fingerprint scan generated using the fingerprint sensor 656, a photograph of the oncologist's face generated using the camera 653, and / or a text password. In some embodiments, the mobile device application can provide the raw fingerprint scan, image, and / or password to the transceiver 16, and another process (e.g., a process within the collaboration server 12) can determine whether the authentication information is sufficient (e.g., whether the fingerprint scan sufficiently matches a predetermined fingerprint scan associated with the oncologist). In other embodiments, the mobile device application can determine whether the authentication information is sufficient and transmit authentication information indicating whether the authentication information is sufficient (e.g., a Boolean yes / no) (668). If the authentication information is sufficient, the collaboration device 20 can resume full operation.

[0316] The mobile device 652 can also transmit user requests (i.e., oncologist requests) to the transceiver for processing by the collaboration server 12 (669). User requests can be notes, clinical reports, molecular reports, patients, cases, etc., product suggestions (e.g., collaboration device 20 features the oncologist wants to add), product fulfillment requests (e.g., ordering a test kit), the status of an ordered test kit (e.g., a liquid or tissue-based biopsy test kit for performing molecular tests), a recommendation for a tumor board session for the patient, or other requests that may be more easily disseminated by a human than a computer process, or a request for a human (e.g., an administrator, or possibly a medical practitioner) to review the request that is not necessarily related to intent fulfillment. Some user requests may be generated and transmitted by the mobile device 652 and / or the collaboration device 20 based on voice signals captured from the oncologist. The AI ​​database 14 can determine the intent of a user request, for example, being a test kit order or a request for manual review of a particular case. The text form of the speech signal and any associated information (e.g., intent) may then be transmitted to collaboration server 12, which may transmit the text form of the speech signal and any associated information to an administrator and / or appropriate computer process. If the user's request includes a recommendation for a patient for a tumor board or clinical trial, associated information about the patient may also be transmitted along with the request, significantly reducing the need to fill out application forms for clinical trials and / or tumor boards.

[0317] In some embodiments, processes performed by the mobile device 652 (eg, authentication processes, transmitting a user request (669), etc.) may be performed by the collaboration device 20.

[0318] 24 and 25, a mobile application screenshot 700 is displayed. The mobile application screenshot 700 may be part of a mobile device application included on the mobile device 652. The mobile application screenshot 700 may include a battery level indicator 702 indicating the battery level of the collaboration device 20, a username 703 corresponding to the current logged-in oncologist, an authentication indicator 704 indicating whether the oncologist has been authenticated by the mobile device 652, a microphone button 706, a night mode button 708, and a mute button 710. The oncologist may select the microphone button 706 instead of speaking a wake-up word or phrase (e.g., “Tempus ONE”) to prompt the collaboration device 20 and / or the mobile device 652 to record an audio signal. The night mode button 708 may control the darkness and / or color displayed by the mobile device application.

[0319] Mobile application screenshot 700 may include a slider 712. An oncologist may activate slider 712 to control the volume of collaboration device 20. Mobile application screenshot 700 may include a suggested questions section 714 that may display example queries and / or general queries that the oncologist may ask collaboration device 20. For example, a first question 716 may show the oncologist how to ask a question about a particular patient, and a second question 718 may show the oncologist how to ask a medical question that is not patient-specific.

[0320] 24, 25, and 26, a screenshot 720 of a second mobile application is shown. The screenshot 720 of the second mobile application may include the suggested questions section 714, first question 716, and second question 718 included in the screenshot 700 of the mobile application shown in FIG. 25. In FIG. 26, the suggested questions section 714 is shown to include additional suggested questions. The screenshot 720 of the second mobile application may include a suggest new feature button 722. An oncologist may, for example, select the suggest new feature button 722 and provide a suggestion for a new feature in a pop-up box. The mobile device 652 may then transmit the suggestion to an administrator.

[0321] The second mobile application screenshot 720 may include a frequently asked questions (FAQ) section 724 that includes general questions regarding the functionality of the collaboration device 20. The second mobile application screenshot 720, and more specifically, the FAQ section 724, may include a search button 726 that the oncologist can select to search a set of FAQs.

[0322] 25 and 27, a screenshot 730 of a third mobile application is shown. The screenshot of the third mobile application may include an answer 732 to the second question 718 shown in FIG. 25. The answer 732 may include text and may be included in a pop-up box that is displayed when the oncologist selects the second question 718.

[0323] 28 , a fifth exemplary collaboration system 750 is shown. The fifth exemplary system 750 can include an administrator device 752, such as a laptop or desktop computer. An administrator can use the administrator device 752 to analyze data aggregated from multiple oncologists, update firmware in the collaboration device 20, analyze requests from oncologists, update intent sets (e.g., in Dialogflow), and perform other appropriate tasks related to collaboration device and / or mobile device operation 652.

[0324] The fifth exemplary system 750 may include a cloud architecture 754 including several modules that may be located remotely (e.g., on one or more servers) relative to the administrator device 752, the collaboration device 20, and / or the mobile device 652. The collaboration device 20 and / or the mobile device 652 may be linked to an IoT core module 758 that may process authentication requests and other communications from the collaboration device 20. The IoT core module 758 may be linked to a pub / sub-module 760 that is linked to an authentication module 762 and a ping module 764. The pub / sub-module 760 may transmit updates (e.g., status updates) to the collaboration device 20 and / or the mobile device. The authentication module 762 may receive authentication requests from the collaboration device 20 and / or the mobile device 652. The pub / sub-module 760 may direct communications from the IoT core module 758 to either the authentication module 762 or the ping module 764, as appropriate. The pub / sub-module 760, the IoT core module 758, the authentication module 762, and / or the ping module 764 may be stored on the first server 756.

[0325] The collaboration device 20 and / or the mobile device 652 can be linked to a gateway module 768, which may be included in a second server 766. The gateway module 768 may include at least some of the processes included in the collaboration server 12. The collaboration device 20 and / or the mobile device 652 can transmit requests (e.g., user requests transmitted from the mobile device 652) and / or voice signal messages to the gateway module 768. The gateway module 768 can be linked to an AI module 774. The AI ​​module 774 can include at least some of the processes included in the AI ​​provider server 14 (e.g., a voice signal extraction process) and may receive voice signals, extract intents from the voice signals, and transmit data responses to the gateway module 768 for transmission to at least one of the collaboration device 20 and the mobile device 652. In some embodiments, the Dialogflow suite can be included in the AI ​​module 774. The gateway module 768 can also be linked to a debug bucket module 778 and a redis module 780, which are included in a third server 776. The gateway module 768 may be linked to an AI demo module 772 contained in a fourth server 770 .

[0326] The third server 776 may be fully accessible only by the administrator device 752 (e.g., configured to allow full control and / or modification of processes) and not accessible by the collaboration device 20 and / or the mobile device 652. The third server 776 may include an administrator module 782 accessible by the administrator device 752 to update firmware of the collaborator device 20, define intents, update intent fulfillment processes, and perform other administrator functions. The administrator module 782 may also process and / or transmit user requests (e.g., orders for test kits) to the administrator device 752. The administrator and / or administrator device 752 may then analyze the user request and proceed accordingly. For example, an order for a test kit may be transmitted to an order fulfillment center. As another example, an oncologist may request a manual review of a particular case, which may be transmitted to an administrator, who may assign the case to a medical practitioner for review within a predetermined timeframe, e.g., within 24 hours.

[0327] The third server 776 may include a console module 786 linked to the administrator device 752. The console module 786 may perform administrative tasks. An administrator database 784 may be linked to the administrator module 782. The console module 786 may be linked to a portal module 790 included in a fifth server 788. The portal module 790 may provide an interface for oncologists to review molecular test reports.

[0328] The fifth system 750, and more specifically, the administrator module 782, administrator device 752, console module 786, and / or administrator database 784, may track a number of collaboration devices 20. More specifically, the fifth system 750 may track relatively static information such as whether each collaboration device 20 is connected (e.g., in contact with the cloud architecture 754), active (e.g., processing queries), the version of firmware each collaboration device 20 is running, and the owner and / or institution associated with the device.

[0329] The administrator module 782 may include processes for analyzing queries from oncologists and generate usage data regarding how oncologists are using the collaboration device 20, how many test kits are ordered for different case types, how frequently questions about specific sections of generated clinical reports are asked about on / off FDA labeled drug questions, treatments associated with specific variants, actions taken by oncologists in different scenarios (e.g., what questions are being asked), and other pertinent data.

[0330] It is understood that servers 756, 766, 770, 776, and 788 can each include multiple servers. Additionally, at least some of the modules and / or processes included in cloud architecture 754 can be implemented using infrastructure-as-code that can be migrated between clouds such as AWS, Google Cloud, Azure, etc.

[0331] The fifth system 750, and more specifically, the administrator module 782, administrator device 752, console module 786, and / or administrator database 784, can track intent being processed across several cases (e.g., thousands of cases) and / or other actions being taken by one or more collaboration devices 20, or developments occurring in the medical community (e.g., new research articles, studies, and / or treatment techniques) to provide “nudges” to the oncologist to potentially make the oncologist aware of information that the oncologist may not potentially be aware of. The fifth system 750 can ask the oncologist for permission to analyze clinical data generated by the oncologist.

[0332] Other data that the administrator module 782, administrator device 752, console module 786, and / or administrator database 784 may track include the number of test kits ordered by an oncologist in a predetermined time frame (which may indicate whether onboarding was successful), answers (e.g., statistics) or other information (e.g., Tempus) provided by the collaboration device 20 that is not included in a clinical report. Insights, actionable mutations, etc.), regularly scheduled surveys of oncologists on the collaboration device 20 and / or mobile application (e.g., responses to "What information is most helpful when making clinical decisions?"), which parts of clinical reports are most frequently asked about (either to an individual oncologist or to multiple oncologists), how oncologists behave on a macroscale (e.g., what tests have other oncologists performed for a given cancer type and / or molecular mutation), how similar patients respond to specific therapies (e.g., how many patients who presented with a XXX molecular mutation also had variant YYY and how many of those had a ZZZ response to therapy AAA over the course of the BBB), or other suitable data.

[0333] In some embodiments, the administrator device 752 may provide at least some of the functionality of the collaboration device 20, while being tailored for an administrator. For example, the administrator device 752 may be appropriately equipped (e.g., with a microphone and speaker) and configured to answer questions such as "Where is sample [x] stored?" or "What is the SOP for scenario y?", which may be relevant only to the administrator. In some embodiments, the administrator may use the collaboration device 20 with a set of administrator-specific intents that may not be available to oncologists (i.e., of limited use). The administrator may, for example, utter "Tempus ONE, where is sample [x] stored?" and the collaboration device can determine that the intent of the query is to know the location (e.g., warehouse) of sample [x] and provide an appropriate visual and / or audible response.

[0334] Some data can then be used to customize the user experience for each oncologist. For example, data regarding which portions of the clinical report are most frequently asked about can be used to customize the report layout and format suggestions that the oncologist can accept (i.e., update the report layout and / or format) or reject (i.e., maintain the same report layout and / or format) after receiving the notification. The report displayed via the portal module 790 or mobile application (e.g., on the touchscreen 660) can then conform to the updated template and / or layout. Additionally or alternatively, the oncologist can provide suggestions regarding the report layout and / or format, and the report layout and / or format can be updated accordingly.

[0335] The fifth system 750 can use data collected by the fifth system 750 to provide nudges to the oncologist using the collaboration device 20 and / or the mobile device 652. The nudges can be provided to the oncologist without the oncologist having to ask. One nudge can include the fifth system 750 determining the most successful treatment for a patient similar to the patient the oncologist is analyzing. The most successful treatment can be determined based on the patient's molecular data (e.g., molecular mutations and / or variants), age, gender, etc., as well as the success rates of various treatments in populations with the same molecular data, age, gender, etc. Another nudge can include informing the oncologist about cancer boards, clinical trials, and other programs for which the patient is eligible within a predetermined radius (e.g., 15 miles from the medical facility where the oncologist is located). Additionally, the fifth system 750 can provide a nudge to the oncologist a predetermined time (e.g., within 24 hours) before the next patient visit, offering only cancer boards, clinical trials, and other programs that were unavailable when the patient last visited and / or the patient's last clinical report was generated. The oncologist can be notified by controlling the indicator light 50 with a predetermined pattern and / or color, outputting a specific sound on the speaker 44, displaying a notification on the mobile device 652, vibrating the collaboration device 20 and / or the mobile device 652 using a haptic signaling component, etc. Furthermore, tumor board and / or exam and / or action plan findings can be provided to the oncologist, or a specific action plan following the tumor board can be cited. The oncologist can then easily retrieve the specific tumor board findings.

[0336] Further nudges may include controlling indicator light 50 with a predetermined pattern and / or color, outputting a particular sound on speaker 44, displaying a notification on mobile device 652, vibrating collaboration device 20 and / or mobile device 652 using a haptic signaling component, etc. to indicate that a new molecular or clinical report is available to the patient.

[0337] Yet another nudge can include notifying oncologists of newly available content (e.g., research papers, articles, journals, posters, etc.) relevant to their practice area or patients. Oncologists can select notifications associated with multiple data sources, content types, cancer subtypes and / or diseases, molecular mutations / variants, treatments (FDA on-label, off-label, investigational, etc.), and clinical trials. A further nudge can include notifying oncologists that ordered tests may be completed more efficiently using an alternative test (e.g., using an xF liquid biopsy test instead of a tissue-specific xT panel). While insufficient tissue may prevent the test kit from being processed, the oncologist only sees the newly proposed test.

[0338] Further nudges can include informing oncologists of various patient tests and orders that other oncologists have performed on similar patients and / or cases. For example, a nudge can include a notification that a peer oncologist (or x% of other oncologists) has placed a test order for a similar patient. In some embodiments, the fifth system 750 can determine whether an oncologist has ordered PDL1 IHC for a particular test and notify the oncologist whether that test is a good option for the patient. An oncologist may not want to use a new vendor initially and may want to consult with a colleague to better understand the type of information returned via PDF reports, through the port AI module 790 or a mobile application on the mobile device 652. Knowing that other oncologist colleagues within the institution have ordered tests for x% of a particular patient cohort can help an oncologist operate with knowledge of how other oncologists are treating similar patients.

[0339] Still with respect to test kits, yet another nudge may include notifying the oncologist that test kit inventory is low (e.g., below a predetermined threshold). Some nudges may include information regarding potential financial assistance available for the test kit. Once the oncologist is informed of test kit options, the oncologist can order the test kit and / or apply for financial assistance by speaking appropriate commands on the collaboration device 20 and / or mobile device 652. The fifth system 750 may then automatically populate a test kit order form and / or a financial assistance application.

[0340] Some nudges could notify oncologists of nearby continuing medical education (CME) courses and / or allow oncologists to enroll in CME credit courses or highlight local and / or online services within a particular specialty and / or area of ​​focus.

[0341] 19, 24, and 28, and 29, a process 1000 for generating supplemental content for physicians based on molecular reports related to a particular patient is shown. Process 1000 can be used to provide patient-specific nudges to oncologists. Process 1000 can identify information that may be relevant to the patient's treatment and that the oncologist may not have considered when querying collaboration device 20. In this manner, collaboration device 20 may assist the oncologist in treating the patient using treatments, medications, clinical tests, and / or other treatment techniques applicable to the particular patient that the oncologist may not have been aware of or previously considered. Process 1000 may also provide the oncologist with information about how other oncologists have treated similar patients (e.g., diagnosed with similar genomic and / or similar cancer types). Process 1000 may be performed by a suitable system, such as the fifth exemplary system 750.

[0342] At 1002, process 1000 can determine a specific patient. At least in some cases, process 1000 can be performed in parallel and / or after process 450 has been performed and / or after process 450 has finished executing. Process 1000 can determine that the specific patient is the same specific patient identified by process 450. At least in some cases, process 1000 can be performed together with process 1000 to effectively form a single process. Process 1000 can then proceed to 1004.

[0343] At 1004, process 1000 can store and maintain a general cancer knowledge database. The general cancer knowledge database can include raw and / or processed data about a large number of patients, including molecular reports, the presence of conditions such as diabetes, heart disease, etc., information about each patient's treatment history, such as medications and / or treatments taken, and response to medications and / or treatments (e.g., the patient was successfully treated using the drug FFF), and / or other suitable data about the patient. The data associated with each patient can be persistently updated as additional information becomes available. The general cancer knowledge database can include non-patient-specific information about a particular topic based on accepted industry standards (e.g., the effectiveness of a particular drug in treating a particular cancer condition, clinical trial information, drug class-mutation interactions, genes, etc.) or empirical information derived by the service provider, as well as information about the service provider's system capabilities (e.g., information about specific tests and activities performed by the provider, testing requirements, etc.). The general cancer knowledge database can include the KDB 504 described above. The general cancer knowledge database can include information about available clinical trials, treatments, research, academic papers, CLE courses, or other available resources. The process 1000 can then proceed to 1006.

[0344] At 1006, the process 1000 can persistently update the molecular report for a particular patient. For example, the process 1000 can update the associated clinical trials included in the molecular report. The process 1000 can then proceed to 1008.

[0345] At 1008, process 1000 can automatically identify at least one intent and associated data action related to a general cancer knowledge database based on the particular patient's molecular report data. The at least one intent can be related to a medication, gene, test method, etc., as described above. The at least one intent can also be related to a particular cancer that the particular patient has been diagnosed with. At least some of the intents may be intents that the oncologist has not previously queried collaboration device 20 about. Process 1000 can then proceed to 1010.

[0346] At 1010, process 1000 can persistently perform associated data operations on the general cancer knowledge database to generate a new set of response data that was not previously generated. In some cases, process 1000 can persistently perform multiple associated data operations on the general cancer knowledge database. By persistently executing the general cancer knowledge database, process 1000 can provide updated information (i.e., a new set of response data) to oncologists. Furthermore, the new set of response data may be used to provide information relevant to a particular patient that the oncologist may not have been previously aware of. For example, the new set of response data can be used to inform the oncologist how various treatment options will perform for other patients with a similar genomic profile. As another example, the new set of data can be used to inform the oncologist of tests ordered for other patients diagnosed with the same cancer as the particular patient and who have a similar genomic profile (e.g., the presence of a particular gene mutation). Process 1000 can then proceed to 1012.

[0347] At 1012, once process 1000 generates a new set of response data, it can use the new set of response data to generate a notification to output to the oncologist. In some cases, the notification can be an audible response file that process 1000 generates based on the new set of response data. In some cases, the notification can be a visual indicator that process 1000 generates based on the new set of response data. The visual indicator can include a question related to the new set of response data. For example, the question can be a suggested question that can be answered using the new set of response data. In this example, if the new set of response data includes information about patient response to a particular treatment (e.g., patients with mutation XXX treated with YYY had a VVV% chance of survival with cancer type WWW), the suggested question can be, "How many patients with mutation XXX survived when treated with YYY?" or "What is the most effective treatment for patients with mutation XXX and cancer type WWW?" Process 1000 can then proceed to 1014.

[0348] At 1014, process 1000 may output the notification generated at 1012 to the oncologist. If the notification is an audible response file, the process may output the audible response file at collaboration device 20 (e.g., speaker 44) and / or mobile device 652 (e.g., speaker 654). If the notification is a visual indicator, the visual indicator may be output at collaboration device 20 (e.g., display screen 48) and / or mobile device 652 (e.g., touch screen 660). If the visual indicator is a suggested question, the suggested question may be displayed in suggested questions section 714 above. The notification may function as a nudge. Thus, process 1000 may generate and provide at least some of the nudges above to the oncologist.

[0349] 19, 24, and 28, and 30, a process 1050 for generating non-patient-specific supplemental content for physicians is shown. Process 1050 can be used to provide non-patient-specific nudges to oncologists. Process 1050 can identify information that may be generally relevant to oncologists, such as newly available treatments, research, and academic papers. Process 1050 can reduce the need for oncologists to search for new developments in their field of practice. For example, if an oncologist specializes in treating breast cancer patients, process 1050 may provide information that may be useful in treating breast cancer patients. Process 1050 may be performed by a suitable system, such as the fifth exemplary system 750.

[0350] At 1052, process 1050 can determine one or more streams of interest for the oncologist. The streams of interest can include newly available clinical trials, treatments, research, journal articles, CLE courses, or other suitable types of information and / or programs related to the cancer type that may be useful to the oncologist. In some embodiments, the oncologist can provide (e.g., audibly) the stream types of interest and / or cancer types of interest to process 1050. Process 1050 may automatically determine the streams of interest based on the oncologist's history. For example, an oncologist may generally treat patients with breast cancer and lung cancer, and process 1050 can select available streams of interest related to those cancer types. Process 1050 can then proceed to 1054.

[0351] At 1054, process 1050 can store and maintain a general cancer knowledge database. The general cancer knowledge database can include raw and / or processed data about a large number of patients, including molecular reports, the presence of conditions such as diabetes, heart disease, etc., information about each patient's treatment history, such as medications and / or treatments taken, and response to medications and / or treatments (e.g., the patient was successfully treated using the drug FFF), and / or other suitable data about the patient. The data associated with each patient can be persistently updated as additional information becomes available. The general cancer knowledge database can include non-patient-specific information about a particular topic based on accepted industry standards (e.g., the effectiveness of a particular drug in treating a particular cancer condition, clinical trial information, drug class-mutation interactions, genes, etc.) or empirical information derived by the service provider, as well as information about the service provider's system capabilities (e.g., information about specific tests and activities performed by the provider, testing requirements, etc.). The general cancer knowledge database can include the KDB 504 described above. The general cancer knowledge database can include information about available clinical trials, treatments, research, academic papers, CLE courses, or other available resources. The process 1050 can then proceed to 1056.

[0352] At 1056, process 1050 can automatically identify at least one intent and associated data operation related to the general cancer knowledge database based on the stream of interest associated with the oncologist. For example, the at least one intent may relate to identifying whether new scientific articles on a particular type of cancer (e.g., breast cancer) are newly available (e.g., published last week), identifying whether new clinical trials for a particular type of cancer (e.g., lung cancer) are newly available, identifying whether new treatment options for a particular type of cancer (e.g., breast cancer), or any other suitable intent. In this example, the associated data operation may include searching for available new scientific articles, clinical trials, and treatment options. At least some of the intents may be intents that the oncologist has not previously queried on collaboration device 20. Process 1050 can then proceed to 1058.

[0353] At 1058, process 1050 can persistently perform associated data operations on the general cancer knowledge database to generate a new set of response data that was not previously generated. In some cases, process 1050 can persistently perform multiple associated data operations on the general cancer knowledge database. By persistently running the general cancer knowledge database, process 1050 can provide updated information (i.e., a new set of response data) to oncologists. The new set of data can be used to inform oncologists of newly available journal articles, clinical trials, available treatment options, etc. Process 1050 can then proceed to 1060.

[0354] At 1060, once process 1050 generates a new set of response data, it can use the new set of response data to generate a notification for output to the oncologist. In some cases, the notification can be an audible response file that process 1050 generates based on the new set of response data. In some cases, the notification can be a visual indicator that process 1050 generates based on the new set of response data. The visual indicator can include a question related to the new set of response data. For example, the question can be a suggested question that can be answered using the new set of response data. In this example, if the new set of response data includes information about a newly available breast cancer treatment (e.g., treatment YYY is now available to breast cancer patients), the suggested question can be, "Are there any new treatment options available for breast cancer patients?" Process 1050 can then proceed to 1062.

[0355] At 1062, process 1050 may output the notification generated at 1060 to the oncologist. If the notification is an audible response file, the process may output the audible response file at collaboration device 20 (e.g., speaker 44) and / or mobile device 652 (e.g., speaker 654). If the notification is a visual indicator, the visual indicator may be output at collaboration device 20 (e.g., display screen 48) and / or mobile device 652 (e.g., touch screen 660). If the visual indicator is a suggested question, the suggested question may be displayed in suggested questions section 714 above.

[0356] Referring now to FIG. 31 , a process 800 that may be used for onboarding an oncologist is shown. At block 802, process 800 may determine that a user (e.g., an oncologist) has opened a mobile application and that the collaboration device is turned on. The mobile application may be a mobile application included on mobile device 652, and the collaboration device may be collaboration device 20 described above. Collaboration device 20 may output, using speaker 44, "Hello, your Tempus ONE is ready for setup. Download the Tempus ONE mobile app to begin setup." Control passes to block 804, where process 800 may display an option to confirm high-level instructions to the oncologist. The option may be displayed on a user interface, such as touchscreen 660. Control passes to block 806, where process 800 may determine whether the oncologist has logged into the mobile application.

[0357] Once the oncologist has logged in, control may pass to block 808, where process 800 may attempt to log the mobile device 652 into the wireless network to which the collaboration device is connected. After the mobile device 652 has logged into the wireless network, control may pass to block 810, where process 800 may display options for configuring security settings to the oncologist in a user interface. Control may pass to block 812, where if the oncologist selects the option to configure security settings (i.e., "yes" at block 812), process 800 proceeds to block 814. If the oncologist does not select the option to configure security settings (i.e., "no" at block 812), control may pass to block 816. In block 814, process 800 may configure security settings of the mobile device 652 and / or the collaboration device 20. For example, process 800 may set the oncologist's authentication preferences (e.g., fingerprint preferences, facial recognition preferences, or entered password preferences).

[0358] Flow then passes to block 816, where process 800 may display an option to open an educational module to the oncologist in the user interface. Control then passes to block 818, where if the oncologist selects the option to open an educational module, process 800 may proceed to block 820. If the oncologist does not select the option to open an educational module, process 800 may proceed to block 822. In block 820, process 800 may display instruction manuals (i.e., a user manual) to the oncologist. Control then passes to block 822, where process 800 may display an FAQ menu, as well as a suggested pathway tutorial option, to the oncologist in the user interface. Control then passes to block 824, where if the oncologist selects the suggested pathway tutorial option, control passes to block 826. If the oncologist does not select the suggested pathway tutorial option, process 800 ends.

[0359] In block 826, process 800 executes at least one tutorial, which may include a tutorial on how to use the collaboration device 20 (e.g., how to change the volume of the collaboration device 20) and suggests “first questions” the oncologist might want to ask the collaboration device. In particular, the tutorial related to the suggested questions may instruct the oncologist on how to query the collaboration device. After the oncologist performs several questions, the collaboration device 20 may terminate the tutorial and allow the oncologist to ask questions independently. The tutorial may be generated by recognizing the type of intent a particular physician might have and predicting questions based on various criteria (e.g., facility, specialty, questions from other physicians in the physician's department, patient molecular / clinical data and past order history, upcoming patients based on EMR scheduling integration, etc.). The tutorial may visually and / or audibly instruct the user on basic voice commands that the collaboration device can recognize, such as “volume up,” “volume down,” “start pairing,” “turn off,” or other appropriate voice commands.

[0360] Referring now to FIG. 32 , a screenshot 1100 is shown that a system administrator uses to specify system intents, intent parameters, and answer formats for provider panel types, consistent with at least some aspects of the present disclosure. As shown, a panel variable module 1104, an intent module 1108, and an answer module 1112 can be used by a user to specify an intent. Note that the modules may have different appearances for ease of identification. For example, modules 1104-1112 can be different shapes. An intent module can have an intent (i.e., “Intent A”) that may require variable inputs to answer. In this example, the variable is the panel type. The panel variable module 1104 corresponding to the type of panel can be linked to the intent module 1108, which can also be linked to the answer module 1112, which can automatically fill in the answer based on the panel variable module 1104. A user can drag and link the modules 1104-1112 using a mouse or touch screen to create intents and associated answers.

[0361] 1 and 33 , an intent extraction architecture 1150 is illustrated consistent with at least some aspects of the present disclosure. The intent extraction architecture 1150 may include an input module 1158 including a microphone and an output module 1162 including a speaker that may be included in the collaboration device 20. A user 1154 may provide an audible query to the input module 1158 and receive an audible answer from the output module 1162. The input module 1158 may process the audible query (e.g., perform text recognition) and transmit a query 1166 to an intent matching module 1174 included in the intent extraction architecture 1150. The intent matching module 1174 may include an intent matching application such as Dialogflow. The intent matching module 1174 may extract an intent from the query and transmit the query 1166 and the intent to a parameter extraction module 1178 included in the intent extraction architecture 1150. The parameter extraction module 1178 may extract any parameters from the query 1166 that are associated with the intent. The parameter extraction module 1178 may then communicate with the API module 1182 and / or the database 1186 included in the intent extraction architecture 1150 to extract information related to the extracted parameters and / or intent from the database 1186. The information may be transmitted to the intent matching module 1174. The intent matching module 1174 may generate actionable data 1170 based on the information from the database 1186. The intent matching module 1174 may then transmit the actionable data 1170 to the output module 1162, which may output an audible answer to the user 1154 based on the actionable data 1170.

[0362] 1 , 28 , and 33 , an exemplary question and answer workflow 1200 consistent with at least some aspects of the present disclosure is shown. The workflow 1200 may include one or more collaboration devices 1204 (e.g., collaboration device 20 and / or mobile device 652), each including an input module 1208 and an output module 1212. The input module 1208 may include at least some of the components of the input module 1158, and the output module 1212 may include at least some of the components of the output module 1162. The input module 1208 may receive an audible query from an oncologist. The audible query may include a single question that may be formulated from a series of prompts displayed on one of the collaboration devices 1204. The input module 1208 may output the audible query (which may include a raw audio file) to an agent module 1216 included in the workflow 1200. The agent module 1216 may include several natural language understanding (NLU) modules that convert text or spoken user requests into actions. The agent module 1216 can convert the audible query into an action and transmit the action to an intent matching module 1224 included in the workflow 1200. The intent matching module 1224 can be substantially the same as the intent matching module 1174. The intent matching module 1224 can communicate with a fulfillment module 1228 included in the workflow 1200. The fulfillment module 1228 can include intent-specific webhooks for agent lookup of business logic. The fulfillment module 1228 can receive intents from the intent matching module 1224 and communicate with the API module 1232 included in the workflow 1200 to extract relevant information from a database linked to the API module 1232.The fulfillment module 1228 may then receive the relevant information from the API module 1232 and transmit the relevant information to the intent matching module 1224. The intent matching module 1224 may then generate a response 1220 based on the relevant information and transmit the response 1220 to the output module 1212. The output module 1212 may then output the response 1220 visually and / or audibly.

[0363] 33, 34, and 35, an example conversation workflow 1250 is shown consistent with at least some aspects of the present disclosure. A collaboration device 1254 (e.g., collaboration device 20) can receive an audible query. For example, the audible query can be, "Hello, Wang, how many patients with identified PIK3CA mutations have you sent to Tempus in the past 60 days?" The collaboration device 1254 can transmit text extracted from the audible query to an intent matching module 1258, which can be substantially similar to the intent matching module 1224. The intent matching module 1258 can extract an intent associated with the audible query from the text. For example, the intent can be "Wang, patient, count." The intent matching module 1258 can transmit the audible query and the text associated with the intent to an entity module 1262, which can be substantially similar to the parameter extraction module 1178. The entity module 1262 can determine one or more parameters based on the text. For example, the entity module 1262 may determine a status parameter (e.g., @status:Sequenced), a mutation parameter (e.g., @mutation:PIK3CA), and a timeframe parameter (e.g., @timeframe:60). The entity module 1262 may transmit the parameters, text, and / or intent to a fulfillment module 1266, which formulates the parameters as a request to a database containing the actual values ​​of the parameters. The fulfillment module 1266 transmits the request to the intent to a database matching module 1270, which extracts the requested values ​​from the database. The intent to the database matching module 1270 can then output the requested values ​​as a response module 1274, which may be included in the intent matching module 125...

Claims

1. 1. A method for audibly broadcasting a response to a user based on a user query about a molecular report of a particular patient, for use in a collaboration device including a processor, a microphone, and a speaker linked to said processor, (i) storing molecular reports of a plurality of patients in a system database; (ii) receiving an audible query from the user via the microphone; (iii) identifying at least one intent associated with the audible query; (iv) identifying at least one data action associated with said at least one intent; (v) accessing the molecular report of said particular patient; (vi) performing at least one of the identified at least one data operation on a first data set included in the molecular report of the particular patient to generate a first set of response data; (vii) using the first set of response data to generate an audible response file; (viii) broadcasting the audible response file through the speaker; A method comprising:

2. 10. The method of claim 1, further comprising identifying qualifying parameters in the audible query, wherein identifying at least one data action comprises identifying the at least one data action based on both the identified intent and the qualifying parameters.

3. The method of claim 2 , wherein at least one of the eligibility parameters includes patient identity.

4. The method of claim 2 , wherein at least one of the eligibility parameters includes a patient condition.

5. 3. The method of claim 2, wherein at least one of the qualifying parameters comprises a genetic variation.

6. The method of claim 2 , wherein at least one of the eligibility parameters includes a type of treatment.

7. 2. The method of claim 1, further comprising identifying qualifying parameters in the molecular report of the particular patient, wherein identifying at least one data action comprises identifying the at least one data action based on both the identified intent and the qualifying parameters.

8. 2. The method of claim 1, further comprising the step of storing a general knowledge database containing non-patient-specific data on a particular topic, wherein the step of identifying at least one data action associated with the at least one intent comprises the step of identifying at least first and second data actions associated with the at least one intent, wherein the first data action is associated with a molecular report of the particular patient and the second data action is associated with the general knowledge database.

9. 9. The method of claim 8, wherein a second data operation associated with the general knowledge database is first performed to generate a second data operation result, the second data operation result is used to define the first data operation, and the first data operation associated with a molecular report of the specific patient is second performed to generate the first set of response data.

10. 9. The method of claim 8, wherein a first data operation associated with the molecular report of the particular patient is first performed to generate a first data operation result, the first data operation result is used to define the second data operation, and the second data operation associated with the general knowledge database is second performed to generate the first set of response data.

11. 2. The method of claim 1, wherein the identifying at least one intent comprises determining that the audible query is associated with the particular patient, accessing a molecular report of the particular patient, determining a cancer status of the particular patient from the molecular report, and then selecting an intent from a pool of intents related to a cancer status.

12. 12. The method of claim 11, further comprising storing a general knowledge database containing non-patient-specific data on a particular topic, and upon determining that the audible query is not associated with a particular patient, selecting an intent associated with the general knowledge database.

13. The method of claim 1 , wherein the collaboration device comprises a portable wireless device including a wireless transceiver.

14. The method of claim 13 , wherein the collaboration device is a handheld device.

15. 14. The method of claim 13, wherein the collaboration device includes at least one visual indicator, and the processor is linked to the visual indicator and is controllable to change at least some aspects of the appearance of the visual indicator to indicate different states of the collaboration device.

16. 2. The method of claim 1, wherein the processor is programmed to monitor a microphone input to identify a "wake up" phrase, and the processor monitors the audible query after the wake up phrase is detected.

17. 10. The method of claim 1, wherein a series of audible queries are received via the microphone, and wherein the at least one of the identified data operations includes identifying a subset of data that can be used in a subsequent audio query to identify an intent associated with the subsequent query.

18. 10. The method of claim 1, further comprising: identifying at least one activity that a collaboration device user may want to perform based on at least one audible query received via the microphone and associated data in a system database; and initiating the at least one activity.

19. 20. The method of claim 18, wherein the initiating the at least one activity comprises generating a second audible response file, broadcasting the second audible response file to the user seeking verification that the at least one activity should be performed, monitoring the microphone for a positive response, and initiating the at least one activity upon receiving a positive response.

20. 20. The method of claim 18, wherein the at least one activity includes periodically capturing health information from electronic health records contained in the system database.

21. 20. The method of claim 18, wherein the at least one activity includes checking the status of an existing clinical or test order.

22. 20. The method of claim 18, wherein the at least one activity includes ordering a new clinical or laboratory order.

23. The method of claim 1 , wherein the collaboration device is one of a smartphone, a tablet computer, a laptop computer, a desktop computer, or an Amazon Echo.

24. 20. The method of claim 18, wherein the initiating the at least one activity comprises automatically initiating the at least one activity without initiating input from the user.

25. 10. The method of claim 1, further comprising the steps of: storing and maintaining a general cancer knowledge database; persistently updating the molecular report of the specific patient; automatically identifying at least one intent and associated data operation related to the general cancer knowledge database based on the molecular report data of the specific patient; persistently performing the associated data operation on the general cancer knowledge database to generate a new set of response data not previously generated; upon generating a new set of response data, using the new set of response data to generate another audible response file; and broadcasting the other audible response file via the speaker.

26. 10. The method of claim 1, also for use with an electronic health record system that maintains health records associated with a plurality of patients, including the particular patient, further comprising identifying at least another data action associated with the at least one intent, and performing the another data action on the particular patient's health record to generate additional response data.

27. 27. The method of claim 26, wherein using the first set of response data to generate an audible response file comprises using the response data and the additional response data to generate the audible response file.

28. 1. A method for audibly broadcasting a response to a user based on a user query about a molecular report of a particular patient, for use in a collaboration device including a processor, a microphone, and a speaker linked to said processor, (i) storing an individual molecular report for each of a plurality of patients in a system database; (ii) storing a general cancer knowledge database containing non-patient-specific data on the topic of cancer; (iii) receiving an audible query from the user via the microphone; (iv) identifying at least one intent associated with the audible query; (v) identifying at least a first data action associated with the at least one intent and the molecular report of the particular patient; (vi) identifying at least a second data action associated with the at least one intent and the general cancer knowledge database; (vii) accessing the molecular report of the specific patient and the general cancer knowledge database; (viii) performing the at least first data operation on a first data set included in the molecular report of the particular patient to generate a first set of response data; (ix) performing said at least a second data operation of said general cancer knowledge database to generate a second set of response data; (x) using at least one of the first and second sets of response data to generate an audible response file; (xi) broadcasting said audible response file through said speaker; A method comprising:

29. 1. A method for audibly broadcasting a response to a user based on a user query about a molecular report of a particular patient, for use in a collaboration device including a processor, a microphone, and a speaker linked to said processor, (i) storing molecular reports of a plurality of patients in a system database; (ii) receiving an audible query from the user via the microphone; (iii) determining that the audible query is associated with the particular patient; (iv) accessing the molecular report of said particular patient; (v) determining the cancer status of the particular patient from the molecular report; (vi) identifying at least one intent from a pool of intents related to the particular patient's cancer status and the audible query; (vii) identifying at least one data action associated with said at least one intent; (viii) performing at least one of the identified at least one data operation on a first dataset included in the molecular report of the particular patient to generate a first set of response data; (ix) using the first set of response data to generate an audible response file; (x) broadcasting said audible response file through said speaker; A method comprising:

30. 1. A method for audibly broadcasting a response to a user based on a user query about a patient, for use in a collaboration device including a processor, a microphone, and a speaker linked to said processor, (i) storing health records of a plurality of patients in a system database and storing a general cancer knowledge database; (ii) receiving an audible query from the user via the microphone; (iii) identifying a particular patient associated with the audible query; (iv) accessing the health record of the particular patient; (v) identifying cancer-related data in the particular patient's health record; (vi) identifying at least one intent associated with the identified cancer-related data; (vii) identifying at least one data action associated with said at least one intent; (viii) performing said at least one data operation against said general cancer knowledge database to generate a first set of response data; (ix) using the first set of response data to generate an audible response file; (x) broadcasting said audible response file through said speaker; A method comprising:

Images