Oncology Workflow for Clinical Decision Support

Abstract

A system and method for managing patient data is provided. The system integrates medical data from multiple sources into an integrated patient database. Structured and unstructured medical data is acquired, enriched (e.g., by specifying data field types, standardizing data types or terminology, etc.), and stored in the integrated patient database. Data retrieved from different sources is stored in data elements in the integrated patient database in a network of connected objects including data related to tumor masses, treatments, reports, medical history, and diagnoses. Data in the integrated patient database is used to display patient data in user-friendly interface views, including a patient journey view that displays patient data in a chronological format organized by data type. Different interface views can be traversed to easily display patient data originating from different sources to improve the clinical decision-making process.

Description

Related Applications

[0001] Cross - References to Related Applications This application claims the benefit of priority to U.S. Patent Application No. 63 / 138,275, filed on January 15, 2021, and U.S. Patent Application No. 63 / 256,476, filed on October 15, 2021, each of which is incorporated herein by reference for all purposes.

Background Art

[0002] Every day, hospitals generate a vast amount of clinical data worldwide. The analysis of this data is important for understanding detailed insights in the provision of healthcare and the quality of care and for providing a basis for improving personalized medicine. Unfortunately, most of the recorded data is difficult to access and analyze because most data is captured in an unstructured format. Unstructured data can include, for example, notes from healthcare providers, imaging or pathology reports, or any other data that is not associated with a structured data model and is not organized in a predefined way to define the context and / or meaning of the data. Data is typically stored in multiple data sources. A clinician attempting to analyze a patient's data to make a decision may need to retrieve data from multiple data sources and then manually parse the data to extract the information necessary to make a clinical decision. However, such methods of obtaining data to make a clinical decision are time - consuming, slow, costly, and error - prone.

Summary of the Invention

[0003] This specification discloses technologies for improving clinicians' access to patient data and enabling them to make clinical decisions, such as those related to oncology. In some examples, a medical data processing system is provided. The medical data processing system can collect patient medical data from multiple data sources, transform the medical data into structured data, and present the structured data in various formats, such as summary and long-term view report formats. The medical data processing system can also support an oncology workflow solution that can assist or perform diagnostic actions on the collected medical data and present the diagnostic results to the clinician. The oncology workflow solution can enable clinicians, such as oncologists or their representatives, to manage cancer patients from suspected to long-term through treatment and follow-up. The oncology workflow solution can also support other medical applications, such as quality of care assessment tools for evaluating the quality of care provided to patients, and medical research tools for determining correlations between various patient information (e.g., demographic information) and patient tumor information (e.g., prognosis or expected survival). This technology is not limited to oncology and may be applied to other types of disease areas.

[0004] In some embodiments, a method for managing medical data includes, by a server computer, creating a patient record for a patient in an integrated patient database, the patient record including a patient identifier and one or more data objects relating to medical data associated with the patient, the integrated patient database creating a patient record including data from multiple sources, reading a medical record for a patient from an external database, receiving an identification of a primary cancer associated with the medical record via a graphical user interface (GUI), creating a primary cancer object in the patient record in response to receiving the identification of a primary cancer, the primary cancer object having a field containing the primary cancer, storing a medical record linked to the primary cancer object in the patient record in the integrated patient database, receiving the patient's medical data via user input to the GUI, determining that the patient's medical data is associated with a primary cancer, and storing the patient's medical data linked to the primary cancer object in the patient record in the integrated patient database.

[0005] In some embodiments, a medical record for a patient is a first format comprising a set of data elements associated with a corresponding data type, and receiving a primary cancer identification comprises identifying the primary cancer by analyzing the data elements and data type, displaying a GUI that includes a prompt for the user to confirm the primary cancer identification, and receiving user confirmation of the primary cancer identification via the GUI.

[0006] In some embodiments, the medical record is a first medical record, and the method further includes receiving a second medical record about a patient, wherein the second medical record is in a second format containing unstructured data; identifying data elements associated with primary cancer from the unstructured data; analyzing the unstructured data to assign the data elements to data types; and storing the data elements linked to primary cancer objects in a patient record in an integrated patient database based on the assigned data types and the identification that the data elements are associated with primary cancer.

[0007] In some embodiments, receiving identification of primary cancers associated with a medical record includes, via a GUI, displaying the medical record and a menu configured to receive user input to select one or more primary cancers, and receiving user input to select primary cancers via a GUI.

[0008] In some embodiments, the method further includes storing a medical record in a patient record, parsing the medical record and determining that the patient record is not associated with a particular primary cancer, and displaying the medical record and menu in response to the determination that the patient record is not associated with a particular primary cancer.

[0009] In some embodiments, the medical record includes unstructured data, the method further includes applying a first machine learning model to identify text within the medical record, and applying a second machine learning model to correlate portions of the identified text with corresponding fields, and storing the medical record further includes storing the identified text in an integrated patient database in association with fields. In some embodiments, the first machine learning model includes an optical character recognition (OCR) model, and the second machine learning model includes a natural language processing (NLP) model.

[0010] In some embodiments, the method further includes retrieving at least a subset of a patient's medical data from an integrated patient database and displaying at least a subset of the patient's medical data via a user interface for performing clinical decision-making. In some embodiments, the external database corresponds to at least one of the following: an EMR (Electronic Medical Records) system, a PACS (Picture Archiving and Communication System), a Digital Pathology (DP) system, a LIS (Laboratory Information System), and a RIS (Radiology Information System). In some embodiments, the medical records are retrieved based on the patient's identifier.

[0011] In some embodiments, a method for managing an integrated patient database is a server computer that stores patient records in the integrated patient database, which include a network of interconnected data objects, wherein the integrated patient database stores patient records that include data from multiple sources, and stores a first data object in the patient records within the integrated patient database that corresponds to a data element of a primary cancer tumor mass, wherein the first data object includes an attribute that specifies the location of the tumor mass, and receives diagnostic information corresponding to primary cancer from a diagnostic computer, analyzes the diagnostic information to identify a correlation between the diagnostic information and the tumor mass, and integrates based on identifying the correlation between the diagnostic information and the tumor mass. The system includes storing a second data object in a patient database that corresponds to diagnostic information, wherein the second data object is connected to a first data object via a network of interconnected data objects; receiving treatment information corresponding to primary cancer from a diagnostic computer; analyzing the treatment information to identify a correlation between the treatment information and tumor masses; and, based on the identification of the correlation between the treatment information and tumor masses, storing a third data object in an integrated patient database that corresponds to treatment information, wherein the third data object is connected to a first data object via a network of interconnected data objects.

[0012] In some embodiments, the method further includes retrieving one or more of the following from an integrated patient database: attributes specifying the location of a tumor mass, diagnostic information, and / or treatment information; and displaying one or more of the following attributes specifying the location of a tumor mass, diagnostic information, and / or treatment information via a user interface for clinical decision-making.

[0013] In some embodiments, the method further includes receiving patient history data from a diagnostic computer, analyzing the patient history data to identify a correlation between the patient history data and tumor masses, and storing a fourth data object in an integrated patient database corresponding to the patient history data, wherein the fourth data object is connected to a first data object via a network of interconnected data objects.

[0014] In some embodiments, the method further includes receiving tumor mass information from a diagnostic computer corresponding to tumor masses at metastatic sites of primary cancer; analyzing the tumor mass information to identify correlations between diagnostic information and tumor masses; and, based on receiving the tumor mass information and identifying a first data object, storing in an integrated patient database a fifth data object corresponding to the tumor mass information, which is connected to the first data object via a network of interconnected data objects. In some embodiments, the second data object includes one or more attributes selected from the stage of primary cancer, biomarkers, and tumor size.

[0015] In some embodiments, the method further includes identifying data elements and data types associated with a patient from an integrated patient database, and transmitting the data elements and data types to an external system in a structured format. In some embodiments, when the method generates a first data object and a second data object, it further includes generating a first timestamp stored in association with the first data object indicating the creation time of the first data object, and a second timestamp stored in association with the second data object indicating the creation time of the second data object.

[0016] In some embodiments, the method further includes updating an integrated patient database by importing medical data from an external database, parsing the imported medical data to identify specific data elements associated with the patient and primary cancer, and associating the specific data elements with a first data object and storing them in a sixth data object.

[0017] In some embodiments, the external database corresponds to at least one of the following: EMR (Electronic Medical Records) system, PACS (Picture Archiving and Communication System), Digital Pathology (DP) system, LIS (Laboratory Information System), and RIS (Radiology Information System).

[0018] In some embodiments, a method for processing medical data to facilitate clinical decision-making includes, by a server computer, receiving identification data that identifies a patient via a graphical user interface; receiving user input to select one of a set of selectable modes of the graphical user interface; reading a set of medical data associated with the patient from an integrated patient database based on the identification data and user input, wherein the set of medical data corresponds to the selected mode; and displaying a set of user-selectable objects in a timeline via the graphical user interface, wherein the objects are organized into rows, each row corresponding to a different category of a plurality of categories, the plurality of categories including lesions, diagnoses, and treatments, and displaying a set of objects.

[0019] In some embodiments, retrieving a set of medical data includes querying an integrated patient database to identify patient records for a patient from the integrated patient database, wherein the patient record includes identifying a patient record that contains patient objects, identifying each of the sets of objects associated with the patient object, and retrieving a predetermined subset of the identified sets of objects for display.

[0020] In some embodiments, the set of medical data corresponds to one or more of the following: a procedure object in an integrated patient database, the procedure object storing the procedure type, date, and response to the procedure; a diagnostic findings object in an integrated patient database, the diagnostic findings object storing biomarker data, staging data, and / or tumor size data; and a history object in an integrated patient database, the history object storing surgical history, allergies, and / or family medical history.

[0021] In some embodiments, the method further includes detecting user interactions with objects from a set of objects, identifying and retrieving corresponding reports from an integrated patient database, and displaying the reports via a graphical user interface. In some embodiments, the graphical user interface further comprises a ribbon displayed above a timeline, the ribbon displaying a subset of objects flagged as significant.

[0022] In some embodiments, the graphical user interface further includes an element for navigating to a second interface view, and the method further includes detecting user interaction with the element for navigating to the second interface view and transitioning to a second interface view displaying oncological summary data.

[0023] In some embodiments, a method for managing patient data includes storing patient records in an integrated patient database, wherein the integrated patient database includes data from multiple sources, and the patient record includes a plurality of data objects, including a first primary cancer data object that stores data elements corresponding to a first tumor mass of the patient, and a second primary cancer data object that stores data elements corresponding to a second tumor mass of the patient; rendering and displaying a graphical user interface that includes a patient summary containing information summarizing the patient data in the patient record in the integrated patient database; detecting user interactions with elements of the graphical user interface; reading data elements from the first primary cancer data object and the second primary cancer data object of the patient record from the integrated patient database in response to the detection of user interactions; rendering a first modal corresponding to the first primary cancer of the patient and a second modal corresponding to the second primary cancer of the patient; and displaying the first modal and the second modal side by side in the graphical user interface.

[0024] In some embodiments, each modal displays a set of biomarkers having a timestamp, disease staging information, and metastatic site information. In some embodiments, multiple sources include two or more of the following: EMR (Electronic Medical Records) systems, PACS (Picture Archiving and Communication Systems), Digital Pathology (DP) systems, LIS (Laboratory Information Systems), RIS (Radiology Information Systems), patient report outcomes, wearable devices, or social media websites.

[0025] In some embodiments, a method for processing medical data to facilitate clinical decision-making includes receiving input medical data of a patient associated with multiple data categories via a portal, wherein the multiple data categories are associated with oncology workflow operations; generating structured medical data of a patient based on the input medical data, wherein the structured medical data is generated to assist oncology workflow operations in generating diagnostic results, wherein the diagnostic results include one of the following: the patient does not have cancer; the patient has primary cancer; the patient has multiple primary cancers; or the patient has cancer at an unknown primary site; and displaying the structured medical data and the history of the patient's diagnostic results over time in the portal via the portal to enable clinical decision-making based on the history of diagnostic results.

[0026] In some embodiments, the portal includes a data entry interface for receiving input medical data and mapping the input medical data to fields to generate structured medical data, the data entry interface organizing the structured medical data into one or more pages, each of which is associated with a specific primary tumor site. In some embodiments, the method further includes receiving a first instruction via the data entry interface that a first subset of medical data entered into a first page of the data entry interface associated with a first primary tumor site belongs to a second primary tumor site, creating a second page for the second primary tumor site based on the first instruction, and entering the first subset of medical data into the second page.

[0027] In some embodiments, the method further includes receiving, via a data input interface, a second indication that a second subset of the medical data input on the first page is related to metastases of the second primary tumor site; and inputting, based on the second indication, the second subset of the medical data on the second page. In some embodiments, the method further includes importing a document file from an integrated patient database; and extracting input medical data from the document file based on at least one of natural language processing (NLP) operations or rule-based extraction operations on the text included in the document file.

[0028] In some embodiments, the method further includes displaying the document file in a document browser of the portal; and highlighting one or more portions of the document file from which the input medical data was extracted. In some embodiments, the method further includes displaying one or more data fields adjacent to the document browser; and displaying, for a subset of the one or more data fields, an indication that input medical data extracted from the one or more highlighted portions of the document file should be input into the subset of the one or more data fields, to indicate a correspondence between the subset of the one or more data fields and the one or more highlighted portions of the document file.

[0029] In some embodiments, the indication includes highlighting a subset of the one or more data fields and surrounding the highlight marking over the one or more highlighted portions of the document file. In some embodiments, the indication is displayed based on receiving an input from a user via the portal. In some embodiments, the one or more highlighted portions are determined based on detecting an input from a user via the portal. In some embodiments, the one or more highlighted portions are determined based on at least one of natural language processing (NLP) operations or rule-based extraction operations.

[0030] In some embodiments, the method further includes determining one or more medical data categories of the extracted input medical data, determining a mapping between one or more fields in the structured medical data and one or more medical data categories based on a structured data list (SDL), and inputting the extracted input medical data into the one or more fields based on the mapping.

[0031] In some embodiments, the mapping includes mapping the input medical data to standardized values. In some embodiments, the input medical data is received from one or more sources including at least one of an EMR (Electronic Medical Record) system, a PACS (Picture Archiving and Communication System), a digital pathology (DP) system, a LIS (Laboratory Information System), a RIS (Radiation Information System), patient report outcomes, a wearable device, or a social media website.

[0032] These and other embodiments of the invention are described in detail below. For example, other embodiments relate to systems, devices, computer products, and computer-readable media associated with the methods described herein.

[0033] A good understanding of the features and advantages of embodiments of the invention can be obtained by reference to the following detailed description and the accompanying drawings.

Brief Description of the Drawings

[0034] The detailed description is described with reference to the accompanying drawings.

[0035] [Figure 1] An example of a conventional clinical decision-making process to be improved by examples of the present disclosure is shown. [Figure 2] A medical data processing system for facilitating clinical judgment according to a specific aspect of the present disclosure is shown. [Figure 3A] An example of a data input interface of the medical data processing system of FIG. 2 according to a specific aspect of the present disclosure is shown. [Figure 3B] Figure 2 shows an example of a data input interface for the medical data processing system relating to a particular aspect of this disclosure. [Figure 3C] Figure 2 shows an example of a data input interface for the medical data processing system relating to a particular aspect of this disclosure. [Figure 3D] Figure 2 shows an example of a data input interface for the medical data processing system relating to a particular aspect of this disclosure. [Figure 3E] Figure 2 shows an example of a data input interface for the medical data processing system relating to a particular aspect of this disclosure. [Figure 3F] Figure 2 shows an example of a data input interface for the medical data processing system relating to a particular aspect of this disclosure. [Figure 3G-1] Figure 2 shows an example of a data input interface for the medical data processing system relating to a particular aspect of this disclosure. [Figure 3G-2] Figure 2 shows an example of a data input interface for the medical data processing system relating to a particular aspect of this disclosure. [Figure 3H-1] Figure 2 shows an example of a data input interface for the medical data processing system relating to a particular aspect of this disclosure. [Figure 3H-2] Figure 2 shows an example of a data input interface for the medical data processing system relating to a particular aspect of this disclosure. [Figure 4A] Figure 2 shows an example of a data summarization interface for the medical data processing system relating to a particular aspect of this disclosure. [Figure 4B] Figure 2 shows an example of a data summarization interface for the medical data processing system relating to a particular aspect of this disclosure. [Figure 4C] Figure 2 shows an example of a data summarization interface for the medical data processing system relating to a particular aspect of this disclosure. [Figure 5A] Figures 4A to 4C show examples of how the data summarization interface works. [Figure 5B] Figures 4A to 4C show examples of how the data summarization interface works. [Figure 5C] Figures 4A to 4C show examples of how the data summarization interface works. [Figure 5D] Figures 4A to 4C show examples of how the data summarization interface works. [Figure 6A] Figure 2 shows additional examples of data extraction interfaces and operations for the medical data processing system relating to a particular aspect of this disclosure. [Figure 6B] Figure 2 shows additional examples of data extraction interfaces and operations for the medical data processing system relating to a particular aspect of this disclosure. [Figure 6C] Figure 2 shows additional examples of data extraction interfaces and operations for the medical data processing system relating to a particular aspect of this disclosure. [Figure 6D-1] Figure 2 shows additional examples of data extraction interfaces and operations for the medical data processing system relating to a particular aspect of this disclosure. [Figure 6D-2] Figure 2 shows additional examples of data extraction interfaces and operations for the medical data processing system relating to a particular aspect of this disclosure. [Figure 7A] Figure 2 shows an example of the data adjustment interface and operation of the medical data processing system relating to a particular aspect of this disclosure. [Figure 7B] Figure 2 shows an example of the data adjustment interface and operation of the medical data processing system relating to a particular aspect of this disclosure. [Figure 8A] This document provides an example of a portal summary view that improves access to patient health data, relating to a specific aspect of this disclosure. [Figure 8B] This document provides an example of a portal summary view that improves access to patient health data, relating to a specific aspect of this disclosure. [Figure 8C] This document provides an example of a portal summary view that improves access to patient health data, relating to a specific aspect of this disclosure. [Figure 9A] This disclosure provides an example of a portal patient journey view that improves access to patient medical data, relating to a specific aspect of this disclosure. [Figure 9B] This disclosure provides an example of a portal patient journey view that improves access to patient medical data, relating to a specific aspect of this disclosure. [Figure 9C] This disclosure provides an example of a portal patient journey view that improves access to patient medical data, relating to a specific aspect of this disclosure. [Figure 9D] This disclosure provides an example of a portal patient journey view that improves access to patient medical data, relating to a specific aspect of this disclosure. [Figure 9E] This disclosure provides an example of a portal patient journey view that improves access to patient medical data, relating to a specific aspect of this disclosure. [Figure 10-1] This document provides an example of a portal report view that improves access to patient medical data, relating to a specific aspect of this disclosure. [Figure 10-2] This document provides an example of a portal report view that improves access to patient medical data, relating to a specific aspect of this disclosure. [Figure 11] This disclosure provides an example of a portal performance metrics view that improves access to patient health data, relating to a specific aspect of this disclosure. [Figure 12-1] This document provides an example of a data schema for patient data relating to a specific aspect of this disclosure. [Figure 12-2] This document provides an example of a data schema for patient data relating to a specific aspect of this disclosure. [Figure 13-1] This shows another example of a data schema for patient data relating to a particular aspect of this disclosure. [Figure 13-2] This shows another example of a data schema for patient data relating to a particular aspect of this disclosure. [Figure 14A] This document illustrates an exemplary summary workflow for patient data management relating to a particular aspect of this disclosure. [Figure 14B]This document illustrates an exemplary summary workflow for patient data management relating to a particular aspect of this disclosure. [Figure 14C-1] This document illustrates an exemplary summary workflow for patient data management relating to a particular aspect of this disclosure. [Figure 14C-2] This document illustrates an exemplary summary workflow for patient data management relating to a particular aspect of this disclosure. [Figure 14D] This document illustrates an exemplary summary workflow for patient data management relating to a particular aspect of this disclosure. [Figure 15] This disclosure describes a method for managing patient data from different sources in an integrated manner, as described in certain aspects of this disclosure. [Figure 16] This disclosure describes alternative methods for managing patient data to improve access to patient data, relating to certain aspects of this disclosure. [Figure 17] This disclosure describes a method for displaying patient data through a graphical user interface to improve access to patient data, as relating to a particular aspect of this disclosure. [Figure 18] This document describes methods for managing and displaying patient data relating to specific aspects of this disclosure. [Figure 19A] Figure 2 illustrates an example of an oncology workflow enabled by the medical data processing system according to a particular aspect of this disclosure. [Figure 19B] Figure 2 illustrates an example of an oncology workflow enabled by the medical data processing system according to a particular aspect of this disclosure. [Figure 20A] Figure 2 illustrates another example of an oncology workflow enabled by the medical data processing system, relating to a particular aspect of this disclosure. [Figure 20B] Figure 2 illustrates another example of an oncology workflow enabled by the medical data processing system, relating to a particular aspect of this disclosure. [Figure 21] This disclosure describes a method for processing medical data to facilitate clinical decision-making, relating to a specific aspect of this disclosure. [Figure 22] This specification shows exemplary computer systems that may be used to implement the technologies disclosed herein. [Modes for carrying out the invention]

[0036] This document describes technologies to improve clinicians' access to patient data for making clinical decisions, such as clinical judgments related to oncology. The medical data processing system can collect patient medical data from multiple data sources, convert the medical data into structured data, and present the structured data in various formats, such as summary formats and long-term view reports. The medical data processing system can also support oncology workflow solutions that assist / perform diagnostic actions on the collected medical data and present the diagnostic results to clinicians. Oncology workflow solutions can enable clinicians, such as oncologists or their representatives, to manage cancer patients long-term, from suspected cancer through treatment and follow-up. For example, a database and a graphical user interface for accessing the database are provided to update and view patient data in oncology, representing the patient's journey for diagnosis and / or treatment. The graphical user interface can be used, for example, by oncologists to manage patient data and obtain a clear view of cancer progression over time and response to treatment.

[0037] In some examples, a medical data processing system includes a data acquisition module, a data summarization module, an enhancement module, a data access module, and a data refining module. The medical data acquisition module can receive or read patient medical data. Patient data can originate from various data sources (one or more healthcare institutions), including, for example, EMR (Electronic Medical Records) systems, PACS (Picture Archiving and Communication Systems), digital pathology (DP) systems, LIS (Laboratory Information Systems) including genomic data, RIS (Radiology Information Systems), patient report outcomes, wearables and / or digital technologies, social media, etc.

[0038] A database system can ingest data from multiple sources. For example, as mentioned above, data may be ingested from one or more external databases, such as an electronic medical record (EMR) repository or a picture archiving and communication system (PACS). Data can also be manually entered through fields within the user interface. The ingested data can include both structured and unstructured data. Unstructured data may originate from unstructured reports, such as PDF files. In the case of unstructured reports, machine learning (e.g., optical character recognition (OCR) and / or natural language processing (NLP)) is used to identify and enter fields. A database system that ingests data from multiple sources and stores the data in a new schema can be called an integrated patient database.

[0039] Within an integrated patient database, data can be stored in a graph structure, where data elements are linked to connect different cancers or other conditions in patients through different treatments, observations, etc. The graph structure can also be used to link different cancers (e.g., primary and metastatic) to each other.

[0040] Data can be ingested and enhanced through a user interface. In particular, an interface for data summarization is provided. In the data summarization process, information can be extracted from reports and used to generate structured medical data by populating fields in an interface that the user can review or edit. In the data enhancement process, enhancement actions are performed to improve the quality of the extracted medical data. Examples of enhancement actions include normalizing various numerical values ​​(e.g., weight, tumor size, etc.), replacing non-standard terms provided by patients with standardized terms, and filling in missing fields that characterize or supplement the data, which may include displaying pull-down menus containing categories, data standardization formats, etc. Fields are populated or updated automatically and / or via user input. For example, a user can interact with interface elements to classify a tumor as primary cancer (also called primary tumor) or metastasis, or to fill in other fields such as date, time, and physician's notes.

[0041] Another interface view may be used in the adjustment process. The adjustment interface view may be triggered when data has been uploaded to the database, but information such as association with primary cancer, stage, or type of surgery is missing from the record. For example, in the adjustment process, a tumor may be associated with one or more primary cancers, which may trigger the tumor data record stored in the integrated patient database by the updated mapping.

[0042] The patient journey can be viewed at any point in the data acquisition, summarization, and adjustment process. The patient journey is a timeline showing various multimodal elements of the patient's oncological journey and medical history in chronological order. This facilitates the visualization of the patient's cancer milestones and progression (e.g., as it metastasizes, recurs, or recurs). The patient journey includes a set of objects within the timeline. These objects can correspond to categories such as lesions, diagnoses, and treatments. Each category may have a row within the timeline, and the objects within that category are displayed chronologically. Each object can be user-selectable. Upon detecting user interaction with an object, the system may read and display supplementary information, reports, etc., via a graphical user interface.

[0043] Furthermore, the technology can improve clinicians' access to patient data, enabling them to make clinical decisions, such as those related to oncology. In some cases, a medical data processing system can collect patient medical data from multiple data sources, transform the medical data into structured data, and present the structured data in various formats, such as summary formats and long-term view reports. A medical data processing system can support oncology workflows, enabling clinicians to make various diagnoses at different stages of the workflow. A medical data processing system can facilitate the input of diagnostic results by clinicians at different stages of the workflow and perform post-processing of the data, both of which enable clinicians to manage cancer patients long-term, from suspected cancer through treatment and follow-up. A medical data processing system can also support other medical applications, such as quality of care assessment tools to evaluate the quality of care provided to patients, and medical research tools to determine correlations between various patient information (e.g., demographic information) and patient tumor information (e.g., prognosis or expected survival). This technology is not limited to oncology and can be applied to other types of disease areas.

[0044] In some cases, the medical data collection module also provides a portal that enables the input and display of structured medical data into the system. Structured medical data can include various pieces of information related to the diagnosis of a tumor, such as tumor site, stage classification, pathology information (e.g., biopsy results), diagnostic procedures, and biomarkers for both the primary tumor and additional tumor sites (e.g., resulting from metastases from the primary tumor). The portal can display the structured data in the form of a patient summary. The portal can also organize the display of structured data into pages, each associated with a specific primary tumor site and containing fields of information about the associated primary tumor site, which may be accessed by tabs. A data input interface can allow users to manually input medical data. Based on the detection of user input in specific fields within the page for the first primary tumor (e.g., designation of an additional tumor site as a new primary tumor), the medical data collection module can create an additional page for the second primary tumor and input fields on the newly created page for the second primary tumor based on the additional tumor site information entered on the page for the first primary tumor. In some cases, the medical data collection module also allows the user to select additional tumor masses found during the primary tumor diagnostic procedure and associate those masses with the second primary tumor to represent metastatic cases. Based on the association detection, the medical data collection module can transfer all diagnostic results for the additional tumors from the first primary tumor page to a newly created page for the second primary tumor.

[0045] Furthermore, the portal also allows users to import document files (e.g., pathology reports, physician's notes, etc.) from the aforementioned data sources. The medical data summarization module can then perform data summarization operations, which extract various medical data from the document files and use it to populate fields in the patient summary to generate structured medical data. In some examples, medical data may be extracted based on performing natural language processing (NLP) operations, rule-based extraction operations, etc., on the text contained in the document files. In some examples, medical data may also be extracted from metadata of the document files, such as the file date, the category of the document file (e.g., pathology report to physician's notes), the clinician who authored / signed off the document file, and the type of procedure associated with the content of the document file (e.g., biopsy, imaging, or other diagnostic step). The extracted medical data can then be used to automatically populate various fields in the patient summary. The medical data summarization module can also highlight the parts of the document files from which structured medical data is extracted, as well as the fields populated by the structured medical data, to allow users to track / verify the results of the data summarization operations. In some examples, the medical data summarization module can also assist with the manual extraction of structured medical data from document files via the portal.

[0046] Furthermore, the augmentation module can perform various augmentation actions to improve the quality of the extracted medical data. One augmentation action may include normalization to correct data errors, or to replace non-standard terms provided by patients with standardized terms based on various medical standards / protocols such as the International Classification of Diseases (ICD) and Systematic Medical Nomenclature (SNOMED), in order to normalize various numerical values ​​(e.g., weight, tumor size, etc.) contained in the extracted medical data to standardized units, or to replace non-standard terms provided by patients with standardized terms based on various medical standards / protocols such as the International Classification of Diseases (ICD) and Systematic Medical Nomenclature (SNOMED). The augmented extracted medical data can then be stored in the integrated patient database as part of the patient's structured medical data (e.g., structured oncology data). In addition, if the portal receives medical data manually entered by a user, the augmentation module can also control the portal to display a pull-down menu containing alternatives of standardized data (e.g., SNOMED terms) that can be selected by the user as input, in order to ensure that the user enters standardized medical data into the medical data processing system.

[0047] The medical data summarization and enhancement modules can be sequentially adapted to improve the extraction and normalization processes. For example, some of the original unstructured patient data from a data source may be manually tagged to indicate a mapping of specific data elements as ground truth. For instance, a series of texts within a physician's notes may be tagged as ground truth indications for adverse effects of a procedure. The tagged physician's notes could then be used, for example, to train the NLP in the data summarization module so that the NLP can extract text indicating adverse effects from other untagged physician's notes. The NLP can also be trained with other training datasets, including, for example, common data models, data dictionaries, and hierarchical data (i.e., dependencies between texts), to extract data elements based on a semantic and contextual understanding of the extracted data. For example, a natural language processor could be trained to select the most semantic candidate from a standardized set of data candidates for data elements in a cancer registry as extracted data. Furthermore, some of the extracted data, such as numerical data, may be updated or validated for consistency with one or more data normalization rules as part of the processing.

[0048] Furthermore, the oncology workflow module can perform / support diagnostic actions based on structured medical data provided by the medical data collection module. For example, a diagnostic action can be performed to verify whether biopsy results are for the same primary tumor or different tumors, and to track the size of the primary tumor to assess the tumor's response to specific treatments. In another example, a diagnostic action can be performed to determine whether a patient has a single primary tumor site, multiple primary tumor sites, or an unknown primary site. The results of the diagnostic action can then be recorded and / or displayed in the portal in terms of time as part of the patient's medical journey, enabling oncologists or their representatives to manage cancer patients from suspected cancer to long-term cancer management through treatment and follow-up. Diagnostic results can also be used to support other medical applications, such as quality of care assessment tools to evaluate the quality of care provided to patients, and medical research tools to determine correlations between various patient information (e.g., demographic information) and patient tumor information (e.g., prognosis or expected survival).

[0049] The disclosed technology enables the aggregation and extraction of medical data to generate patient summaries and display the data on a portal. By providing all relevant medical data to the portal and organizing the data according to tumor site, access to medical data by clinicians can be significantly improved, facilitating clinician decision-making and management of patient care. Furthermore, as part of the oncology workflow, automated diagnostic actions can be performed that mimic parts of a clinician's diagnosis, which can reduce the workload on clinicians. In addition, displaying diagnostic results rather than raw medical data as part of the patient journey on the portal allows clinicians to better visualize the patient's medical status. This enables oncologists or their representatives to manage cancer patients from suspected to long-term through treatment and follow-up. All of these aspects can improve the quality of care provided to patients. I. Clinical decision making

[0050] Figure 1 is Chart 100, which illustrates a conventional clinician decision-making process. As shown in Figure 1, a clinician 102 can obtain patient medical data 104, which may include structured medical data 106 and unstructured medical data 108, in order to generate a clinical decision 110. Structured medical data 106 may include data in various categories, such as patient demographic information (age, sex, etc.), diagnostic results described in relation to various standardized codes such as the International Classification of Diseases (ICD), Diagnostic Related Groups (DRG), Current Treatment Terms (CPT), and SNOMED codes, medication history (e.g., autopsy therapeutic chemicals (ATC)), and clinical chemistry and immunochemistry results. Furthermore, unstructured medical data 108 may include data in various categories, such as various medical reports, such as pathology reports, radiology reports, sequencing laboratory reports, surgical reports, admission reports, discharge reports, and physician's notes. The clinical decision 110 may include, for example, medication, physiotherapy (e.g., radiation), and surgery administered to the patient. Medical data 104 is typically stored in different data sources such as EMR (Electronic Medical Records) systems, PACS (Picture Archiving and Communication Systems), digital pathology (DP) systems, and LIS (Laboratory Information Systems).

[0051] A clinician 102 may need access to every category of data listed in medical data 104 to make decisions. For example, a clinician 102 may need access to pathology and surgical reports to obtain information about a tumor. A clinician 102 may also need access to radiology reports to determine whether the tumor is localized or if cancel cells have spread, and to sequencing laboratory reports to obtain biomarker information. A clinician 102 may also need access to physician's notes to obtain information about the patient's treatment history by another clinician, for example. All of this data is important in deciding how to treat the patient. For example, based on radiology reports, a clinician may determine that the tumor is localized and administer a specific physiotherapy (e.g., radiotherapy) to target the localized tumor. Furthermore, based on the presence of specific biomarkers, a specific drug may be administered to target the site.

[0052] While clinicians have access to a large and diverse set of medical data to make clinical decisions, sourcing medical data from different data sources can be extremely cumbersome. The lack of structured and standardized medical data also makes sourcing difficult. For example, clinicians may need to read and interpret numerous medical reports to obtain the information they are looking for. Clinicians may also need to consider the physicians' habits in writing the reports in order to properly interpret them. All of this is not only cumbersome but also prone to errors, impacting clinicians' ability to determine and administer high-quality care to patients. II. Medical Data Processing System

[0053] Figure 2 shows an example of a medical data processing system 200 that can address at least some of the above problems. The medical data processing system 200 can collect patient medical data 242 and convert the medical data 242 into structured patient data 202. The medical data processing system 200 can also store the structured patient data 202 in an integrated patient database 204. The integrated patient database 204 can store data read from various sources in an integrated manner. The data may originate from one or more patient data sources 240. The patient data sources 240 may include one or more external databases or other sources, such as an electronic medical record (EMR) repository, a picture archiving and communication system (PACS), a digital pathology (DP) system, a laboratory information system (LIS) including genomic data, a radiology information system (RIS), patient report outcomes, wearables and / or digital technologies, and social media. The data stored in the integrated patient database 204 may include unstructured data such as images of PDFs or scanned documents, as well as information directly entered into the medical data processing system 200 via a portal 220. The integrated patient database 204 can store multiple records, each corresponding to a specific patient. Each patient record may contain a network of interconnected data objects. The data schema for use in the integrated patient database 204 is described in more detail below with reference to Figures 12 and 13.

[0054] When medical data is focused on oncology, structured patient data 202 can include various data categories such as patient history information 212, tumor diagnostic information 214, treatment history 216, and biomarkers 218. Tumor diagnostic information 214 can further include various data subcategories or data types within specific data categories such as tumor site 214a, staging classification 214b, pathology information 214c (e.g., biopsy results), and diagnostic procedure 214d. The medical data processing system 200 further includes a portal 220, which can present structured data in various formats such as summary format and long-term view report format, as shown in Figures 3A to 11. In some implementations, the portal 220 is displayed on a display component of a computing device separate from the medical data processing system 200. For example, a diagnostic computer (not shown) displays the portal 220 and receives user input such as medical data 242.

[0055] Furthermore, the medical data processing system 200 can support the oncology workflow application 222. The oncology workflow application 222 can determine the data to be collected by the medical data processing system 200 to support the oncology workflow. Furthermore, as described later, the oncology workflow application 222 can perform (or assist in) analysis on the collected medical data and generate analysis results 224. The analysis may include determining the patient's tumor status, for example, whether the patient has a single tumor or multiple tumors, or whether the patient has metastases, based on structured patient data 202. The analysis results may be updated whenever new data (e.g., new diagnosis, new biopsy results, etc.) is added about the patient. In some implementations, the oncology workflow application 222 runs on a diagnostic computer.

[0056] The analysis results presented in Portal 220 can enable clinicians, such as oncologists or their representatives, to manage cancer patients from suspected to long-term through treatment and follow-up. The results of the diagnostic actions can then be recorded and / or displayed in the portal in terms of time as part of the patient's medical journey. Portal 220 can enable oncologists or their representatives to manage cancer patients from suspected to long-term through treatment and follow-up. The analysis results can also be used to support other medical applications, such as quality of care assessment tools to evaluate the quality of care provided to patients, and medical research tools to determine correlations between various patient information (e.g., demographic information) and patient tumor information (e.g., prognosis or expected survival). The medical data processing system 200 can store structured patient data 202, as well as analysis results 224, in an integrated patient database 204, from which the structured data and analysis results can be accessed by other medical applications.

[0057] As shown, the medical data processing system 200 includes a portal 220, a data acquisition module 230, a data summarization module 232, an enhancement module 234, and a data access module 236. The data acquisition module 230 can receive medical data 242 from a user via the data input interface of the portal 220, and the user can input the data into various fields, and structured patient data 202 can be created through mapping between the fields and the input data.

[0058] Furthermore, the data collection module 230 can also receive medical data 242 directly from the portal 220, which can provide a document summarization interface that allows the user to import document files 244 (e.g., pathology reports, physician's notes, etc.) from the patient data source 240. From the document files 244, the data summarization module 232 can perform a summarization operation, which extracts medical data from the document files and maps the extracted data to various data categories. The mapping can be based on a master structured data list (SDL) 246 that defines a list of data categories for the document types of the document files 244 to support the oncology workflow application 222. The patient data source 240 (in one or more healthcare institutions) can include, for example, an EMR (Electronic Medical Record) system, a PACS (Picture Archiving and Communication System), a digital pathology (DP) system, a LIS (Laboratory Information System) including genomic data, a RIS (Radiology Information System), patient report results, wearables and / or digital technologies, social media, etc. After the summarization operation, the user can edit and / or review the data extracted from the documents.

[0059] Furthermore, the augmentation module 234 can perform various augmentation operations to improve the quality of the extracted medical data, such as performing normalization operations. Normalization operations may be performed, for example, to normalize various numerical values ​​(e.g., weight, tumor size, etc.) contained in the extracted medical data to standardized units, to correct data errors, or to replace non-standard terms provided by patients with standardized terms based on various medical standards / protocols such as the International Classification of Diseases (ICD) and Systematic Medical Nomenclature (SNOMED). As described below, the augmentation module 234 can perform normalization operations on data received from the data acquisition module 230 and / or the data summarization module 232. The augmented extracted medical data can then be stored in the integrated patient database 204 as part of the patient's structured patient data 202 (e.g., structured oncology data). The augmentation module 234 can also work with the portal 220 to provide interface elements, such as pull-down menus containing alternatives to standardized data that can be selected by the user as input, to ensure that the user inputs standardized medical data into the medical data processing system.

[0060] The data access module 236 provides temporary storage for data received from the data acquisition module 230 and the data summarization module 232, and can update the data in the temporary storage based on edits made to the data by the user via the portal 220. After receiving confirmation from the user via the portal 220 that the data has been finalized and can be released back to the integrated patient database 204, the data access module 236 can release the data to the integrated patient database 204 as structured patient data 202. Furthermore, the data access module 236 can provide access to the data in the temporary storage to various applications, such as the oncology workflow application 222. This allows the data acquisition module 230 and the data summarization module 232, which support the workflow application, to provide users with information to track and manage data entry and data summarization operations.

[0061] The data adjustment module 238 can identify data elements in the integrated patient database 204 that are missing information necessary for properly storing and displaying patient data. For example, if a data record for a particular cancerous tumor is not associated with a primary cancer site, this tumor may be flagged for adjustment. The data adjustment module 238 can provide a UI element that prompts the user to input the necessary information (for example, to associate the cancerous tumor with a primary cancer, e.g., as a new primary cancer or as a metastasis of another primary cancer). The data adjustment module 238 can read the user input and modify the data record for the cancerous tumor to associate the tumor with the primary cancer identified via the user input to the UI. III. Exemplary Interfaces

[0062] Figures 3A to 11 illustrate various interfaces that can be used to display patient data and facilitate the ingestion and organization of patient data for clinical decision-making. The data entry interfaces in Figures 3A to 7B can be used to import and organize data stored in an integrated patient database. The view interfaces in Figures 8A to 11 can be used to read and display data from the integrated patient database for use in clinical decision-making. A. Data Entry Interface

[0063] Figures 3A, 3B, 3C, 3D, 3E, 3F, 3G, and 3H show an example of portal 220. The example provides an interface for managing the medical data of an exemplary patient. 1. Summary page

[0064] As shown in Figure 3A, the portal 220 can provide a data entry interface 300 for entering data to support the oncology workflow application 222. The data entry interface 300 can guide the user to manually enter data and / or approve or edit automatically extracted data. Data received through the data entry interface 300 of the portal 220 can be stored in the integrated patient database 204 in an appropriate manner based on the fields 308 of the data entry interface 300, using a data schema as described below with respect to Figures 12 and 13. The data from the integrated patient database 204 can then be read to display further interface views, such as a patient journey view showing a long-term view report of patient data over time, as shown in Figures 9A-9E.

[0065] The data entry interface 300 includes various fields for various information related to the diagnosis of a tumor, such as a field 302 for tumor site, a field 304 for staging, a field 306 for pathological information (e.g., biopsy results), a field 308 for diagnostic procedures, and a field 310 for biomarkers. Fields 302-310 can form a patient summary page 311 for a specific tumor site. In addition to the patient summary page 311, the data entry interface 300 may include fields for other information, such as a patient report 312, oncological treatment information 314 for a series of oncological procedures the patient has received, current medication information 316 for current medications the patient has received, and patient history information 318 for various patient histories (e.g., medical history, surgical history, family history, social history, and substance use history). The data entry interface 300 provides an interface for aggregating different modalities of patient data and then converting the data into structured patient data 202. The fields and various options provided to the data entry interface 300 may be defined based on the oncology workflow application 222.

[0066] Each of the patient summary page 311, patient report 312, oncological treatment information 314, current medication information 316, and patient history information 318 further includes a publish button. For example, the patient summary page 311 includes a publish button 319. As described above, once the data entry interface 300 receives data entered into various fields, the data access module 236 can store the data in temporary storage and hold the data from the integrated patient database 204. Activating the publish button 319 prompts the data access module 236 to send the data as structured patient data 202 to the integrated patient database 204.

[0067] The data entry interface 300 can provide various methods for entering data for most fields, including manual data entry and summaries from document files. For example, in the field of oncological treatment information 314, links 315a and 315b may be provided. Activating link 315a brings up the display of a data summary portal (as described below with reference to Figures 4A to 6D, for example), allowing the extraction of oncological treatment information data from document fields. Activating link 315b brings up the display of text boxes and / or pull-down menus, allowing the user to manually enter oncological treatment information data, as described with reference to Figures 3B to 3F. 2. How the summary page works

[0068] Figures 3B to 3F illustrate examples of how the patient summary page 311 behaves when receiving data manually entered by the user. Referring to behavior 320 in Figure 3B, the primary tumor region 302 may receive the input text “right upper lobe of the lung” (e.g., location), but the diagnosis has not yet been confirmed and is still pending, and the “diagnosis pending” flag 321 is asserted. The title of the patient summary page 311 remains “Unnamed Primary.” Furthermore, the diagnostic procedure field 308 may receive input text indicating that positron emission tomography-computed tomography (PET-CT) was performed as part of the diagnostic procedure and that a mass consistent with a lung neoplasm and liver metastasis was found. The input text further indicates the size of the masses found in the lung and liver. In behavior 322, the “diagnosis pending” flag 321 is de-asserted in the primary tumor region 302, confirming that the mass in the right upper lobe of the lung is the primary tumor. Additionally, the lesion field 306 is populated with further information. Such specifications can be imported by the medical data processing system 200 and stored in the integrated patient database 204 according to structured fields established via the interface.

[0069] Referring to operation 324 in Figure 3C, after detecting that the “Diagnosis Pending” flag has been asserted, the data entry interface 300 can change the title of the patient summary page 311 from “Unnamed Primary Tumor” to “Right Right Lobe of the Lung” to reflect that the information in fields 302-310 belongs to the tumor in the right right lobe of the lung. Furthermore, referring to operation 326 in Figure 3C, upon detecting that the add icon 325 has been activated, the data entry interface 300 can display an additional set of fields for the user to enter information about a new diagnostic procedure. This information may include, for example, the date of the new diagnostic procedure, the name of the procedure, and the findings. Additionally, a pull-down menu 332 is provided to select the location of the tumor mass found in the new diagnostic procedure in field 334. The candidates listed in the pull-down menu 332 may be provided as standardized terms by the enhancement module 234 so that only standardized terms are entered into field 334. As shown in Figure 3C, in operation 326, an additional tumor mass (ascending colon mass) is added as a result of the new diagnostic procedure.

[0070] Figures 3D, 3E, and 3F illustrate examples of the operation for creating a new page for a second primary tumor after data has been entered into page 311 (for a primary tumor in the right upper lobe of the lung). Referring to Figure 3D, in operation 340, when the data entry interface 300 detects that an additional tumor mass listed in the new diagnostic procedure has been selected, it can provide a pull-down menu 342. The pull-down menu 342 includes an option 344 that allows the user to specify the newly added tumor mass (ascending colon mass) as the new primary tumor. Referring to Figure 3E, in operation 350, when the selection to specify the newly added tumor site in the colon as the new primary tumor is detected, the data entry interface 300 can create a new page 352 for the ascending colon primary tumor, in addition to page 311 for the primary tumor in the right upper lobe of the lung. The enhancement module 234 can also add the standardized term "adenocarcinoma" to the primary tumor site information on page 352 as a supplement to the ascending colon. Furthermore, fields 302-310 on page 352 are populated with information from page 311, including new diagnostic procedures added again in operation 326 in Figure 3C. As a result of operation 340, the data acquisition module 230 can create a first data structure for the primary tumor site in the right upper lobe of the lung and a second data structure for the primary tumor site in the ascending colon as part of the patient's structured patient data 202, each data structure containing tumor diagnostic information, treatment history, and a set of biomarkers.

[0071] After creating page 352 for the second primary tumor site (ascending colon), a specific diagnostic result on page 311 (primary tumor in the right upper lobe of the lung) can be linked to the second primary tumor site. For example, referring to Figure 3F, the diagnostic result on page 311 includes information 360 about an additional tumor mass in the right upper lobe of the lung. In operation 362, the data input interface 300 can detect the selection of information 360 and output a menu 364 that includes an option 366 to associate the additional tumor mass with the ascending colon as the second primary tumor site. Upon detecting the selection of option 366, the data acquisition module 230 can move information 360 to page 352 for the second primary tumor site to indicate that the additional tumor mass in the right upper lobe of the lung is a result of metastasis from the second primary tumor site in the ascending colon. 3. Addition of various categories of medical data

[0072] Figure 3G shows the patient summary view 370 of portal 220. The patient summary view 370 is a graphical user interface view for displaying and modifying patient data. The patient summary view 370 includes an add button 372. In response to detection of user interaction with the add button 372, a data add modal 374 is displayed. The add data modal 374 can be a web page element that is displayed before other page content. While the add data modal 374 is displayed, it may deactivate the page content outside of the add data modal 374. The add data modal 374 includes a list of data types and data categories in which data can be entered and stored. The data types and data categories shown in Figure 3G include allergens, biomarkers, environmental risks, family history, current illness history, medical history, medications, metastatic sites, oncological treatments, radiation, surgery, systemic antineoplasms 375, oncological summary, performance status, primary cancer, social history, staging, substance use history, and surgical history. A data category may contain data types within that data category. For example, Radiation, Surgery, and Systemic Antineoplasm 374 are data types within the data category of oncological procedures in this example. The data types and data categories shown in the additional data modal 374 may correspond to data objects stored in a map structure within the integrated patient database, where the data types and data categories organize and label the corresponding data elements. For example, a data object may include a patient root data object 1201 mapped to related data objects, including a tumor mass data object 1202, a diagnostic findings data object 1205, a procedure data object 1208, and a history data object 1210, as shown in Figure 12. This data schema facilitates the display of the patient summary view, and information entered via the patient summary view can be used to modify data in the integrated patient database, as further described in Section IV below.

[0073] Each of these data types and data categories can correspond to a different set of configured data fields. In response to user interaction with either a displayed data type or data category, the portal 220 can transition to a data input view 380 containing the data fields corresponding to the selected data type, as shown in Figure 3H. As shown in Figure 3G, the cursor 376 indicates user interaction with the displayed data type systemic antineoplasm 375. When hovered over, systemic antineoplasm 375 is highlighted. Clicking on systemic antineoplasm 375 transitions the interface to a data input view 380 containing the data fields corresponding to systemic antineoplasm 375.

[0074] Figure 3H shows a data entry view 380 of the portal 220 according to several embodiments. The data entry view 380 may be used to receive patient medical data via the portal 220. The data is stored in an integrated patient database within patient records, which can be organized into a data graph that maps data elements to each other based on configured data types (for example, once entered into the interface), as shown in Figure 12. Menu 382 includes a set of fields that can accept user input to manually provide information corresponding to each field. These fields may include both drop-down menus from which the type of treatment, primary cancer, status, or outcome can be selected, and fields configured to accept entered user input such as the number of cycles, start date, end date, responsible person, and additional notes. In response to detection of user interaction with the save button 384, the system saves the data entered into the fields. For example, data elements entered into each field may be stored in an integrated patient database 204 organized based on the data type corresponding to that field. B. Interface for managing data ingestion from unstructured reports

[0075] Figures 4A to 6D show examples of interfaces for managing data from unstructured reports. Figures 4A to 4C show examples of document summarization interfaces for importing information from report files. Figures 5A to 5D show examples of how to extract data from a report using the summarization interface. Figures 6A to 6D show different examples of interfaces for extracting fields from a report. 1. Extracting data from report files

[0076] In addition to manual data entry, the portal 220 also allows users to import document files 244 (e.g., pathology reports, physician's notes, etc.) from patient data sources 240, and the data summarization module 232 can extract various structured medical data from the document files. Figures 4A, 4B, and 4C show examples of document summarization interfaces 400 that can be part of the portal 220.

[0077] Figure 4A shows a document summarization interface 400 that may be used to guide a user to review or update data extracted from a document. As shown in Figure 4A, the document summarization interface 400 includes a document directory 402, a document browser 404, and an extracted medical data section 406. The document directory 402 may display a list of selectable icons, including an icon 407 representing the document to be selected (or selected) for performing medical data extraction and summarization operations. Furthermore, the document browser 404 may display the selected document. As will be discussed later, the document summarization interface 400 may highlight the portion of the document from which medical data was extracted from the document browser 404, which allows the user to track the source of the extracted medical data. The extracted medical data section 406 may include a report page 408 and a results page 410. The report page 408 may include a list of metadata extracted from the selected document, including, for example, the document name 408a, the report date 408b, and the document type 408c. The results page 410 contains a set of fields corresponding to a set of categories of data extracted from the selected documents or entered by the user. In some examples, the results page 410 may be part of the patient summaries shown in Figures 3A–3H.

[0078] As described above, the set of fields included in the results page 410 may be defined based on the master structured data list (SDL) 246, and the data summarization module 232 may be selected based on the document type 408c. Figures 4B and 4C show examples of data categories extracted for different document categories. Figure 4B shows an exemplary results page 411 of a pathology report that provides information on the diagnosis of cancer. As shown in Figure 4B, various categories of data may be extracted from a pathology report, including diagnostic information 412, staging information 414, and additional notes 416. Furthermore, the diagnostic information 412 may include various fields such as tumor site information 412a, histological type 412b, histological malignancy grade 412c, and biomarker information 412d, while the staging information 414 may include various fields describing the stage of the tumor. Furthermore, Figure 4C shows an exemplary results page 420 of a cytology report that provides information on the examination of cells from a patient's body. As shown in Figure 4C, various categories of data can be extracted from cytology reports, such as tumor site information 420a and biomarker information 420b. The data categories shown in Figure 4B may be defined based on the SDL246 selected by the data summarization module 232 based on the document type 408c of the selected document indicating that the document is a pathology report. 2. Extraction of results

[0079] Figures 5A, 5B, 5C, and 5D illustrate the exemplary operation of the document summarization interface 400 for pathology reports. The document summarization interface 400 can be used to guide the user to identify the data types of data to be integrated into the integrated patient database, such as in fields automatically populated using machine learning. Referring to Figure 5A, the data summarization module 232 can parse text strings from a selected document (e.g., obtained from optical character recognition (OCR) processing of the document) and detect text strings containing the data to be extracted, including metadata, data, and various categories of medical data. The data summarization module 232 can then populate the extracted data into corresponding fields in the report page 408 and the results page 410. The data summarization module 232 can also display highlight markings such as highlight markings 502, 504, 506, 508, 510, and 512 in the document browser 404. Highlight marking 502 can correspond to text indicating document type 408c (e.g., pathology report), while highlight marking 504 can correspond to text indicating the date of the pathology report, both of which can be extracted from the pathology report metadata. Fields 520 (report date) and 522 (document type) on the results page 420 are populated with the report date and the extracted document type 408c, respectively.

[0080] Furthermore, highlight marking 506 can correspond to text describing the procedure involved (e.g., right breast tumor excision), highlight marking 508 can correspond to text describing clinical data (e.g., a 2.5 cm right breast tumor was detected by diagnostic mammography and fine-needle aspiration (FNA) of the right breast tumor was performed), highlight marking 510 can correspond to text describing the right breast volume (e.g., a single fragment of soft tissue received in formalin), while highlight marking 512 can correspond to microscopic examination details of the right breast volume (e.g., tumor size of 1.9 × 1.6 × 1.4 cm). Next, the fields 524 (e.g., procedure label), 526 (e.g., clinical data label), and 528 (tumor size label) of the results page 420 are populated with the text highlighted by highlight markings 506, 508, and 510, respectively. Additional display effects may also be provided to indicate links between the fields and the highlighted portions of the document. For example, in Figure 5A, based on the user's selection of field 524, the highlight marking 506 may be enclosed by a line boundary, but the line of field 524 is also highlighted to show the correspondence between field 524 and the data covered by the highlight marking 506. After the user confirms the entered data and activates the publish button 529, the data access module 236 can release the data to the integrated patient database 204.

[0081] The data summarization module 232 can detect text containing medical data and extract the medical data from the text based on various techniques. For example, detection can be based on natural language processing (NLP) operations on text contained in document files, rule-based extraction operations, etc. As another example, the data summarization module 232 can detect selection and drag operations on a document via the document browser 404, and detection can be based on text selected by the user. After detecting text strings containing medical data, the data summarization module 232 can determine the data categories of the medical data and their associated data values, while the enhancement module 234 can convert the data values ​​to standardized and / or normalized values, or provide the option to include normalized / standardized values ​​selected by the user. NLP and rules can be obtained from training operations based on other medical documents that include data category tags as ground truth. For example, the document used for training may include a series of texts tagged as a procedure, such as "mammary gland, right, tumor excision," which allows the data summarization module 232 to determine that those texts also refer to a procedure in the document shown in Figure 5A. As another example, a document used for training might include a sequence of text "Total tumor size" followed by another sequence of text indicating the size of the tumor. This allows the data summarization module 232 to determine that the sequence of text "1.9 × 1.6 × 1.4 cm" with the highlight marking 512 represents the size of the tumor. The enhancement module 234 can then convert the data values ​​to standardized and / or normalized values ​​as needed. For example, if the sequence of text under the highlight marking 512 is "1.9 × 1.6 × 1.4 m", the enhancement module 234 can determine that the unit (meters) is not a standard unit and may replace the unit with another unit established as a standard unit (e.g., centimeters (cm), millimeters (mm), etc.).

[0082] Next, the data collection module 230 can populate the fields in the results page 410 with the extracted and / or normalized values ​​of the corresponding data categories. In some examples, field input can be automated based on the mapping between data categories and fields defined in SDL246. In some examples, field input can be based on user selection.

[0083] Figure 5B shows an exemplary sequence of actions on the document summarization interface 400 for selecting text from the document browser 404. Referring to Figure 5B, action 530 begins with displaying the document in the document browser 404. In action 532, the document browser 404 can receive a select and drag action to select a portion of the document to perform a data summarization action, and a highlight 533 is displayed to indicate the extent of the select and drag action and the portion of the document selected at a given point. In action 534, the document browser 404 can receive a click action from the user to indicate that the select and drag action is complete and the selected portion of the document has been confirmed. The document browser 404 can then display a border 535 around the highlight 533 to indicate that the selected text will be processed by the data summarization module 232 to extract medical data. In action 536, field 526 on the results page 410 can receive a click action from the user to indicate that the highlight 533 is mapped to field 526 and that the medical data extracted from the portion of the document below the highlight 533 will be entered into field 526. The document summarization interface 400 can also display a line 537 on field 533 to indicate that field 526 has been selected to map to the highlight. In operation 538, after input is complete, the document browser 404 can remove the boundary 535 from the highlight 533 and the line 537 from field 526. Displaying the boundary 535 and line 537 allows the user to easily visualize which highlighted portion of the results page 410 is mapped to which field when the user has made a mapping decision, which can help the user track mapping decisions and reduce mapping errors, especially when multiple portions of a document are mapped to multiple fields, as shown in Figure 5A.

[0084] Figure 5C shows an example of operation on the document summarization interface 400 after the text of a highlighted portion of a document has been mapped to a field on the results page 410, in order to help the user track the source of the data in a field. As shown in Figure 5C, in operation 540, the document summarization interface 400 detects a click operation on field 526. Upon detecting a click operation, the document summarization interface 400 can display a line 537 on field 526. Furthermore, the document browser 404 can also automatically scroll to the highlight 533 and display a border 535 around the highlight 533 to indicate that the text in field 526 originates from the highlight 533. Furthermore, in operation 542, the document summarization interface 400 detects a click operation on the highlight 533. Upon detecting a click operation, the document summarization interface 400 can display a border 535 around the highlight 533. Furthermore, the extracted medical data section 406 can also automatically scroll the results page 410 to field 526 to indicate that the text in field 526 originates from the highlight 533.

[0085] In some examples, the data summarization module 232 can automatically detect text that may contain medical data and extract the medical data from the text, as further described below with respect to Figure 15. The enhancement module 234 can determine one or more candidate data values ​​for the extracted medical data in a particular field based on the SDL 246. The document summarization interface 400 can then provide the candidate data values ​​as options that can be selected by the user for the field.

[0086] Figure 5D illustrates a series of operations on the document summarization interface 400, including automatic text detection. The document summarization interface 400 can guide the user to provide or confirm information for use when entering structured data into the integrated patient database. As shown in Figure 5D, in operation 550, the data summarization module 232 detects the text "cm" (centimeters) and causes the document browser 404 to display a highlight 552 and a border 554 above the text "cm" to indicate that the data summarization module 232 has processed the text. As a result of the processing, field 556 on the results page 410 may display a dropdown menu 558 containing two candidate values, "cm" and "mm" (millimeters), to be selected by the user. The document summarization interface 400 may display a line 560 on field 556 to indicate that the field is mapped to text below the highlight 552. In operation 570, the document summarization interface 400 may receive the selection of the candidate value "cm" for entering into field 556.

[0087] Furthermore, referring back to Figure 2, the medical data processing system 200 can support the oncology workflow application 222. The oncology workflow application 222 can determine the data to be collected by the medical data processing system 200 to support the oncology workflow, and then determine the fields to be displayed on the results page 410 and the categories of data to be received. Furthermore, as will be described later, the oncology workflow application 222 can perform analysis on the collected medical data and generate analysis results 224. 3. Extracting data from the report

[0088] Figures 6A to 6D illustrate examples of adding interface views for extracting and importing data from unstructured reports, according to several embodiments. As shown in Figures 6A and 6B, different types of reports are associated with different fields, which can be automatically populated by the system using machine learning and can be populated by the user via parallel views that show both the fields and the reports, or a combination of the two.

[0089] These reports may originate from external systems such as EMR. Some of the information used to ultimately generate the patient journey interfaces in Figures 9A–9E and the patient summary interfaces in Figures 8A–8B may originate from EMR in structured form. In other cases, the information is embedded in the reports. Information embedded in these reports may not be available for visualization or analysis because it is not in structured fields. Using the interfaces in Figures 6A–6D, the user is shown a list of data fields, allowing the user to input information into this structured dataset. The user can manually input some of the information while viewing the report, the information may be used automatically to input fields when ingested in structured form, and / or machine learning may be used to scan the document and match the information with the corresponding fields.

[0090] All information from these different sources can be aggregated in a single location, namely an integrated patient database. Data can originate from external sources such as EMR, and data can be manually entered and / or machine learning such as NLP may be used to suggest values ​​that can be presented to the user for verification. All this data is aggregated and enhanced within the medical data processing system 200.

[0091] Figure 6A shows an interface view 600 that includes report 602 alongside a data entry panel 603. The data entry panel 603 includes a set of fields 604-620 identified by the system based on an identified type of report that can be stored in the report itself, e.g., document type 606. As shown in Figure 6A, report 602 is a surgical lesion report associated with a specific set of fields corresponding to surgical lesions. As shown in the example in Figure 6A, these fields are accessible via a dropdown 604 labeled with report information. Other selection mechanisms may be used besides the dropdown list. The fields include document type 606, document title 608, report ID 610, report date 6012, sample collection date 614, sample collection method 616, author 618, and anatomical site 620. As described above with respect to Figures 3G and 3H, each of these fields can correspond to a data category or data type used to organize and manage patient data. Based on the fields, the provided data can be stored in corresponding data objects in the data graph. This can also include the data object of the report itself. Examples of such data objects are shown in Figures 12 and 13, and are described below.

[0092] In the exemplary interface view 600 shown in Figure 6A, the fields are configured to accept user input via interface elements including drop-down menus 606-616, a text input field 618, and radio buttons 620. The interface view 600 displays the report 602 alongside the data input panel 603, allowing the user to easily enter information to fill in the fields while viewing the report. For example, the drop-down 606 may be populated with each possible report type previously configured for the system (e.g., radiology report, pathology report, etc.). The user can click the drop-down 606 to display the possible report types and select a surgical pathology report, which is then used to populate the corresponding object on the backend.

[0093] When a user enters information, the save button 622 may be activated, and in response to the detection of user interaction with the save button 622, the entered data is saved to the integrated patient database 204.

[0094] Figure 6B shows an interface view 625 that includes report 626 alongside a data input panel 627. The data input panel 627 includes a set of fields 628–644. The fields may be identified by the system based on the identified type of report. For example, a report for MRI may be expected to have certain fields, and a report for mammogram may be expected to have other fields. As described above, the fields of a given report may be identified based on a master structured data list (SDL) 246 that defines a list of data categories for the document type of document file 244.

[0095] In the example shown in Figure 6B, information is read from an external system such as an EMR containing structured data. Some of the information is already available in the EMR or other external system in a structured format. This information can be analyzed and associated with a report (for example, by matching the report metadata with the structured data when reading data from the EMR). The interface may include instructions indicating that data corresponding to specific fields is received from the EMR and that a given report is associated with these fields. In some implementations, data linked to a report via information read from a trusted source such as an EMR may be locked for editing, but the user can fill in any missing information fragments. The UI shown in Figure 6B facilitates augmenting or enriching a dataset read from an external system by allowing the user to add missing information and incorporate it into the medical data processing system 200.

[0096] As shown in Figure 6B, report 626 is a radiology report associated with a specific set of fields corresponding to radiology. As shown in Figure 6B, these fields are accessible for viewing via a dropdown 629 labeled with report information. The fields include report type 628, report title 630, report ID 632, report date 634, sample collection date 636, sample collection method 638, author 640, and anatomical site 644.

[0097] In the example shown in Figure 6B, fields 628-640 are highlighted. Certain colors may be used to highlight fields and indicate that the system has read data to populate these fields from EMR or another external database. Such fields may be locked for user editing. Field 644 is shown in white, meaning it needs to be manually entered by the user. Assigning anatomical sites is a diagnostic task that may be most suitable for users such as physicians. The user can select radio button 642 for either an existing selection or creating a new one. In the example shown in Figure 6B, an existing selection is selected, and an anatomical site dropdown menu is displayed, which the user can interact with to select an existing anatomical site. Alternatively, the user can select the Create New button, which displays a text input field for entering the name of the new anatomical site.

[0098] Figure 6C shows an interface view 645 that includes report 646 alongside a data entry panel 647. The data entry panel 647 includes fields identified by the system based on report 646. As shown in Figure 6C, these fields are accessible via a dropdown 604 labeled "Report Information". A first set of fields 650, 654, 656, and 658 are configured to be entered via user input (as described above, for example, with respect to Figures 6A and 6B). Once the user enters information, a save button 659 may be activated, and in response to detection of user interaction with the save button 659, the entered data is saved to the integrated patient database 204.

[0099] In the example shown in Figure 6C, a second set of fields 652 is highlighted. The highlighting may be a different color than that used to highlight the fields shown in Figure 6C to indicate different states of data to be entered into these fields. The highlighted fields 652 correspond to the highlighted fields 648 in report 646. These are fields proposed using machine learning, which the user can review, confirm, or modify. In some implementations, the fields are configured to display data automatically extracted from report 646. One or more machine learning models, including optical character recognition (OCR) and natural language processing (NLP) models, can be used to identify text data from the report, analyze the report, and identify data corresponding to specific fields. The medical data processing system 200 may utilize models trained on labeled data that identify different terms associated with a given given field. In the example shown in Figure 6C, biomarkers are automatically detected by the system. The medical data processing system 200 can input data elements detected using machine learning. The user may be prompted to confirm via an interface, and the user may, in some cases, modify the data elements to be entered into a given field. Over time, the medical data processing system 200 can learn and update the machine learning models used to detect data. Using these techniques, the system can provide recommendations to reduce the user's data input burden. The techniques for applying machine learning to extract and classify medical data are described in more detail in International Publication No. 2021 / 046536, filed on 8 September 2020, entitled "Automated Information Extraction And Enrichment In Pathology Report Using Natural Language Processing," which is incorporated herein by reference.

[0100] Figure 6D shows a set of interface elements illustrating a data entry workflow 660 that can be performed using the interfaces shown in Figures 6A to 6C. The interface elements shown in Figure 6D include interface element 662 for inputting primary tumor information, interface elements 666, 668, and 669 for reading primary tumor information, and interface elements 670 and 674 for editing primary tumor information.

[0101] In Figure 6D, interface element 662 for inputting primary tumor information includes a set of fields for accepting user input of information related to the primary tumor. The fields include interface elements for adding information about the anatomical location (e.g., the right upper lobe of the lung, which is selected from a dropdown menu when the “Select Existing” radio button is selected). The fields further include the histological type and histological malignancy. The user can fill in diagnostic information. Interface element 662 further includes user-selectable checkboxes 663 which can be checked to set the diagnosed primary tumor as a patient symptom for discussion. As indicated by the cursor and highlighted on the save button 664, the user can click the save button 664 to save the entered information to the integrated patient database 204.

[0102] In Figure 6D, interface elements 666, 668, and 669 for reading primary tumor information display information entered via interface element 662. Interface element 666 temporarily highlights recently entered information. Interface element 668 no longer highlights entered information after 5 seconds (or another appropriate time frame). Interface elements 666 and 668 flag diagnoses as pending diagnoses. Interface element 669 does not mark primary tumors as pending diagnoses, and there is no pending diagnose flag.

[0103] In Figure 6D, interface elements 670 and 674 are for editing primary tumor information. The user can interact with interface elements for reading primary tumor information, such as interface element 666. As shown in interface element 670, the cursor is clicking on the highlighted primary tumor diagnosis. The highlighting is maintained until editing is complete. Upon clicking, the color becomes focused and the edit drawer 674 opens. The edit drawer 674 is an interface element, such as a modal, that opens when it detects user interaction, such as a click. The edit drawer 674 contains fields for accepting user input to edit previously entered information (e.g., via interface element 662). The components of the drawer 674 include data input fields that can be used to edit fields such as date, diagnosis, pending diagnosis 676, anatomical site 680, and histological type 682. The edit drawer 674 further includes radio buttons 678 for selecting an existing anatomical site or creating a new one. These interface elements can be used to retrieve data to update data stored in the integrated patient database 204. The retrieved data can be used, additionally or alternatively, to train machine learning models used for automated data extraction from documents (for example, if a medical data processing system identifies that a field has been incorrectly entered by the model based on user corrections, it can be used to update the model's training data). C. Interface for adjusting unmapped data

[0104] Figures 7A and 7B show examples of interface views for reconciling unmapped data according to several embodiments. Reconciliation may be initiated when data is not mapped to the data fields where it is expected to be needed, such as the association between a cancerous tumor and a primary cancer (e.g., as a new primary cancer or as a metastasis from another primary cancer). The interfaces shown in Figures 7A and 7B can be used to manage such a reconciliation process, prompting the user for the necessary information even after a record has been saved for the cancerous tumor in question. For example, missing information may be flagged in the integrated patient database 204 for reconciliation, prompting the workflow described later.

[0105] In the case of data from source systems such as EMR, some data elements may lack relationships between them. For example, a patient may have two primary cancers and metastatic sites. While the primary cancers and metastatic sites are read from the EMR report, the primary site associated with the metastatic site is unknown. The clinician may be aware of information not read from the EMR. In such use cases, the system needs to coordinate unmapped data functions.

[0106] In adjustment, specific data is summarized, but the system needs to determine where the data belongs in the UI, for example, which primary condition the data should be mapped to. The adjustment UI may prompt the user to provide input to associate a particular anatomical site with the correct primary cancer. For a given primary cancer, specific fields may be associated with the primary cancer. The adjustment UI prompts the user to uniquely associate different types of information, such as the primary site, with relevant findings such as histology, biomarkers, stage, and metastatic sites, with primary cancer or other data elements. The adjustment UI may also be used to map specific medical interventions, such as oncological or non-oncological surgical history, or specific drugs as antineoplasms or non-cancerous.

[0107] In some cases, external systems such as EMR provide information indicating the primary cancer and where this primary cancer has metastasized. In this case, the association is known and no additional work may be required. In other cases where coordination is necessary, the external system has not captured the association or transmitted that information to the medical data processing system 200. If such an association is needed, the medical data processing system 200 may use the coordination process to determine where in an interface, such as the patient summary or patient journey view, to indicate metastasis (e.g., against the right or left breast). To enable the information to be presented in a clinically accurate manner, the coordination process allows the user to provide guidance on where to indicate this association, which affects the data mapping applied to the integrated patient database 204.

[0108] In some cases, reconciliation may be triggered when an external database, such as EMR, transmits data related to a specific site, but lacks information indicating other sites to which that site relates. Using the reconciliation interface, users can provide information to associate the site with a specific cancer, and after the update, the site begins to appear associated with the correct primary cancer in the interface view and the integrated patient database. In other examples, reports may be received from external databases without structured information, in which case several detailed points may be missing. Such details may be provided by the user via an interface such as the one shown in Figure 7A.

[0109] Figure 7A shows an interface summary view 700 that includes data adjustment elements 702, 704, and 706. In some implementations, the interface summary view 700 provides data adjustment elements 702, such as buttons or dropdown menus, for interacting with data that has not been mapped for adjustment. In the upper right corner of the screen, this data adjustment element 702, the "Unmapped" button, allows the user to open unadjusted items that are not related to any cancer or lack mapping information. The user can provide data specifying the missing relationships and save the updated data. Once the user adjusts this data, it begins to appear in the portal 220.

[0110] User interaction with adjustment element 702 may trigger the display of data adjustment elements 704 and 706. Data adjustment element 704 includes a notice that is displayed prominently (e.g., highlighted with a warning sign). In the example shown in Figure 7A, the notice displayed in data adjustment element 704 states, "There is not enough information to place these items in the patient summary and journey views." Data adjustment element 706 displays information about the items that need adjustment. In this example, cancerous masses and iliac crest structures lack the information necessary to add them to the patient summary and journey views. Data adjustment element 706 further provides additional information about cancerous masses, namely "right" and "fetched from integration on November 27, 2020." Such information may be read from the integrated patient database according to the data type of the mapping within it (e.g., "fetched from integration date" may be based on a timestamp, and "right" may be based on a location data type). When user interaction with data adjustment element 706 occurs, the interface may transition to the interface view shown in 7B for adjustment.

[0111] Figure 7B shows an interface view 720 for data adjustment. In some implementations, the interface summary view 720 includes a report 721 and a drawer 723 (e.g., a modal with elements for accepting user input) for receiving data for adjustment. Within the drawer 723 is a heading 722 labeled “Mapping Anatomical Sites”. The drawer 723 indicates that the missing information to be adjusted is to associate the iliac crest structure 726 with primary cancer or metastasis, or to mark it as benign. The drawer 723 also includes a warning 724 similar to the data adjustment element 704 described above with respect to Figure 7A. The drawer 723 of the interface view 720 further includes a set of checkboxes that the user can use to specifically associate the iliac crest structure 726 with primary cancer or metastasis, or to mark the iliac crest structure 726 as benign. In some implementations, the integrated patient database stores patient records with objects corresponding to different cancer sites. Based on the anatomical site mapping established using interface view 720, the object for the iliac crest structure 726 may be linked to other objects accordingly. For example, if the iliac crest structure 726 is marked by the user as a metastasis of right breast cancer, the iliac crest structure object will be linked to the right breast cancer object. The received designation of the iliac crest structure as primary, metastatic, or benign may be stored in the integrated patient database in relation to the “characteristics” data type within the tumor mass data object, as further described below with respect to Figure 12.

[0112] As shown, possible options include setting the iliac crest structure as the primary or metastatic site of a new primary cancer, which may trigger the display of additional interface elements for establishing a new primary cancer. Possible options further include setting the iliac crest structure as the primary site. This causes the iliac crest structure to be stored in the integrated patient database as a primary cancer object with its own set of linked objects, as shown in Figure 12. Alternatively, the iliac crest structure is set as a metastasis of a pre-established cancer, namely right or left breast cancer. This causes the iliac crest structure to be stored in the integrated patient database as an object linked to a type metastasis object and linked to another data object corresponding to the selected primary cancer. Another checkbox is provided to mark the iliac crest structure 726 as benign, which hides the iliac crest structure from the summary. In such a case, the iliac crest structure 726 may be stored in the integrated patient database as an object linked to a benign type object and not linked to any object corresponding to a primary cancer. When the user selects a cancer association, the refresh button 730 is activated. The user can interact with an update button to trigger the system to store the provided adjustment data in the integrated patient database 204. The data schema for storing data objects in response to the selected anatomical site mapping will be described in further detail later in Section IV with respect to Figures 12 and 13. D. Patient Portal Interface

[0113] Figures 8A, 8B, 8C, 9A, 9B, 9C, 9D, 9E, 10, and 11 show examples of portal 220 that provide a centralized view of patient data. Portal 220 can display various interface views, including a patient summary interface view as shown in Figures 8A–8C, a patient journey interface view as shown in Figures 9A–9E, a report interface view as shown in Figure 10, and a care quality metrics interface view as shown in Figure 11. 1. Patient Summary Interface

[0114] Figures 8A–8C show examples of patient summary interface views according to several embodiments. The patient summary interface displays patient summary data. The patient summary interface view may be used to display data that allows the user to view detailed information and oncological summary information about various primary cancer sites, as well as to provide a launch pad for performing data entry and adjustment via portal 220.

[0115] Referring to Figure 8A, the portal 220 can display the patient's current diagnosis (adenocarcinoma of the lung), date of diagnosis, notes from the last visit, upcoming visits, and current treatment. The portal 220 can receive structured patient data 202 from an integrated patient database 204, which is either manually entered via the data entry interface 300 or automatically sourced and summarized from medical reports by the data summarization module 232.

[0116] Figure 8B shows another implementation of the patient summary interface 800 of the portal 220. The patient summary interface 800 displays summary information about a specific patient that can be fetched from the integrated patient database 204 for display. The upper ribbon 801 can display patient information such as the patient's name, age, date of birth, gender, and patient identifier.

[0117] The patient summary interface 800 includes a primary cancer element 802. The primary cancer element 802 includes tabs 803A and 803B, corresponding to different primary cancers: breast cancer (within tab 803A) and lung cancer (within tab 803B). The primary cancer element 802 contains information about each primary cancer, including events, associated biomarkers, staging, and metastatic sites.

[0118] In Figure 8B, the patient summary interface 800 further includes an oncological summary element 804, an oncological treatment element 806, and a medication element 808, displaying information related to each. The patient summary interface also includes a patient history element 810 that displays patient history information, including medical history, surgical history, family history, and social history.

[0119] The patient summary interface 800 also includes user-selectable elements that can be used to navigate to other interface views. The patient journey element 811 may be selected to transition to the patient journey view, as shown in Figures 9A-9E. The report element 812 may be selected to transition to the report view 1000, as shown in Figure 10. The unmapped data element 813 may be selected to transition to the adjustment view shown in Figure 7B. The add element 814 may be selected to transition to a view for adding data (for example, directly to the portal 220 or by uploading a report). A summary element 815 is also included and can be used to transition to the patient summary view from other views. Thus, the patient summary view can be used to transition to various views of the portal 220. The primary cancer information element 805 may be selected to display a modal, overlaid on the patient summary view 800, and contain additional information about one or more primary cancers, as shown in Figure 8C. Based on the selected view or mode, data is read from the integrated patient database 204 according to the data connections and type mappings within it.

[0120] Referring to Figure 8C, an example of a patient summary view 820 is shown, having two modals 822 and 824 corresponding to two primary cancers. In response to the detection of a user interaction with the primary information element 805, this example displays modals for both primary cancers associated with the patient. Modal 822 (e.g., the first modal) is about right breast cancer and contains information about right breast cancer, including a set of associated biomarkers with timestamps. Modal 824 (e.g., the second modal) is about left breast cancer and contains information about left breast cancer primary cancer, including a set of associated biomarkers with timestamps.

[0121] The first and second modals are displayed side-by-side in the graphical user interface. Advantageously, this side-by-side display allows the user to view more detailed information on multiple primary cancers at once without leaving the summary interface screen. Furthermore, because each modal corresponds to a different primary site, data can be efficiently retrieved site by site, potentially providing parallel analysis. This organization of the database (e.g., the data schema described in Section IV) enables data retrieval and visualization via the graphical interface.

[0122] In some implementations, the integrated patient database stores data objects corresponding to primary right breast cancer and data objects corresponding to primary left breast cancer. These data objects are timestamped and linked to various other data objects that describe different events associated with the primary cancer. For example, the right breast cancer primary object is linked to one or more biomarker data objects, and the left breast cancer primary object is linked to one or more biomarker data objects. In response to the detection of a user interaction with the primary information element 805, the system queries the integrated patient database to identify the right breast cancer primary data object and the left breast cancer primary object. The objects linked to each primary cancer data object are identified based on the mapping in the integrated patient database between the identified primary cancer data object and its respective child data objects. The information is read in association with the identified linked object. The identified linked object is used to populate modals 822 and 824 with respect to method 1800 in Figure 18, as further described below.

[0123] Right breast cancer and left breast cancer can be different cancers located in different, unrelated parts of the body. The patient summary view 820 may be used to display information related to each primary cancer, showing different information side by side for each primary. Using the interface view 820, clinicians can observe information on multiple primary cancers and compare information such as diagnosis, date of onset, location of each primary site, key biomarkers for this patient, stage classification, and any metastases. This may be achieved by organizing data according to primary cancer designation using the specialized data schema described herein, which can be fetched to display the interface view 820 displaying the primarys side by side. 2. Patient Process Interface

[0124] Figures 9A–9E show examples of patient journal interface views according to several embodiments. The patient journey interface displays patient-related data in chronological order. Using the patient journey interfaces shown in Figures 9A–9E, cancer progression and available information about the cancer can be viewed in an organized chronological format. Users can click on objects in the timeline to see how the cancer progressed.

[0125] The patient journey interface view can display patient information from suspected cancer to diagnosis, treatment planning, monitoring, and survival. Cancer care is inherently multi-institutional, encompassing many specialties. Generally, patient information can be scattered across different systems. By extracting and integrating information from reports and other data read from different sources, a healthcare data processing system can construct inter-institutional patient journeys, allowing users to view a comprehensive patient journey across data points collected across different service providers and service types.

[0126] Once this information enters the patient journey, any other user following the patient journey can see the illustrated information in exactly the same way. This is advantageous from a care coordination perspective. With conventional technologies, over-reliance on medical notes often results in different providers carrying out different information about what is happening to the patient based on different note-taking styles. Therefore, it is difficult to gain a common understanding of what is really happening to the patient using conventional systems. The patient journey interface shown in Figures 9A-9E solves these and other problems by enabling any user to see an integrated view of the patient's treatment history. The patient journey can be viewed and entered by cross-functional teams such as radiation oncologists, medical oncologists, surgical oncologists, and / or attending physicians. These users can interact with the patient summary UI and view patient data in a user-friendly and integrated way to observe and understand the progression of the patient's condition, such as cancer.

[0127] To display a patient journey view, the system may receive patient-identifying data (e.g., patient ID, name, etc.) via a graphical user interface. Based on the patient-identifying data, the system may read patient-related medical data from an integrated patient database and display a set of user-selectable objects on a timeline via a graphical user interface, where the objects include multiple categories organized into rows, and the categories include lesions, diagnoses, and treatments, as shown in Figures 9A to 9E. Techniques for inputting the patient journey view are further described below with respect to Method 1700 in Figure 17.

[0128] Referring to Figure 9A, portal 220 can display a timeline view of the patient journey. The timeline view can show various examination and imaging results, as well as diagnostic results provided by the oncology workflow module 222, in relation to time.

[0129] Figure 9B shows another implementation of the patient journey interface view 900. The patient journey interface view 900 in portal 220 includes a summary ribbon 902 and an adjustable timeline 908.

[0130] The summary ribbon 902 can be a ribbon that appears above the timeline. The summary ribbon 902 can display a subset of objects flagged as important relevant information. Users can bookmark objects displayed in the summary ribbon 902. Given that patient journeys can be long and contain many objects, the summary ribbon 902 is useful for bringing important objects to the forefront. Users can also remove objects from bookmarks when they are no longer important, and the objects will be removed and disappear from this ribbon. The summary ribbon 902 itself can function as a mini-journal highlighting important objects that occurred in this patient.

[0131] The adjustable timeline 908 contains information about the patient's oncological history. The adjustable timeline 908 displays the information chronologically, with older objects on the left and newer objects on the right. The period displayed can be controlled using the start date element 904 and the end date element 905, as well as the adjustable scroll bar 906 to select the time window in which objects appear on the timeline 908.

[0132] Information within the timeline 908 is displayed in sets of rows corresponding to different data categories, including events 910, pathology 912, imaging and procedures 914, treatments 916, biomarkers 918, and response assessments 920. Within each category, relevant information may be color-coded (e.g., orange events, red lesions). Each row may display information collected about a patient within the corresponding data category. A given row may contain multiple entries at a given time, as shown in Figure 9B. For example, in event 910, multiple events in January correspond to two different cancers.

[0133] The Event 910 data category includes events (e.g., the category of the displayed object) that correspond to information about the progression of the cancer itself. For example, Event 910 includes Invasive Breast Cancer 922 dated January 2020. This may correspond to the date when this primary cancer was diagnosed and added to the patient record. When the user clicks on Event 922, the system displays additional information about Event 922, as shown in Figures 9C and 9D. The rows in Event 910 can show the cancer diagnosis and progression for each different cancer the patient has. For example, as shown in Figure 9C, Event 922 is invasive ductal carcinoma of the right breast at the 6 o'clock position. When the user clicks on these different items in the Interface View 900, the user can see how the cancer has evolved. For example, when the cancer first started, there were no metastases. The user can scroll through the timeline to see if the cancer has metastasized elsewhere after one or two years, and if so, it will be visible in a specific box. Thus, the Patient Progress Interface View 900 allows the user to see the progression of cancer over time. This information can be read from tumor mass data object 1202 and / or cancer symptom data object in the integrated patient database, according to the data schema shown in Figure 12.

[0134] The pathology data category 912 includes objects corresponding to pathology reports, displayed chronologically. If there are multiple reports associated with a date, multiple pathology reports may be displayed stacked on top of each other. Pathology reports may be associated with events 910 used to diagnose a particular cancerous mass, for example. Examples of pathology reports include biopsy reports, cytology reports, genomic reports, and surgical resection reports. Through the patient process interface view 900, users can drill down into a specific pathology report to identify information such as how far the cancer has spread, what its size is, what stage it is, and key biomarkers to test for from the sample taken. This information may be read from the diagnostic findings data object 1205 in the integrated patient database, according to the data schema shown in Figure 12.

[0135] The data category for diagnostic imaging and procedures 914 includes objects corresponding to diagnostic imaging such as MRI and CT scans. For example, diagnostic imaging procedures 914 includes MRI 924 from January 14, 2020, as shown in Figure 9B. These objects are displayed in chronological order. Objects can be linked to diagnostic imaging reports, such as MRI reports or reports on several lesions. A clinician viewing this information should be able to confirm that an MRI was performed on this patient on a given date in the timeline and drill down to view the results of that MRI by opening the report. For example, if a user clicks on MRI (924) from January 14, 2020, the report may be opened directly from the patient procedure interface view 900. Advantageously, the user does not need to navigate to another system to find reports that would be required without the technology of this disclosure. This information can be read from the diagnostic findings data object 1205 in the integrated patient database, according to the data schema shown in Figure 12.

[0136] The data category for procedure 916 includes objects corresponding to the procedure given to the patient. As seen in Figure 9B, the procedure may last for several months. This information can be read from procedure data object 1208 in the integrated patient database according to the data schema shown in Figure 12.

[0137] The biomarker 918 data category includes objects corresponding to biomarkers associated with a patient. These may include genomic markers, diagnostic markers, prognostic markers, therapeutic markers, etc. These biomarkers may originate from various types of reports but are treated similarly within the system. For example, biomarkers may originate from cytology reports, genomic reports, etc. All of these various types of biomarker objects are shown in the biomarker 918 row. This information can be retrieved from the diagnostic findings data object 1205 (e.g., molecular / biomarker object) in the integrated patient database, according to the data schema shown in Figure 12.

[0138] The response evaluation data category 920 includes objects corresponding to clinician assessments of patient response. At each step of disease management, clinicians assess the patient's tumor status and clinical symptoms to determine the effectiveness of treatment and whether and how to continue the current treatment plan. The determination of treatment effectiveness is a complex judgment based on elements of clinical response, radiological response, molecular response, and serological response. The patient journey interface view 900 brings this together into a single, highly telegraphic icon view on a timeline, allowing clinicians to see it chronologically along with treatments, scans, and other data. For example, a physician might note that a patient was given a certain number of cycles of a particular drug along with radiotherapy, and the patient responded partially. Using the patient journey view, if a clinician wants to record how this particular cancer is progressing at any given point in time, the user can input an assessment of the response, such as whether it is a partial response, whether the cancer is stable, how the patient is feeling, whether there are any adverse events, and whether the cancer progression is not problematic. In some implementations, the patient journey is entirely read-only, with the exception of this one response field. A crucial role of oncologists is to manage toxicity during procedures and monitor patient responses; therefore, response assessment by clinicians is important in many situations.

[0139] In Figure 9C, the cursor 923 is hovering over the invasive element 922 of the breast cancer. This causes the system to zoom in on the field of view so that additional text can be seen in Breast Cancer, Invasive Element 922 - Ductal Carcinoma, Right Breast (6:00).

[0140] In Figure 9D, when the user then clicks on breast cancer, invasive element 922, a pop-up 927 appears, displaying further information such as date and location, as shown in Figure 9D.

[0141] In Figure 9E, the adjustable timeline 908 is adjusted to show different time windows (for example, via a slider). Figure 9E shows a time window from October 1, 2020 to January 1, 2021. Thus, through user interaction with the GUI, the user can scroll to display different objects on the timeline by moving the slider 906 to view the timeline over a longer period, or by focusing on a period of interest.

[0142] In some implementations, reports can be previewed from the patient summary view. The system can detect user interaction with objects displayed in the patient summary view, then identify and retrieve the corresponding report from the integrated patient database and display the report via a graphical user interface (e.g., as a popup on the patient summary view). The user can then navigate to the report view for a more detailed view of the report.

[0143] The patient journey view can be used to see how a patient's cancer has evolved over time. For example, at the first time point, the patient has one primary cancer site (for example, in the example shown in Figure 9A). At the second time point, one primary site is still visible in the patient journey view. At the third time point, two different primary sites can be seen (for example, in the example shown in Figure 9B). Thus, in this particular example shown in Figures 9B-9E, two primary cancers, left and right breast cancers, are displayed in the patient journey. 3. Report View Interface

[0144] Figure 10 shows an example of a report interface view 1000 according to several embodiments. The report interface view 1000 includes a list 1001 of reports associated with a patient. One of the listed reports 1002 is selected, and that report 1003 is displayed on the right. Interface elements are also provided that allow the user to click and open the full report.

[0145] In some implementations, the top-level Patient Journey, Summary, and Report tabs (1005) are used to allow navigation between the respective interface views. For example, in the Patient Journey view, the system detects user interaction with the Summary tab and transitions to the Summary view to display oncological summary data. 4. Quality Care Metric Interface

[0146] Figure 11 shows another interface view for displaying quality care metrics. As shown in Figure 11, the portal 220 can display care quality metrics such as the Quality Oncology Practice Initiative (QOPI) over time for different patients. The metrics can be calculated based on structured patient data 202 at different time points. IV. Exemplary schema for an integrated patient database

[0147] Figures 12 and 13 show exemplary data schemas for use in constructing data stored in the integrated patient database. The patient summary and patient journey interfaces described above are enabled by retrieving interconnected data elements associated with the patient, which are timestamped and hierarchically linked. These data elements are dynamically updated and enhanced. This is made possible by using a dedicated data schema for the integrated patient database. Figure 12 shows examples of different types of data objects connected to each other within the patient data map. Figure 13 shows examples of specific data objects that can be stored and modified. A. Data schema for patient data elements

[0148] Figure 12 shows an exemplary data schema 1200 for patient data elements according to several embodiments. The data schema 1200 is used to decompose different data elements read by the medical data processing system 200 and generate individual data objects (also called data entities). These data objects store the various data elements associated with the patient data. Relationships between these data entities are maintained and updated. This data schema allows the system to continuously maintain up-to-date images of the patient, and detailed data elements are stored in a structured manner. In some implementations, the data schema is based on the HL7 FHIR (Fast Healthcare Interoperability Resources) standard, as described in "Welcome to FHIR," https: / / www.hl7.org / fhir / (2019).

[0149] In this example, data schema 1200 contains a set of data objects (illustrated in boxes such as tumor mass data object 1202, diagnostic findings data object 1205, etc.). The integrated patient database may store patient records for each patient. A patient record contains a network of interconnected data objects, each of which may contain a set of configured data elements of a data type corresponding to a given data object. Each box shown in Figure 12 is a data object and can be implemented as a resource using the HL7 FHIR standard. Alternatively, data objects can be implemented as tables using a relational database.

[0150] As shown in Figure 12, each data object has associated attributes in the form of a set of data types corresponding to data objects of that type. For example, the data schema 1200 includes a tumor mass data object 1202 that can store data elements corresponding to data types such as history, anatomical site, site description, and characteristics, as shown in Figure 12. The data types may correspond to fields shown in the interface view (e.g., fields 606-620 shown in Figure 6A, fields 628-644 shown in Figure 6B, etc.).

[0151] Each data object, such as the tumor mass data object 1202, the diagnostic findings data object 1205, and the patient route data object 1201, represents a clinical data entity. These data objects can be related to one another, facilitating the management of graphs of patient data, which are networks of interconnected data objects. For example, a given data object may contain information including data elements such as "colon," in relation to a corresponding data type that characterizes or classifies data elements such as "location."

[0152] In Figure 12, the lines 1220 connecting data objects indicate relationships between elements. For example, a cancer symptom data object must be linked to a patient data object and one or more tumor mass data objects, and may optionally be linked to one or more oncological treatment data objects. Circles 1222 indicate optional relationships, a single solid bar 1224 indicates a one-to-one relationship, a V-shaped symbol indicates a one-to-many relationship, and a circle with a V-shaped symbol (e.g., the intermediate connector in report 1204) indicates zero or many relationships. Links (connections) between objects can be specified in various ways within the integrated patient database. For example, a master list may be stored for patient records that identify each object linked to another object. The direction of the link, for example, the report in which the tumor mass was formed, may be specified.

[0153] The root data object 1201 is a patient data object and may contain information such as the patient's name, date of birth, sex, and identifier. As shown by the connection line 1220, the root data object 1201 is connected to various other data objects corresponding to the patient's oncology data. The data objects may be categorized with respect to diagnosis, treatment, history, or other appropriate data categories. Each of the other data objects may be linked to the patient root data object 1201. Information from the patient root data object 1201 may be displayed along the top of the interface view of the portal 220 (e.g., the patient ribbon 801 in Figure 8B). The patient root data object 1201 may be used to identify and traverse patient data records to identify additional information for viewing and editing via the portal 220.

[0154] Various data objects corresponding to data types are connected to the patient's root data object 1201. All data objects are related to the patient root data object in some way. For example, the diagnostic-related data object 1203 is a data object used to describe the patient's diagnostic information. Each diagnostic-related data object 1203 is connected to the patient root data object 1201. The diagnostic findings data object 1205 is a data object corresponding to a diagnosis, connected to the tumor mass data object 1202. This includes various types of diagnostic findings data objects 1205, including TNM staging data objects, molecular / biomarker data objects, tumor size data objects, and other pathology / imaging findings data objects, as shown at the top of Figure 12.

[0155] Each of these data objects can store the corresponding data elements of the configured data type. For example, a tumor size data object is configured to store data elements corresponding to the data type maximum dimension, additional dimensions, units, and date. Other pathology / imaging findings data objects may be configured to store data elements corresponding to the data type, value, and date. Findings can be any kind of information about an anatomical site obtained from one or more samples from a site, which may originate from a report. For example, findings such as histological malignancy may be extracted from a pathology report, and findings such as tumor size may be extracted from an imaging report, and so on. In data schema 1200, findings are generally associated with a specific site, but some findings may be directly related to the cancer symptom itself rather than a specific site. For example, the stage of cancer may be defined at a higher level rather than at an individual site. In the patient summary UI shown in Figures 9B to 9E, these diagnostic data objects correspond to data category events 910 displayed at the top, one example being cancer diagnostic event 922.

[0156] Another data object within the diagnosis category 1203 is the tumor mass data object 12002. The tumor mass data object stores data elements that characterize the tumor, organized according to the data types histology, anatomical site, site description, and characterization. For example, the tumor mass data object 1202 includes a structured field for the data type "characterization" indicating whether the tumor is a primary tumor, a metastatic tumor, or a benign tumor.

[0157] For multiple tumor masses, there may be separate data objects connected to the root data object 1201. Multiple instances of the tumor mass object may exist, each corresponding to a different tumor mass identified at a different location in the patient. A tumor mass may be designated as a primary cancer or metastasis, affecting network interconnections to other objects. Thus, the data object may include a data object corresponding to the primary cancer, another data object corresponding to the metastasis of that primary cancer, and so on. As shown in Figure 12, for each tumor mass, the data object may include information such as histology, anatomical site, site description, and characteristics. This data object is linked to various other diagnostic-related data objects 1203, including cancer symptoms, diagnostic findings 1205, and reports 1204.

[0158] One or more procedure-related data objects correspond to a procedure and are connected to the patient route data object 1201 and / or tumor mass data object 1202. The procedure-related data objects include the oncological procedure data object 1208. The oncological procedure data object 1208 is configured to store data elements such as procedure type, date, and response, and can be linked to associated reports. The oncological procedure data object 1208 can be used to input procedure 916 lines in the patient route interface shown in Figures 9B–9E.

[0159] One or more report data objects 1204 may be connected to a patient root data object 1201 and / or a diagnostic findings data object 1205. Reports from EMR or other sources may also be stored as report data objects 1204. As shown in Figure 12, a report data object 1204 is configured to store data type status, category, title, date, and attachments. A report data object 1204 may contain attachments or addendums in PDF or image format. Addendums are issued when changes are made to a patient's clinical documentation and medical records. They may contain information that was not available at the time of original entry or may contain corrections to previously issued medical information. It is important for clinicians to know if a particular patient report has been updated or added, and to view the entire report. A report data object 1204 may also contain text data extracted from the report (e.g., using OCR).

[0160] One or more history-related data objects 1210 may be stored and connected to a patient root data object 1201 and / or a tumor mass data object 1202. The history-related data object 1210 can contain various different types of data objects with corresponding attributes, as shown in Figure 12. For example, the data schema 1200 can contain drug data objects, comorbidity data objects, family medical history data objects, surgical history data objects, allergy data objects, substance abuse data objects, performance status data objects, environmental risk data objects, social history data objects, and other history findings data objects, as shown in Figure 12. The history-related data object 1210 may store history-related data elements of various data types, as shown in Figure 12. The data mapping shown in Figure 12 can be used to establish where the corresponding data elements are displayed in various interface views.

[0161] In the data schema 1200, each data object may be stored in association with one or more timestamps. Timestamps can track when an event occurred. For example, a given data object may include timestamps corresponding to the date and / or time of a diagnosis, treatment, sample collection, treatment date, report issuance, or other event. Timestamps can further track when the data was integrated into the integrated patient database. For example, when data is stored in the integrated patient database, the medical data processing system 200 generates and stores a timestamp indicating the time the data was incorporated into the integrated patient database. B. Example of a data schema

[0162] Figure 13 shows an exemplary data schema 1300 according to several embodiments. The data schema 1300 includes data objects for different cancer sites. Cancer 1 1302 and Cancer 2 1307 are primary cancers. Each of these is stored as its own data object along with relevant information such as stage and diagnosis.

[0163] At the first time point T1, cancer 1 may be associated with multiple data objects in the data schema 1200 of the integrated patient database shown in Figure 12, including tumor mass 1202. Other objects for findings related to cancer 1 are linked to the tumor mass, including TNM staging objects, biomarker objects, tumor size objects, etc.

[0164] Later, other sites may be found and associated with primary cancers, such as cancer 1 or cancer 2. When a new site (tumor mass) is identified, the new tumor mass object can be linked to the data model. For example, as shown in Figure 13, tumor 1 data object 1306 is stored in association with primary cancer 1 data object 1302. Tumor 2 data object 1308 and tumor 3 data object 1310 can be two data objects stored in association with cancer 2 data object 1304. Tumor 1 data object 1306, tumor 2 data object 1308, and tumor 3 data object 1310 can correspond to multiple tumor mass objects 1202 linked to the same patient object 1201, as shown in Figure 12.

[0165] The example shown in Figure 13 illustrates how the data schema in Figure 12 is used to handle a possible diagnostic process that a cancer patient might experience, which could include various tests, imaging, and other diagnoses, with new information coming in over time. This data schema is configured to be updateable while maintaining complex relationships between different data types from different data sources that come in at different times.

[0166] As shown on the right, each of the tumor objects 1306, 1308, and 1310 has associated data elements for storing information such as location, size, liver, and site, and if the data originates from a report, that report is also stored as a data element for that data object (e.g., reports 1312, 1214, 1316, and 1318, as well as associated data attributes that can be extracted from these reports). For example, as shown in Figures 3A to 7B, information is extracted from reports using the interfaces described above with respect to these. Data elements may be assigned data categories using NLP, which is then used to populate the appropriate data objects.

[0167] Each of the data objects 1306, 1308, and 1310 can correspond to three virtual time points. Each time point represents the time when the data that inputs the corresponding data object was acquired. For example, data object 1306 may contain data derived from a radiology report PDF 1312 acquired on a given date, data object 1308 may contain data derived from a pathology report 1314 acquired on a later date, and data object 1310 may contain data derived from another pathology report 1316 acquired on a given date. Each of these can be imported into the integrated patient database at their respective different time points and tracked by the timestamps stored in each data object.

[0168] For example, at the initial stage, a lung mass is found in the patient's lung based on radiology report 1312. At this point, other tests are pending. The data schema 1300 may be updated as additional information becomes available. Initial data objects may correspond to initial assumptions about the patient's diagnosis. For example, there is a 2 cm mass in the lung and another 2 cm mass in the liver. A primary diagnosis is entered that there is primary lung cancer that may have metastasized to the liver. The physician may order additional tests. In this example, report 1312 is linked to two masses 1306 and 1308, indicating that at the time report 1312 was obtained, both masses 1306 and 1308 were included in the radioanalysis and corresponding data was extracted.

[0169] At time 2, when additional test results 1314 and 1316 are returned, two more reports are added to data schema 1300. Report 1314 is a pathology report concerning tumor 1 1306. At time 2, pathology report 1314 is imported into the system, NLP is used to identify data categories corresponding to data fields extracted from radiology report 1314, and corresponding data objects are entered, including tumor mass data object 1202 corresponding to tumor 1 1306 and linked data objects corresponding to related findings. Report 1316 is a pathology report concerning tumor 1 1306. At time 2, pathology report 1316 is also imported into the system, NLP is used to identify data categories corresponding to data fields extracted from pathology report 1316, and corresponding data objects are entered, including tumor mass data object 1202 corresponding to tumor 2 1308 and linked data objects corresponding to related findings.

[0170] At time 3, when the colonoscopy report 1318 is read by the medical data processing system 200, additional colonoscopy findings are summarized from the report. This helps the user to make further diagnoses, such as confirming that the colon mass and liver mass found during the colonoscopy are consistent. A final image of the patient's diagnosis may then be created. In this example, the diagnosis involves lung cancer and metastatic colon cancer, with two different primary diseases occurring simultaneously.

[0171] The data schema shown in Figures 12 and 13 facilitates the representation of all three states as snapshots over time, while also allowing users to modify the relationships between entities as new information becomes available from new reports. The data schema provides a representation of the report itself, as well as a representation of the individual findings summarized from the report. The data schema also provides a representation of each cancer and anatomical site, as well as the attributes of these sites. Each data object is associated with one or more timestamps, so that the patient's journey can be tracked over time to facilitate the clinical decision-making process. The data schema links sites, findings, and reports, while allowing sites relevant to the most recent information. Some of these relationships can be modified individually without affecting the rest of the graph of data elements and attributes. Once these associations are created, a timestamp associated with the association exists. Thus, the data schema facilitates an interface view that provides visibility not only when a report was created, but also of new associations, old associations, and changes to associations over time. The data schema can also track origin information (e.g., who edited something and where). V. Method A. Overview of Medical Data Workflow

[0172] Figures 14A–14D provide an overview of the oncology workflow for capturing, modifying, and displaying patient data. The workflow in Figures 14A–14D involves collecting data and storing it in an integrated patient database 1409 (e.g., integrated patient database 204 shown in Figure 2). This data may include relevant radiographs, procedures, and pathological findings associated with one or more primary tumors and their associated metastatic lesions, which may be updated throughout the course of cancer treatment and other phases of the patient journey. The data may also be read from the integrated patient database 204 and displayed in a set of interface views that facilitate the management, care, and diagnosis of clinical patients. Figures 14A–14D provide an overview of the operation, which will be described in more detail with respect to the methods in Figures 15–21.

[0173] In Figure 14A, data is collected and stored in the integrated patient database 1409. Initially, a patient record is created, which can be initiated via input from user 1401 and / or EMR integration 1401. The user can manually create a new patient in 1403. The EMR can send selected patients to the system in 1404. This data can originate from the EMR, or other external databases such as the laboratory system, or other data systems within the hospital. This can result in patient data such as patient identifiers and other data types, which can be stored in the patient root data object 1201, as shown in Figure 12. The data collected in 1403 and 1404 includes patient data that identifies the patient. If there is no existing record for a patient, a new record is created.

[0174] Subsequently, for example, when additional data is collected and / or periodically, additional data may be stored in the integrated patient database. In 1408, the EMR sends a report to the system. The system generates structured data from the report and sends the structured data to the integrated patient database 1409 for storage in association with patient records. This is shown in Figures 4A to 7B and can be performed using the interfaces described above with respect to these. In 1406, the user manually adds structured data to be stored in the integrated patient database 1409. This is shown in Figures 3A to 3H and can be performed using the interfaces described above with respect to these. The data may be stored according to the data schema described above with reference to Figures 12 and 13. Thus, the system can collect both structured and unstructured data from different sources and store them in the integrated patient database 1409 in an integrated manner.

[0175] The data stored in the integrated patient database 1409 may include identifying information about patients and patient demographics. The data stored in the integrated patient database may include structured data such as patient diagnosis, medication, and medical history. The data stored in the integrated patient database may also include unstructured data such as pathology reports, imaging reports, and clinical notes. For example, as shown in Figure 12, data may be stored in data objects that are mapped to each other and can be updated and modified over time. As shown in Figure 13, specific instances of these data objects may store the reports themselves in association with data extracted from those reports.

[0176] If all data stored in the integrated patient database is in a structured format, that data can be used to generate various analyses or visualizations as described above in Section III, such as patient summaries and patient journey views. Before this can be achieved, data augmentation operations are performed on the data from EMR or other external databases / systems.

[0177] In Figure 14B, data summarization of report 1412 is performed. Report 1412 may include a pathology report, a procedure report, etc., as shown in Figure 14B. In 1414, the user opens the report. The report may or may not contain structured data. The user may open the report for viewing. Depending on which report is opened, the list of fields that can be populated using the information present in this report may change. For example, as shown in Figure 6A, interface 600 for data summarization shows a surgical pathology report and a corresponding set of fields to be populated associated with the surgical pathology report. As shown in Figure 6B, interface 625 for data summarization shows a radiology report and a corresponding set of fields to be populated associated with the radiology report.

[0178] In step 1416, summarization is performed. Specific fields or medical concepts may be highlighted in the data summarization UI to provide the user with information such as diagnoses and notes. In step 1418, the user fills in any missing information. The structured data is mapped to terms, aided by OCR and NLP where possible. This process generates structured data from the unstructured report, and the structured data is sustained in step 1419. Once the user saves all of this information, it is immediately sent to the integrated patient database. This data is enhanced by adding more structured information extracted from this report and sending it back to the integrated patient database.

[0179] Figure 14C shows further details regarding the data summarization process. In 1421, the user summarizes findings related to an anatomical site from the report. In 1422, the system determines whether the site is already associated with any primary cancer. This can be achieved, for example, through user input to an interface that provides or confirms the association. If the site is already associated with a primary cancer, in 1424, the system preserves the existing association by saving the findings of the anatomical site associated with that primary cancer upon saving. If the site is not yet associated with a primary cancer, in 1423, the system displays the anatomical site in an adjustment area, allowing the user to associate the anatomical site with a primary cancer. The system then proceeds to the adjustment process described later for Figure 14D.

[0180] In 1425, the system determines whether the user wishes to add / update an association. If the user does not wish to add or update an association, the add / update process is skipped. If the user wishes to add or update an association, in 1426, for each finding associated with the anatomical site, the system displays an association menu. Through the association menu, the user can associate the site as the primary site of any one primary cancer, or the user can associate the site as a metastasis to one or more primary cancers. In some embodiments, an anatomical site may be a primary cancer site of only one primary at a given time. An anatomical site may be associated with two or more primary sites at any given time for various reasons, such as a pending diagnosis or medical judgment, or not being important to the patient's treatment process.

[0181] There are several options for these association updates. In 1428, the site is previously labeled as a primary cancer site, and then labeled as a metastasis. Next, in 1432, the user can proceed only if there is no stage associated with the primary. In 1433, the system displays the finding as a metastasis to a newly associated primary and updates the data object with updated information on the finding, including biomarkers and pathology / radiology reports. Tracked biomarkers appear accordingly. The system also allows the user to select and track which of many biomarkers are important to the explanation of the cancer. Biomarker information may be presented before the patient summary view. In addition to the patient summary interface views shown in Figures 8A–8C, the patient summary interface may further include interface views that display relevant biomarkers and accept user input to add, modify, or drill down to view more detailed biomarker data.

[0182] In 1429, as described in 1427, sites are labeled, and associations are updated for each finding / site. For example, if a site was previously designated as a metastasis, this is updated so that the site is associated with the primary site. In 1434, the anatomical site appears as the metastatic portion of the newly associated primary cancer. The system moves the anatomical site and corresponding biomarkers and pathology / radiology report findings to the correct primary site. Information such as biomarkers and findings can be stored in association with different primary cancer objects using the data object connections described above with respect to Figures 12 and 13. Any biomarker appears accordingly, associated with the updated primary cancer object. For example, if a stage is associated with a finding, the new primary site is updated by that stage.

[0183] In step 1430, the user retains the current association for the finding. If the current association is retained, in step 1435, the system updates any information about that finding obtained from this new report. The primary cancer association is retained. Tracked biomarkers or stages appear accordingly. If the pathology report has any stage associated with the finding, the stage information does not need to be shown in the patient summary unless the site is the primary site.

[0184] At 1431, the user marks an anatomical site as benign. If the user marks an anatomical site as benign, at 1436, the benign site is no longer relevant to a cancer diagnosis and is therefore removed from the patient summary and patient pathway visualization. At 1437, the report ends and the interface returns to the patient summary view.

[0185] Figure 14D shows the data visualization portion of the workflow. This may include retrieving patient-related data from an integrated patient database and displaying a user interface such as a patient journey interface or patient summary interface.

[0186] In 1442, data is read from the integrated patient database 1409 and displayed in the patient journey. Based on the patient identifier, the patient record in the integrated patient database is identified. This may include a patient route data object 1201, which can be identified by querying the integrated patient database and identifying the patient object corresponding to that identifier. As shown in Figure 12, the patient route data object 1201 is mapped to various different data objects that can be time-stamped and used to visualize the patient journey over time. Examples of patient journey interfaces are shown in Figures 9A–9E.

[0187] In step 1440, data is read from the integrated patient database 1409 and displayed in the patient summary. Based on the patient identifier, the patient record in the integrated patient database is identified. This may include a patient root data object 1201, which can be identified by querying the integrated patient database and identifying the patient object corresponding to that identifier. As shown in Figure 12, the patient root data object 1201 is mapped to various different data objects that are timestamped and can be used to input the patient summary interface. Examples of the patient summary interface are shown in Figures 8A-8C. From the patient summary, the user can perform an update, for example, in step 1444, if the user wants to update the associations from the transfer section of the patient summary. This can trigger the display of the modified UI in step 1446. If the site is created manually, the report is not displayed, and only the association UI is shown. If the site is derived from a report, the report is also viewable.

[0188] In step 1450, data reconciliation is performed. The user can interact with the GUI to establish missing relationships (e.g., associating identified cancerous tumors with specific primary sites). Findings that are not associated are flagged to be reconciled later. Reconciliation identifies data that is not mapped. In one example, a cancerous tumor does not specify an associated primary site. In another example, a surgical history record does not specify whether it is an oncological or non-oncological surgical history. In yet another example, reconciliation can be used to identify the stage of cancer. Reconciliation can be used both to identify and fill in missing relationships, as well as to determine where in the UI it is appropriate to display certain information. Reconciliation can be guided using the interface shown in Figures 7A and 7B. B. Data Management Technology

[0189] Figure 15 illustrates Method 1500 for managing patient data from different sources in an integrated manner. Method 1500 can be performed, for example, by the medical data processing system 200 in Figure 2. Method 1500 can be used to integrate both structured and unstructured data from various sources into an integrated patient database in an integrated and organized manner, and as a result the data can be used to generate the useful visualizations described herein.

[0190] In step 1502, the medical data processing system 200 creates a patient record for the patient in the integrated patient database. The patient record includes a patient identifier and one or more data objects related to the medical data associated with the patient. The patient identifier may be, for example, the patient's name or an alphanumeric identifier for the patient. As described above with respect to Figure 12, the integrated patient database can store multiple data objects of different types that organize different types of medical data associated with the patient. For example, a patient record may include a data object corresponding to a tumor mass, a data object corresponding to a procedure given to the patient, and so on.

[0191] The integrated patient database contains data from multiple sources (for example, data may be ingested from EMR, RIS, user entries, wearable devices, etc.). As mentioned above with respect to Figure 14A, patient records may be created via user input or from information read from external databases such as EMR. Creating a patient record may involve generating and storing data objects, tables, or other records for that patient. The data stored may include information such as patient identifiers, demographic information, and date of birth.

[0192] In step 1504, the medical data processing system 200 retrieves medical records about a patient from an external database. Medical records may include unstructured data such as reports in PDF or image format. Alternatively or additionally, medical records may include structured data such as tables. Medical records may be retrieved from one or more external databases, including, for example, EMR (Electronic Medical Record) systems, PACS (Picture Archiving and Communication Systems), digital pathology (DP) systems, LIS (Laboratory Information Systems) including genomic data, RIS (Radiological Information Systems), patient report outcomes, wearable and / or digital technologies, social media, etc. Medical records may include information such as a name identifying a particular cancerous tumor, a timestamp associated with the report, and other information, as described herein. In some implementations, the medical data processing system 200 retrieves medical records based on a patient identifier. For example, the medical data processing system 200 queries an integrated patient database to identify records that include or are indexed by a patient identifier. Alternatively or additionally, the medical data processing system 200 may periodically read medical records (for example, by downloading data from an external database in batches).

[0193] Medical records may contain structured and / or unstructured data. For example, a medical record about a patient may be structured (e.g., in the first format). Structured data may include a set of data elements associated with a corresponding data type. Data elements may include words or groups of words corresponding to elements in the medical record, such as "MRI on January 5, 2021." Each data element may be labeled and / or stored in association with a corresponding data type that characterizes the data element, such as "primary tumor" or "procedure." Alternatively or additionally, a medical record about a patient may be unstructured (e.g., in the second format). Unstructured data may include data elements without specifying a data type.

[0194] In some embodiments, medical records include unstructured data. The medical data processing system 200 can identify text from unstructured data such as PDFs or images. The medical data processing system 200 can apply a first machine learning model to identify text within medical records. For example, the first machine learning model is or includes an optical character recognition (OCR) model, and the text is identified using OCR.

[0195] The medical data processing system 200 may apply a second machine learning model to correlate identified portions of text with corresponding fields. The medical data processing system 200 may use the second machine learning model to identify data elements such as words or sets of words, analyze unstructured data, and assign data elements to data types. For example, if the data element "colon cancer" is identified, the surrounding words and phrases themselves are analyzed, and the data type "diagnosis" is assigned to the data element.

[0196] In some embodiments, the second machine learning model is or includes a natural language processing (NLP) model. The trained NLP model identifies the data types of text in the unstructured report (for example, the NLP model determines that the text "January 10, 2020" corresponds to the "Date" data type / field, and the text "Radiation" corresponds to the "Procedure Type" data type / field). The medical data processing system 200 may, for example, use NLP to recognize entities from input text strings. The NLP model may identify entities corresponding to predefined medical categories and classifications, such as medical diagnoses, procedures, medications, or specific locations / organs within a patient's body. This can be done in some implementations where a named entity recognition unit trained on medical data recognizes entities corresponding to data types of interest. Each entity may be labeled by a data type indicating a category / classification, specifying a data element or value corresponding to the data being classified. The medical data processing system 200 can then generate structured medical data that associates data types with data elements based on the mapping. Techniques for processing unstructured medical data using machine learning are described in more detail in the above-mentioned International Publication No. 2021 / 046536 brochure.

[0197] In some implementations, the medical data processing system 200 is communicatively coupled to multiple external databases / systems, including EMR, PACS, DP, etc. When these systems make changes to the data associated with one or more patients managed by the medical data processing system 200, the data is transmitted to the medical data processing system 200. The medical data processing system 200 can periodically retrieve medical records from one or more external databases to periodically update the integrated patient database.

[0198] In step 1506, the medical data processing system 200 receives identification of primary cancers associated with a medical record via a graphical user interface (GUI). For example, summarization processing may be performed using an association interface such as the one shown in Figure 7B. The user can associate the identified cancerous masses in the report with specific primary cancer sites. In some embodiments, receiving identification of primary cancers associated with a medical record includes displaying the medical record and a menu configured to receive user input to select one or more primary cancers via the GUI, and receiving user input to select primary cancers via the graphical user interface.

[0199] In some cases, such user selections are performed during the adjustment process, as described above with respect to Figures 7A and 7B. For example, medical records are stored in patient records. The medical data processing system 200 parses the medical records to determine that the patient records are not associated with a particular primary cancer. In response to determining that the patient records are not associated with a particular primary cancer, the medical data processing system 200 displays the medical records and menus and prompts the user to adjust the data via an interface as shown in Figures 7A and 7B.

[0200] Alternatively or additionally, the medical data processing system 200 receives identifications of potential primary cancers associated with medical records from an external database (e.g., EMR). Such identifications received from a remote database may, in some cases, be confirmed via user input to a GUI. For example, the medical data processing system 200 identifies primary cancers by analyzing data elements and data types. A specific data element (e.g., "left breast cancer") may be labeled by a data type that indicates, for example, that the data element corresponds to a primary cancer (e.g., "primary cancer"). In some implementations, a data summarization module 232 extracts medical data from document files and maps the extracted data to specific primary cancers. The mapping may be based on a master structured data list (SDL) that defines a list of data categories for document types of documents.

[0201] The medical data processing system 200 may display a GUI with prompts for the user to confirm the identification of primary cancer (e.g., having pre-filled fields that are highlighted and / or flagged with text prompting the user to confirm or correct the designation of primary cancer). The medical data processing system 200 may then receive user confirmation of primary cancer identification via the GUI. Alternatively or additionally, the medical data processing system 200 may, in some cases, identify primary cancer without user intervention. For example, data elements may be stored in an integrated patient database labeled with data types (e.g., structured fields) indicating the "character" of the tumor, indicating whether the tumor is a primary tumor, a metastatic tumor, or a benign tumor, as shown in Figure 12.

[0202] In some cases, identifying primary cancer may involve the analysis of unstructured data by the medical data processing system 200. For example, medical records are received in an unstructured format that includes unstructured data. The medical data processing system 200 identifies data elements associated with primary cancer from the unstructured data and analyzes the unstructured data to assign the data elements to data types. This can be done using one or more machine learning models, as described above with respect to step 1504.

[0203] In step 1508, the medical data processing system 200 stores medical records linked to the primary cancer object in patient records within the integrated patient database. Storing medical records may include storing identified text in the integrated patient database in relation to identified fields, using the data schema described above with respect to Figures 12 and 13.

[0204] In step 1510, the medical data processing system 200 receives the patient's medical data via user input to the GUI. This may be medical data that is entered directly using the interface described above with respect to Figures 3A to 3H. For example, the user may enter treatment information, diagnostic information, information on metastasis of primary cancer, etc., into the corresponding fields of the GUI.

[0205] In step 1512, the medical data processing system 200 determines that the patient's medical data is associated with primary cancer. For example, data entered by the user into the GUI may be entered into a field designated as primary cancer. As another example, data read from an external database indicates that the patient's medical data is associated with primary cancer. The medical data processing system 200 may compare the field received in 1510 with the corresponding stored data element in the integrated patient database corresponding to the medical record read in 1504.

[0206] In step 1514, the medical data processing system 200 stores the patient's medical data linked to the primary cancer object in the patient record within the integrated patient database. The data elements may be linked using the data schema described above with respect to Figures 12 and 13.

[0207] Data stored in an integrated patient database can be efficiently read and displayed for the user. For example, the medical data processing system 200 reads at least a subset of a patient's medical data from the integrated patient database. The medical data processing system 200 displays at least a subset of the patient's medical data via a user interface to perform clinical decision-making. Displaying may include displaying the user interface on the display components of the medical data processing system 200 itself, or sending commands usable by an external computing device to display the user interface. The displayed information is presented in a user-friendly manner to facilitate clinical decision-making via an interface such as those shown in Figures 7A to 10. C. Technologies for Data Management

[0208] Figure 16 illustrates a method 1600 for managing an integrated patient database using a data schema as shown in Figure 12. The data schema is used to manage patient data, facilitate the efficient generation of the interface views shown herein to support clinical decision-making, and facilitate the export of structured medical data. Method 1600 can be implemented, for example, by the medical data processing system 200 shown in Figure 2.

[0209] In step 1602, the medical data processing system 200 stores patient records, including a network of interconnected data objects, in an integrated patient database. As described above, the integrated patient database may include data from multiple sources, such as data integrated from an EMR system provided to an interface on a remote computer via user input, and data collected from wearable devices.

[0210] In step 1604, the medical data processing system 200 stores a first data object in the patient record within the integrated patient database that corresponds to the data element of a primary cancer tumor mass, the first data object containing attributes that specify the location of the tumor mass. In some implementations, initial data is uploaded to the integrated patient database from one or more of several sources. Once a given data element (e.g., information corresponding to a specific area, such as information characterizing the tumor mass) is taken into the system, the medical data processing system 200 creates a data object to store this information. The data object may be created in response to data obtained from different sources. For example, a user may input data from a user interface. Some data may be automatically taken in from an external system such as EMR. Additional structured data may be automatically summarized from documents (e.g., PDFs) and validated by the user. The data object may further contain one or more data attributes, including those that specify the location of the tumor mass (e.g., right lung, left breast, etc.).

[0211] In step 1606, the medical data processing system 200 receives diagnostic information corresponding to primary cancer from the diagnostic computer. The medical data processing system 200 may also receive information entered by a physician into a user interface provided by the medical data processing system 200 via a network. As a specific example, a physician may enter diagnostic information such as findings and biomarkers using a GUI as shown in Figures 4B and 4C. Such information may be collected in a structured manner based on the input data fields of the GUI.

[0212] In step 1608, the medical data processing system 200 analyzes the diagnostic information to identify correlations between the diagnostic information and tumor masses. This may include, for example, traversing data received from a GUI. Specifically, as shown in Figure 4C, the GUI includes fields for tumor site information 420a and biomarker information 420b. When the medical data processing system 200 receives data from such a GUI, it can determine that a tumor site (e.g., a primary tumor mass) is associated with a biomarker. Alternatively or additionally, the diagnostic information may originate from an unstructured report, and the medical data processing system 200 may apply one or more machine learning models to identify data types and correlations, as described above with respect to step 1504 in Figure 15.

[0213] In step 1610, based on the identification of the correlation between the diagnostic information and the tumor mass, the medical data processing system 200 stores a second data object in the integrated patient database corresponding to the diagnostic information, and the second data object is connected to the first data object via a network of interconnected data objects. The second data object may include one or more attributes such as the stage of the primary cancer, biomarkers, and / or tumor size. The medical data processing system 200 can store the data objects connected to the first data object using the data schema described above with respect to Figures 12 and 13.

[0214] In step 1612, the medical data processing system 200 receives treatment information corresponding to the primary cancer from the diagnostic computer. Treatment information may be received from the diagnostic computer in a similar manner to the diagnostic information, as described above in step 1606. For example, treatment information may be read from the diagnostic computer by input into a GUI, analysis of an unstructured report, or other appropriate means.

[0215] In step 1614, the medical data processing system 200 analyzes the treatment information to identify the correlation between the treatment information and the tumor mass. The medical data processing system 200 may, for example, analyze structured fields and / or perform NLP on text data in a manner similar to that described above with respect to step 1608.

[0216] In step 1616, based on the identification of the correlation between the treatment information and the tumor mass, the medical data processing system 200 stores a third data object corresponding to the treatment information in the integrated patient database, and the third data object is connected to the first data object via a network of interconnected data objects.

[0217] The medical data processing system 200 may also receive and store patient history data, such as surgical history, comorbidities, medications, and other family history, as described above with respect to Figure 12. For example, the medical data processing system 200 receives patient history data. Patient history data may be received from a diagnostic computer (e.g., via direct user input). Alternatively or additionally, patient history data may be received from an external computing system such as an EMR. The medical data processing system 200 analyzes the patient history data to identify correlations between the patient history data and tumor masses (e.g., in a similar manner to described above with respect to step 1608). Based on identifying the correlations between the patient history data and tumor masses, the medical data processing system 200 stores a fourth data object corresponding to the patient history data in the integrated patient database. The fourth data object is connected to the first data object via a network of interconnected data objects.

[0218] The medical data processing system 200 may also receive and store information about additional tumor masses, such as tumor masses at metastatic sites of primary cancer, tumor masses associated with other primary cancers, etc. For example, the medical data processing system 200 receives tumor mass information from a diagnostic computer corresponding to tumor masses at metastatic sites of primary cancer. The user may input the tumor mass information into a GUI or upload a document, and the data may be transmitted to the medical data processor via the GUI in a manner similar to that described above with respect to step 1606. The medical data processing system 200 analyzes the tumor mass information to identify correlations between diagnostic information and tumor masses (for example, in a manner similar to that described above with respect to step 1608). Based on receiving the tumor mass information and identifying the first data object, the medical data processing system 200 stores in the integrated patient database a fifth data object in which the tumor mass information is connected to the first data object via a network of interconnected data objects.

[0219] The medical data processing system 200 may then update the integrated patient database. For example, the medical data processing system 200 may import medical data from an external database. The external database may correspond to one or more of the following: EMR (Electronic Medical Record) systems, PACS (Picture Archiving and Communication Systems), Digital Pathology (DP) systems, LIS (Laboratory Information Systems), and / or RIS (Radiology Information Systems). In some examples, the medical data processing system 200 parses the imported data to identify specific data elements associated with patients and primary cancers. The medical data processing system 200 may, for example, parse structured data received from an EMR or other source to identify fields where it is noted that the data elements describe procedures, tumor masses, or other types of medical data. The structured data may further describe the primary cancer corresponding to the first data object (for example, a field in the imported structured data may indicate that a procedure was applied to a primary cancer). The medical data processing system 200 may then store specific data elements in association with the first data object (for example, a sixth data object). For example, the sixth data object is linked to the first data object in a data schema similar to that depicted in Figure 12.

[0220] As described above with respect to Figure 12, data stored in the integrated patient database can be indexed using timestamps. Timestamps can track when an event occurred (e.g., the day and / or time an MRI was performed, a procedure was received, or a diagnosis was made). Timestamps can further track when the data was integrated into the integrated patient database. For example, when a first data object and a second data object are generated, the medical data processing system 200 generates a first timestamp stored in association with the first data object, indicating the creation time of the first data object. The medical data processing system 200 generates a second timestamp stored in association with the second data object, indicating the creation time of the second data object. These timestamps can then be stored in their respective data objects and used to show the history of database entries. Timestamps tracking when each event occurred can be further used to generate time-series visualizations, such as the patient journey views shown in Figures 9A-9E.

[0221] Data stored in the integrated patient database can be efficiently read and displayed for the user. For example, the medical data processing system 200 reads one or more of the following from the integrated patient database: attributes specifying the location of a tumor mass, diagnostic information, and / or treatment information. Reading attributes may involve querying the integrated patient database. In some embodiments, the medical data processing system 200 traverses connections between data objects to identify associated data objects. For example, the medical data processing system 200 may identify a pointer from a data object corresponding to a tumor mass to another data object corresponding to the treatment of the tumor mass, and read treatment information from there.

[0222] The medical data processing system 200 may display one or more of the following for clinical decision-making: attributes specifying the location of a tumor mass, diagnostic information, and / or treatment information, via a user interface (e.g., a GUI as shown in Figures 8A to 9E). For example, referring to Figure 8B, the patient summary interface 800 displays an attribute in 802 specifying the location of the tumor mass, "Right breast 2:00 location." The GUI 800 also displays diagnostic information on the left side, such as the stage and "invasive ductal carcinoma." The patient summary interface 800 also displays treatment information, such as oncological treatment, in 806. Similarly, in patient journey views 9A to 9E, information including tumor mass location information, diagnostic information, treatment information, and other information may be displayed in a timeline view. Displaying this information may include displaying the GUI on the display components of the medical data processing system 200 itself, or sending commands usable by an external computing device to display the GUI. The displayed information is presented in a user-friendly manner to facilitate clinical decision-making, allowing healthcare professionals to view all the information in one place in an organized way that shows patient responses over time.

[0223] Data stored in an integrated patient database can also be efficiently provided to external systems such as EMRs in a structured format. For example, a medical data processing system 200 identifies data elements and data types associated with a patient from the integrated patient database. The medical data processing system 200 transmits the data elements and data types to the external system in a structured format. As described above, some data objects or data fields may be input by the integration, but these data can often be unstructured or semi-structured. Using the techniques described herein, users and / or machine learning can add more detail or relationships between data objects (e.g., via reconciliation or summarization tools). This facilitates storing data in an integrated patient database with structured information (e.g., characterizing different data elements as different data types). Such structured data can be used to transmit structured data to external systems such as EMRs as needed. D. Technology for displaying patient data via a patient journey interface

[0224] Figure 17 illustrates a method 1700 for displaying patient data to facilitate navigation and presentation via a patient process interface view, as shown in Figures 9A to 9E. The patient process view can provide a view of how the patient responded to treatment over time, with its different types of data organized into rows, which helps clinicians better understand and manage patient treatment. Method 1700 can be implemented, for example, by the medical data processing system 200 in Figure 2.

[0225] In step 1702, the medical data processing system 200 receives patient identification data via a graphical user interface. For example, the user may enter the patient's name or identifier into the portal or select a patient identifier from a displayed menu.

[0226] In step 1704, the medical data processing system 200 receives user input to select a mode from a set of selectable modes of the graphical user interface. For example, as shown in Figures 8A to 10, various modes or interface views are available, including patient journey mode, summary view, and report view. For example, the user selects the patient journey view. The medical data processing system 200 detects that the user clicks the patient journey tab shown near the top of the interface 800 in Figure 8B and transitions the view to the patient journey view.

[0227] In step 1706, based on the identification data and user input, the medical data processing system 200 reads a set of medical data associated with the patient from the integrated patient database. The set of medical data corresponds to the selected mode. For example, the set of medical data corresponds to the patient process mode. The medical data processing system 200 may query the integrated patient database to identify the patient's record (for example, by identifying the patient data record as shown in Figure 12).

[0228] Retrieving a set of medical data may involve querying an integrated patient database to identify patient records for a patient from the integrated patient database. Patient records may include patient objects, such as the patient root data object 1201 shown in Figure 12. Based on the patient object, objects connected to the patient object are identified. Some or all of these objects may then be retrieved for display. For example, as shown in Figure 12, the patient root data object 1201 is connected to many different data objects, each capable of storing various data elements. A patient journey view may be configured to display some of this information, and the system can identify these based on pre-configured data object types and / or elements. A list of data object types displayed in the patient journey may be stored in a configuration file. Based on the object types in the list, instances of these object types may be retrieved, for example, as long as they have a timestamp within a specified time window. For example, as shown in Figures 9A-9E, some data is not within the currently displayed time window and is not fetched for display at a given time. Other data, such as benign tumors, may be stored in the integrated patient database but are not displayed in the patient journey UI.

[0229] The retrieved data may include various data objects and elements described herein with respect to, for example, Figure 12. For example, a set of medical data may correspond to a procedure object in the integrated patient database (for example, retrieved from or in association with), where the procedure object stores the procedure type, date, and response. Alternatively or additionally, a set of medical data may correspond to a diagnostic findings object in the integrated patient database, where the diagnostic findings object stores biomarker data, staging data, and / or tumor size data. Alternatively or additionally, a set of medical data may correspond to a history object in the integrated patient database, where the history object stores surgical history, allergies, and / or family medical history.

[0230] In step 1708, the medical data processing system 200 displays a user-selectable set of objects on a timeline via a graphical user interface, where the objects are organized into rows, each row corresponding to a different category from several categories, the categories including lesions, diagnoses, and treatments. This may correspond to the patient process views shown in Figures 9A–9E. The medical data processing system 200 may read this information from an integrated patient database and use it to display the patient process views. For example, based on the object types defined above with respect to Figure 12, the corresponding row in the patient process interface is identified for a particular object. Based on the timestamp associated with that object, the object is placed at a specific time on the timeline of the patient process view for the identified row. As a concrete example, a biomarker object is placed at a specific time in the biomarker row. This is repeated across each element of the medical data read in 1606, resulting in GUI views such as those shown in Figures 9A–9E.

[0231] The graphical user interface may further include a ribbon displayed above the timeline, which shows a subset of objects flagged as significant. For example, as shown in Figure 9B, there is a summary ribbon 902 that highlights important events. The summary ribbon can highlight important events in an easily visible location, and the user can drill down to see the events and the order in which they occurred in more detail using the timeline view below. This can provide an improved user experience and help facilitate clinical decision-making by giving the user an important event and a temporal view of those events.

[0232] The graphical user interface can receive user interaction and prompt the display of additional information, including reports. Reports can be viewed in detailed or simplified formats. In some implementations, the user can hover over an object in the timeline (e.g., MRI924 shown in Figure 9B), and the medical data processing system 200 reads and displays the report. The medical data processing system 200 can detect user interaction with an object in a set of objects, such as MRI924 shown in Figure 9B. The medical data processing system 200 identifies and reads the corresponding report from the integrated patient database. For example, as shown in Figure 12, a data record may include a report 1204 linked to different data objects, such as a patient root data object 1201 for the patient, and one for diagnostic findings 1205. The medical data processor can traverse such connections in the integrated patient database to identify reports associated with objects in the patient summary graphical user interface. The medical data processor can then display the report via the graphical user interface. This provides a convenient way to drill down into different objects displayed in the patient summary view.

[0233] From the patient journey view, the user can switch to other available views, such as the patient summary view or the patient report view. For example, the graphical user interface further includes elements for navigating to a second interface view, such as the selectable report element 812 and summary element 815 shown in Figure 8B. The medical data processing system 200 detects user interaction with elements for navigating to a second interface view. For example, the second interface view is a summary view, as shown in Figures 7A and 8A-8B, and the second interface view displays oncological summary data. As another example, the second interface view is a report view, which displays a specific report or a list of reports, as shown in Figure 10, for example. E. Technology for managing and displaying data from multiple tumor masses.

[0234] Figure 18 illustrates a method 1800 for displaying patient data to facilitate navigation and presentation via a parallel tumor mass view, such as that shown in Figure 8C. Method 1800 can be performed, for example, by the medical data processing system 200 shown in Figure 2.

[0235] In step 1802, the medical data processing system 200 stores patient records in an integrated patient database. A patient record includes multiple data objects, including a first primary cancer data object that stores data elements corresponding to a first tumor mass of the patient, and a second primary cancer data object that stores data elements corresponding to a second tumor mass of the patient. For example, one object may be stored in association with a primary cancer in the right breast, and another data object may be stored in association with another primary cancer in the right lung. As shown in Figure 13, the data schema used by the medical data processing system 200 may include multiple objects for multiple primary cancers, each having its own data object such as cancer 1 1302 and cancer 2 1304.

[0236] As mentioned above, the integrated patient database includes data from multiple sources, including EMR (Electronic Medical Records) systems, PACS (Picture Archiving and Communication Systems), Digital Pathology (DP) systems, LIS (Laboratory Information Systems), RIS (Radiology Information Systems), patient report outcomes, wearable devices, and social media websites.

[0237] In step 1804, the medical data processing system 200 renders and displays a graphical user interface. The graphical user interface includes a patient summary. As shown in Figures 8A to 8C, the patient summary view may include information that summarizes patient data in patient records within the integrated patient database. The patient summary view may be displayed as described above with respect to Figure 17.

[0238] In step 1806, the medical data processing system 200 detects user interaction with elements of the graphical user interface. For example, the patient summary view displays information about primary cancers and elements for displaying more information about one or more primary cancers. As a specific example, the patient summary view shown in Figure 8B includes a box 802 containing information about two primary cancers, "breast cancer" and "lung cancer," along with an element 805 that the user can interact with to view more information. In some implementations, there is an element configured to initiate the display of information about multiple primary cancers. Alternatively, the graphical user interface may display a first element 805 when viewing a first primary cancer (e.g., breast cancer) and a second element 805 when viewing a second primary cancer (e.g., lung cancer). In this case, the user can click each of the two buttons in turn.

[0239] In some implementations, the medical data processing system 200 identifies several primary cancers and displays information about each identified primary cancer. For example, the medical data processing system 200 stores each tumor mass, represented as a separate data object with structured data fields indicating the characteristics of the tumor mass, as shown in Figures 12 and 13. This data schema allows the medical data processing system 200 to count the number of primary or metastatic tumors as needed.

[0240] In step 1808, in response to the detection of user interaction, the medical data processing system 200 reads data elements from the integrated patient database, specifically from the first primary cancer data object and the second primary cancer data object of the patient record. Based on the elements interacted with in step 1806, the medical data processing system 200 may identify one or more primary cancer data objects and query the integrated patient database to read data elements associated with the corresponding primary cancer data object. In some implementations, each tumor mass is represented as an independent data object having structured data fields that indicate information about the tumor mass, such as its characteristics (e.g., primary, metastatic, etc.), as shown in Figure 12. This allows the medical data processing system 200 to identify primary cancer data objects within the integrated patient database.

[0241] In step 1810, the medical data processing system 200 renders a first modal corresponding to the patient's first primary cancer and a second modal corresponding to the patient's second primary cancer. Rendering modals may include generating graphics to overlay on the current GUI (for example, as a popup on the patient summary view).

[0242] In step 1812, the medical data processing system 200 displays a first modal and a second modal side by side in the graphical user interface. The side-by-side modals may include two pop-up windows superimposed on the patient summary view, as shown in Figure 8C. As shown in Figure 8C, the first and second modals can provide summaries of key information about each primary cancer. The information displayed in the first and second modals may include a set of biomarkers with timestamps, staging information, and metastatic site information, as shown in Figure 8C. Presenting primary cancers side by side in a summary can help clinicians, such as oncologists, see how multiple primary cancers are progressing simultaneously. Displaying modals may include displaying modals on the display components of the medical data processing system 200 itself, or sending commands usable by an external computing device to display the GUI. F. Overview of the Diagnostic Workflow

[0243] Figures 19A and 19B illustrate an example of an oncology workflow that can be implemented by the oncology workflow application 222. The goal of the workflows in Figures 19A and 19B is to maintain a detailed, curated table of relevant radiation, treatment, and pathological findings related to the primary tumor and its associated metastatic lesions, which are updated throughout the course of cancer treatment and other phases of the patient's journey. The primary tumor and metastatic lesions are treated as target lesions. Measurements can be incorporated as structured data, which can make decisions about tumor response or progression more objective and better inform decisions about the patient's clinical status. As findings change, they are recorded iteratively. Figure 19A shows an exemplary chart 1900 showing the change in lesion size over time for different target lesions that may be obtained from the exemplary workflow.

[0244] Figure 19B shows flowchart 1901 of an example oncology workflow that enables a medical oncologist to select a target lesion for monitoring and response assessment. Referring to Figure 19B, in step 1902, a diagnostic procedural finding, characteristics of the finding, and the operator's comments / diagnostic interpretation of the finding are recorded based on data received from the medical data processing system 200. In step 1904, a decision is made as to whether the finding (e.g., a lesion) indicates a primary tumor. If the lesion is neither a primary tumor (step 1904) nor a metastasis (step 1906), the iteration can be terminated, and step 1902 is then repeated to record a new diagnostic procedural finding, characteristics of the finding, and the operator's comments / diagnostic interpretation of the finding. If the lesion is a metastasis (in step 1906), in step 1908, the finding may be assigned as a metastasis in the data entry interface 300. Metastasis assignment can be performed on patient summary page 311 as shown in Figure 3F, the patient's oncological data can be updated, and then the iteration can be terminated, and step 1902 can be repeated.

[0245] On the other hand, if the lesion is a primary tumor (step 1904), in step 1910, the lesion is assigned as a primary tumor via the data input interface 300, as shown in operation 340 in Figure 3D. If the diagnosis is confirmed (in step 1912), the patient's oncology data may be updated. If the diagnosis is not confirmed (in step 1912), in step 1914, the “Diagnosis Pending” flag 321 in Figure 3B may remain asserted. In both cases, the iteration can be terminated, and step 1902 can be repeated. Furthermore, from step 1910 to step 1920, a decision can be made regarding whether a biopsy was performed. If the findings were biopsied (step 1920), in step 1922, the pathological findings may be recorded as part of the patient's structured data. Subsequently, step 1902 is repeated to record new diagnostic findings, characteristics of the findings, and the practitioner's comments / diagnostic interpretation of the findings.

[0246] Figures 20A and 20B show flowchart 2000 of another exemplary oncology workflow. The oncology workflow in flowchart 2000 enables oncologists (and their representatives) to manage cancer patients from suspected to long-term through treatment and follow-up by leveraging the full context of patient information. Referring to Figure 20A, in step 2002, the data acquisition module 230 can collect medical data of a patient suspected of having cancer via the portal 220. In step 2004, the oncologist can analyze the data to determine whether the patient has cancer. If cancer is not confirmed (in steps 2006 and 2008), the oncology workflow can be terminated. However, if cancer is confirmed, in step 2010, a decision is made as to whether the clinical findings suggest a single primary cancer.

[0247] Referring to Figure 20B, if clinical findings suggest a single primary cancer (step 2010), in step 2012, biopsy and post-processing data can be analyzed to confirm the primary tumor. If there is no evidence of metastasis (step 2014), in step 2016, it may be concluded that the patient has a single primary cancer. On the other hand, if there is evidence of metastasis (in step 2014) and all metastases are associated with a known primary (in step 2018), in step 2020, it may be concluded that the metastases originate from a single primary cancer.

[0248] If clinical findings do not suggest a single primary cancer (step 2010), or if metastases are not associated with a known primary site (step 2018), in step 2022, biopsy and post-processing data can be analyzed to determine whether there are multiple primary sites. If the biopsy and post-processing data in step 2022 confirm the presence of only a single primary site, in step 2020, it may be determined that the metastases originate from a single primary cancer. However, if the biopsy and post-processing data in step 2022 cannot confirm the presence of only a single primary site, the workflow can proceed through a different route. For example, if clinical data suggest a carcinoma at an unknown primary site (step 2026) and the biopsy shows similar histology (step 2028), in step 2020, it may be determined that the metastases originate from a single primary cancer. However, if clinical data suggest a carcinoma at an unknown primary site (step 2026) and the biopsy shows a different histological type (step 2028), in step 2030, it may be determined that the patient has a carcinoma at an unknown primary site. Furthermore, returning to step 2024, if the biopsy shows two histological types suggesting two different primary sites (step 2032), and in step 2034, the user flags two primary cancers (for example, by assigning masses to primary cancer sites as shown in Figures 3D and 3E), then in step 736, it may be determined that the patient has two primary cancers. A specific diagnostic result (e.g., discovery of a tumor mass) may be associated with a second primary tumor, as shown in Figure 3F. G. Methods for processing medical data to facilitate clinical decision-making.

[0249] Figure 21 shows a method 2100 for processing medical data to facilitate clinical decision-making. Method 2100 can be performed, for example, by the medical data processing system 200 shown in Figure 2.

[0250] In step 2102, the medical data processing system 200 receives the input medical data of the patient via a portal (for example, portal 220). The patient data can be derived from various data sources (one or more medical institutions) including, for example, an EMR (Electronic Medical Record) system, a PACS (Picture Archiving and Communication System), a digital pathology (DP) system, a LIS (Laboratory Information System) including genomic data, a RIS (Radiation Information System), patient report outcomes, wearable and / or digital technologies, social media, etc.

[0251] In some examples, the portal can provide a data input interface including various fields for receiving the input medical data, and structured medical data can be generated based on the mapping between the fields and the data. The structured medical data can include various information related to the diagnosis of tumors, such as tumor site, stage classification, pathological information (for example, biopsy results), diagnostic procedures, and biomarkers for both the primary tumor and additional tumor sites (for example, due to metastasis from the primary tumor). The portal can display the structured data in the form of a patient summary. The portal can also organize the display of the structured data into pages, each page being associated with a specific primary tumor site and including fields of information for the associated primary tumor site, and can be accessed by tabs. Based on the detection of user input in a specific field within the page of the first primary tumor (for example, designation of an additional tumor site as a new primary tumor), the portal can create an additional page for the second primary tumor and input the fields of the newly created page for the second primary tumor based on the additional tumor site information entered in the page of the first primary tumor. In some examples, the portal processor can also enable the user to select an additional tumor mass found during the diagnostic procedure of the primary tumor and associate that mass with the second primary tumor to represent a metastatic case. Based on the detection of the association, the medical data processor can transfer all the diagnostic results of the additional tumor from the page of the first primary tumor to the newly created page of the second primary tumor.

[0252] In some examples, the portal also enables a user to import document files (e.g., pathology reports, doctor's notes, etc.) from the aforementioned data sources. Then, the medical data summarization module can extract various structured medical data from the document files. The structured medical data can be extracted, for example, based on performing natural language processing (NLP) operations, rule-based extraction operations, etc. on the text included in the document files. The medical data summarization module also enables manual extraction of structured medical data from document files via the portal. And the portal can display the medical data extracted in addition to the document files.

[0253] For example, the portal can overlay the text of the file by highlighting marking. The portal can also display a text box containing medical data extracted from the text on the highlighted text. Further, the structured medical data can also be extracted from various metadata of the document file such as the date of the file, the category of the document file (e.g., pathology report for a clinician's note), the clinician who authored / signed off on the document file, and the type of procedure associated with the content of the document file (e.g., biopsy, imaging, or other diagnostic steps). Then, the portal can input various fields of the page based on the extracted data. Various enhancement operations can also be performed on the extracted data to improve the quality of the extracted medical data. One enhancement operation can include normalization operations to normalize various numerical values (e.g., weight, tumor size, etc.) included in the extracted medical data to standardized units, to correct data errors, or to replace non-standard terms provided by the patient with standardized terms based on various medical standards / protocols such as the International Classification of Diseases (ICD) and the Systematized Nomenclature of Medicine (SNOMED). Then, the enhanced extracted medical data can be stored in the integrated patient database as part of the patient's structured medical data (e.g., structured oncology data).

[0254] In step 2104, structured medical data is generated based on the input medical data. The structured medical data is generated to support oncology workflow operations for generating diagnostic results that include one of the following: patients without cancer, patients with primary cancer, patients with multiple primary cancers, or patients with cancer at an unknown primary site. Examples of oncology workflows are shown in Figures 19A to 20B. The oncology workflow can also perform diagnostic operations based on the structured medical data. In one example, a diagnostic operation can be performed to determine whether biopsy results are for the same primary tumor or different tumors, and to track the size of the primary tumor to assess the tumor's response to a particular treatment. In another example, a diagnostic operation can be performed to determine whether the patient has a single primary tumor site, multiple primary tumor sites, or an unknown primary site. The results of the diagnostic operation can then be recorded and / or displayed in the portal in terms of time as part of the patient's medical journey, enabling oncologists or their representatives to manage cancer patients from suspected cancer to cancer patients over the long term through treatment and follow-up. Diagnostic results can also be used to support other medical applications, such as quality of care assessment tools to evaluate the quality of care provided to patients, and medical research tools to determine correlations between various patient information (e.g., demographic information) and patient tumor information (e.g., prognosis or expected survival).

[0255] In step 2106, the portal can display a history of the patient's diagnostic results over time to enable clinical decisions to be made based on the diagnostic history. For example, as shown in Figures 9A to 9E, the portal can display a timeline representing the patient's medical journey, which may include a history of primary tumor size, a history of other diagnostic results, etc. This enables clinicians to make clinical decisions, for example, about the procedures administered to the patient. V. Exemplary Computer Systems

[0256] Any computer system referred to herein may utilize any appropriate number of subsystems. An example of such subsystems is shown in Figure 22 in computer system 2200. In some embodiments, the computer system includes a single computer device, where subsystems are components of the computer device. In other embodiments, the computer system may include multiple computer devices, each having internal components, each being a subsystem. The computer system may include desktop and laptop computers, tablets, mobile phones, and other mobile devices. In some embodiments, cloud infrastructure (e.g., Amazon Web Services), graphical processing units (GPUs), etc., may be used to implement the disclosed technology.

[0257] The subsystems shown in Figure 22 are interconnected via a system bus 75. Additional subsystems are shown, including a printer 74, a keyboard 78, a storage device 79, and a monitor 76 connected to a display adapter 82. External and input / output (I / O) devices coupled to the I / O controller 71 can be connected to the computer system by any number of means known in the art, such as I / O ports 77 (e.g., USB, FireWire®). For example, an I / O port 77 or an external interface 81 (e.g., Ethernet, Wi-Fi, etc.) can be used to connect the computer system 10 to a wide area network such as the Internet, a mouse input device, or a scanner. The interconnection via the system bus 75 allows the central processor 73 to communicate with each subsystem and control the execution of multiple instructions from the system memory 72 or storage device 79 (e.g., a fixed disk such as a hard drive, or an optical disk), as well as the exchange of information between subsystems. The system memory 72 and / or storage device 79 may embody computer-readable media. Another subsystem is a data acquisition device 85, such as a camera, microphone, accelerometer, and others. Any of the data described herein may be output from one component to another and to the user.

[0258] A computer system may include multiple identical components or subsystems that are connected together, for example, by an external interface 81 or an internal interface. In some embodiments, computer systems, subsystems, or devices may communicate over a network. In such examples, one computer may be considered a client, another a server, and each of them may be part of the same computer system. Clients and servers may each include multiple systems, subsystems, or components.

[0259] Aspects of the embodiments may be implemented in the form of control logic using hardware (e.g., application-specific integrated circuits or field-programmable gate arrays) and / or computer software with a generally programmable processor in a modular or integrated form. As used herein, the processor includes a single-core processor on a single integrated chip, a multi-core processor, or a multiprocessing unit on a single circuit board or a networked unit. Based on the disclosures and teachings provided herein, those skilled in the art will know and understand other means and / or methods for carrying out embodiments of the invention using hardware and combinations of hardware and software.

[0260] Any software component or function described in this application may be implemented as software code executed by a processor using any suitable computer language, such as Java, C, C++, C#, Objective-C, Swift, or a scripting language, such as Perl or Python, using conventional or object-oriented techniques. The software code may be stored as a set of instructions or commands on a computer-readable medium for storage and / or transmission. Suitable non-temporary computer-readable media may include random access memory (RAM), read-only memory (ROM), hard drives, magnetic media such as floppy disks, optical media such as compact discs (CDs) or DVDs (digital multipurpose discs), or flash memory. The computer-readable medium may be any combination of such storage devices or transmission devices.

[0261] Such programs may also be encoded and transmitted using carrier signals adapted for transmission over wired, optical, and / or wireless networks compliant with various protocols, including the Internet. Thus, computer-readable media may be created using data signals encoded with such programs. Computer-readable media encoded with program code may be packaged with compatible devices or provided separately from other devices (e.g., via internet download). Any such computer-readable media may be present on or within individual computer products (e.g., hard drives, CDs, or complete computer systems), and may also reside on or within different computer products within a system or network. Computer systems may include monitors, printers, or other suitable displays for providing users with any of the results described herein.

[0262] Any of the methods described herein may be performed entirely or partially using a computer system including one or more processors that can be configured to perform the steps described above. Thus, each embodiment may target a computer system configured to perform any of the steps of the methods described herein using different components that potentially perform each of the steps or each of the groups of steps. Although presented as referenced steps, the steps of the methods herein may be performed simultaneously or in different orders. Furthermore, some of these steps may be used in conjunction with some of other steps from other methods. Also, all or some of the steps may be optional. Furthermore, any step of any method may be performed using a module, unit, circuit, or other means for performing these steps.

[0263] The specific details of particular embodiments may be combined in any suitable manner without departing from the spirit and scope of the embodiments of the present invention. However, other embodiments of the present invention may focus on specific embodiments relating to each individual aspect, or on specific combinations of these individual aspects.

[0264] The above description of exemplary embodiments of the present invention is presented for illustrative and illustrative purposes only. It is not intended to be exhaustive or to limit the invention to the form described, and numerous modifications and variations are possible in light of the teachings given above.

[0265] The use of “a,” “an,” or “the” is intended to mean “one or more” unless otherwise explicitly stated. The use of “or” is intended to mean “inclusive OR” rather than “exclusive OR” unless otherwise explicitly stated. A reference to a “first” component does not necessarily require a second component to be produced. Furthermore, a reference to a “first” or “second” component does not limit the referred component to a specific position unless explicitly stated.

[0266] All patents, patent applications, publications, and descriptions described herein are incorporated herein by reference in their entirety for all purposes. None of them are considered prior art.

Claims

1. A method for managing medical data, which is performed by a server computer. Creating a patient record for a patient in an integrated patient database, wherein the patient record includes a patient identifier and one or more data objects relating to medical data associated with the patient, and the integrated patient database creates a patient record that includes data from multiple sources. Retrieving medical records for the patient from an external database, wherein the medical records for the patient are in a first format including a set of data elements associated with a corresponding data type, Receiving the identification of the primary cancer associated with the medical record via a graphical user interface (GUI), Identifying the primary cancer by analyzing the aforementioned data elements and data types, The GUI includes a prompt for the user to confirm the identification of the primary cancer, Receiving user confirmation of the identification of the primary cancer via the GUI, Including receiving, In response to receiving the identification of the primary cancer, create a primary cancer object in the patient record, wherein the primary cancer object has a field containing the primary cancer. The medical record linked to the primary cancer object in the patient record within the integrated patient database is stored. The medical data of the patient is received via user input to the GUI, The determination that the patient's medical data is associated with the primary cancer, A method comprising storing the medical data of the patient linked to the primary cancer object in the patient record in the integrated patient database.

2. The medical record is the first medical record, and the method further, Receiving a second medical record for the patient, wherein the second medical record is in a second format including unstructured data, From the aforementioned unstructured data, identify the data elements associated with the primary cancer, Analyzing the unstructured data in order to assign the data elements to data types, The method according to claim 1, comprising storing the data element linked to the primary cancer object in the patient record in the integrated patient database, based on the assigned data type and the identification that the data element is associated with the primary cancer.

3. Receiving the identification of the primary cancer associated with the medical record, The GUI is configured to display a menu that receives user input to select the medical record and one or more primary cancers, The method according to claim 1, comprising receiving user input to select the primary cancer via the GUI.

4. The patient record is to store the medical record, This further includes parsing the medical records and determining that the patient records are not associated with a specific primary cancer, The method according to claim 3, wherein displaying the medical record and the menu is in response to determining that the patient record is not associated with a particular primary cancer.

5. The aforementioned medical record includes unstructured data, The method described above is Applying a first machine learning model to identify the text within the medical record, The method further includes applying a second machine learning model to correlate the identified portion of the text with a corresponding field. The method according to claim 1, further comprising storing the medical record in association with the identified text and storing it in the integrated patient database.

6. The first machine learning model includes an optical character recognition (OCR) model, The method according to claim 5, wherein the second machine learning model includes a natural language processing (NLP) model.

7. Reading at least a subset of the patient's medical data from the integrated patient database, The method according to claim 1, further comprising displaying at least a subset of the patient's medical data via a user interface in order to perform clinical decision-making.

8. The aforementioned external database corresponds to at least one of the following: EMR (Electronic Medical Records) system, PACS (Picture Archiving and Communication System), Digital Pathology (DP) system, LIS (Laboratory Information System), and RIS (Radiology Information System). The method according to claim 1, wherein the medical record is read based on the patient's identifier.

9. A computer-readable medium for storing a plurality of instructions for controlling a computer system to perform the method according to any one of claims 1 to 8.

10. It is a system, A system comprising one or more processors for executing instructions stored in a computer-readable medium as described in claim 9.

Images