Systems and methods for processing electronic image s for computational assessment of disease

Abstract

To provide systems and methods for determining cancer detection results on the basis of digital pathology images.SOLUTION: Systems and methods includes: receiving a digital image corresponding to a target specimen associated with a pathology category, the digital image being an image of tissue specimen; determining a detection machine learning model, the detection machine learning model being generated by processing a plurality of training images to output cancer qualification and further cancer quantification if the cancer qualification is a confirmed cancer qualification; providing the digital image as an input to the detection machine learning model; receiving one of a pathological complete response (pCR) cancer qualification and a confirmed cancer quantification as an output from the detection machine learning model; and outputting the pCR cancer qualification or the confirmed cancer quantification.SELECTED DRAWING: None

Description

[Technical Field]

[0001] (Related Applications) This application claims priority to U.S. Provisional Application No. 62 / 957,523, filed January 6, 2020, the entire disclosure of which is incorporated herein by reference in its entirety.

[0002] Various embodiments of the present disclosure generally relate to determining the presence or absence of disease, such as cancer cells. More specifically, certain embodiments of the present disclosure relate to determining at least one of pathological complete response (pCR) or minimal residual disease (MRD) based on cells in whole slide images (WSI). [Background technology]

[0003] Pathological complete response (pCR) can refer to the absence of residual invasive and intraepithelial cancer cells on histology microscope slides of excised tissue samples. pCR can be used as a surrogate endpoint to determine whether a patient is responding to therapy (e.g., therapy related to breast cancer, prostate cancer, bladder cancer, colon cancer, etc.). For example, pCR for breast cancer can be defined as the absence of all signs of invasive cancer in the breast tissue and lymph nodes removed during post-treatment surgery.

[0004] Minimal residual disease (MRD) can refer to submicroscopic disease, such as disease that remains dormant within a patient but may eventually lead to recurrence. In cancer treatment, MRD can provide information about whether treatment has eliminated cancer or whether traces remain. Currently, pCR / MRD is determined manually by pathologists examining tissue samples under a microscope to determine whether cancer cells remain or whether all cancer cells have been eliminated. This detection task can be subjective and difficult due to various definitions of pCR / MRD and the therapeutic effects of neoadjuvant therapy, which can alter the morphology of cancerous and benign tissue. The level of subjectivity and difficulty can increase when treatment damage is present.

[0005] The foregoing general description and the following detailed description are exemplary and explanatory only and are not limiting of the present disclosure. The background provided herein is generally for the purpose of providing a context for the present disclosure. Unless otherwise indicated herein, the material described in this section is not prior art to the claims in this application, and is not admitted to be prior art or an indication of prior art by inclusion in this section. Summary of the Invention [Means for solving the problem]

[0006] According to one aspect of the present disclosure, a system and method for determining cancer detection results based on digital pathology images is disclosed.

[0007] A method for outputting a cancer detection result includes receiving a digital image corresponding to a target sample associated with a pathology category, the digital image being an image of a tissue sample; determining a detection machine learning model, the detection machine learning model being generated by processing a plurality of training images, outputting a cancer qualification and, if the cancer qualification is a confirmed cancer qualification, further outputting a cancer quantification; providing the digital image as an input to the detection machine learning model; receiving one of a pathological complete response (pCR) cancer qualification or a confirmed cancer quantification as an output from the detection machine learning model; and outputting the pCR cancer qualification or the confirmed cancer quantification.

[0008] A system for outputting cancer detection results includes a memory that stores instructions; and a processor that executes the instructions to perform a process including receiving a digital image corresponding to a target sample associated with a pathology category, the digital image being an image of a tissue sample; determining a detection machine learning model, the detection machine learning model being generated by processing a plurality of training images, outputting a cancer qualification and, if the cancer qualification is a confirmed cancer qualification, further outputting a cancer quantification; providing the digital image as an input to the detection machine learning model; receiving one of a pathological complete response (pCR) cancer qualification or a confirmed cancer quantification as an output from the detection machine learning model; and outputting the pCR cancer qualification or the confirmed cancer quantification.

[0009] A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method for generating a specialized machine learning model, the method including: receiving a digital image corresponding to a target sample associated with a pathology category, the digital image being an image of a tissue sample; determining a detection machine learning model, the detection machine learning model being generated by processing a plurality of training images, outputting a cancer qualification and, if the cancer qualification is a confirmed cancer qualification, further outputting a cancer quantification; providing the digital image as an input to the detection machine learning model; receiving one of a pathological complete response (pCR) cancer qualification or a confirmed cancer quantification as an output from the detection machine learning model; and outputting the pCR cancer qualification or the confirmed cancer quantification. The present invention provides, for example, the following items. (Item 1) 1. A computer-implemented method for processing an electronic image, the method comprising: receiving a digital image corresponding to a target sample associated with a pathology category, the digital image being an image of a tissue sample; determining a detection machine learning model, the detection machine learning model being generated by processing a plurality of training images, outputting a cancer qualification and, if the cancer qualification is a confirmed cancer qualification, further outputting a cancer quantification; providing the digital image as an input to the detection machine learning model; receiving one of a pathological complete response (pCR) cancer qualifier or a confirmed cancer quantifier as output from the detection machine learning model; outputting said pCR cancer qualification or said confirmed cancer quantification; 11. A computer-implemented method comprising: (Item 2) 2. The computer-implemented method of claim 1, wherein receiving confirmed cancer quantification comprises receiving minimal residual disease (MRD) cancer quantification. (Item 3) Item 3. The computer-implemented method of item 2, wherein the MRD cancer qualifier is protocol-specific. (Item 4) Item 3. The computer-implemented method of item 2, wherein the MRD cancer qualifier corresponds to the number of cancer cells below an MRD threshold. (Item 5) The pCR cancer qualifier corresponds to the digital image having zero detectable cancer cells. Item 1. The computer-implemented method of item 1. (Item 6) Item 10. The computer-implemented method of item 1, wherein the plurality of training images comprises images with a therapeutic effect. (Item 7) Item 10. The computer-implemented method of item 1, wherein the detection machine learning model comprises a treatment effect machine learning model. (Item 8) 8. The computer-implemented method of claim 7, wherein the treatment effect machine learning model is initialized by using a trained machine learning model trained based on the plurality of training images, the plurality of training images excluding images with a treatment effect. (Item 9) 2. The computer-implemented method of claim 1, wherein the digital image is from a pathology category, and the pathology category is selected from one or more of histology, cytology, frozen section, immunohistochemistry (IHC), immunofluorescence, hematoxylin and eosin (H&E), hematoxylin only, molecular pathology, or 3D imaging. (Item 10) 1. A system for processing electronic images, said system comprising: at least one memory for storing instructions; at least one processor, wherein the at least one processor executes the instructions; receiving a digital image corresponding to a target sample associated with a pathology category, the digital image being an image of a tissue sample; determining a detection machine learning model, the detection machine learning model being generated by processing a plurality of training images, outputting a cancer qualification and, if the cancer qualification is a confirmed cancer qualification, further outputting a cancer quantification; providing the digital image as an input to the detection machine learning model; receiving one of a pathological complete response (pCR) cancer qualifier or a confirmed cancer qualifier as output from the detection machine learning model; outputting the pCR cancer qualification or the confirmed cancer qualification; at least one processor that performs operations including A system comprising: (Item 11) 11. The system of claim 10, wherein receiving a confirmed cancer quantification further comprises receiving a minimal residual disease (MRD) cancer quantification. (Item 12) Item 12. The system of item 11, wherein MRD cancer qualifiers are protocol specific. (Item 13) Item 11. The system of item 10, wherein the confirmed cancer qualifi- cation corresponds to detecting a threshold number of cancer cells. (Item 14) Item 11. The system of item 10, wherein the plurality of training images comprises images with therapeutic effects. (Item 15) Item 11. The system of item 10, wherein the detection machine learning model comprises a treatment effect machine learning model. (Item 16) Item 16. The system of item 15, wherein the treatment effect machine learning model is initialized by using a trained machine learning model trained based on the plurality of training images, the plurality of training images excluding images with treatment effect. (Item 17) 11. The system of claim 10, wherein the digital image is from a pathology category, and the pathology category is selected from one or more of histology, cytology, frozen section, immunohistochemistry (IHC), immunofluorescence, hematoxylin and eosin (H&E), hematoxylin only, molecular pathology, or 3D imaging. (Item 18) A non-transitory computer-readable medium having instructions stored thereon that, when executed by a processor, cause the processor to perform operations for processing an electronic image, the operations including: receiving a digital image corresponding to a target sample associated with a pathology category, the digital image being an image of a tissue sample; determining a detection machine learning model, the detection machine learning model being generated by processing a plurality of training images, outputting a cancer qualification and, if the cancer qualification is a confirmed cancer qualification, further outputting a cancer quantification; providing the digital image as an input to the detection machine learning model; receiving one of a pathological complete response (pCR) cancer qualifier or a confirmed cancer quantifier as output from the detection machine learning model; outputting said pCR cancer qualification or said confirmed cancer quantification; 1. A non-transitory computer-readable medium comprising: (Item 19) 20. The non-transitory computer-readable medium of item 18, wherein receiving confirmed cancer quantification comprises receiving minimal residual disease (MRD) cancer quantification. (Item 20) 20. The non-transitory computer-readable medium of claim 18, wherein the detection machine learning model comprises a treatment effect machine learning model. [Brief explanation of the drawings]

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various exemplary embodiments and, together with the description, serve to explain the principles of the disclosed embodiments.

[0011] [Figure 1A] FIG. 1A illustrates an example block diagram of a system and network for implementing a detection tool involving digital images, according to an example embodiment of the present disclosure.

[0012] [Figure 1B] FIG. 1B illustrates an example block diagram of a machine learning module according to an example embodiment of the present disclosure.

[0013] [Figure 2] FIG. 2 is a flowchart illustrating an example method for using a detection machine learning model, according to an example embodiment of the present disclosure.

[0014] [Figure 3] FIG. 3 illustrates an example block diagram of a training module according to an example embodiment of the present disclosure.

[0015] [Figure 4] FIG. 4 illustrates a schematic diagram for detecting cancer cells using a detection module, according to an exemplary embodiment of the present disclosure.

[0016] [Figure 5] FIG. 5 is a flowchart of an exemplary embodiment of a detection implementation, according to an exemplary embodiment of the present disclosure.

[0017] [Figure 6] FIG. 6 is a schematic illustration of experimental results using a detection model, according to an exemplary embodiment of the present disclosure.

[0018] [Figure 7] FIG. 7 depicts an example system that may implement the techniques presented herein. DETAILED DESCRIPTION OF THE INVENTION

[0019] Description of the embodiment Reference will now be made in detail to the exemplary embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.

[0020] The systems, devices, and methods disclosed herein are described in detail by way of example and with reference to the Figures. The examples discussed herein are examples only and are provided to aid in the explanation of the apparatus, devices, systems, and methods described herein. None of the features or components shown in the drawings or discussed below should be construed as essential for any particular implementation of any of these devices, systems, or methods, unless specifically designated as essential.

[0021] Also, with respect to any method described, whether the method is described in conjunction with a flow diagram or not, unless otherwise specified or required by context, it should be understood that any explicit or implicit ordering of steps performed in the execution of the method does not imply that the steps must be performed in the order presented, but may instead be performed in a different order or in parallel.

[0022] As used herein, the term "exemplary" is used in the sense of "example," rather than "ideal." Additionally, the terms "a" and "an" are used herein not to denote a limitation of quantity, but rather to denote the presence of one or more of the referenced item. In the discussion that follows, relative terms such as "about," "substantially," and "approximately" are used to indicate a possible variation of ±10% or less from a stated value, numerical value, or other value.

[0023] Pathology refers to the study of disease. More specifically, pathology refers to the performance of tests and analyses used to diagnose disease. For example, a tissue sample may be placed on a slide to be viewed under a microscope by a pathologist (e.g., a medical doctor who is an expert in analyzing tissue samples and determining whether any abnormalities are present). That is, a pathology specimen may be cut into multiple sections, stained, and prepared as a slide for the pathologist to examine and render a diagnosis. When diagnostic uncertainty is found on a slide, the pathologist may prescribe additional sections, stains, or other tests to gather more information from the tissue. A technician may then create a new slide, which may contain additional information for the pathologist to use in making a diagnosis. This process of creating additional slides can be time-consuming, not only because it may involve taking a block of tissue, cutting it, creating a new slide, and then staining the slide, but also because it may be bulky for multiple orders. This can significantly delay the final diagnosis rendered by the pathologist. Additionally, even after a delay, there may still be no guarantee that the new slides will have enough information to render a diagnosis.

[0024] Pathologists can evaluate cancer and other disease pathology slides for cancer detection. This disclosure presents automated methods for identifying cancer cells, performing cancer qualification, and, where applicable, cancer quantification. In particular, this disclosure describes various exemplary AI tools that can be integrated into the pathologist's workflow to facilitate and improve their work.

[0025] For example, a computer may be used to analyze images of tissue samples to rapidly identify whether the tissue sample contains one or more cancer cells to determine cancer qualitation (e.g., the presence or absence of cancer) and cancer quantitation (e.g., the extent of cancer present). Thus, the process of reviewing stained slides and tests may be performed automatically before, instead of, or in conjunction with review by a pathologist. When paired with automatic slide review and cancer detection, this may provide a fully automated slide preparation and evaluation pipeline.

[0026] Such automation has the benefit of at least (1) minimizing the amount of time wasted by pathologists determining slide findings by manually detecting cancer cells, (2) minimizing the (average total) time from specimen acquisition to diagnosis by avoiding additional time spent performing manual analysis or suspicious slides, (3) reducing the amount of repeat tissue evaluation based on missed or difficult-to-detect tissue areas, (4) reducing the cost of repeat biopsies and pathologist reviews by taking treatment efficacy into account, (5) eliminating or reducing the need for a second or subsequent pathologist diagnostic review, (6) reducing the probability of an incorrect diagnosis, (7) increasing the probability of a correct diagnosis, and / or (8) identifying or verifying the correct characteristics (e.g., pCR, MRD, etc.) of digital pathology images.

[0027] The process of using computers to assist pathologists is called computational pathology. Computing methods used for computational pathology may include, but are not limited to, statistical analysis, autonomous or machine learning, and AI. AI may include, but is not limited to, deep learning, neural networks, classification, clustering, and regression algorithms. The use of computational pathology can save lives by helping pathologists improve their diagnostic accuracy, reliability, efficiency, and accessibility. For example, computational pathology may be used to assist in detecting slides that are suspicious for cancer, thereby allowing pathologists to check and confirm their initial assessment before rendering a final diagnosis.

[0028] Histopathology refers to the study of specimens mounted on slides. For example, a digital pathology image may consist of a digitized image of a microscope slide containing a specimen (e.g., a smear). One method a pathologist may use to analyze the image on a slide is to identify nuclei and classify them as normal (e.g., benign) or abnormal (e.g., malignant). To assist pathologists in identifying and classifying nuclei, histological stains may be used to visualize cells. Many dye-based staining systems have been developed, including periodic acid-Schiff reaction, Masson's trichrome, Nissl and methylene blue, and hematoxylin and eosin (H&E). For medical diagnosis, H&E is a widely used dye-based method; hematoxylin stains cell nuclei blue, eosin stains cytoplasm and extracellular matrix pink, and other tissue regions take on variations of these colors. However, in many cases, histological preparations with H&E staining do not provide sufficient information for pathologists to visually identify biomarkers that could aid in diagnosis or guide treatment. In this situation, techniques such as immunohistochemistry (IHC), immunofluorescence, in situ hybridization (ISH), or fluorescence in situ hybridization (FISH) may be used. IHC and immunofluorescence involve the use of antibodies, for example, that bind to specific antigens within tissue and allow visual detection of cells expressing specific proteins of interest, which may reveal biomarkers that are not reliably identifiable to trained pathologists based on analysis of H&E-stained slides. ISH and FISH may be employed to assess the number of gene copies or the abundance of specific RNA molecules, depending on the type of probe employed (e.g., DNA probes for gene copy number and RNA probes for assessing RNA expression). When these methods also fail to provide sufficient information to detect some biomarkers, genetic testing of tissue may be used to confirm the presence of biomarkers (e.g., overexpression of specific proteins or gene products in tumors, amplification of a given gene in cancer).

[0029] Digitized images may be prepared to represent stained microscope slides, allowing pathologists to manually view the images on the slides and estimate the number of stained abnormal cells within the image. However, this process can be time-consuming and can lead to errors in identifying abnormalities because some abnormalities are difficult to detect. Computational processes using machine learning models and devices may be used to assist pathologists in detecting abnormalities that may otherwise be difficult to detect. For example, AI may be used to detect cancer cells from prominent areas (e.g., because they may be distinguishable from non-cancerous cells) within digital images of tissue stained using H&E and other dye-based methods. The images of tissue may be whole slide images (WSIs), images of tissue cores in microarrays, or selected areas of interest within tissue sections. Using staining methods such as H&E, these cancer cells may be difficult for humans to visually detect or quantify without the assistance of additional testing. Using AI to detect these cancer cells from digital images of tissue has the potential to improve patient treatment while also being faster and less expensive.

[0030] As described above, the computational pathology process and device of the present disclosure provide an integrated platform, enabling a fully automated process, including data capture, processing, and viewing of digital pathology images via a web browser or other user interface, while integrating with a laboratory information system (LIS). Additionally, clinical information may be aggregated using cloud-based data analytics of patient data. Data may originate from hospitals, clinics, field researchers, etc., and may be analyzed by machine learning, computer vision, natural language processing, and / or statistical algorithms to provide real-time monitoring and prediction of health patterns at multiple levels of geographic specificity.

[0031] Implementations of the disclosed subject matter include systems and methods for using a detection machine learning model to determine the presence or absence of cancer cells in a WSI. The detection machine learning model may be generated to determine a cancer qualifier. The cancer qualifier may include an indication of whether cells represented in a digital image of a tissue sample are cancer cells or whether no cancer cells are identified in the digital image. According to some implementations, the cancer qualifier may also include the type of cancer (e.g., breast, prostate, bladder, colon, etc.). If the cancer qualifier is a confirmed cancer qualifier, a cancer quantifier may also be output by the detection machine learning model. The cancer quantifier may indicate the number, proportion, or extent of cancer cells identified from the digital image, or may be a minimal residual disease (MRD) designation based on established MRD criteria (e.g., 1 cell per million or less). If the cancer qualifier output by the detection machine learning model does not indicate any cancer cells, a pathological complete response (pCR) cancer qualifier may be output.

[0032] The detection machine learning model may be trained based on supervised, semi-supervised, weakly supervised, or unsupervised training, including, but not limited to, multiple-instance learning. The training images may be from the same pathology category as the individual digital images input to the detection machine learning model. According to some implementations, multiple different training images from multiple pathology categories may be used to train the detection machine learning model across pathology categories. According to this implementation, the input to the detection machine learning model may include the pathology category of the digital image. The pathology category may include, but is not limited to, histology, cytology, frozen section, immunohistochemistry (IHC), immunofluorescence (IF), hematoxylin and eosin (H&E), hematoxylin only, molecular pathology, 3D imaging, or the like. The detection machine learning model may be trained to detect cancer cells, for example, based on training images having tagged cancer cells. The detection machine learning model may adjust the weights in one or more layers to identify regions that are likely to have cancer cells based on known or determined cancer type, and may further adjust the weights in one or more layers based on identifying cancer cells or not finding cancer cells within those regions.

[0033] According to one implementation, the detection machine learning model may be trained using training digital images depicting tissue exhibiting a therapeutic effect. Treatments that may result in a therapeutic effect include, but are not limited to, neoadjuvant therapies such as hormone therapy (androgen deprivation therapy (ADT), nonsteroidal antiandrogens (NSAA)), radiation therapy, chemotherapy, or the like. Such treatments may cause therapeutic damage and alter the morphology of cancerous and benign cells, thus making detection-based assessment more difficult than such assessment without the therapeutic effect. The therapeutic effect may be the result of a treatment applied to a patient from which a tissue sample corresponding to the digital image was obtained. Treatments often alter the morphology of patient tissue, which is commonly known as a "therapeutic effect," and can often cause an analysis that determines cancer cells to differ from an analysis of tissue that does not exhibit a therapeutic effect. The training digital images depicting tissue exhibiting a therapeutic effect may or may not be tagged as being digital images corresponding to tissue with a therapeutic effect. A therapeutic effect machine learning model may be trained based on images exhibiting a therapeutic effect and may be part of the detection machine learning model. By utilizing the treatment detection machine learning model, the detection machine learning model's qualification and potential quantification of cancer may be informed by the treatment detection machine learning model output, providing an indication of the success or failure of a given treatment. The treatment effect machine learning model may be initialized by using a base detection machine learning model (i.e., a trained machine learning model trained based on multiple training images without treatment effect). Similarly, the pCR and / or MRD detection component of the detection machine learning model may be initialized by using the base detection machine learning model.

[0034] Notifications, visual indicators, and / or reports may be generated based on the output of the detection machine learning model. Reports may be based on individual digitized images, or on multiple digitized images, either over a given period of time, or generally retrospectively.

[0035] The systems disclosed herein may be implemented locally (e.g., on-premise) and / or remotely (e.g., cloud-based). The systems may or may not have a user interface and workflow that can be directly accessed by a pathologist (e.g., a downstream oncologist may be flagged based on cancer qualification or quantification, etc.). Thus, the implementations disclosed herein may be used as standalone operations or within a digital workflow.

[0036] Although the disclosed subject matter is described as being implemented based on oncology applications, they may also be used for other forms of cell detection (e.g., infectious disease cells, cystic fibrosis cells, sickle cell anemia, etc.) In addition to providing cancer detection benefits, the described implementations may be used to train health care professionals (e.g., slide technicians, pathologists, etc.) to practice cell qualification or quantification and / or diagnostic determinations while reducing the risk of patient harm.

[0037] 1A illustrates a block diagram of a system and network for determining sample property or image property information for a digital pathology image using machine learning, according to an exemplary embodiment of the present disclosure. As further disclosed herein, the system and network of FIG. 1A may include a machine learning module 100 with a detection tool 101 for providing a cancer qualifier output, and potentially, a cancer quantifier output.

[0038] 1A illustrates an electronic network 120 that may be connected to servers in a hospital, laboratory, and / or doctor's office, etc. For example, a physician server 121, a hospital server 122, a clinical trial server 123, a research laboratory server 124, and / or a laboratory information system 125, etc., may each be connected to electronic network 120, such as the Internet, through one or more computers, servers, and / or handheld mobile devices. According to one implementation, electronic network 120 may also be connected to server system 110, which may include a processing device configured to implement machine learning module 100 according to an exemplary embodiment of the disclosed subject matter.

[0039] The physician server 121, hospital server 122, clinical trial server 123, research laboratory server 124, and / or laboratory information system 125 may create or otherwise acquire images of one or more categories of pathology samples, including patient cytology samples, histopathology samples, cytology sample slides, histology, immunohistochemistry, immunofluorescence, digitized images of histopathology sample slides, or any combination thereof. The physician server 121, hospital server 122, clinical trial server 123, research laboratory server 124, and / or laboratory information system 125 may also acquire any combination of patient-specific information, such as age, medical history, cancer treatment history, family history, previous biopsies, or cytology information. The physician server 121, hospital server 122, clinical trial server 123, research laboratory server 124, and / or laboratory information system 125 may transmit the digitized slide images and / or patient-specific information to the server system 110 via the electronic network 120. Server system 110 may include one or more storage devices 109 for storing images and data received from at least one of physician server 121, hospital server 122, clinical trial server 123, research laboratory server 124, and / or laboratory information system 125. Server system 110 may also include a processing device for processing images and data stored in storage device 109. Server system 110 may further include one or more machine learning tools or capabilities via machine learning module 100. For example, the processing device may include detection tool 101, shown as machine learning module 100, according to one embodiment. Detection tool 101 may include one or more other components, such as detection and treatment effect machine learning models as disclosed herein, a quantification module, or the like. Alternatively, or in addition, the present disclosure (or portions of the systems and methods of the present disclosure) may be implemented on a local processing device (e.g., a laptop).

[0040] Physician server 121, hospital server 122, clinical trial server 123, research laboratory server 124, and / or laboratory information system 125 refer to systems used by pathologists to review images of slides. In a hospital setting, tissue type information may be stored within laboratory information system 125.

[0041] FIG. 1B illustrates an example block diagram of a machine learning module 100 for determining tissue sample property or image property information for a digital pathology image using machine learning.

[0042] 1B depicts components of a machine learning module 100, according to one embodiment. For example, machine learning module 100 may include a detection tool 101, a data capture tool 102, a slide import tool 103, a slide scanning device 104, a slide manager 105, a storage device 106, and a viewing application tool 108. For clarity, machine learning module 100 shown in FIGS. 1A and 1B is a previously trained and generated machine learning model (e.g., a detection machine learning model, which may include a treatment effect machine learning model). Additional disclosure is provided herein for training and generating different types of machine learning models that may be used as machine learning module 100.

[0043] The detection tool 101 refers to a process and system for determining cancer qualifiers and, if confirmed, cancer quantifiers. Cancer qualifiers may be confirmed cancer qualifiers, pCR cancer quantifiers (e.g., no cancer detected), or the like. Confirmed cancer qualifiers may indicate that one or more cancer cells were detected in a digital image of a tissue sample. Cancer quantifiers may indicate the number of cancer cells detected, the ratio of cancer cells to non-cancerous cells, or the extent of cancer. A subset of cancer quantifiers is MRD cancer qualifiers, which may indicate whether the number of cancer cells is below an MRD threshold. The MRD threshold may be protocol-specific, cancer-type-specific, institution-specific, pathologist-specific, or the like. The detection tool 101 may include multiple machine learning models or load one machine learning model at a time. For example, the detection tool 101 may include a treatment efficacy machine learning model, which may be trained based on a different or additional training dataset than the detection machine learning models disclosed herein.

[0044] Data capture tools 102 refer to processes and systems for facilitating the transfer of digital pathology images to various tools, modules, components, and devices of machine learning module 100 used to characterize and process the digital pathology images, according to an exemplary embodiment.

[0045] Slide capture tool 103 refers to a process and system for scanning pathology images and converting them into digital form, according to an exemplary embodiment. Slides may be scanned using slide scanning device 104, and slide manager 105 may process the images on the slides into digitized pathology images and store the digitized images in storage device 106.

[0046] The viewing application tool 108, according to an exemplary embodiment, refers to a process and system for providing a user (e.g., a pathologist) with characterization or image quality information regarding a digital pathology image. The information may be provided through various output interfaces (e.g., a screen, a monitor, a storage device, and / or a web browser, etc.). As an example, the viewing application tool 108 may apply an overlay layer over the digital pathology image, which may highlight areas of significant consideration. The overlay layer may be, or may be based on, the output of the detection tool 101 of the machine learning module 100. As discussed further herein, the viewing application tool 108 may be used to indicate specific areas of the digital image that correspond to cancerous cells or areas that may be more likely to contain cancerous cells.

[0047] The detection tool 101 and its components may each transmit and / or receive digitized slide images and / or patient information to and / or from the server system 110, the physician server 121, the hospital server 122, the clinical trial server 123, the research laboratory server 124, and / or the laboratory information system 125 via the network 120. Additionally, the server system 110 may include a storage device for storing images and data received from at least one of the detection tool 101, the data capture tool 102, the slide import tool 103, the slide scanning device 104, the slide manager 105, and the viewing application tool 108. The server system 110 may also include a processing device for processing images and data stored in the storage device. The server system 110 may further include one or more machine learning tools or capabilities, for example, due to the processing device. Alternatively, or in addition, the present disclosure (or portions of the systems and methods of the present disclosure) may be implemented on a local processing device (e.g., a laptop).

[0048] The detection tool 101 may provide output (e.g., cancer qualification, cancer quantification, pCR qualification, MRD qualification, etc.) of the machine learning module 100. As an example, the slide capture tool 103 and the data capture tool 102 may receive input to the machine learning module 100, and the detection tool 101 may identify cancer cells in the slide based on the data and output an image highlighting the cancer cells or associated areas via the viewing application tool 108.

[0049] Any of the above devices, tools, and modules may be located on devices that may be connected to an electronic network 120, such as the Internet or a cloud service provider, through one or more computers, servers, and / or handheld mobile devices.

[0050] FIG. 2 shows a flowchart 200 for outputting cancer qualification and quantification based on a digital image, according to an exemplary embodiment of the disclosed subject matter. At 202 of FIG. 2, a digital image corresponding to a target sample associated with a pathology category may be received. The digital image may be a digital pathology image captured using the slide capture tool 103 of FIG. 1B. At 204, a detection machine learning model may be determined (e.g., in the machine learning module 100). The detection machine learning model may be trained by processing multiple training images that are from the same pathology category as the digital image received at 202. The pathology category may include, but is not limited to, histology, cytology, frozen section, or immunohistochemistry. According to one implementation, the detection machine learning model may be trained using training images from multiple different pathology categories. The digital image corresponding to the target sample as received at 202 may also be received along with its pathology category, which may be provided as input to the detection machine learning model. At 206, the digital image from 202 may be provided to the detection machine learning model as input to the model. One or more other attributes may also be provided as input to the detection machine learning model. The one or more other attributes may include, but are not limited to, pathology category, slide type, glass type, tissue type, tissue area, chemical used, amount of stain, time applied, scanning device type, date, or the like. At 208, the detection machine learning model may output a cancer qualifier (e.g., pCR cancer qualifier) ​​or a confirmed cancer quantifier (e.g., amount of cancer). According to some implementations, a confirmed cancer qualifier (e.g., presence of cancer) may be received in addition to or instead of a confirmed cancer quantifier. The cancer qualifier may determine whether a cancer quantifier is generated. For example, if the confirmed cancer qualifier indicates the presence of cancer cells, a cancer quantifier (e.g., number of cancer cells) may be determined. A qualifier indicating the absence of cancer cells may not require a cancer quantifier unless a cancer qualifier threshold of zero cancer cells is exceeded.At 210, the cancer qualification (e.g., pCR cancer qualification, confirmed cancer qualification, etc.), and, if applicable, the cancer quantification, may be output as a data signal, report, notification, alert, visual output (e.g., via the visual application tool 108), or the like.

[0051] Traditional techniques for detecting cancer cells, such as through manual pathologist review, can be subjective and difficult due to tissue complexity, various attributes of qualification and quantification (e.g., pCR / MRD), definition of pCR / MRD, and / or treatment effects that may alter the morphology of cancerous and benign tissues, for example, due to neoadjuvant therapy. Techniques and systems based on the process described in flowchart 200 of FIG. 2 enable robust, objective, and more accurate calculated assessment of cancer qualification or quantification, including pCR and MRD. MRD and pCR designations may be used as surrogate endpoints for assessment of disease-free survival and overall survival (e.g., for breast cancer) to accelerate clinical trials. The automated nature of the process described in FIG. 2 may expedite outcomes and / or treatment approval.

[0052] As shown in FIG. 2 , a digital image corresponding to a target sample associated with a pathology category may be received at 202. The target sample may be a biopsy or otherwise obtained tissue sample taken from a patient. The target sample may be taken during a surgical procedure in which a portion of the patient's tissue is removed from the patient's body for analysis. The target sample may be a portion or subset of the total amount of tissue extracted from the patient, such that multiple sample slides may be generated from the tissue extracted from a single procedure.

[0053] The target sample may be associated with at least one pathology category or technique, such as histology, cytology, frozen section, H&E, hematoxylin only, IHC, molecular pathology, 3D imaging, or the like, as disclosed herein. According to some implementations, the pathology category and digital image or other image information about the target sample may also be received. The image information may include, but is not limited to, slide type, glass type, tissue type, tissue area, chemicals used, and amount of stain.

[0054] At 204, a detection machine learning model may be determined. The detection machine learning model may be trained and generated within the machine learning module 100, or may be trained and generated externally and received at the machine learning module 100. The detection machine learning model may be trained by processing a plurality of training images, at least some of which are from the same pathology category as the digital image received at 202. The pathology categories may include, but are not limited to, histology, cytology, frozen section, H&E, hematoxylin only, IHC, molecular pathology, 3D imaging, or the like. The detection machine learning model may be instantiated using one or more of deep learning, including but not limited to, deep neural networks (DNNs), convolutional neural networks (CNNs), fully convolutional networks (FCNs), and recurrent neural networks (RCNs); probabilistic models, including but not limited to, Bayesian networks and graphical models; and / or discriminative models, including but not limited to, decision forests and maximum margin methods, or the like. These models may be trained in a supervised, semi-supervised, weakly supervised, or unsupervised manner, such as using multiple-instance learning. A quantification component of the detection machine learning model or a separate machine learning model may also use machine learning techniques to generate an output related to the quantification of cancer in the digital image (e.g., the number of cancer cells in the image). The quantification component may be trained using, without limitation, deep learning, CNN, multiple-instance learning, or the like, or a combination thereof.

[0055] The detection machine learning model may be trained to output cancer qualification and quantification as disclosed herein. The cancer qualification may be output for one or more of a plurality of different cancer types. The detection machine learning model may be trained using images from one or more of a plurality of different cancer types. For example, the training images may include images related to breast cancer, prostate cancer, and lung cancer. Thus, the generated detection machine learning model may receive digital images at 202 of FIG. 2 and may qualify the images as representing tissue containing cancer cells, the number of cancer cells, and / or the type of cancer cells. The detection machine learning model may output cancer qualification and / or quantification based on weights and / or layers trained during its training process. Based on the weights and / or layers, the detection machine learning model may identify regions of the digital image that may be more strongly used as evidence regarding the presence or absence of cancer and further the extent of cancer. The model may then evaluate some or all of those regions and determine the presence, absence, and / or extent of cancer cells based on training with the images that provided it. Feedback (e.g., pathologist confirmation, corrections, adjustments, etc.) may further train the detection machine learning model during operation of the model.

[0056] To generate the detection machine learning model at 204, a training dataset including a large number of digital pathology images of pathology samples (e.g., histology, cytology, frozen section, H&E, hematoxylin only, IHC, molecular pathology, 3D imaging, etc.) may be applied. The digital pathology images may be digital images generated based on physical biopsy samples as disclosed herein, or may be images algorithmically generated to replicate tissue samples (e.g., human, animal, etc.) by, for example, a rendering system or a generative adversarial model. Image or sample association information (e.g., slide type, glass type, tissue type, tissue region, chemicals used, amount of stain, time applied, scanner type, date, etc.) may also be received as part of the training dataset. Additionally, as part of training the detection machine learning model, each image may be paired with output information for known or hypothesized cancer qualifiers and, if applicable, cancer quantifiers. Such output information may include an indication of cancer presence or absence, type of cancer, and / or extent of cancer. The detection machine learning model may learn from a plurality of such training images and associated information such that the detection machine learning model is trained by modifying one or more weights based on the qualification and quantification associated with each training image. Although supervised training is provided as an example, it should be understood that the training of the detection machine learning model can be supervised, semi-supervised, weakly supervised, or unsupervised.

[0057] Training datasets including digital pathology images, image or sample association information, and / or output information may be generated and / or provided by one or more of system 110, physician server 121, hospital server 122, clinical trial server 123, research laboratory server 124, and / or laboratory information system 125. Images used for training may be derived from real sources (e.g., humans, animals, etc.) or synthetic sources (e.g., graphics rendering engines, 3D models, etc.). Examples of digital pathology images may include (a) digitized slides stained with various dyes, such as, but not limited to, H&E, hematoxylin only, IHC, molecular pathology, etc., and / or (b) digitized tissue samples from 3D imaging devices, such as microCT.

[0058] The detection machine learning model may be generated based on applying a digital pathology image, optionally with associated information paired with output information as applied by a machine learning algorithm. The machine learning algorithm may receive as input the pathology sample, associated information, and output information (e.g., cancer qualification, cancer quantification, etc.) and may implement training using one or more techniques. For example, the detection machine learning model may be trained in one or more deep learning algorithms, such as, but not limited to, DNN, CNN, FCN, RCN, CNN with multiple instance learning or multi-label multiple instance learning, recurrent neural network (RNN), long short-term memory RNN (LSTM), gated recurrent unit RNN (GRU), graph convolutional network, or the like, or a combination thereof. Convolutional neural networks can directly learn the image feature representations necessary to discriminate characteristics, which can work very well when there is a large amount of data to train on per sample, while other methods can be used in conjunction with traditional computer vision features, such as either SURF or SIFT, or learned embeddings (e.g., descriptors) generated by a trained convolutional neural network, which can provide advantages when there is only a small amount of data to train on. The trained detection machine learning model may be configured to provide cancer qualification and / or cancer quantification outputs based on digital pathology images.

[0059] 3 shows an example training module 300 for training a detection machine learning model. As shown in FIG. 3, training data 302 may include one or more of a pathology image 304 (e.g., a digital representation of a biopsy image), input data 306 (e.g., a cancer type, a pathology category, etc.), and a known outcome 308 (e.g., a quality designation) associated with the pathology image 304. The training data 302 and a training algorithm 310 may be provided to a training component 320, which may apply the training data 302 to the training algorithm 310 to generate a detection machine learning model.

[0060] At 206, the detection machine learning model may be provided with inputs including patient-based digital pathology images (e.g., digital images of pathology samples (e.g., histology, cytology, immunohistochemistry, etc.)) and, optionally, associated information. Internal weights and / or layers of the detection machine learning model may be applied to the digital pathology images and associated information to determine cancer qualification, and, if applicable, cancer quantification.

[0061] The cancer qualifier may be the presence or absence of cancer. The cancer qualifier may be a binary decision, such that detecting a single cancer cell may correspond to the presence of cancer, and not detecting a single cancer cell may correspond to the absence of cancer. The pCR cancer qualifier may correspond to the absence of cancer, and may be output when no cancer cells are detected.

[0062] According to some implementations, cancer presence and / or pCR cancer qualification may be protocol specific, such that a protocol may define one or more thresholds for cancer qualification. As an example, a protocol may indicate that a minimum of 5 cancer cells per million cells is required to output a cancer presence, and that fewer than 5 cancer cells per million are sufficient for pCR cancer qualification.

[0063] Additionally, as disclosed herein, the detection machine learning model may be configured to output a cancer type based on image data received as input. The cancer type output may be an indication of a determined cancer type or a probability of a cancer type. The cancer type output may be informed by inputs in addition to the tissue sample-based digital image and may include tissue characteristics, slide type, glass type, tissue type, tissue region, chemicals used, and / or amount of stain.

[0064] Cancer quantification may be an output when the presence of cancer is detected. Cancer quantification may include the number of cancer cells (e.g., cancer cells per million), the density of cancer cells, or an indication that the number of cancer cells exceeds a threshold amount (e.g., an MRD threshold). For example, cancer quantification may be or include MRD cancer quantification, which may refer to submicroscopic disease, such as disease that remains dormant within a patient but may eventually lead to recurrence. In cancer treatment, MRD may provide information about whether the treatment has eliminated the cancer or whether any traces remain.

[0065] The output of the detection machine learning model (i.e., cancer qualification and, if applicable, cancer quantification) may be provided at 210 to storage device 109 of FIG. 1A (e.g., cloud storage, hard drive, network drive, etc.). The output of the detection machine learning model may be cancer qualification or cancer quantification, or may also be a notification based thereon. The notification may include information about the cancer qualification, cancer quantification, cancer type, location of cancer cells, or the like. The notification may be provided via any applicable technique, such as a notification signal, a report, a message via an application, a notification via a device, or the like. The notification may be provided to any applicable device or personnel (e.g., a histotechnician, a scanning device operator, a pathologist, a recorder, etc.). As an example, the output of the detection machine learning model may be integrated with the history of the corresponding target sample in, for example, a laboratory information system 125 that stores records of patients and associated target samples.

[0066] According to some implementations, the output of the detection machine learning model may be a report based on cancer qualification, cancer quantification, cancer type, location of cancer cells, changes in any such factors over time, or the like. The report may be in any applicable format, such as PDF format, HTML format, in-app format, or the like.

[0067] According to some implementations, the output of the detection machine learning model may be or may include a visual indicator at 210. The visual indicator may be provided, for example, via the viewing application tool 108 of FIG. 1B. The visual indicator may provide a visual representation of the digital image and may provide an indication regarding the cells used in determining cancer qualification or cancer quantification.

[0068] FIG. 4 shows an exemplary digital image 400 enlarged to provide a close-up view 400A. The digital image 400 in FIG. 4 is an H&E slide with a therapeutic effect resulting from hormone therapy for prostate cancer. As shown in the close-up view 400A, an area of ​​cells not relied upon for cancer qualification and / or cancer quantification is shown in area 402. Area 402 may correspond to an area that the detection machine learning model has determined to be less relevant for detection purposes. Area 402 corresponds to an area that the detection machine learning model has determined to be a benign tissue region in which no cancer cells are detected. An area of ​​cells relied upon for cancer qualification and / or cancer quantification is shown in area 404. Compared to the digital image 400 of the tissue sample, area 404 relied upon for cancer qualification and / or cancer quantification may be relatively small. Area 404 may correspond to an area that the detection machine learning model has determined to be more relevant for detection purposes. Area 404 corresponds to the area that the detection machine learning model determines as the cancerous tissue region in which at least one cancer cell is detected.

[0069] According to some implementations, the detection machine learning algorithm may also be trained based on and / or receive as input clinical information (e.g., patient information, surgical procedure information, diagnostic information, etc.), laboratory information (e.g., processing time, personnel, tests, etc.), etc. The detection machine learning algorithm may provide cancer qualification and / or cancer quantification based on such input.

[0070] According to some implementations, the detection machine learning model may include a treatment effect machine learning model, as disclosed herein. The treatment effect may correspond to an oncology treatment based on a pharmaceutical drug, hormone therapy, chemotherapy, etc. Tissue samples from patients treated with a therapy (e.g., a cancer therapy) may have different properties than tissue samples from patients not treated with a similar therapy.

[0071] The treatment effect machine learning model may be generated using low-shot or transfer learning methods and may be initialized by using a base detection machine learning model (i.e., a trained machine learning model trained based on multiple training images that exclude the treatment effect). For example, a sample detection machine learning model as disclosed herein may be trained using digital images from tissue samples from patients who have not received the treatment and / or whose tissue samples do not exhibit the treatment effect. The treatment machine learning model may be initialized using the sample detection machine learning model such that the weights and / or one or more layers associated with the sample detection machine learning model are retained, and additional weights, weight modifications, layers, and / or layer modifications are applied when generating the treatment effect machine learning model. Similarly, the pCR and / or MDR detection component of the detection machine learning model may be initialized by using the base detection machine learning model.

[0072] 5 shows an exemplary implementation of the detection analysis as disclosed herein with reference to FIG. 2. FIG. 5 shows a flowchart for the process of predicting pCR and / or MRD in prostate cancer with therapeutic effect. In the prostate cancer example, pCR and MRD may be used as endpoints for clinical trials assessing neoadjuvant treatment. However, a major challenge is the presence of therapeutic effect when assessing cancer.

[0073] At 502, a detection machine learning model may be trained using images to output a cancer qualification and, if applicable, a cancer quantification if the cancer qualification is a confirmed cancer qualification. Training may include inputting one or more digital images of prostate tissue (e.g., histopathology, H&E, IHC, 3D imaging, etc.) containing an indication of the presence or absence of cancer. The detection training model may be trained using one or more machine learning algorithms and / or formats (e.g., deep learning, DNN, CNN, FCN, RCN, probabilistic model, discriminative model, etc.) as disclosed herein. The detection learning model may be trained to output a cancer qualification, a cancer quantification, and an assessment of pCR and / or MRD, which may be protocol-specific, as disclosed herein. The training images may include images that may be or be tagged as one of pCR or MRD.

[0074] At 504, a quantification module may be trained based on the digital images applied at 502. To train the quantification module, quantification of the amount of prostate cancer (e.g., the number of cells exhibiting prostate cancer) may be included with all or a subset of the digital images used to train the model. It should be understood that the quantification module may be part of the overall detection model trained at 502.

[0075] At 506, a treatment effect module may be trained based on the digital images applied at 502. All or a subset of the digital images applied at 502 may depict tissues that exhibit a treatment effect. The treatment effect may be inherent in the image or may be tagged such that the tag is used as part of the training.

[0076] At 508, a digital image of a pathology sample may be received as input to one or more of the trained detection machine learning model at 502, the quantification module at 504, and the treatment effect module at 506.

[0077] At 510, the detection machine learning model at 502, the quantification module at 504, and / or the treatment efficacy module at 506 may be used to determine a pCR cancer qualifier, a confirmed cancer qualifier. If a pCR cancer qualifier is determined, the pCR cancer qualifier may be output at 512 (e.g., via a notification, report, visual indication, etc.). If a confirmed cancer qualifier is determined, the confirmed cancer qualifier may be output at 518. Additionally or alternatively, cancer quantification may be determined. For example, an MRD value may be determined at 514 and output at 516.

[0078] Figure 6 illustrates an experiment based on the process disclosed in flowchart 200 of Figure 2. Figure 6 relates to the pathological evaluation of prostate cancer treated with neoadjuvant hormonal therapy in radical prostatectomy whole mount sections. Such evaluation traditionally poses significant challenges due to the morphological changes in tumors combined with the possibility of very small foci of residual tumor in a large amount of non-neoplastic tissue.

[0079] The detection machine learning model applied in this experiment may utilize a multiple instance learning approach to train a whole-slide image classifier using the SE-ResNet50 convolutional neural network. In this example, the model was trained on 36,644 WSIs (7,514 of which had cancerous lesions) with a reduced embedding size to accommodate the large number of patch instances in whole-mount slides. The detection machine learning model was then fine-tuned in a fully supervised context on a small annotated set of radical prostatectomy samples taken from prostate cancer patients treated in the neoadjuvant setting to further improve performance on radical prostatectomy data. This produced an AUC of 0.99 for antiandrogen-treated cases.

[0080] The detection machine learning model demonstrated an AUC of 0.99 for cases involving anti-androgen receptor neoadjuvant therapy. After training, the exemplary system was evaluated on 40 WSI images of H&E-stained whole-mount prostatectomy slides from 15 prostatectomy specimens collected from patients after neoadjuvant treatment with anti-androgen therapy. Ground truth was established through pathologist annotation. All 37 malignant WSIs (3 WSIs containing tumors <5 mm and 34 WSIs containing tumors >5 mm) were correctly classified by the system as occult, treated cancer. Of the three benign WSIs, the detection machine learning model incorrectly classified one benign lesion as cancer, while the other two were correctly classified as benign tissue.

[0081] Thus, accurate slide-level classification of H&E-stained slides from radical prostatectomy specimens has the potential to improve the accuracy and efficiency of histopathological evaluation of whole-mount sections from radical prostatectomy specimens of patients who received neoadjuvant therapy prior to surgery. As shown in Figure 6, digital image 600 includes a visual indication of the cells relied upon in area 602. Enlargement 600A more clearly shows the cells relied upon by the detection machine learning model, and also shows a specific area 604 within area 602, indicating cancer cells within area 604. Further enlargement 600B even more clearly shows the cells relied upon by the detection machine learning model, and also shows a specific area 606 within specific area 604, indicating cancer cells and / or benign cells within area 606.

[0082] 7 may correspond to hardware utilized by the server system 110, the hospital server 122, the research laboratory server 124, the clinical trial server 123, the physician server 121, the laboratory information system 125, and / or a client device, etc. The device 700 may include a central processing unit (CPU) 720. The CPU 720 may be any type of processor device, including, for example, any type of special-purpose or general-purpose microprocessor device. As will be understood by those skilled in the art, the CPU 720 may also be a single processor in a multi-core / multi-processor system, such as a system operating alone or within a cluster of computing devices operating within a cluster or server farm. The CPU 720 may be connected to a data communications infrastructure 710, for example, a bus, a message queue, a network, or a multi-core message passing scheme.

[0083] The device 700 may also include a primary memory 740, e.g., random access memory (RAM), and may also include a secondary memory 730. The secondary memory 730, e.g., read-only memory (ROM), may be, for example, a hard disk drive or a removable storage drive. Such a removable storage drive may comprise, for example, a floppy disk drive, a magnetic tape drive, an optical disk drive, a flash memory, or the like. The removable storage drive, in this embodiment, reads from and / or writes to a removable storage unit in a well-known manner. The removable storage unit may comprise a floppy disk, magnetic tape, optical disk, etc., which is read by and written to by the removable storage drive. As will be appreciated by those skilled in the art, such removable storage units generally include computer-usable storage media having computer software and / or data stored therein.

[0084] In alternative implementations, secondary memory 730 may include other similar means for allowing computer programs or other instructions to be loaded into device 700. Examples of such means may include program cartridges and cartridge interfaces (such as those found in video game devices), removable memory chips (such as EPROMs or PROMs) and associated sockets, and other removable storage units and interfaces that allow software and data to be transferred from removable storage units to device 700.

[0085] Device 700 may also include a communications interface (“COM”) 760. Communications interface 760 allows software and data to be transferred between device 700 and external devices. Communications interface 760 may include a modem, a network interface (such as an Ethernet card), a communications port, a PCMCIA slot and card, or the like. The software and data transferred via communications interface 760 may be in the form of signals, which may be electronic, electromagnetic, optical, or other signals capable of being received by communications interface 760. These signals may be provided to communications interface 760 over a communications path in device 700, which may be implemented using, for example, wire or cable, fiber optics, a phone line, a cellular phone link, an RF link, or other communications channel.

[0086] The hardware elements, operating systems, and programming languages ​​of such devices are conventional in nature and are assumed to be reasonably familiar to those skilled in the art. Device 700 may also include input and output ports 750 for connecting with input and output devices such as a keyboard, mouse, touch screen, monitor, display, etc. Of course, various server functions may be implemented in a distributed manner on several similar platforms to distribute the processing load. Alternatively, the server may be implemented by appropriate programming of one computer hardware platform.

[0087] Throughout this disclosure, references to components or modules generally refer to items that can be logically grouped together to perform a function or group of related functions. Like reference numbers are generally intended to refer to the same or similar components. Components and modules can be implemented in software, hardware, or a combination of software and hardware.

[0088] The tools, modules, and functions described above may be implemented by one or more processors. A "storage" type medium may include any or all of the tangible memory of a computer, processor, or the like, or its associated modules, such as various semiconductor memories, tape drives, disk drives, and the like, which may provide non-transitory storage for software programming from time to time.

[0089] The software may be communicated through the Internet, a cloud service provider, or other telecommunications network. For example, the communication may allow the software to be loaded from one computer or processor into another. As used herein, unless limited to non-transitory tangible "storage" media, terms such as computer or machine "readable medium" refer to any medium that participates in providing instructions to a processor for execution.

[0090] The foregoing general description is exemplary and explanatory only and is not a limitation of the present disclosure. Other embodiments of the present invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. It is intended that the specification and examples be considered as exemplary only.

Claims

1. A computer-implemented method for processing an electronic image, said method comprising: receiving a digital image corresponding to a target sample associated with a pathology category, the digital image being an image of a tissue sample; determining a detection machine learning model, the detection machine learning model being generated by processing a plurality of training images, outputting a cancer qualifier, and if the cancer qualifier is a confirmed cancer quantifier, further outputting a cancer quantifier; providing the digital image as an input to the detection machine learning model; receiving a confirmed cancer quantification including minimal residual disease (MRD) as output from the detection machine learning model, wherein the detection machine learning model comprises a treatment effect machine learning model, the treatment effect machine learning model initialized with one of the layers and / or weights from a trained version of the detection machine learning model; outputting the MRD cancer qualifier based on the treatment effect machine learning model; and 11. A computer-implemented method comprising:

2. The computer-implemented method of claim 1, further comprising receiving one of a pathological complete response (pCR) cancer qualifier.

3. The computer-implemented method of claim 1, wherein the MRD cancer qualifier is protocol-specific.

4. The computer-implemented method of claim 1, wherein the MRD cancer qualification corresponds to the number of cancer cells below an MRD threshold.

5. The computer-implemented method of claim 1, wherein the MRD cancer qualification identifies one or more diseases that remain latent within the patient but may eventually lead to recurrence.

6. The computer-implemented method of claim 1, wherein the treatment effect machine learning model is trained based on tagged treatment effects in the plurality of training images.

7. The computer-implemented method of claim 1, wherein receiving the confirmed cancer quantification also includes receiving a type of cancer when the output of the detection machine learning model includes a confirmed cancer quantification.

8. The computer-implemented method of claim 7, wherein the type of cancer is determined based on the digital image and one or more of tissue characteristics, slide type, glass type, tissue type, tissue area, chemicals used, or amount of staining.

9. The computer-implemented method of claim 1, wherein the digital images are from a pathology category, and the pathology category is selected from one or more of histology, cytology, frozen section, immunohistochemistry (IHC), immunofluorescence, hematoxylin and eosin (H&E), hematoxylin only, molecular pathology, and / or 3D imaging.

10. A system for processing electronic images, said system comprising: at least one memory for storing instructions; at least one processor, wherein said at least one processor executes said instructions; receiving a digital image corresponding to a target sample associated with a pathology category, the digital image being an image of a tissue sample; determining a detection machine learning model, the detection machine learning model being generated by processing a plurality of training images, outputting a cancer qualifier, and if the cancer qualifier is a confirmed cancer quantifier, further outputting a cancer quantifier; providing the digital image as an input to the detection machine learning model; receiving a confirmed cancer quantification including minimal residual disease (MRD) as output from the detection machine learning model, wherein the detection machine learning model comprises a treatment effect machine learning model, the treatment effect machine learning model initialized with one of the layers and / or weights from a trained version of the detection machine learning model; outputting the MRD cancer qualifier based on the treatment effect machine learning model; at least one processor that performs operations including A system comprising:

11. The system described in claim 10, further comprising receiving one of a pathological complete response (pCR) cancer qualifier.

12. The system described in claim 10, wherein the MRD cancer qualifier is protocol specific.

13. The system described in claim 10, wherein the MRD cancer qualification corresponds to the number of cancer cells below an MRD threshold.

14. The system of claim 10, wherein the MRD cancer qualification identifies one or more diseases that remain latent within the patient but may eventually lead to recurrence.

15. The system of claim 10, wherein receiving the confirmed cancer quantification also includes receiving a type of cancer when the output of the detection machine learning model includes a confirmed cancer quantification.

16. The system of claim 15, wherein the type of cancer is determined based on the digital image and one or more of tissue characteristics, slide type, glass type, tissue type, tissue area, chemicals used, or amount of staining.

17. The system of claim 10, wherein the digital images are from a pathology category, the pathology category being selected from one or more of histology, cytology, frozen section, immunohistochemistry (IHC), immunofluorescence, hematoxylin and eosin (H&E), hematoxylin only, molecular pathology, and / or 3D imaging.

18. A non-transitory computer-readable medium, the non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations for processing an electronic image, the operations comprising: receiving a digital image corresponding to a target sample associated with a pathology category, the digital image being an image of a tissue sample; determining a detection machine learning model, the detection machine learning model being generated by processing a plurality of training images, outputting a cancer qualifier, and if the cancer qualifier is a confirmed cancer quantifier, further outputting a cancer quantifier; providing the digital image as an input to the detection machine learning model; receiving a confirmed cancer quantification including minimal residual disease (MRD) as output from the detection machine learning model, wherein the detection machine learning model comprises a treatment effect machine learning model, the treatment effect machine learning model initialized with one of the layers and / or weights from a trained version of the detection machine learning model; outputting the MRD cancer qualifier based on the treatment effect machine learning model; and 1. A non-transitory computer-readable medium comprising:

19. The non-transitory computer-readable medium of claim 18, further comprising receiving one of a pathological complete response (pCR) cancer qualifier.

20. The non-transitory computer-readable medium of claim 18, wherein the MRD cancer qualifier is protocol-specific.

Images