Systems and methods for artificial intelligence-enabled molecular workflows that verify the quality of test slides and blocks

JP2024510394A5Active Publication Date: 2024-11-20PAIGE AI INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023551707
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-03-09
Filing Date
2021-12-07
Publication Date
2024-11-20
Estimated Expiration
2041-12-07

AI Technical Summary

Technical Problem

Current tumor genome sequencing workflows are inefficient and laborious, requiring pathologists to manually select high-quality tumor slides or sections, which is time-consuming and prone to errors.

Method used

A computer-implemented method using machine learning models to analyze digital histopathology images, segment tissue regions, and determine a quality score for tumor slides and blocks, facilitating automated selection of high-quality samples for further testing.

Benefits of technology

The method significantly reduces the time and effort required for selecting high-quality tumor samples, enhancing the accuracy and efficiency of tumor genome sequencing by ensuring sufficient tumor content and purity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

A system and method for verifying the quality of a test slide or block is disclosed. The method may include receiving a collection of one or more digital images in a digital storage device. The collection may be associated with a tissue block and correspond to an instance. The method may include applying a machine learning model to the collection to identify the presence or absence of an attribute, determining an amount or percentage of tissue having the attribute from the digital images in the collection indicating the presence of the attribute, and outputting a quality score corresponding to the determined amount or percentage.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] Related Applications This application claims priority to U.S. Provisional Application No. 63 / 158,781, filed March 9, 2021, the entire disclosure of which is incorporated herein by reference.

[0002] Various embodiments of the present disclosure relate generally to image processing methods, and more specifically, certain embodiments of the present disclosure relate to systems and methods for verifying the quality of test slides and blocks. [Background technology]

[0003] Current workflows for tumor genomic sequencing have many inefficiencies. For example, the current workflow process requires a pathologist to review material to select the best slides or sections of tumor areas for sequencing, which can be tedious and time-consuming.

[0004] Difficulties with current technology include identifying the best tissue blocks and / or slides from a human or animal patient for testing (e.g., genomic sequencing) and ensuring that the selected tissue blocks and / or slides from the patient contain the appropriate tumor tissue.

[0005] The discussion of the background of the present specification is intended to generally indicate the context of the present disclosure. Unless specifically stated herein, the matter described in this section is not prior art to the claims of this application, and inclusion in this section is not an admission that it is prior art or suggestive thereof. Summary of the Invention [Means for solving the problem]

[0006] According to one aspect of the present disclosure, a system and method for verifying the quality of test slides and blocks is disclosed.

[0007] A method for verifying quality of test slides and blocks may be computer implemented. The method may include receiving a collection of one or more digital images in a digital storage device. The collection may be associated with a tissue block and correspond to an instance. The method may include applying a machine learning model to the collection to identify the presence or absence of an attribute, determining an amount or percentage of tissue having the attribute from the digital images in the collection indicating the presence of the attribute, and outputting a quality score corresponding to the determined amount or percentage.

[0008] Determining the amount or proportion may include summing and normalizing the digital images of the set indicating the presence of the attribute by the total amount of tissue. The digital images may be digital histopathology images.

[0009] The method may include dividing each digital image of the collection of digital images into a collection of tiles, detecting and / or segmenting tissue regions from a background of the digital images to create a tissue mask, and removing all tiles of the collection of tiles that contain the background. The detecting and / or segmenting may include performing a connected components algorithm using one or more threshold-based methods. The detecting and / or segmenting may include using one or more segmentation algorithms.

[0010] The method may include determining the tissue block with the highest quality score for subsequent examination. The method may include suggesting to the user that the tissue block has at least one additional slide prepared for examination. The method may include determining whether the quality score is likely to be low enough. The method may include suggesting to the user that a new block for examination is prepared if it is determined that the quality score is likely to be low enough.

[0011] The method may include outputting a function of at least one variable corresponding to the quality score. The function of the at least one variable may be a linear function. The function of the at least one variable may be a non-linear function.

[0012] The method may include outputting a binary image indicating where the attributes are located. The method may include receiving a summary annotation. The summary annotation may include one or more labels for each digital image. The one or more labels may be pixel level labels, tile level labels, slide level labels, and / or subspecimen level labels.

[0013] A system for verifying quality of test slides and blocks using machine learning models includes at least one memory storing instructions and at least one processor configured to execute the instructions to perform operations. The operations may include receiving a collection of one or more digital images in a digital storage device. The collection may be associated with a tissue block and correspond to an instance. The operations may include applying the machine learning model to the collection to identify the presence or absence of an attribute, determining an amount or percentage of tissue having the attribute from the images in the collection that indicate the presence of the attribute, and outputting a quality score corresponding to the determined amount or percentage.

[0014] Determining the amount or proportion may include summing and normalizing the set of digital images indicating the presence of the attribute by the total amount of tissue.

[0015] A non-transitory computer readable medium may store instructions that, when executed by a processor, perform a method of verifying quality of test slides and blocks using a machine learning model. The method may include receiving a collection of one or more digital images in a digital storage device. The collection may be associated with a tissue block and correspond to an instance. The method may include applying the machine learning model to the collection to identify the presence or absence of an attribute, determining an amount or percentage of tissue having the attribute from the digital images in the collection that indicate the presence of the attribute, and outputting a quality score corresponding to the determined amount or percentage.

[0016] The method may further include dividing each digital image in the collection of digital images into a collection of tiles, detecting and / or segmenting tissue regions from a background in the digital images to create a tissue mask, and removing all tiles in the collection of tiles that contain the background.

[0017] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments.

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various exemplary embodiments and, together with the description, serve to explain the principles of the disclosed embodiments. [Brief description of the drawings]

[0019] [Figure 1A] FIG. 1 illustrates an exemplary block diagram of a system and network for verifying the quality of test slides and blocks according to an exemplary embodiment of the present disclosure.

[0020] [Figure 1B] FIG. 1 illustrates an exemplary block diagram of a disease detection platform according to an exemplary embodiment of the present disclosure.

[0021] [Figure 1C]FIG. 1 illustrates an exemplary block diagram of a slide analysis tool according to an exemplary embodiment of the present disclosure.

[0022] [Diagram 2] 1 is a flow chart illustrating a method for verifying quality of test slides and blocks according to an exemplary embodiment of the present disclosure.

[0023] [Figure 3A] 1 is a flowchart illustrating a method for training and using a machine learning model to output a quality score function according to an exemplary embodiment of the present disclosure. [Figure 3B] 1 is a flowchart illustrating a method for training and using a machine learning model to output a quality score function according to an exemplary embodiment of the present disclosure.

[0024] [Figure 4A] 1 is a flow chart illustrating a method for training and using a machine learning model to determine whether a formalin-fixed, paraffin-embedded tissue (FFPE) should have additional slides prepared for examination according to an exemplary embodiment of the present disclosure. [Figure 4B] 1 is a flow chart illustrating a method for training and using a machine learning model to determine whether a formalin-fixed paraffin-embedded tissue (FFPE) should have additional slides prepared for examination according to an exemplary embodiment of the present disclosure.

[0025] [Figure 5A] 1 is a flow chart illustrating a method for training and using a machine learning model to determine whether a new FFPE block of tissue should be prepared for examination according to an exemplary embodiment of the present disclosure. [Figure 5B] 1 is a flow chart illustrating a method for training and using a machine learning model to determine whether a new FFPE block of tissue should be prepared for examination according to an exemplary embodiment of the present disclosure.

[0026] [Figure 6]1 is an exemplary use of a machine learning model for assessing block quality according to an exemplary embodiment of the present disclosure.

[0027] [Figure 7] 1 illustrates an example of a system capable of implementing the techniques of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Reference will now be made in detail to the exemplary embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.

[0029] The systems, devices, and methods disclosed herein will be described in detail below with reference to the drawings. The embodiments described herein are merely examples and are provided to assist in the description of the devices, devices, systems, and methods of the present specification. Furthermore, none of the features or components shown in the drawings or described below should be considered essential to the implementation of any of the features of these devices, systems, or methods, unless specifically designated as essential.

[0030] Additionally, for any method of description, whether or not the method is described using a flow diagram, unless otherwise specified or required by context, it should be understood that any explicit or implicit ordering of steps performed in performing the method does not imply that the steps must be performed in the order presented, but instead may be performed in a different order or in parallel.

[0031] As used herein, the term "exemplary" is used in the sense of "exemplary" rather than "ideal." Additionally, the words "a" and "an" do not denote a limitation of quantity but rather denote the presence of one or more of the referenced item.

[0032] The technology disclosed herein may use AI techniques to identify the best tissue block and / or slide for testing and to verify that the selected tissue block and / or slide taken from the patient contains the appropriate amount of tumor. Data and predictions may be instantly aggregated and made available via any user interface (e.g., via digital pathology viewing systems, reports, laboratory information systems, etc.). Machine learning algorithms may rapidly and simultaneously assess validity, classify samples into diagnostic categories, and screen for the most likely molecular alterations, limiting total molecular testing to tumor and increasing the likelihood of a valid molecular result with sufficient tumor volume.

[0033] FIG. 1A illustrates a block diagram of a system and network for verifying slide and block quality using machine learning according to an exemplary embodiment of the present disclosure.

[0034] 1A illustrates an electronic network 120 that may be connected to servers at a hospital, a testing laboratory, and / or a doctor's office, etc. For example, a doctor server 121, a hospital server 122, a clinical trial server 123, a laboratory server 124, and / or a testing laboratory information system 125, etc. may each be connected to the electronic network 120, such as the Internet, via one or more computers, servers, and / or mobile devices. According to an exemplary embodiment of the present application, the electronic network 120 may also be connected to a server system 110 that may include a processing device configured to implement a disease detection platform 100 that includes a slide analysis tool 101 for analyzing tissues in a WSI in accordance with an exemplary embodiment of the present disclosure.

[0035] The doctor server 121, the hospital server 122, the clinical trial server 123, the laboratory server 124 and / or the laboratory information system 125 may create or otherwise obtain images of cytology sample(s), histopathology sample(s) and slide(s) of cytology sample(s), digitized images of slide(s) of histopathology sample(s), or combinations thereof for one or more patients. The doctor server 121, the hospital server 122, the clinical trial server 123, the laboratory server 124 and / or the laboratory information system 125 may also obtain any combination of patient-specific information such as age, medical history, cancer treatment history, family history, and previous biopsy or cytology information. The doctor server 121, the hospital server 122, the clinical trial server 123, the laboratory server 124 and / or the laboratory information system 125 may transmit the digitized slide images and / or patient-specific information to the server system 110 via the electronic network 120. The server system 110 may include one or more storage devices 109 for storing images and data received from at least one of the physician server 121, the hospital server 122, the clinical trial server 123, the laboratory server 124, and / or the laboratory information system 125. The server system 110 may also include a processing device for processing the images and data stored in the one or more storage devices 109. The server system 110 may further include one or more machine learning tool(s) or functionality. For example, the processing device may include machine learning tools for the disease detection platform 100 according to an embodiment. Alternatively or additionally, the present disclosure (or parts of the systems and methods of the present disclosure) may be implemented on a local processing device (e.g., a laptop).

[0036] Physician server 121, hospital server 122, clinical trial server 123, lab server 124 and / or laboratory information system 125 refer to systems used by pathologists to review images of slides. In a hospital environment, tissue type information may be stored in the laboratory information system 125. According to exemplary embodiments of the present disclosure, cells in a WSI that share similar targets may be grouped together without needing to access the laboratory information system 125. Additionally, access to laboratory information system content may be restricted due to its sensitive content.

[0037] FIG. 1B shows an example block diagram of a disease detection platform 100 that uses machine learning to verify slide and block quality.

[0038] 1B illustrates components of a disease detection platform 100 according to one embodiment. For example, the disease detection platform 100 may include a slide analysis tool 101, a data acquisition tool 102, a slide capture tool 103, a slide scanner 104, a slide manager 105, a storage 106, and a viewing application tool 108.

[0039] Slide analysis tool 101, as described below, refers to a process and system for grouping cells in a WSI that share similar targets according to an exemplary embodiment.

[0040] Data acquisition tools 102 refer to processes and systems that facilitate the transfer of digital pathology images to various tools, modules, components, and devices used to classify and process the digital pathology images, according to an example embodiment.

[0041] Slide capture tool 103 refers to a process and system for scanning and converting pathology images into digital form according to an exemplary embodiment. Slides may be scanned with slide scanner 104, and slide manager 105 may process the images on the slides into digitized pathology images and store the digitized images in storage 106.

[0042] The viewing application tool 108 refers to processes and systems that provide specimen characteristic or image characteristic information regarding digital pathology image(s) to a user (e.g., a pathologist) according to an exemplary embodiment. The information may be provided via a variety of output interfaces (e.g., a screen, a monitor, a storage device, and / or a web browser, etc.).

[0043] The slide analysis tool 101 and each of its components may transmit and receive digitized slide images and / or patient information to the server system 110, the physician server 121, the hospital server 122, the clinical trial server 123, the laboratory server 124, and / or the laboratory information system 125 via the electronic network 120. The server system 110 may also include a storage device to store images and data received from at least one of the slide analysis tool 101, the data acquisition tool 102, the slide importer tool 103, the slide scanner 104, the slide manager 105, and the viewing application tool 108. The server system 110 may also include a processing device to process the images and data stored in the storage device. The server system 110 may further include one or more machine learning tool(s) or functionality, such as by the processing device. Alternatively or in addition, the present disclosure (or portions of the systems and methods of the present disclosure) may be implemented on a local processing device (e.g., a laptop).

[0044] Any of the above devices, tools and modules may be located on devices that may be connected to an electronic network 120, such as the Internet or a cloud service provider, via one or more computers, servers and / or mobile devices.

[0045] 1C illustrates an example block diagram of a slide analysis tool 101 according to an example embodiment of the present disclosure. The slide analysis tool 101 may include a training image platform 131 and / or a target image platform 135.

[0046] According to one embodiment, the training image platform 131 may include a training image capture module 132 , a data analysis module 133 , and a cell identification module 134 .

[0047] The training image platform 131 according to an embodiment may create or receive training images used to train the machine learning model to efficiently analyze and classify digital pathology images. For example, the training images may be received from any one or any combination of the server system 110, the physician server 121, the hospital server 122, the clinical trial server 123, the laboratory server 124, and / or the laboratory information system 125. The images used for training may be from real sources (e.g., humans, animals, etc.) or synthetic sources (e.g., graphic rendering engines, 3D models, etc.). The digital pathology images may include (a) digitized slides stained with various stains such as hematoxylin and eosin (H&E), hematoxylin alone, IHC, molecular pathology, etc., and / or (b) digitized tissue samples from 3D imaging devices such as micro-CT.

[0048] The training image capture module 132 may create or receive a dataset including one or more training datasets corresponding to one or more health variables and / or one or more data variables. For example, the training dataset may be received from any one or any combination of the server system 110, the physician server 121, the hospital server 122, the clinical trial server 123, the laboratory server 124, and / or the laboratory information system 125. The dataset may be held in a digital storage device. The data analysis module 133 may identify whether a set of individual cells belongs to the cells of interest or belongs to the background of the digitized image. The cell identification module 134 may analyze the digitized image and determine whether individual cells in the cytology sample need to be further analyzed. Identifying whether individual cells need to be further analyzed and useful for aggregating these regions may trigger a user alert.

[0049] According to an embodiment, the target image platform 135 may include a target image capture module 136, an analyte detection module 137, and an output interface 138. The target image platform 135 may receive a target image and apply a machine learning model to the received target image to determine a characteristic of the target dataset. For example, the target data may be received from any or a combination of the server system 110, the physician server 121, the hospital server 122, the clinical trial server 123, the laboratory server 124, and / or the laboratory information system 125. The target image capture module 136 may receive a target dataset corresponding to a target health variable or data variable. The analyte detection module 137 may apply a machine learning model to the target dataset to determine a characteristic of the target health variable or data variable. For example, the analyte detection module 137 may detect a trend of the target relationship. The analyte detection module 137 may also apply a machine learning model to the target dataset to determine a quality score of the target dataset. The analyte detection module 137 may also apply a machine learning model to the target image to determine whether a target element is present in the determined relationship.

[0050] The output interface 138 may be used to output information regarding the target data and the determined relationships (eg, to a screen, monitor, storage device, web browser, etc.).

[0051] Currently, molecular laboratories may evaluate tumor genomes using tissue samples from unstained formalin-fixed paraffin-embedded tissue (FFPE) slides or blood using next-generation sequencing (NGS) to identify gene-level mutations, fusions, or deletions. This evaluation may identify tumor origin, estimate prognosis, guide treatment strategies (such as targeted therapy, immuno-oncology therapy, or basket trials), and / or evaluate minimal residual disease (MRD). While this evaluation may be relatively simple for blood samples, it is a complex multi-step process for FFPE samples.

[0052] For FFPE, the current workflow may begin with a primary diagnosis of the tumor by a pathologist, either from a biopsy or resection slide. Tissue samples from the patient may be embedded into FFPE blocks, portions of each block may be sliced ​​to prepare slides, the slides may be used to perform the diagnosis, and the remainder of the FFPE block may be used for genomic sequencing.

[0053] After the primary diagnosis, the treating oncologist may request pan-tumor or specific genetic testing. The request may be presented to a signed-out pathologist who reviews slides from each FFPE block containing the optimal tumor section(s). To identify the optimal tumor section(s), the block containing the existing slides with the highest tumor purity and the least necrosis and / or inflammation may be selected. This step may be done via manual re-review of slides, which can be very laborious. After the block is identified, 11 unstained slides may be excised from the block into the workflow for the pan-tumor NGS panel. The 11th slide may then be stained with H&E and evaluated for residual tumor to ensure that the previous 10 unstained slides have sufficient tumor volume. These 10 unstained slides may be sent to the molecular testing lab along with a requisition / form containing basic patient information (age, sex, topline diagnosis).

[0054] Upon arrival at the molecular testing lab, the first unstained slide may be stained with H&E and the exact location of the tumor may be assessed under a microscope by a technician. The tumor may be annotated by the technician using either a diamond pencil or a marker. This location may be crudely marked on the remaining 9 unstained slides so that the technician may "macro-excise" these tumor-rich areas from the slide. Macro-excision involves scraping the unstained FFPE tissue from the surface of the slide with a blade and aspirating the FFPE tissue so that the tissue can be submerged and DNA can be extracted. Polymerase chain reaction (PCR) testing may then be performed on those tumor sections and the results may be passed to a robust bioinformatics data pipeline. A molecular pathologist may analyze the results and classify the variants into various stages of clinical significance and action.

[0055] If there is enough tumor for molecular testing, molecular testing may be performed on the sample after it is sent to a laboratory. The laboratory may generate a comprehensive report that is sent back to the primary diagnostic pathologist, and this report may be attached to the original diagnostic report for subsequent review by the oncologist. This process may take up to two weeks, but may take longer if there is insufficient tumor, and may take an additional two weeks.

[0056] Identifying the quality of one or more slides from an FFPE block for further testing (e.g., application of molecular / genomic testing) using AI The disclosed systems and methods may facilitate the selection of the highest quality FFPE blocks by using artificial intelligence (AI) to provide a quality score Q for each FFPE block based on available information for each block. The quality score Q may indicate the quality of each block (e.g., tumor content, tumor purity, necrotic content, etc.) and may be used to determine the best or most suitable block for further testing. Various methods for determining the quality score Q are described below. The quality and / or quality score Q of a block may depend on tumor content, tumor purity, necrotic content, etc. The AI ​​system and / or pathologist may then select the block with the highest quality. The AI ​​system may also facilitate a validation process to ensure that the blocks are of sufficient quality by evaluating the quality of the last (e.g., eleventh) slide in each block and determining if the quality of that slide meets certain quality and / or quantity thresholds and / or selecting a new block if the slide does not meet quality and / or quantity thresholds.

[0057] 2 is a flow chart illustrating an exemplary method for verifying the quality of test slides and blocks according to an exemplary embodiment of the present disclosure. For example, the exemplary method 200 (e.g., steps 202-212) may be performed by the slide analysis tool 101 automatically or in response to a request from a user.

[0058] Step 202 may include receiving a collection of digital histopathology images associated with the tissue block in an electronic storage device (eg, a hard drive, a network drive, cloud storage, system memory, etc.).

[0059] In step 204, the method may include dividing each of the digital histopathology images into a set of tiles associated with the digital histopathology image.

[0060] In step 206, the method may include providing or determining a quality score Q for each of the FFPE blocks based on information available for each block. The quality of each FFPE block may depend on tumor content, tumor purity, amount of necrosis, etc. Determining the quality score Q may be performed using an AI system that can measure attributes such as these attributes from all slides prepared from each block and then determine a quality metric for each slide in the block based on the measured attributes. The AI ​​system may be run on slides from each block and output a quality score Q for each block. The AI ​​system may also be configured to prioritize or rank the blocks based on the determined quality scores Qs.

[0061] In step 208, the method may include evaluating the quality of each block based on a quality score Q and selecting or filtering the optimal blocks for use in the inspection. For example, selecting the optimal blocks for use in the inspection may include selecting the blocks with the highest quality scores Q, or selecting the blocks with quality scores Q that exhibit the highest quality across the slide. This evaluation and / or selection may be performed by a user (e.g., a pathologist evaluating the quality scores Q of the output) or may be performed using an AI system.

[0062] For example, tumor block selection may be performed by a signed-out pathologist whose criteria for optimal block selection include tumor quantity and quality (i.e., minimal changes at the biopsy site, inflammation, etc.) Algorithms designed or configured to detect and quantify tumors may be complemented with algorithms that detect blocks of tumor most likely to yield positive molecular findings.

[0063] In step 210, the method may include selecting or preparing N slides from the block of selection and evaluating the quality of the Nth or last slide (e.g., the 11th slide). This evaluation may be performed by an AI system. Alternatively, N slides may be prepared from the block of selection and the Nth or last slide (e.g., N=11) may be stained, such as with H&E, to verify that the block is of sufficient quality. Alternatively, N-1 slides may be stained. This verification may include manual inspection.

[0064] In step 212, the method may include determining whether the Nth slide meets the quality threshold. If the Nth slide does not meet the quality threshold, the method may include selecting a new block for evaluation.

[0065] During step 206 of evaluating the quality score Q and / or step 210 of evaluating the quality of the Nth slide, the quality assessment system may be described by a function Q(I1, I2, ..., IN, C), where the function may output a number indicating the quality, with a higher number indicating a higher quality. Each IN may be a digital image of the slide in a block. C may describe a specific constraint, such as "don't select blocks with more than p% necrosis", or C may describe the weights to use for each of the relevant variables for quality, e.g. the quality is a weighted sum of various measures such as the percentage of necrosis. There may be multiple ways to generate Q.

[0066] Using AI systems to infer attributes The AI ​​system may estimate n attributes vi, such as the percentage of necrosis v1, the percentage of tumor v2, etc. After computing the attributes vi, the attributes vi are integrated linearly or nonlinearly to compute a quality score Q for the block.

[0067] 3A and 3B are flow charts illustrating an exemplary method for verifying the quality of test slides and blocks according to an exemplary embodiment of the present disclosure. For example, the exemplary method 300 shown in FIG. 3A (e.g., steps 302-312) or the exemplary method 320 shown in FIG. 3B (e.g., steps 322-334) may be performed automatically by the slide analysis tool 101 or in response to a request from a user.

[0068] In step 302, the method may include receiving a collection of digital tissue images into a digital storage device (eg, a hard drive, a network drive, cloud storage, RAM, etc.).

[0069] Step 304 may include receiving summary annotations including one or more labels for each slide image or set of slide images. These labels may be at pixel level, tile level, slide level, or sub-specimen level. Labels may be binary (or multi-label binary), categorical, ordinal, or real value. These variables may describe, for example, the presence of quality-related attributes such as invasive cancer, necrosis presence and / or percentage, etc.

[0070] In step 306, the method may include dividing each slide image into a set of tiles.

[0071] In step 308, the method may include detecting and / or segmenting tissue regions from the background of each slide image to create a tissue mask and removing all non-tissue tiles. This can be done in a variety of ways, including but not limited to: a. Followed by the execution of connected component algorithms, e.g., threshold-based methods based on color / luminance, texture features, Otsu's method, etc. b. Segmentation algorithms, e.g., k-means, graph cut, Mask R-CNN, etc.

[0072] In step 310, the method may include training a machine learning model to estimate quality related attributes taking as input all locations on the slide, except for locations that have been removed as background. The model may be a Support Vector Machine (SVM), a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Transformer, a Graph Neural Network (GNN), a Multilayer Perceptron (MLP), a Relational Network, etc. The system may be trained to generate an image for each relevant attribute, such as a binary image indicating where necrosis is found, or it may be trained to directly output the amount of the relevant variable present, such as a number indicating the percentage of necrosis, another number indicating tumor purity, or whether there is an amount of the relevant variable, such as tumor tissue above a predefined threshold.

[0073] Step 312 may include storing parameters of the trained machine learning model in electronic storage.

[0074] Referring to FIG. 3B, at step 322, the method may include receiving the collection of digital tissue pathology images in a digital storage device (eg, a hard drive, a network drive, cloud storage, RAM, etc.).

[0075] In step 324, the method may include dividing each slide image into a set of tiles.

[0076] In step 326, the method may include detecting and / or segmenting tissue regions from the background of each slide image to create a tissue mask and removing all non-tissue tiles. This can be done in a variety of ways, including but not limited to: a. Followed by the execution of connected component algorithms, e.g., threshold-based methods based on color / luminance, texture features, Otsu's method, etc. b. Segmentation algorithms, e.g., k-means, graph cut, Mask R-CNN, etc.

[0077] In step 328, the method may include running the trained machine learning model on each set of digital tissue images corresponding to the instances.

[0078] In step 330, the method may include summing any digital histopathology images showing the presence of each attribute and normalizing them by the total amount of tissue to generate a percentage of that variable as an output, or outputting each relevant variable. It may be determined whether the total amount of tissue meets or exceeds a predefined threshold. The system may be trained to generate an "image" showing the presence of each attribute, for example an image showing where necrosis is found. Alternatively, the system may be trained to directly output each relevant variable, where the i-th relevant variable is V i It may be shown as:

[0079] In step 332, the method may include outputting a quality score that is a function of the relevant variables, either as a linear or non-linear function. This function may be a linear combination specified by the values ​​in the configuration C, for example, where N is the total number of relevant variables, the output Q may be w1v1+w2v2+...+w N v N This score may also be non-linear, for example if necrosis is greater than p% and quality is 0, otherwise it is equal to w1v1+w2v2+...+w N v N is a linear combination specified by the configuration C given by

[0080] In step 334, the method may include writing the quality score of each block to electronic storage.

[0081] Use of systems that directly learn quality This method uses an AI system that directly infers, without necessarily generating intermediate variables, whether the slides in a block are adequate for testing or whether the slides in a block are insufficient.

[0082] 4A and 4B are flow charts illustrating an exemplary method for verifying the quality of test slides and blocks according to an exemplary embodiment of the present disclosure. For example, the exemplary method 400 shown in FIG. 4A (e.g., steps 402-412) or the exemplary method 420 shown in FIG. 4B (e.g., steps 422-434) may be performed automatically by the slide analysis tool 101 or in response to a request from a user.

[0083] In step 402, the method may include receiving a collection of digital tissue images into a digital storage device (eg, a hard drive, a network drive, cloud storage, RAM, etc.).

[0084] In step 404, the method may include receiving a binary value indicating whether the slide images in each FFPE block were suitable or unsuitable for review.

[0085] In step 406, the method may include dividing each slide image into a set of tiles.

[0086] In step 408, the method may include detecting and / or segmenting tissue regions from the background of each slide image to create a tissue mask and removing all non-tissue tiles. This can be done in a variety of ways, including but not limited to: a. Followed by the execution of connected component algorithms, e.g., threshold-based methods based on color / luminance, texture features, Otsu's method, etc. b. Segmentation algorithms, e.g., k-means, graph cut, Mask R-CNN, etc.

[0087] In step 410, the method may include training a machine learning model to estimate whether all locations on the slide, other than the removed locations, were suitable, required additional testing, or were not suitable, and output an image of each relevant attribute or a quantity of relevant variables present. The model may be a support vector machine (SVM), a convolutional neural network (CNN), a recurrent neural network (RNN), a transformer, a graph neural network (GNN), a multi-layer perceptron (MLP), a relational network, a semantic segmentation network, an instance segmentation network (e.g., Mask R-CNN), an object detection CNN (e.g., Fast R-CNN), etc. The system may be trained to generate an image for each relevant attribute, such as a binary image indicating where necrosis was found, or it may be trained to directly output the quantity of relevant variables present, such as a number indicating the percentage of necrosis, another number indicating tumor purity, etc.

[0088] In step 412, the method may include saving the trained model in electronic storage.

[0089] In step 422, the method may include receiving the collection of digital tissue images into a digital storage device (eg, a hard drive, a network drive, cloud storage, RAM, etc.).

[0090] In step 424, the method may include dividing each slide image into a set of tiles.

[0091] In step 426, the method may include detecting and / or segmenting tissue regions from the background of each slide image to create a tissue mask and removing all non-tissue tiles. This can be done in a variety of ways, including but not limited to: a. Followed by the execution of connected component algorithms, e.g., threshold-based methods based on color / luminance, texture features, Otsu's method, etc. b. Segmentation algorithms, e.g., k-means, graph cut, Mask R-CNN, etc.

[0092] In step 428, the method may include running the trained machine learning model for each set of slides corresponding to a block to output a quality score and storing a quality score in electronic storage for each block.

[0093] In step 430, the method may include selecting the block with the highest quality score for subsequent inspection. This may be done manually by a user or automatically by the slide analysis tool 101.

[0094] In step 432, the method may include prompting the user whether the block should prepare additional slides for review.

[0095] Use to verify that the last slide from a block of selections is of adequate quality 5A and 5B are flow charts illustrating an exemplary method for verifying the quality of test slides and blocks according to an exemplary embodiment of the present disclosure. For example, the exemplary method 500 shown in FIG. 5A (e.g., steps 502-512) or the exemplary method 520 shown in FIG. 5B (e.g., steps 522-534) may be performed by the slide analysis tool 101 automatically or in response to a request from a user.

[0096] In step 502, the method may include receiving a collection of digital tissue images into a digital storage device (eg, a hard drive, a network drive, cloud storage, RAM, etc.).

[0097] In step 504, the method may include receiving a binary value indicating whether the slide images in each FFPE block were from an FFPE block that was suitable for inspection or was unsuitable for inspection.

[0098] In step 506, the method may include dividing each slide image into a set of tiles.

[0099] In step 508, the method may include detecting and / or segmenting tissue regions from the background of each slide image to create a tissue mask and removing all non-tissue tiles. This can be done in a variety of ways, including but not limited to: a. Followed by the execution of connected component algorithms, e.g., threshold-based methods based on color / luminance, texture features, Otsu's method, etc. b. Segmentation algorithms, e.g., k-means, graph cut, Mask R-CNN, etc.

[0100] In step 510, the method may include training a machine learning model to estimate whether all locations on the slide, other than the removed locations, were suitable, required additional testing, or were not suitable, and output an image of each relevant attribute or a quantity of relevant variables present. The model may be an SVM, CNN, RNN, transformer, graph neural network, MLP, relational network, semantic segmentation network, instance segmentation network (e.g., Mask R-CNN), object detection CNN (e.g., Fast R-CNN), etc. The system may be trained to generate an image for each relevant attribute, such as a binary image indicating where necrosis was found, or it may be trained to directly output the quantity of relevant variables present, such as a number indicating the percentage of necrosis, another number indicating tumor purity, etc.

[0101] In step 512, the method may include saving the trained model in electronic storage.

[0102] In step 522, the method may include receiving the collection of digital tissue images into a digital storage device (eg, a hard drive, a network drive, cloud storage, RAM, etc.).

[0103] In step 524, the method may include dividing each slide image into a set of tiles.

[0104] In step 526, the method may include detecting and / or segmenting tissue regions from the background of each slide image to create a tissue mask and removing all non-tissue tiles. This can be done in a variety of ways, including but not limited to: a. Followed by the execution of connected component algorithms, e.g., threshold-based methods based on color / luminance, texture features, Otsu's method, etc. b. Segmentation algorithms, e.g., k-means, graph cut, Mask R-CNN, etc.

[0105] In step 528, the method may include running the trained machine learning model on each of the set of slides corresponding to the block to output a quality score and storing a quality score in electronic storage for each block.

[0106] In step 530, the method may include determining whether the quality of the set of slides is sufficiently low (e.g., below a predetermined quality) and, if the quality of the set of slides is sufficiently low, indicating to a user that a new block should be prepared for inspection.

[0107] 6 is an exemplary use of a machine learning model to assess block quality. A machine learning model 604 may have an input 602. In an exemplary illustration, the input 602 is represented by a number of tissue blocks. The input 602 is analyzed for quality by the machine learning model 604 using variables 608 (e.g., tumor content, tumor purity, and / or necrosis rate), although a different number of variables may be used. A quality metric 610 may be output from the analysis using the variables 608 and then used to sort the input 602 (blocks) in step 612. The output 606 may be the blocks sorted in step 612.

[0108] Sequential recurrence scores for invasive breast cancer After invasive breast cancer is detected, genomic assays of the tumor may be performed to determine whether to forgo further treatment and place the patient on endocrine (hormonal) therapy, adjuvant chemotherapy, or some other therapy. These tests may assess the risk of disease recurrence and metastasis after removal of the primary tumor using a continuous scoring system. Additionally, tests may look for genomic information regarding proliferation, invasion, metastasis, stromal integrity, and angiogenesis.

[0109] For example, the EndoPredict (EPclin) test is based on ribonucleic acid (RNA) expression of 12 genes and combines this genomic information with additional clinical features to predict 10-year distant recurrence (DR). A score of 1 to 6 may be assigned to the input tissue, with 6 indicating high risk and 1 indicating low risk. Another exemplary test is MammaPrint, a 70-gene assay that uses formalin-fixed paraffin-embedded tissue (FFPE) or fresh tissue. RNA is isolated from tumor samples and used to predict a continuous score, with values ​​greater than 0 indicating a low risk of cancer recurrence and values ​​less than 0 indicating a high risk of recurrence, suggesting the need for adjuvant chemotherapy.

[0110] Another exemplary test is the Breast Cancer Index (BCI) test, which analyzes seven genes to predict cancer recurrence. Two scores are output: the BCI Prognostic Score and the BCI Predictive Score. The BCI Prognostic Score estimates the likelihood of cancer recurrence 5-10 years after diagnosis on a continuous scale of 0-10, with scores of 5.1-10 indicating a high risk of recurrence. The BCI Predictive Score estimates the likelihood of benefit from an additional 5 years of endocrine therapy versus a total of 10 years of endocrine therapy.

[0111] The Tumor-Type DX Recurrence Score is another such analysis that examines the expression of 21 genes in tumors. An output number between 0 and 100 indicates risk of cancer recurrence, a score greater than 31 indicates high risk of metastasis and the need for adjuvant chemotherapy with endocrine therapy, a score between 26 and 30 indicates an uncertain benefit of adjuvant chemotherapy when used with endocrine therapy, and a score less than 26 indicates that postoperative treatment with endocrine therapy alone is sufficient.

[0112] The Prosigna Breast Cancer Prognostic Gene Signature Assay (i.e., PAM50 Gene Signature) assesses the risk of distant recurrence in hormone receptor-positive breast cancer using RNA from FFPE samples. It generates a continuous score from 0 to 100, with higher scores indicating higher risk of recurrence, to guide treatment.

[0113] All of these tests may require selecting the appropriate block, verifying that an unstained slide from that block has the appropriate tumor tissue, and determining the location of the tumor in the slide before applying the genomic assay. With current technology, these steps are manual, time-consuming, and error-prone. The techniques described herein may select and verify high-quality slides with sufficient tumor content for genomic testing. The techniques described herein may include steps that are more automated, faster, and more accurate than current techniques.

[0114] Continuous Scoring for Recurrence of Noninvasive Breast Cancer Following a diagnosis of non-invasive breast cancer, patients may require adjuvant treatment after undergoing lumpectomy or mastectomy. This treatment may include endocrine therapy or radiation therapy to reduce the risk of recurrence, but these treatments have side effects. Genomic assays are developed to determine the benefit patients may derive from these treatments.

[0115] The most common form of non-invasive breast cancer is ductal carcinoma in situ (DCIS). Currently, the primary genomic test to determine treatment options for DCIS is Oncotype DX DCIS, a 12-panel genomic test. This test generates a continuous score from 0 to 100 to determine the risk of breast cancer recurrence, with higher scores indicating the need for adjuvant treatment to prevent recurrence.

[0116] The systems and methods of the present disclosure may be used to select and verify high quality slides with sufficient tumor content for genomic testing.

[0117] Prostate Cancer Treatment Recommendation Continuous Scoring Workflow To diagnose prostate cancer, men may undergo a prostate biopsy. The biopsy sample may then be processed and visually reviewed by a pathologist to determine the presence and extent of disease. However, prostate cancer treatments, such as removal of the prostate, hormone therapy, and radiation therapy, may negatively impact a man's quality of life, and some patients may not require aggressive treatment.

[0118] Instead of using only pathological evaluation of prostate tissue samples, genomic assays are used to predict tumor aggressiveness. For example, the Oncotype DX Genomic Prostate Score examines 17 genes to determine the aggressiveness of prostate cancer with a continuous score of 0 to 100. Patients with values ​​closer to 0 may be recommended active surveillance, whereas patients with high scores should receive early aggressive treatment to reduce the risk of adverse outcomes (e.g., death or metastasis). Another test is the Prolaris Assay, which combines genomic evaluation with other measurements to determine a continuous score where higher scores indicate more aggressive cancer. This test may help determine whether a man can opt for active surveillance for his prostate cancer instead of aggressive treatment.

[0119] The systems and methods of the present disclosure may be used to select and verify high quality slides with sufficient tumor content for genomic testing.

[0120] Workflow for continuous scoring of malignancy probability Tumors are abnormal masses of cells, either benign or malignant. Benign tumors lack the ability to metastasize or invade surrounding tissues, whereas malignant tumors do. In some cases, pathology evaluation is insufficient to determine whether a tumor is malignant or benign. In these types of scenarios, a continuous score may be used to facilitate the determination.

[0121] For example, the Myriad myPath Melanoma test measures 23 genes associated with cell differentiation, cell signaling and immune response signals to generate a continuous score on a scale of -16 to 10, with a score greater than 0 indicating the skin tumor is likely malignant and in need of aggressive treatment, and a score less than -2 indicating the tumor is likely benign.

[0122] The systems and methods of the present disclosure may be used to select and verify high quality slides with sufficient tumor content for genomic testing.

[0123] As shown in Figure 7, device 700 may include a central processing unit (CPU) 720. CPU 720 may be any type of processor device, including any type of dedicated or general purpose microprocessor device, etc. CPU 720 may also be a single processor in a multi-core / multi-processor system, such as a system operating alone, or a single processor in a cluster of computing devices operating in a cluster or server farm, as will be appreciated by those skilled in the art. CPU 720 may be connected to a data communications infrastructure 710, such as a bus, message queue, network, or multi-core message passing scheme.

[0124] The device 700 may include a main memory 740, e.g., a random access memory (RAM), and may include a secondary memory 730. The secondary memory 730, e.g., a read only memory (ROM), may be, e.g., a hard disk drive or a removable storage drive. Such removable storage drives may include, e.g., a floppy disk drive, a magnetic tape drive, an optical disk drive, a flash memory, and the like. The removable storage drive of this embodiment reads from and / or writes to the removable storage unit in a well-known manner. The removable storage unit may include a floppy disk, a magnetic tape, an optical disk, and the like that is read from and written to by the removable storage drive. As will be appreciated by those skilled in the art, such removable storage units generally include computer usable storage media having computer software and / or data stored thereon.

[0125] In alternative embodiments, secondary memory 730 may include similar means for allowing computer programs or other instructions to be loaded into device 700. Such means include program cartridges and cartridge interfaces (such as those present in video game devices), removable memory chips (such as EPROM or PROM) and associated sockets, and other removable storage units and interfaces that allow software and data to be transferred from removable storage units to device 700.

[0126] The device 700 may also include a communications interface (COM) 760. The communications interface 760 allows software and data to be transferred between the device 700 and external devices. The communications interface 760 may include a modem, a network interface (such as an Ethernet card), a communications port, a PCMCIA slot and card, or the like. The software and data transferred through the communications interface 760 may be in the form of electronic, electromagnetic and optical signals, or other signals receivable by the communications interface 760. These signals may be provided to the communications interface 760 over a communications path in the device 700, which may be implemented using, for example, wire or cable, fiber optics, a telephone line, a cellular phone link, an RF link, or other communications channel.

[0127] The hardware elements, operating systems, and programming languages ​​of such equipment are conventional in nature and are assumed to be well-known to those skilled in the art. The device 700 may also include input / output ports 750 for connecting input / output devices such as a keyboard, mouse, touch panel, monitor, display, etc. Of course, various server functions may be distributed across multiple similar platforms to distribute the processing load. Alternatively, the server may be implemented by appropriately programming a single computer hardware platform.

[0128] Throughout this disclosure, a component or module generally refers to an item that can be logically grouped together to perform a function or related functions. Like reference numbers are generally intended to refer to the same or similar components. Components and modules may be implemented in software, hardware, or a combination of software and hardware.

[0129] The tools, modules and functions described above may be executed by one or more processors. A "storage" type medium may include any or all of the tangible memory of a computer and processor, or its associated modules, such as various semiconductor memories, tape drives, disk drives, etc., that may provide non-transitory storage for software programming at any time.

[0130] The software may be communicated via the Internet, a cloud service provider, or other communications network. For example, the communication may enable the software to be loaded from one computer or processor to another. As used herein, unless limited to non-transitory tangible "storage" media, terms such as computer or machine "readable medium" refer to any medium that participates in providing instructions to a processor for execution.

[0131] The foregoing general description is exemplary and explanatory only and is not intended to limit the present disclosure. Other embodiments of the present invention will be apparent to those skilled in the art from the specification and embodiments of the invention disclosed herein, and it is intended that the specification and examples be considered as exemplary only.

Claims

1. A computer-implemented method for verifying the quality of test slides and blocks, comprising: receiving in a digital storage device a collection of one or more digital images associated with the tissue block and corresponding to the instance; applying a machine learning model to the collection to identify the presence or absence of necrosis in the tissue block from digital images in the collection; determining an amount or percentage of tissue having necrosis from the digital images in the collection that indicates the presence of necrosis; and outputting a quality score corresponding to the determined amount or percentage of tissue with necrosis being less than a predetermined value; Including, If the determined amount or percentage of tissue having necrosis is greater than the predetermined value, the quality score is equal to zero, and if the determined amount or percentage of tissue having necrosis is less than or equal to the predetermined value, the quality score is a linear combination of two or more attribute values.

2. The computer-implemented method of claim 1, wherein determining the amount or proportion includes summing and normalizing the digital images of the collection indicating the presence of necrosis or tumor by a total amount of tissue.

3. The method of claim 2, further comprising: dividing each digital image of the set of digital images into a set of tiles; detecting and / or segmenting tissue regions from a background of the digital image to create a tissue mask; removing all tiles from the set of tiles that contain the background; The computer-implemented method of claim 1 , further comprising:

4. The computer-implemented method of claim 3, wherein the detecting and / or segmenting includes executing a connected components algorithm using one or more threshold-based methods.

5. The computer-implemented method of claim 3, wherein the detecting and / or segmenting includes using one or more segmentation algorithms.

6. The computer-implemented method of claim 1, further comprising determining the tissue block having the highest quality score for subsequent testing.

7. The computer-implemented method of claim 6, further comprising presenting to a user that the tissue block has at least one additional slide prepared for examination.

8. The computer-implemented method of claim 6, further comprising determining whether the quality score is sufficiently low.

9. The computer-implemented method of claim 8, further comprising prompting a user to prepare a new block for inspection if the quality score is determined to be sufficiently low.

10. The computer-implemented method of claim 1, further comprising outputting a function of at least one variable that corresponds to the quality score.

11. The computer-implemented method of claim 10, wherein the function of the at least one variable is a linear function.

12. The computer-implemented method of claim 10, wherein the function of the at least one variable is a nonlinear function.

13. The computer-implemented method of claim 1, further comprising outputting a binary image indicating where the necrosis or tumor is located.

14. The computer-implemented method of claim 1, further comprising receiving a summary annotation including one or more labels for each digital image.

15. The computer-implemented method of claim 14, wherein the one or more labels are pixel-level labels, tile-level labels, slide-level labels, and / or partial specimen-level labels.

16. The computer-implemented method of claim 1, wherein the digital image is a digital pathology tissue image.

17. A system for verifying the quality of test slides and blocks using a machine learning model, comprising: at least one memory for storing instructions; at least one processor configured to execute the instructions to perform operations; Equipped with The operation is receiving in a digital storage device a collection of one or more digital images associated with the tissue block and corresponding to the instance; applying a machine learning model to the collection to identify the presence or absence of necrosis in the tissue block from digital images in the collection; determining an amount or percentage of tissue having necrosis from the digital images in the collection that indicates the presence of necrosis; and outputting a quality score corresponding to the determined amount or percentage of tissue with necrosis being less than a predetermined value; Including, If the determined amount or percentage of tissue having necrosis is greater than the predetermined value, the quality score is equal to zero, and if the determined amount or percentage of tissue having necrosis is less than or equal to the predetermined value, the quality score is a linear combination of two or more attribute values.

18. The system of claim 17, wherein determining the amount or proportion includes summing and normalizing the digital images of the collection indicating the presence of necrosis or tumor by a total amount of tissue.

19. A non-transitory computer readable medium storing instructions that, when executed by a processor, perform a method for verifying quality of test slides and blocks using a machine learning model, the method comprising: receiving in a digital storage device a set of one or more digital images associated with the tissue block and corresponding to the instance; applying a machine learning model to the collection to identify the presence or absence of necrosis in the tissue block from digital images in the collection; determining an amount or percentage of tissue having necrosis from the digital images in the collection that indicates the presence of necrosis; and outputting a quality score corresponding to the determined amount or percentage of tissue with necrosis being less than a predetermined value; Including, If the determined amount or percentage of tissue having necrosis is greater than the predetermined value, the quality score is equal to zero, and if the determined amount or percentage of tissue having necrosis is less than or equal to the predetermined value, the quality score is a linear combination of two or more attribute values.

20. The method according to claim 1, further comprising: dividing each digital image of the set of digital images into a set of tiles; detecting and / or segmenting tissue regions from a background of the digital image to create a tissue mask; removing all tiles from the set of tiles that contain the background; 20. The method of claim 19, further comprising: