AI-powered system and method for molecular workflows to verify the quality of inspection slides and blocks.

Abstract

A system and method for verifying the quality of a test slide or block is disclosed. The method may include receiving a collection of one or more digital images in a digital storage device. The collection may be associated with a tissue block and correspond to an instance. The method may include applying a machine learning model to the collection to identify the presence or absence of an attribute, determining an amount or percentage of tissue having the attribute from the digital images in the collection indicating the presence of the attribute, and outputting a quality score corresponding to the determined amount or percentage.

Description

Technical Field

[0001] Related Applications This application claims priority to U.S. Provisional Application No. 63 / 158,781, filed on March 9, 2021, the entire disclosure of which is incorporated herein by reference.

[0002] Various embodiments of the present disclosure generally relate to image processing methods. More specifically, certain embodiments of the present disclosure relate to systems and methods for verifying the quality of inspection slides and blocks.

Background Art

[0003] Current workflows for tumor genome sequencing have many inefficiencies. For example, in current workflow processing, a pathologist may need to review materials to select the best slide or section of a tumor region for sequencing, which can be time-consuming and laborious.

[0004] Difficulties in current technologies include identifying the best tissue blocks and / or slides taken from human or animal patients for inspection (e.g., genome sequencing), and confirming that the selected tissue blocks and / or slides taken from the patient contain appropriate tumor tissue.

[0005] The description of the background art in this specification is for the purpose of generally indicating the context of the present disclosure. Unless specifically mentioned herein, the matters described in this section are not prior art to the claims of this application, and by being included in this section, it is not admitted to be prior art and its implications.

Summary of the Invention

Means for Solving the Problems

[0006] According to an aspect of the present disclosure, a system and method for verifying the quality of inspection slides and blocks are disclosed.

[0007] A method for verifying the quality of inspection slides and blocks may be performed on a computer. The method may include receiving a set of one or more digital images in a digital storage device. The set may be associated with tissue blocks and correspond to instances. The method may include applying a machine learning model to the set to identify the presence or absence of attributes, determining the amount or proportion of tissue having the attributes from the digital images in the set that indicate the presence of attributes, and outputting a quality score corresponding to the determined amount or proportion.

[0008] Determining the quantity or proportion may include summing and normalizing the digital images of sets indicating the presence of an attribute by the total amount of tissue. The digital images may be digital pathological tissue images.

[0009] The method may include dividing each digital image in a set of digital images into a set of tiles, detecting and / or segmenting tissue regions from the background of the digital images to create a tissue mask, and removing all tiles from the set of tiles that include the background. Detection and / or segmentation may include performing a connected component algorithm using one or more threshold-based methods. Detection and / or segmentation may also include using one or more segmentation algorithms.

[0010] The method may include determining the tissue block with the highest quality score for subsequent inspection. The method may also include informing the user that the tissue block has at least one additional slide prepared for inspection. The method may also include determining whether the quality score is likely to be sufficiently low. If it is determined that the quality score is likely to be sufficiently low, the method may include informing the user to prepare a new inspection block.

[0011] The method may include outputting a function of at least one variable corresponding to the quality score. The function of at least one variable may be a linear function. The function of at least one variable may be a nonlinear function.

[0012] The method may include outputting a binary image showing where the attributes are located. The method may also include receiving summary annotations. The summary annotations may include one or more labels for each digital image. The one or more labels may be pixel-level labels, tile-level labels, slide-level labels, and / or subsample-level labels.

[0013] A system for verifying the quality of inspection slides and blocks using a machine learning model includes at least one memory for storing instructions and at least one processor configured to execute instructions and perform operations. Operations may include receiving a set of one or more digital images in a digital storage device. The set may be associated with tissue blocks and correspond to instances. Operations may include applying a machine learning model to the set to identify the presence or absence of attributes, determining the amount or proportion of tissue having the attributes from the images in the set indicating the presence of attributes, and outputting a quality score corresponding to the determined amount or proportion.

[0014] Determining a quantity or proportion may include summing and normalizing the digital images of sets indicating the presence of an attribute by the total quantity of the organization.

[0015] A non-transitory computer-readable medium may store instructions that, when executed by a processor, perform a method for verifying the quality of inspection slides and blocks using a machine learning model. The method may include receiving, in a digital storage device, a set of one or more digital images. The set may be associated with tissue blocks and may correspond to instances. The method may include applying a machine learning model to the set to identify the presence or absence of an attribute, determining, from the digital images in the set that indicate the presence of the attribute, the amount or percentage of tissue having the attribute, and outputting a quality score corresponding to the determined amount or percentage.

[0016] The method may further include dividing each digital image of the set of digital images into a set of tiles, detecting and / or segmenting tissue regions from the background of the digital images to create a tissue mask, and removing all tiles of the set of tiles that include the background.

[0017] It should be understood that neither the foregoing description nor the following detailed description is restrictive but merely exemplary and explanatory of the disclosed embodiments.

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various exemplary embodiments and, together with the description herein, serve to explain the principles of the disclosed embodiments. The present invention provides, for example, the following: (Item 1) A computer implementation method for verifying the quality of inspection slides and blocks, Receiving a set of one or more digital images associated with an organizational block and corresponding to an instance in a digital storage device, Applying a machine learning model to the aforementioned set to identify the presence or absence of attributes, Determining the amount or proportion of tissue having the attribute from the digital image in the set that shows the presence of the attribute, Outputting a quality score corresponding to the determined quantity or percentage, The computer implementation method, including the above. (Item 2) The computer implementation method according to item 1, wherein determining the amount or proportion includes summing and normalizing the digital images of the set indicating the presence of the attribute by the total amount of the organization. (Item 3) Dividing each digital image in the aforementioned set of digital images into a set of tiles, The process involves detecting and / or segmenting tissue regions from the background of the aforementioned digital image to create a tissue mask. Removing all tiles from the set of tiles including the background, The computer implementation method described in item 1, further including the method described in item 1. (Item 4) The computer implementation method described in item 3, wherein the detection and / or segmentation includes performing a connection component algorithm using one or more threshold-based methods. (Item 5) The computer implementation method according to item 3, wherein the detection and / or segmentation includes using one or more segmentation algorithms. (Item 6) The computer implementation method described in item 1 further includes determining the tissue block having the highest quality score for subsequent inspection. (Item 7) The computer implementation method according to item 6 further includes presenting to the user that the organizational block has at least one additional slide prepared for examination. (Item 8) The computer implementation method according to item 6, further comprising determining whether the quality score is sufficiently low. (Item 9) The computer implementation method described in item 8, further comprising presenting the user with the option to prepare a new test block if the aforementioned quality score is determined to be sufficiently low. (Item 10) The computer implementation method of item 1, further comprising outputting a function of at least one variable corresponding to the aforementioned quality score. (Item 11) The computer implementation method according to item 10, wherein the function of the at least one variable is a linear function. (Item 12) The computer implementation method according to item 10, wherein the function of the at least one variable is a nonlinear function. (Item 13) The computer implementation method according to item 1, further comprising outputting a binary image showing where the aforementioned attribute is located. (Item 14) The computer implementation method described in item 1, further comprising receiving summary annotations containing one or more labels for each digital image. (Item 15) The computer implementation method according to item 14, wherein the one or more labels are pixel-level labels, tile-level labels, slide-level labels and / or partial sample-level labels. (Item 16) The aforementioned digital image is a digital pathological tissue image, as described in item 1, computer implementation method. (Item 17) A system for verifying the quality of inspection slides and blocks using a machine learning model, At least one memory for storing instructions, At least one processor configured to execute the aforementioned instructions and perform operations, Equipped with, The aforementioned operation is, Receiving a set of one or more digital images associated with an organizational block and corresponding to an instance in a digital storage device, Applying a machine learning model to the aforementioned set to identify the presence or absence of attributes, Determining the amount or proportion of tissue having the attribute from the digital image in the set that shows the presence of the attribute, Outputting a quality score corresponding to the determined quantity or percentage, The system including the above. (Item 18) The system according to item 17, wherein determining the quantity or proportion includes summing and normalizing the digital images of the set indicating the presence of the attribute by the total amount of the organization. (Item 19) A non-temporary computer-readable medium that stores instructions for performing a method of verifying the quality of inspection slides and blocks using a machine learning model when executed by a processor, wherein the method is Receiving a set of one or more digital images associated with an organizational block and corresponding to an instance in a digital storage device, Applying a machine learning model to the aforementioned set to identify the presence or absence of attributes, Determining the amount or proportion of tissue having the attribute from the digital image in the set that shows the presence of the attribute, Outputting a quality score corresponding to the determined quantity or percentage, The non-temporary computer-readable medium, including the said non-temporary computer-readable medium. (Item 20) The aforementioned method, Dividing each digital image in the aforementioned set of digital images into a set of tiles, The process involves detecting and / or segmenting tissue regions from the background of the aforementioned digital image to create a tissue mask. Removing all tiles from the set of tiles including the background, The method described in item 19, further including the method described in item 19. [Brief explanation of the drawing]

[0019] [Figure 1A] An exemplary block diagram of a system and network for verifying the quality of inspection slides and blocks according to an exemplary embodiment of this disclosure is shown.

[0020] [Figure 1B] An exemplary block diagram of a disease detection platform according to an exemplary embodiment of this disclosure is shown.

[0021] [Figure 1C] An exemplary block diagram of a slide analysis tool according to an exemplary embodiment of the present disclosure is shown.

[0022] [Figure 2] This flowchart shows a method for verifying the quality of inspection slides and blocks according to exemplary embodiments of the present disclosure.

[0023] [Figure 3A]This flowchart shows a method for training and using a machine learning model according to an exemplary embodiment of the present disclosure to output a quality score function. [Figure 3B] This flowchart shows a method for training and using a machine learning model according to an exemplary embodiment of the present disclosure to output a quality score function.

[0024] [Figure 4A] This flowchart shows a method for determining whether formalin-fixed paraffin-embedded tissue (FFPE) should have an additional slide prepared for examination, using a machine learning model trained according to an exemplary embodiment of the present disclosure. [Figure 4B] This flowchart shows a method for determining whether formalin-fixed paraffin-embedded tissue (FFPE) should have an additional slide prepared for examination, using a machine learning model trained according to an exemplary embodiment of the present disclosure.

[0025] [Figure 5A] This flowchart shows a method for determining whether a new FFPE block in an organization should be prepared for inspection by training and using a machine learning model according to an exemplary embodiment of the present disclosure. [Figure 5B] This flowchart shows a method for determining whether a new FFPE block in an organization should be prepared for inspection by training and using a machine learning model according to an exemplary embodiment of the present disclosure.

[0026] [Figure 6] This is an exemplary use of a machine learning model for evaluating block quality according to an exemplary embodiment of the present disclosure.

[0027] [Figure 7] An example of a system capable of implementing the technology described herein is shown. [Modes for carrying out the invention]

[0028] Exemplary embodiments of the present disclosure are described below in detail, and examples thereof are shown in the accompanying drawings. Wherever possible, the same reference numerals are used throughout the drawings to indicate identical or similar parts.

[0029] The systems, apparatus, and methods disclosed herein will be described in detail below with reference to the drawings. The examples described herein are merely illustrative and are provided to aid in the description of the apparatus, apparatus, systems, and methods herein. Furthermore, none of the features or components shown or described below should be considered essential for the implementation of any of these apparatus, systems, or methods unless specifically designated as essential.

[0030] Furthermore, regardless of the method of explanation, and regardless of whether the method is explained using a flowchart, unless otherwise specified or required by the context, the explicit or implicit ordering of the steps performed in the execution of the method does not mean that those steps must be performed in the order presented. Instead, they may be performed in a different order or in parallel.

[0031] As used herein, the term “exemplary” means “example” and not “ideal.” Furthermore, “a” and “an” in this specification do not indicate a quantitative limitation, but rather indicate the existence of one or more of the items being referenced.

[0032] The techniques disclosed herein may use AI technology to identify the best tissue blocks and / or slides for examination and to verify that selected tissue blocks and / or slides taken from a patient contain an appropriate amount of tumor. The data and predictions may be instantly aggregated and made available through any user interface (e.g., via a digital pathology browsing system, report, laboratory information system, etc.). Machine learning algorithms may limit total molecular testing for tumors to rapidly and simultaneously assess validity, classify samples into diagnostic categories, and screen for the most likely molecular changes, thereby increasing the likelihood of valid molecular results from a sufficient amount of tumor.

[0033] Figure 1A shows a block diagram of a system and network for verifying the quality of slides and blocks using machine learning according to an exemplary embodiment of the present disclosure.

[0034] Specifically, Figure 1A shows an electronic network 120 which may be connected to servers in a hospital, laboratory and / or physician's office, etc. For example, a physician server 121, a hospital server 122, a clinical trial server 123, a laboratory server 124 and / or a laboratory information system 125, etc., may each be connected to the electronic network 120, such as the Internet, via one or more computers, servers and / or mobile terminals. According to exemplary embodiments of the present application, the electronic network 120 may also be connected to a server system 110 which may include a processing unit configured to implement a disease detection platform 100, including a slide analysis tool 101 for analyzing tissue in a WSI according to exemplary embodiments of the present disclosure.

[0035] The physician server 121, hospital server 122, clinical trial server 123, laboratory server 124, and / or laboratory information system 125 may create, or otherwise acquire, images of cytology specimens(multiple), pathology tissue specimens(multiple), and slides(multiple) of cytology specimens(multiple), digitized images of slides(multiple) of pathology tissue specimens(multiple), or combinations thereof, for one or more patients. The physician server 121, hospital server 122, clinical trial server 123, laboratory server 124, and / or laboratory information system 125 may also acquire any combination of patient-specific information such as age, medical history, cancer treatment history, family history, and past biopsy or cytology information. The physician server 121, hospital server 122, clinical trial server 123, laboratory server 124, and / or laboratory information system 125 may transmit digitized slide images and / or patient-specific information to the server system 110 via the electronic network 120. The server system 110 may include one or more storage devices 109 for storing images and data received from at least one of the physician server 121, hospital server 122, clinical trial server 123, laboratory server 124, and / or laboratory information system 125. The server system 110 may also include a processing device for processing the images and data stored in the one or more storage devices 109. The server system 110 may further include one or more machine learning tools or functions. For example, the processing device may include a machine learning tool for a disease detection platform 100 according to one embodiment. Alternatively, or in addition, the Disclosure (or parts of the systems and methods of the Disclosure) may be implemented on a local processing device (e.g., a laptop).

[0036] The physician server 121, hospital server 122, clinical trial server 123, laboratory server 124, and / or laboratory information system 125 refer to systems used by pathologists to review slide images. In a hospital environment, tissue type information may be stored in the laboratory information system 125. According to exemplary embodiments of this disclosure, cells in WSI that share similar targets and do not require access to the laboratory information system 125 may be grouped together. Access to laboratory information system content may also be restricted due to its sensitive nature.

[0037] Figure 1B shows an exemplary block diagram of a disease detection platform 100 that uses machine learning to verify the quality of slides and blocks.

[0038] Specifically, Figure 1B shows the components of a disease detection platform 100 according to one embodiment. For example, the disease detection platform 100 may include a slide analysis tool 101, a data acquisition tool 102, a slide import tool 103, a slide scanner 104, a slide manager 105, storage 106, and a browsing application tool 108.

[0039] The slide analysis tool 101 refers to a process and system for grouping cells in a WSI that share similar targets, as described later in an exemplary embodiment.

[0040] The data acquisition tool 102 refers to processes and systems that facilitate the transfer of digital pathology images to various tools, modules, components, and devices used for classifying and processing digital pathology images, according to exemplary embodiments.

[0041] The slide acquisition tool 103 refers to a process and system for scanning pathological images and converting them into digital forms according to an exemplary embodiment. The slides may be scanned with a slide scanner 104, and the slide manager 105 may process the images on the slides into digitized pathological images and store the digitized images in storage 106.

[0042] The viewing application tool 108 refers to a process and system for providing a user (e.g., a pathologist) with specimen characteristics or image characteristics information relating to a digital pathology image (or more) according to an exemplary embodiment. The information may be provided via various output interfaces (e.g., a screen, monitor, storage device and / or web browser, etc.).

[0043] Each of the slide analysis tool 101 and its components may send and receive digitized slide images and / or patient information to and from the server system 110, physician server 121, hospital server 122, clinical trial server 123, laboratory server 124 and / or laboratory information system 125 via the electronic network 120. The server system 110 may also include a storage device for storing images and data received from at least one of the slide analysis tool 101, data acquisition tool 102, slide capture tool 103, slide scanner 104, slide manager 105, and browsing application tool 108. The server system 110 may also include a processing device for processing the images and data stored in the storage device. The server system 110 may further include one or more machine learning tools or functions by means of a processing device, etc. Alternatively, the present disclosure (or parts of the systems and methods of the present disclosure) may be implemented on a local processing device (e.g., a laptop).

[0044] Any of the above-mentioned devices, tools, and modules may be placed in a device that is connected to an electronic network 120 such as the Internet or a cloud service provider via one or more computers, servers, and / or mobile terminals.

[0045] Figure 1C shows an exemplary block diagram of a slide analysis tool 101 according to an exemplary embodiment of the present disclosure. The slide analysis tool 101 may include a training image platform 131 and / or a target image platform 135.

[0046] According to one embodiment, the training image platform 131 may include a training image acquisition module 132, a data analysis module 133, and a cell identification module 134.

[0047] In one embodiment, the training image platform 131 may create or receive training images used to train a machine learning model to efficiently analyze and classify digital pathology images. For example, training images may be received from any one or any combination of the server system 110, physician server 121, hospital server 122, clinical trial server 123, laboratory server 124, and / or laboratory information system 125. The images used for training may be from real-world sources (e.g., humans, animals, etc.) or from synthetic sources (e.g., graphic rendering engines, 3D models, etc.). Examples of digital pathology images include (a) digitized slides stained with various stains such as hematoxylin and eosin (H&E), hematoxylin alone, IHC, molecular pathology, etc., and / or (b) digitized tissue samples from a 3D imaging device such as a micro-CT scanner.

[0048] The training image acquisition module 132 may create or receive a dataset containing one or more training datasets corresponding to one or more health variables and / or one or more data variables. For example, the training dataset may be received from one or any combination of the server system 110, physician server 121, hospital server 122, clinical trial server 123, laboratory server 124, and / or laboratory information system 125. This dataset may be stored in a digital storage device. The data analysis module 133 may identify whether a set of individual cells belongs to cells of interest or to the background of the digitized image. The cell identification module 134 may analyze the digitized image and determine whether individual cells in the cytology sample need further analysis. Identifying whether individual cells need further analysis and aggregating these areas is useful, and this identification may trigger a user alert.

[0049] According to one embodiment, the target image platform 135 may include a target image acquisition module 136, a sample detection module 137, and an output interface 138. The target image platform 135 may receive target images and apply a machine learning model to the received target images to determine the characteristics of the target dataset. For example, target data may be received from any or a combination thereof of the server system 110, physician server 121, hospital server 122, clinical trial server 123, laboratory server 124, and / or laboratory information system 125. The target image acquisition module 136 may receive target datasets corresponding to target health variables or data variables. The sample detection module 137 may apply a machine learning model to the target dataset to determine the characteristics of the target health variables or data variables. For example, the sample detection module 137 may detect trends in target relationships. The sample detection module 137 may also apply a machine learning model to the target dataset to determine the quality score of the target dataset. Furthermore, the sample detection module 137 may apply a machine learning model to the target image to determine whether or not a target element exists in a determination relationship.

[0050] The output interface 138 may be used to output target data and information regarding the determined relationship (for example, to a screen, monitor, storage device, web browser, etc.).

[0051] Currently, molecular laboratories may evaluate tumor genomes using tissue samples from unstained formalin-fixed paraffin-embedded (FFPE) slides or blood samples using next-generation sequencing (NGS) to identify gene-level mutations, fusions, or deletions. This evaluation may identify tumor origin, estimate prognosis, guide treatment strategies (target therapies, immuno-oncology therapies, or basket testing, etc.), and / or assess minimal residual disease (MRD). While this evaluation may be relatively simple for blood samples, it is a complex, multi-step process for FFPE samples.

[0052] For FFPE, the current workflow may begin with a primary diagnosis of the tumor by a pathologist using either a biopsy or excised slide. The tissue sample from the patient may be embedded in an FFPE block, or a portion of each block may be sliced ​​to prepare slides, which may be used for diagnosis, and the remainder of the FFPE block may be used for genome sequencing.

[0053] Following the primary diagnosis, the oncologist may request pan-tumor testing or specific genetic testing. The request may be presented to a signed-out pathologist who reviews slides from each FFPE block containing the optimal tumor section(s). To identify the optimal tumor section(s), a block containing existing slides with the highest tumor purity and the least necrosis and / or inflammation should be selected. This step may be performed via manual re-review of the slides, which can be very time-consuming. After identifying the block, 11 unstained slides may be cut from that block for a workflow for a pan-tumor NGS panel. The 11th slide may then be stained with H&E and evaluated for residual tumor to confirm that the previous 10 unstained slides contained a sufficient amount of tumor. These 10 unstained slides may be sent to a molecular laboratory along with a request / form containing basic patient information (age, sex, top-line diagnosis).

[0054] Upon reaching the molecular laboratory, the first unstained slide may be stained with H&E and the precise location of the tumor may be assessed by a technician under a microscope. The tumor may be annotated by the technician using either a diamond pencil or a marker. This location may be roughly marked on the remaining nine unstained slides, allowing the technician to "macro-excise" these tumor-rich areas from the slides. Macro-excision involves scraping the unstained FFPE tissue from the surface of the slide with a blade and aspirating the FFPE tissue, allowing the tissue to be immersed and DNA extracted. Polymerase chain reaction (PCR) testing may then be performed on these tumor sections, and the results may be fed into a robust bioinformatics data pipeline. A molecular pathologist may analyze the results and classify the variants into clinically significant and various stages of action.

[0055] If there is sufficient tumor for molecular testing, the molecular testing may be performed after the sample has been sent to the laboratory. The laboratory may prepare a comprehensive report that is returned to the primary diagnostic pathologist, and this report may be attached to the original diagnostic report for subsequent review by the oncologist. This process may take two weeks, but may take longer, possibly another two weeks, if there is insufficient tumor.

[0056] Identifying the quality of one or more slides from an FFPE block for additional AI-based testing (e.g., application of molecular / genomic testing). The systems and methods of this disclosure may facilitate the selection of the highest quality FFPE blocks by using artificial intelligence (AI) to provide a quality score Q for each FFPE block based on available information for each block. The quality score Q may indicate the quality of each block (e.g., tumor content, tumor purity, necrotic content, etc.) and may be used to determine the best or most suitable block for further testing. Various methods for determining the quality score Q are described below. The quality of the block and / or the quality score Q may depend on the tumor content, tumor purity, necrotic content, etc. The AI ​​system and / or pathologist may then select the highest quality block. The AI ​​system may also facilitate a verification process to ensure that the block is of sufficient quality by evaluating the quality of the last (e.g., 11th) slide in each block, determining whether the quality of that slide meets a certain quality and / or mass threshold, and / or selecting a new block if the slide does not meet the quality and / or quantity threshold.

[0057] Figure 2 is a flowchart illustrating an exemplary method for verifying the quality of inspection slides and blocks according to an exemplary embodiment of the present disclosure. For example, exemplary method 200 (e.g., steps 202-212) may be performed automatically or in response to a user request by the slide analysis tool 101.

[0058] Step 202 may also include receiving a collection of digital pathological tissue images associated with tissue blocks in an electronic storage device (e.g., a hard drive, network drive, cloud storage, system memory, etc.).

[0059] In step 204, the method may include dividing each digital histopathological image into a set of tiles associated with the digital histopathological image.

[0060] In step 206, the method may include providing or determining a quality score Q for each FFPE block based on the available information for each block. The quality of each FFPE block may depend on tumor content, tumor purity, necrotic volume, etc. Determining the quality score Q may be done using an AI system that can measure attributes such as these from all slides prepared from each block, and then determine the quality metric for each slide within the block based on the measured attributes. The AI ​​system may be run on the slides from each block and output a quality score Q for each block. The AI ​​system may also be configured to prioritize or rank the blocks based on the determined quality scores Qs.

[0061] In step 208, the method may include evaluating the quality of each block based on a quality score Q and selecting or sorting the best block to use for testing. For example, selecting the best block to use for testing may include selecting a block with the highest quality score Q, or a block with a quality score Q that shows the highest quality across the slide. This evaluation and / or selection may be performed by a user (e.g., a pathologist evaluating the quality score Q of the output) or by an AI system.

[0062] For example, tumor block selection may be performed by a signed-out pathologist whose criteria for optimal block selection include the quantity and quality of the tumor (i.e., minimal change at the biopsy site, inflammation, etc.). Algorithms designed or configured to detect and quantify tumors may be complemented by algorithms that detect tumor blocks most likely to yield positive molecular findings.

[0063] In step 210, the method may include selecting or preparing N slides from a block of selection and evaluating the quality of the Nth or last slide (e.g., the 11th slide). This evaluation may be performed by an AI system. Alternatively, the N slides may be prepared from a block of selection, and the Nth or last slide (e.g., N=11) may be stained with H&E, etc., to verify that the block is of sufficient quality. Alternatively, slide N-1 may be stained. This verification may include manual inspection.

[0064] In step 212, the method may include determining whether the Nth slide meets a quality threshold. If the Nth slide does not meet the quality threshold, the method may include selecting a new block for evaluation.

[0065] Between step 206, which evaluates the quality score Q, and / or step 210, which evaluates the quality of the Nth slide, the quality evaluation system may be described by a function Q(I1, I2, ..., IN, C), where the function may output a number indicating quality. A higher number indicates higher quality. Each IN may be a digital image of the slide within a block. C may describe a specific constraint, such as "do not select blocks where necrosis exceeds p%", or C may describe the weights to be used for each of the variables related to quality, for example, quality is a weighted sum of various measurements such as the percentage of necrosis. There may be multiple ways to generate Q.

[0066] Use of AI systems to estimate attributes The AI ​​system may estimate n attributes vi, such as the necrosis rate v1 and the tumor rate v2. After calculating these attributes vi, the attributes vi are integrated linearly or nonlinearly to calculate the block quality score Q.

[0067] Figures 3A and 3B are flowcharts illustrating exemplary methods for verifying the quality of inspection slides and blocks according to exemplary embodiments of the present disclosure. For example, exemplary method 300 shown in Figure 3A (e.g., steps 302-312) or exemplary method 320 shown in Figure 3B (e.g., steps 322-334) may be performed automatically by the slide analysis tool 101 or in response to a user request.

[0068] In step 302, the method may include receiving a collection of digital pathological tissue images into a digital storage device (e.g., a hard drive, network drive, cloud storage, RAM, etc.).

[0069] Step 304 may include receiving summary annotations containing one or more labels for each slide image or set of slide images. These labels may be pixel-level, tile-level, slide-level, or sub-sample-level. The labels may be binary (or multi-labeled binary), categorical, ordinal, or real-value. These variables may describe the presence of quality-related attributes such as invasive cancer, necrosis, and / or percentage.

[0070] In step 306, the method may include dividing each slide image into a set of tiles.

[0071] In step 308, the method may include detecting and / or segmenting tissue regions from the background of each slide image to create a tissue mask, and removing all non-tissue tiles. This can be done in a variety of ways, including, but not limited to, the following: a. A threshold-based method based on connected component algorithms, e.g., color / luminance, texture features, Otsu method, etc. b. Segmentation algorithms such as k-means, graph cuts, and masked R-CNN.

[0072] In step 310, the method may include training a machine learning model to estimate quality-related attributes using all locations on the slide, except those removed as background, as input. The model may be a support vector machine (SVM), convolutional neural network (CNN), recurrent neural network (RNN), transformer, graph neural network (GNN), multilayer perceptron (MLP), relational network, etc. The system may be trained to generate images for each related attribute, such as binary images showing the location where necrosis was found, or it may be trained to directly output whether there is an amount of an existing related variable, such as a number indicating the percentage of necrosis, another number indicating tumor purity, or an amount of a related variable such as tumor tissue exceeding a predetermined threshold.

[0073] Step 312 may include saving the parameters of the trained machine learning model to electronic storage.

[0074] Referring to Figure 3B, in step 322, the method may also include receiving the collection of digital pathological tissue images on a digital storage device (e.g., a hard drive, network drive, cloud storage, RAM, etc.).

[0075] In step 324, the method may include dividing each slide image into a set of tiles.

[0076] In step 326, the method may include detecting and / or segmenting tissue regions from the background of each slide image to create a tissue mask, and removing all non-tissue tiles. This can be done in a variety of ways, including, but not limited to, the following: a. A threshold-based method based on connected component algorithms, e.g., color / luminance, texture features, Otsu method, etc. b. Segmentation algorithms such as k-means, graph cuts, and masked R-CNN.

[0077] In step 328, the method may include running a trained machine learning model on each set of digital pathological tissue images corresponding to an instance.

[0078] In step 330, the method may include summing any digital pathological tissue images showing the presence of each attribute by the total amount of tissue, normalizing the sum, and generating the proportion of that variable as output, or outputting each relevant variable. It may also determine whether the total amount of tissue meets or exceeds a predetermined threshold. The system may be trained to generate “images” showing the presence of each attribute, for example, images showing the location where necrosis was found. Alternatively, the system may be trained to output each relevant variable directly, where the i-th relevant variable is V i It may also be shown as follows.

[0079] In step 332, the method may include outputting a quality score, which is a function of the related variables, as either a linear or nonlinear function. This function may be a linear combination identified by the values ​​in configuration C, for example, if the total number of related variables is N, the output Q is w1v1 + w2v2 + ... + w N v N This is equal to w1v1+w2v2+...+w N v N It is a linear combination specified by the configuration C given by .

[0080] In step 334, the method may include writing the quality score of each block to electronic storage.

[0081] Use of a system that directly learns quality This method uses an AI system that directly infers whether the slides within a block are appropriate for the test or insufficient, without necessarily generating intermediate variables.

[0082] Figures 4A and 4B are flowcharts illustrating exemplary methods for verifying the quality of inspection slides and blocks according to exemplary embodiments of the present disclosure. For example, exemplary method 400 shown in Figure 4A (e.g., steps 402-412) or exemplary method 420 shown in Figure 4B (e.g., steps 422-434) may be performed automatically by the slide analysis tool 101 or in response to a user request.

[0083] In step 402, the method may include receiving a collection of digital pathological tissue images into a digital storage device (e.g., a hard drive, network drive, cloud storage, RAM, etc.).

[0084] In step 404, the method may include receiving a binary value indicating whether the slide image within each FFPE block was suitable or unsuitable for examination.

[0085] In step 406, the method may include dividing each slide image into a set of tiles.

[0086] In step 408, the method may include detecting and / or segmenting tissue regions from the background of each slide image to create a tissue mask, and removing all non-tissue tiles. This can be done in a variety of ways, including, but not limited to, the following: a. A threshold-based method based on connected component algorithms, e.g., color / luminance, texture features, Otsu method, etc. b. Segmentation algorithms such as k-means, graph cuts, and masked R-CNN.

[0087] In step 410, the method may include training a machine learning model to take all positions on the slide other than the removed position as input and estimate whether they were appropriate, required further examination, or inappropriate, and to output an image of each relevant attribute or the quantity of the relevant variable present. The model may be a support vector machine (SVM), convolutional neural network (CNN), recurrent neural network (RNN), transformer, graph neural network (GNN), multilayer perceptron (MLP), relational network, semantic segmentation network, instance segmentation network (e.g., masked R-CNN), object detection CNN (e.g., fast R-CNN), etc. The system may be trained to generate an image for each relevant attribute, such as a binary image showing the location where necrosis was found, or it may be trained to directly output the quantity of the relevant variable present, such as a number indicating the percentage of necrosis, another number indicating tumor purity, etc.

[0088] In step 412, the method may include saving the trained model to electronic storage.

[0089] In step 422, the method may include receiving a collection of digital pathological tissue images into a digital storage device (e.g., a hard drive, network drive, cloud storage, RAM, etc.).

[0090] In step 424, the method may include dividing each slide image into a set of tiles.

[0091] In step 426, the method may include detecting and / or segmenting tissue regions from the background of each slide image to create a tissue mask, and removing all non-tissue tiles. This can be done in a variety of ways, including, but not limited to, the following: a. A threshold-based method based on connected component algorithms, e.g., color / luminance, texture features, Otsu method, etc. b. Segmentation algorithms such as k-means, graph cuts, and masked R-CNN.

[0092] In step 428, the method may include running a trained machine learning model for each set of slides corresponding to a block to output a quality score, and storing the quality store in electronic storage for each block.

[0093] In step 430, the method may include selecting the block with the highest quality score for subsequent inspection. This may be done manually by the user or automatically by the slide analysis tool 101.

[0094] In step 432, the method may include informing the user whether or not the block should prepare additional slides for inspection.

[0095] Use to verify that the last slide from the block of selection is of appropriate quality. Figures 5A and 5B are flowcharts illustrating exemplary methods for verifying the quality of inspection slides and blocks according to exemplary embodiments of the present disclosure. For example, exemplary method 500 shown in Figure 5A (e.g., steps 502-512) or exemplary method 520 shown in Figure 5B (e.g., steps 522-534) may be performed automatically by the slide analysis tool 101 or in response to a user request.

[0096] In step 502, the method may include receiving a collection of digital pathological tissue images into a digital storage device (e.g., a hard drive, network drive, cloud storage, RAM, etc.).

[0097] In step 504, the method may include receiving a binary value indicating whether the slide image in each FFPE block belonged to an FFPE block suitable for inspection or an FFPE block unsuitable for inspection.

[0098] In step 506, the method may include dividing each slide image into a set of tiles.

[0099] In step 508, the method may include detecting and / or segmenting tissue regions from the background of each slide image to create a tissue mask, and removing all non-tissue tiles. This can be done in a variety of ways, including, but not limited to, the following: a. A threshold-based method based on connected component algorithms, e.g., color / luminance, texture features, Otsu method, etc. b. Segmentation algorithms such as k-means, graph cuts, and masked R-CNN.

[0100] In step 510, the method may include training a machine learning model to take all locations on the slide other than the removed locations as input to estimate whether they were appropriate, required further examination, or inappropriate, and to output an image of each relevant attribute or the quantity of the relevant variable present. The model may be an SVM, CNN, RNN, trans, graph neural network, MLP, relational network, semantic segmentation network, instance segmentation network (e.g., masked R-CNN), object detection CNN (e.g., fast R-CNN), etc. The system may be trained to generate an image for each relevant attribute, such as a binary image showing the location where necrosis was found, or it may be trained to directly output the quantity of the relevant variable present, such as a number indicating the percentage of necrosis, another number indicating tumor purity, etc.

[0101] In step 512, the method may include saving the trained model to electronic storage.

[0102] In step 522, the method may include receiving a collection of digital pathological tissue images into a digital storage device (e.g., a hard drive, network drive, cloud storage, RAM, etc.).

[0103] In step 524, the method may include dividing each slide image into a set of tiles.

[0104] In step 526, the method may include detecting and / or segmenting tissue regions from the background of each slide image to create a tissue mask, and removing all non-tissue tiles. This can be done in a variety of ways, including, but not limited to, the following: a. A threshold-based method based on connected component algorithms, e.g., color / luminance, texture features, Otsu method, etc. b. Segmentation algorithms such as k-means, graph cuts, and masked R-CNN.

[0105] In step 528, the method may include running a trained machine learning model for each set of slides corresponding to a block to output a quality score, and storing the quality store in electronic storage for each block.

[0106] In step 530, the method may include determining whether the quality of the slide set is sufficiently low (e.g., below a certain quality) and, if the quality of the slide set is sufficiently low, informing the user that a new block should be prepared for inspection.

[0107] Figure 6 shows an exemplary use of a machine learning model for evaluating block quality. The machine learning model 604 may have an input 602. In this exemplary description, the input 602 is represented by multiple tissue blocks. The input 602 is analyzed for quality by the machine learning model 604 using variables 608 (e.g., tumor content, tumor purity, and / or necrosis rate), although a different number of variables may be used. A quality metric 610 is output from the analysis using variables 608 and may then be used in step 612 to sort the input 602 (blocks). The output 606 may be the sorted blocks in step 612.

[0108] Sequential recurrence score for invasive breast cancer After invasive breast cancer is detected, a tumor genome assay may be performed to determine whether to postpone further treatment, and whether to administer patient-administered endocrine (hormone) therapy, patient-administered adjuvant chemotherapy, or some other therapies. These tests may also be used to assess the risk of disease recurrence and metastasis after primary tumor resection using a serial scoring system. Furthermore, the tests may examine genomic information regarding proliferation, invasion, metastasis, stromal integrity, and angiogenesis.

[0109] For example, the EndoPredict (EPclin) test is based on the ribonucleic acid (RNA) expression of 12 genes and combines this genomic information with additional clinical features to predict the 10-year distant recurrence rate (DR). A score from 1 to 6 may be assigned to the input tissue, with 6 indicating a high risk and 1 indicating a low risk. Another exemplary test is MammaPrint, a 70-gene assay using formalin-fixed paraffin-embedded tissue (FFPE) or fresh tissue. RNA is isolated from tumor samples and used to predict a serial score, with values ​​greater than 0 indicating a low risk of cancer recurrence and values ​​less than 0 indicating a high risk of recurrence, suggesting the need for adjuvant chemotherapy.

[0110] Another exemplary test is the Breast Cancer Index (BCI) test, which analyzes seven genes to predict cancer recurrence. Two scores are output: a BCI prognosis score and a BCI prediction score. The BCI prognosis score estimates the likelihood of cancer recurrence 5 to 10 years after diagnosis, on a continuous scale from 0 to 10, with scores of 5.1 to 10 indicating a high risk of recurrence. The BCI prediction score estimates the potential benefit of administering an additional 5 years of endocrine therapy compared to a total of 10 years of endocrine therapy.

[0111] The Tumor Type DX Recurrence Score is another such analysis target that examines the expression of 21 genes within the tumor. Output numbers between 0 and 100 indicate the risk of cancer recurrence; scores greater than 31 indicate a high risk of metastasis and the need for adjuvant chemotherapy with endocrine therapy; scores between 26 and 30 indicate the uncertain benefit of adjuvant chemotherapy when used in conjunction with endocrine therapy; and scores less than 26 indicate that postoperative treatment with endocrine therapy alone is sufficient.

[0112] The Prosigna Breast Cancer Prognostic Gene Signature Assay (i.e., PAM50 gene signature) uses RNA from FFPE samples to assess the risk of distant recurrence in hormone receptor-positive breast cancer. It generates a continuous score from 0 to 100, with higher scores indicating a higher risk of recurrence, to guide treatment decisions.

[0113] All of these tests may require selecting an appropriate block, confirming that an unstained slide from that block contains appropriate tumor tissue, and determining the location of the tumor in the slide before applying the genomic assay. Using current techniques, these steps are manual, time-consuming, and prone to error. The techniques described herein may involve selecting and validating high-quality slides with sufficient tumor content for genomic testing. The techniques described herein may include steps that are automated, faster, and more accurate than current techniques.

[0114] Continuous score for recurrence of non-invasive breast cancer Following a diagnosis of non-invasive breast cancer, patients may require adjuvant therapy after undergoing a mammary tumor removal or mastectomy. This therapy may include endocrine therapy or radiation therapy to reduce the risk of recurrence, but these therapies have side effects. Genomic assays are being developed to determine the benefits patients may receive from these therapies.

[0115] The most common form of non-invasive breast cancer is ductal carcinoma in situ (DCIS). Currently, the primary genomic test for determining treatment options for DCIS is Oncotype DX DCIS, a 12-panel genomic test. This test generates a continuous score from 0 to 100 to determine the risk of breast cancer recurrence, with higher scores indicating the need for adjuvant therapy to prevent recurrence.

[0116] The systems and methods of this disclosure may be used to select and validate high-quality slides having sufficient tumor content for genomic testing.

[0117] Workflow for continuous scoring of prostate cancer treatment recommendations To diagnose prostate cancer, men may undergo a prostate biopsy. The biopsy sample may then be processed and visually reviewed by a pathologist to determine the presence and extent of the disease. However, prostate cancer treatments such as prostatectomy, hormone therapy, and radiation therapy can negatively impact a man's quality of life, and some patients do not require aggressive treatment.

[0118] Instead of relying solely on pathological evaluation of prostate tissue samples, genomic assays are used to predict tumor aggressiveness. For example, the Oncotype DX Genomic Prostate Score examines 17 genes to determine the aggressiveness of prostate cancer on a continuous score from 0 to 100. Patients with scores close to 0 may be advised to undergo active monitoring, while those with high scores should receive early, aggressive treatment to reduce the risk of adverse outcomes (e.g., death or metastasis). Another test is the Prolaris assay, which combines genomic evaluation with other measurements to determine a continuous score indicating higher cancer aggressiveness, with higher scores indicating greater cancer aggressiveness. This test may help men determine whether they should choose active monitoring for prostate cancer instead of aggressive treatment.

[0119] The systems and methods of this disclosure may be used to select and validate high-quality slides having sufficient tumor content for genomic testing.

[0120] Continuous scoring workflow for the likelihood of malignant tumors Tumors are abnormal masses of cells and can be either benign or malignant. Benign tumors lack the ability to metastasize or invade surrounding tissues, while malignant tumors possess this ability. In some cases, pathological evaluation is insufficient to determine whether a tumor is malignant or benign. In these types of scenarios, a sequential score may be used to facilitate the determination.

[0121] For example, the Myriad myPath melanoma test measures 23 genes associated with cell differentiation, cellular signaling, and immune response signaling to generate a continuous score on a scale of approximately -16 to 10. A score greater than 0 indicates that the skin tumor is likely malignant and requires aggressive treatment, while a score less than -2 indicates that the tumor is likely benign.

[0122] The systems and methods of this disclosure may be used to select and validate high-quality slides having sufficient tumor content for genomic testing.

[0123] As shown in Figure 7, the apparatus 700 may include a central processing unit (CPU) 720. The CPU 720 may be any type of processor, including any type of dedicated or general-purpose microprocessor. The CPU 720 may also be a single processor in a multicore / multiprocessor system, such as a system operating independently, or a single processor in a cluster of arithmetic units operating in a cluster or server farm, as will be understood by those skilled in the art. The CPU 720 may be connected to, for example, a bus, message queue, network, or data communication infrastructure 710 of a multicore message passing scheme.

[0124] The device 700 may include, for example, a main memory 740 which is random access memory (RAM), and may also include a secondary memory 730. The secondary memory 730, for example, read-only memory (ROM), may be, for example, a hard disk drive or a removable storage drive. Such a removable storage drive may include, for example, a flexible disk drive, a magnetic tape drive, an optical disk drive, and flash memory. The removable storage drive in this embodiment reads from and / or writes to a removable storage unit in a well-known manner. The removable storage unit may include a floppy disk, magnetic tape, optical disk, etc., which are read from and written to by the removable storage drive. As those skilled in the art will understand, such a removable storage unit generally includes a computer-usable storage medium storing computer software and / or data.

[0125] In alternative embodiments, the secondary memory 730 may include similar means that enable loading computer programs or other instructions into the device 700. Such means include program cartridges and cartridge interfaces (such as those found in video game devices), removable memory chips (such as EPROMs or PROMs) and associated sockets, and other removable storage units and interfaces that enable the transfer of software and data from removable storage units to the device 700.

[0126] The device 700 may also include a communication interface (COM) 760. The communication interface 760 enables the transfer of software and data between the device 700 and an external device. The communication interface 760 may include a modem, a network interface (such as an Ethernet® card), a communication port, a PCMCIA slot and card, etc. The software and data transferred via the communication interface 760 may be in the form of electronic signals, electromagnetic signals and optical signals, or other signals that can be received by the communication interface 760. These signals may be supplied to the communication interface 760 via a communication path of the device 700, which may be implemented using, for example, wires or cables, optical fibers, telephone lines, cell phone links, RF links or other communication channels.

[0127] The hardware elements, operating systems, and programming languages ​​of such equipment are inherently common and presumably well-known to those skilled in the art. The device 700 may also include input / output ports 750 for connecting input / output devices such as keyboards, mice, touch panels, monitors, and displays. Of course, various server functions may be distributed across multiple similar platforms to distribute the processing load. Alternatively, the server may be implemented by appropriately programming a single computer hardware platform.

[0128] Throughout this disclosure, a component or module generally refers to an item that can be logically grouped together to perform a function or a set of related functions. Similar reference numerals are generally intended to refer to identical or similar components. Components and modules may be implemented in software, hardware, or a combination of software and hardware.

[0129] The above tools, modules, and functions may be executed by one or more processors. The “storage” type medium may include any or all of the tangible memory of a computer and processor, or its associated modules, such as various semiconductor memory, tape drives, disk drives, etc., which can provide non-temporary storage for software programming at any time.

[0130] The software may be communicated via the Internet, a cloud service provider, or other communication network. For example, the communication may enable the software to be loaded from one computer or processor to the other. Unless limited to non-temporary tangible “storage” media as used herein, terms such as computer or machine “readable media” refer to any media involved in providing and executing instructions to a processor.

[0131] The general descriptions set forth herein are illustrative and descriptive, and do not limit the present disclosure. Other embodiments of the present invention will be apparent to those skilled in the art from the description of the invention and embodiments disclosed herein, and this specification and the examples are intended to be considered illustrative only.

Claims

1. A computer implementation method for verifying the quality of inspection slides and tissue blocks, Receiving a set of one or more digital images associated with an organizational block and corresponding to an instance in a digital storage device, Applying a machine learning model to the aforementioned set to identify the presence or absence of necrosis in the tissue block from the digital images in the aforementioned set, Determining the amount or percentage of tissue having necrosis from the digital images in the set showing the presence of necrosis, Outputting a quality score corresponding to the determined amount or percentage of necrotic tissue that is below a predetermined value, Includes, If the determined amount or percentage of tissue having necrosis is greater than the predetermined value, the quality score is equal to zero, and if the determined amount or percentage of tissue having necrosis is less than or equal to the predetermined value, the quality score is a linear combination of two or more attribute values. A computer implementation method in which two or more attribute values ​​are selected from a group consisting of the percentage of tissue with necrosis, the percentage of tissue with tumors, and a number indicating tumor purity.

2. The computer implementation method according to claim 1, wherein determining the amount or proportion includes summing and normalizing the digital images of the set indicating the presence of the necrosis or tumor by the total amount of tissue.

3. Dividing each digital image in the aforementioned set of digital images into a set of tiles, The process involves detecting and / or segmenting tissue regions from the background of the aforementioned digital image to create a tissue mask, Removing all tiles from the set of tiles including the background, The computer implementation method according to claim 1, further comprising:

4. The computer implementation method according to claim 3, wherein the detection and / or segmentation includes executing a connected component algorithm using one or more threshold-based methods.

5. The computer implementation method according to claim 3, wherein the detection and / or segmentation includes using one or more segmentation algorithms.

6. The computer implementation method according to claim 1, further comprising determining the tissue block having the highest quality score for subsequent inspection.

7. The computer implementation method according to claim 6, further comprising presenting to the user that the tissue block has at least one additional slide prepared for inspection.

8. The computer implementation method according to claim 6, further comprising determining whether the quality score is less than a predetermined quality score.

9. The computer implementation method according to claim 8, further comprising presenting the user with the option to prepare a new tissue block for inspection if the quality score is determined to be less than the predetermined quality score.

10. The computer implementation method according to claim 1, further comprising outputting a function of at least one variable corresponding to the quality score.

11. The computer implementation method according to claim 10, wherein the function of the at least one variable is a linear function.

12. The computer implementation method according to claim 10, wherein the function of the at least one variable is a nonlinear function.

13. The computer implementation method according to claim 1, further comprising outputting a binary image showing the location of the necrotic or tumor.

14. The computer implementation method according to claim 1, further comprising receiving summary annotations, each of which includes one or more labels for each digital image.

15. The computer mounting method according to claim 14, wherein the one or more labels are pixel-level labels, tile-level labels, slide-level labels and / or partial sample-level labels.

16. The computer implementation method according to claim 1, wherein the digital image is a digital pathological tissue image.

17. A system for verifying the quality of inspection slides and tissue blocks using a machine learning model, At least one memory for storing instructions, At least one processor configured to execute the aforementioned instructions and perform operations, Equipped with, The aforementioned operation is, Receiving a set of one or more digital images associated with an organizational block and corresponding to an instance in a digital storage device, Applying a machine learning model to the aforementioned set to identify the presence or absence of necrosis in the tissue block from the digital images in the aforementioned set, Determining the amount or percentage of tissue having necrosis from the digital images in the set showing the presence of necrosis, Outputting a quality score corresponding to the determined amount or percentage of necrotic tissue that is below a predetermined value, Includes, If the determined amount or percentage of tissue having necrosis is greater than the predetermined value, the quality score is equal to zero, and if the determined amount or percentage of tissue having necrosis is less than or equal to the predetermined value, the quality score is a linear combination of two or more attribute values. The system is selected from a group consisting of the percentage of tissue with necrosis, the percentage of tissue with tumors, and a number indicating tumor purity.

18. The system according to claim 17, wherein determining the amount or proportion includes summing and normalizing the digital images of the set indicating the presence of the necrosis or tumor by the total amount of tissue.

19. A computer, A procedure for receiving a set of one or more digital images associated with an organizational block and corresponding to an instance in a digital storage device, A procedure for applying a machine learning model to the set to identify the presence or absence of necrosis in the tissue block from the digital images in the set, A procedure for determining the amount or percentage of tissue having necrosis from the digital images in the set showing the presence of necrosis, A procedure for outputting a quality score corresponding to the determined amount or percentage of necrotic tissue that is below a predetermined value, A computer-readable recording medium containing a program for executing a program, If the determined amount or percentage of tissue having necrosis is greater than the predetermined value, the quality score is equal to zero, and if the determined amount or percentage of tissue having necrosis is less than or equal to the predetermined value, the quality score is a linear combination of two or more attribute values. A computer-readable recording medium in which the two or more attribute values ​​are selected from a group consisting of the percentage of necrotic tissue, the percentage of tumorous tissue, and a number indicating tumor purity.

20. The program is installed on the computer. A procedure for dividing each digital image in the aforementioned set of digital images into a set of tiles, A procedure for detecting and / or segmenting tissue regions from the background of the aforementioned digital image to create a tissue mask, A procedure for removing all tiles from the set of tiles including the background, A computer-readable recording medium according to claim 19, which is for further execution of the above.

Images