Automatic estimation of tumor cell properties using the DPI AI platform

The DPI AI platform with DCNNs for automatic tumor cellularity estimation addresses the inefficiencies of manual methods, enhancing reproducibility and accuracy by classifying patches and estimating scores without segmentation, thus improving neoadjuvant therapy assessment.

JP7714689B2Active Publication Date: 2025-07-29SONY GROUP CORP +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023572606
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-06-15
Filing Date
2022-05-27
Publication Date
2025-07-29
Estimated Expiration
2042-05-27

AI Technical Summary

Technical Problem

Current manual methods for estimating tumor cellularity in digital pathology are time-consuming and prone to variability among evaluators, reducing the prognostic predictive power in neoadjuvant therapy.

Method used

Utilizing a DPI AI platform with two deep convolutional neural networks (DCNNs) to automatically classify patches as normal or suspicious and estimate cellularity scores, one using image classification and the other using ordinal regression, without requiring nuclear or cell segmentation.

Benefits of technology

This approach reduces manual effort, increases reproducibility and diagnostic accuracy, and improves agreement among observers by providing efficient and reliable cellularity estimation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007714689000002
    Figure 0007714689000002
  • Figure 0007714689000003
    Figure 0007714689000003
  • Figure 0007714689000004
    Figure 0007714689000004
Patent Text Reader

Abstract

A method for automatically estimating cellularity in a digital pathology slide image includes extracting patches of interest from the digital pathology slide image; for each patch, operating using a trained first deep convolutional neural network (DCNN) to classify the patch as either a normal patch having an estimated cellularity of 0% or a suspicious patch having an estimated cellularity estimated to be higher than 0%; for each suspicious patch, operating using a second DCNN trained using a deep ordinal regression model to determine an estimated cellularity score for the suspicious patch; and combining the estimated cellularity scores of the patches of interest to provide an estimated cellularity for the digital pathology slide image at a patch-by-patch level.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Patent Application No. 17 / 348,436, filed June 15, 2021, entitled "AUTOMATIC ESTIMATION OF TUMOR CELLULARITY USING A DPI AI PLATFORM" (Client Reference Number: SYP339215US01), which is incorporated herein by reference in its entirety for all purposes as if fully set forth herein. [Background technology]

[0002] In general, in the field of cancer treatment, neoadjuvant therapy (NAT), in which therapeutic drugs are administered before surgery, has been very successful in downstaging tumors to allow for tissue-conserving surgery rather than more drastic surgical options. For example, in breast cancer, one of the most common cancers in women, NAT can significantly reduce the amount of normal breast tissue that would otherwise be removed during surgery.

[0003] After NAT, the amount of remaining cancer tissue, i.e., residual cancer burden or residual tumor burden, is an indicator of the effectiveness of NAT in that individual case, and this indicator has been found to be a useful prognostic factor for long-term survival. The "gold standard" for assessing residual tumor is pathological examination of tissue sections to assess tumor cellularity, defined as the percentage of cancerous cells rather than normal cells found in the tissue section.

[0004] In current clinical practice, tumor cellularity (which may also be simply referred to as "cellularity" elsewhere in this disclosure) is manually estimated by a pathologist after sectioning a patient's tissue sample and staining it (usually with hematoxylin and eosin, i.e., H&E). In FIG. 1, a slice of a suspected tissue 110 containing cancer cells stained with H&E is removed from the surrounding normal tissue 120 and divided into sections (A1 to A5 in the illustrated example), and each of these is mounted on a corresponding slide (such as 141 to 145) and observed through a microscope, schematically showing part of the process.

[0005] At this time, the pathologist can examine one or more regions of the tissue section on the slide and compare the proportion of the residual tumor bed area containing cancer with standard cellularity criteria such as set 200 shown in FIG. 2. Set 200 includes, on the left side of the figure, images of microscope fields observed for tissue sections having cellularity values from 1% to 30%, and these images include a subset showing the stained cell distribution in a collective pattern as seen in fields 211 to 215 and a subset showing the stained cell distribution in a scattered pattern as seen in fields 221 to 225. Set 200 also includes, on the right side, images of microscope fields observed for tissue sections having high cellularity values from 40% to 95%. In these higher ranges, since it is not very meaningful to distinguish the distribution pattern into two types, only one reference image is shown for each cellularity value as seen in fields 231 to 237.

[0006] The pathologist can derive an estimated cellularity value from the proportion of the area occupied by tumor cells relative to normal cells in the section by comparing the appearance of each observed tissue section with the reference set. That is, in the case of FIG. 1, the pathologist can generate five "local" cellularity values, one for each of slides 141 to 145, and then calculate the average or overall cellularity of the entire tissue sample 110.

[0007] Such manual cellularity estimation, each involving a series of visual comparison or matching operations, requires a great deal of time. Further, the quality and reliability of manual estimations performed by different people, i.e., different evaluators, are naturally subject to variability among evaluators, which may reduce the prognostic predictive power in NAT tests and normal patient care. Summary of the Invention Problems to be Solved by the Invention

[0008] Therefore, there is a need for a time - efficient method for estimating cellularity that reduces manual effort and does not rely on the skills and consistency of individual pathologists. Such a method would ideally utilize the technological advancements in digital pathology to save time, reduce the impact of human error, increase agreement among observers, and improve reproducibility and diagnostic accuracy. Means for Solving the Problems

[0009] Embodiments generally relate to systems and methods for automatically estimating cellularity in digital pathology slide images. In one embodiment, the method includes extracting a patch of interest from a digital pathology slide image; for each of the extracted patches, operating using a trained first deep convolutional neural network (DCNN) to classify the patch as either a normal patch having an estimated cellularity of 0% or a suspicious patch having an estimated cellularity greater than 0%; for each of the suspicious patches, operating using a second DCNN trained using a deep ordinal regression model to determine an estimated cellularity score for the suspicious patch; and combining the estimated cellularity scores of the patches of interest to provide an estimated cellularity of the digital pathology slide image at a per - patch level.

[0010] In another embodiment, a method of training first and second deep convolutional neural networks to automatically estimate the cellularity of a digital pathology slide image includes training a first deep convolutional neural network (DCNN) to classify each of a first plurality of training digital pathology images input to the first DCNN as either normal with an estimated cellularity of 0% or suspect with an estimated cellularity higher than 0%, and training a second DCNN to estimate a cellularity score for each of a second plurality of training digital pathology images using an ordinal regression model.

[0011] In yet another embodiment, an apparatus for automatically estimating cellularity in a digital pathology slide image includes one or more processors and logic encoded on one or more non-transitory media for execution by the one or more processors. The logic, when executed, extracts a region of interest patch from the digital pathology slide image and operates on each of the extracted patches using a trained first deep convolutional neural network (DCNN) to classify the patch as either a normal patch with an estimated cellularity of 0% or a suspect patch with an estimated cellularity estimated to be higher than 0%, and operates on each of the suspect patches using a second DCNN trained using a deep ordinal regression model to determine an estimated cellularity score for the suspect patch, and combines the estimated cellularity scores of the regions of interest patches to provide an estimated cellularity of the digital pathology slide image at a per-patch level.

[0012] The characteristics and advantages of the specific embodiments disclosed herein can be further understood by reference to the remainder of the specification and the attached drawings.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

[0014] The embodiments described herein relate to automatic cellularity estimation in digital pathology slide images. The present invention performs the estimation using a DPI AI platform that is trained and operates according to the methods and systems described below

[0015] FIG. 3 schematically shows a system 300 for cellularity estimation according to some embodiments of the present invention. The system input 310 is a digital pathology slide image of a stained tissue section of interest. The first operation performed by the system 300 is to extract a patch set 320 from 310 and keep the data load to be processed in each of the extracted patches (such as 320-D) of the patch set 320 manageably low. In some cases, this extraction can optionally include a background removal step that excludes regions of the patch that do not show signs of staining

[0016] Each of the extracted patches can be considered to generally correspond to a microscopic field image of the tissue on the slide in a prior art method such as the method of FIG. 1 (e.g., like slide 141)

[0017] The next operation is to supply patches one by one to the deep convolutional neural network (DCNN) 330. The DCNN 330 operates on any input patch using a relatively conventional AI image classification method and is pre-trained to classify that input patch as either a normal patch, i.e., one with 0% estimated cellularity, or a suspicious patch, i.e., one with an estimated cellularity higher than 0% by a relatively rough estimation process. When the trained DCNN 330 subsequently receives an input patch for use within the system 300, if it determines that the patch is normal, it generates an output in OPA, while if it determines that the patch is suspicious, it transmits this patch on output OPB. A trained DCNN such as DCNN 330 can be trusted in that it does not misclassify a patch with a cellularity exceeding 0% as normal, but it has been experimentally found that it misclassifies a small number of normal patches as suspicious, i.e., a small portion of the patches in the stream OPB actually have a cellularity of 0%, i.e., they are normal. Therefore, the term "suspicious" is more accurate than the term "cancerous" for describing OPB patches.

[0018] In the regular operation of the system 300, the OPB stream supplies suspicious patches one by one to a DCNN 340 that has been pre-trained to operate on any input patch using a deep ordinal regression model (described in more detail below) to estimate the corresponding cellularity score within the range of 0% to 100%. When the trained DCNN 340 subsequently receives a stream of suspicious input patches through OPB for use within the system 300, it estimates the cellularity score for each patch and generates an output stream containing the corresponding estimated cellularity score for each patch at OPC.

[0019] In this way, a cellularity score can be provided for all input patches, enabling a detailed analysis across the entire whole-slide image 310.

[0020] Figure 4 is a flowchart of steps in a cellularity estimation method 400 according to some embodiments of the present invention. In step 410, a digital pathology image, such as that shown in Figure 3 as 310, is input. In step 420, a patch extraction process is performed on 310 to remove optional regions of no interest and divide the remaining region into patches of manageable data content (such as patch 320D).

[0021] In step 430, a trained DCNN, such as that shown as 330 in FIG. 3, is further operated on each patch to classify it as either normal, i.e., its cellularity is estimated to be 0%, or suspicious, i.e., its cellularity is estimated to be higher than 0%. The classification of each patch is checked in step 435, and for each patch classified as normal, a corresponding indication of normality, i.e., a cellularity score of 0%, is provided as output in step 440. As described above in FIG. 3, this DCNN is assumed to have been pre-trained using conventional AI image classification methods that have proven very successful in not misclassifying suspicious patches as normal. However, these classifiers typically have a significant failure rate in the reverse direction, i.e., a significant proportion (on the order of 5%) of patches they classify as suspicious actually have 0% cellularity and should ideally have been classified as normal.

[0022] If a patch is found in step 435 that has been classified as suspicious but that actually has a small chance of being normal as described above, then a second DCNN is run on this patch that is trained by an ordinal regression modeling method that, unlike the DCNN used in step 430, does not assign different patches to different categories solely based on apparent cellularity. This second DCNN instead runs in step 445 to estimate a specific cellularity score for each suspicious patch, and this cellularity score is provided as an output in step 450.

[0023] The two DCNNs used in embodiments of the present invention are trained using training DPI images of similar size and quality to those of the image patches these DCNNs are expected to process later. Figure 5 shows an example of one such training image 511, captured from a stained tissue section (typically from an organ or organs of interest) with ground truth cellularity assessed by some reliable means, typically manually by one or more expert pathologists. Obtaining a sufficient number of such images for successful training is facilitated by data augmentation, a well-known technique in machine learning for processing “natural” images, which we adapt to DPI images. In Figure 5, the initial image 511 serves as a “seed” image for an entire batch of images 510 derived from 511 through simple image manipulations such as color perturbations, rotations, and flips. For example, image 515 is derived by rotating 511 180 degrees and varying brightness and contrast. In some cases, blurring or other types of image degradation or enhancement may also be employed in addition to or in combination with such adjustments. By generating a set of training images by such means and using them as training input, the entire cellular range expected to be encountered can be covered without each training image having to be obtained from a unique tissue section image.

[0024] In some embodiments, a different training image set is used for the DCNN used to classify images as normal or suspicious (i.e., a DCNN performing the functions of DCNN 330) than for the DCNN used to estimate cellularity scores (i.e., a DCNN performing the functions of DCNN 340), with the former set likely having a higher proportion of images with 0% cellularity than the latter. In other embodiments, the same training image set can be used for both types of DCNN.

[0025] In the training of the first DCNN of the system of the present invention, such as the DCNN 330 of the system 300, a relatively simple iterative process of a type well-known to those skilled in the art is used. Since the purpose of this DCNN is to simply classify an image as either normal (cellularity 0%) or suspicious (cellularity > 0%), the initial classification of each image in the training set is compared with a rough classification based on the ground truth of that image or the seed image from which that image is derived, and then the weights and connectivity parameters of the internal DCNN backbone can be iteratively adjusted until an acceptable matching rate is achieved.

[0026] A completely different approach is adopted in the training of the second DCNN of the system of the present invention, such as the DCNN 340 in FIG. 3. Instead of a classification method that sorts images into different categories based on an evaluation of whether the number of cells in the image falls within one range or another, an ordinal regression method is used that involves an intermediate step of determining a series of ordinal values and then combining them to provide an estimated value of the specific cellularity score for each image. FIG. 6 shows an overview of how this method is implemented.

[0027] Consider the digital pathology image 611 supplied to the DCNN model 630. The image 611 represents an image that is considered to be classified as suspicious by the first DCNN of the system of the present invention (such as 330 of the system 300). The DCNN model 630 operates on the image 611 in a relatively conventional manner to extract image features 650 from the input image. Such feature extraction methods are well-known in the processing of natural images, but in the present invention, they are adapted for the processing of DPI images.

[0028] Next, the network branches into a stack of K output layers (660) where each output layer (or binary classifier) contains two neurons and corresponds to a binary classification task. The k-th task is that the cellularity of the input image is rank C kIt is to predict whether it is larger than. Each binary classifier in the stack 660 outputs "1" or "0" according to the comparison. For example, one classifier outputs "1" when it predicts that the cellularity of the input image 611 is higher than 5%, while another classifier can output "1" when it predicts that the cellularity of the input image 611 is higher than 10% and so on. Then, in the fusion element 670 that weights the outputs of different classifiers in different forms, the binary outputs (OK for the k-th classifier) generated by the output layer 660 are "fused" or mathematically combined to provide a fused cellularity score "c". When comparing this cellularity score with the ground truth cellularity score of the input image 611, an iterative process (shown by a dashed line in the figure) can be executed to adjust the parameters in the output layer 660 and the parameters in the DCNN model 630 so as to optimize the loss function of the binary classifier. As a result, these parameters and weights are fixed, and the DCNN is trained so that the fusion score is regarded as the desired output.

[0029] Figure 7 shows a part of the mathematical details involved in the operation of the binary classifier, the fusion element, and the loss function in one embodiment of the present invention. Equation 705 represents the output of the k-th binary classifier in the output layer 760 (corresponding to 660 in FIG. 6), where C k represents the cellularity rank, c represents the cellularity of the input image, and p(c > C k ) represents the estimated probability that c is greater than C k . The fusion element 770 corresponding to 670 in FIG. 6 operates according to Equation 715 to generate a fused cellularity score TIFF0007714689000001.tif6150. In Equation 725, the loss to be optimized by training the neural network model is defined as the sum of the cross-entropies of a series of binary classifiers, where ok is the output probability of the k-th binary classifier defined in Equation 705, and y k is the label of the input image for the k-th binary classifier, and is actually a true / false comparison with the ground truth. For example, when the cellularity c > C k of the input image, then y k= 1, otherwise y k = 0. The feedback loop, shown by the dashed line in Figure 6, then operates as described above to minimize this loss by adjusting the parameters in the fully connected output layer and the parameters of the DCNN model.

[0030] There are several significant differences between the present invention and prior art methods, including those that apply DCNNs to the problem of cell estimation. One such difference is that the present invention does not require nuclear or cell segmentation of the input image patch. Almost all prior art methods rely on one or both of these as an initial step, where all nuclei (or cells) in the image patch are identified and classified to distinguish between normal and cancerous nuclei / cells, and a corresponding region of interest is defined for further analysis. In the present invention, the only type of segmentation that can optionally occur is a rough tumor segmentation that defines a peripheral boundary around the entire tumor region and removes the surrounding area as a simple background removal step. Avoiding the need for nuclear or cell-based segmentation is a valuable simplification that the present invention brings.

[0031] Another difference is that prior art methods, including those that bypass a nuclear or cell segmentation step, rely on a type of classification in which a cellularity score—i.e., the percentage of cellular area likely occupied by cancer cells—is assigned based on finding the closest visual match to one in a set of predefined category “bins.” For example, if an input image “looks more similar to” an image with a ground truth cellularity score within 20% ± 5% than an image in any other “bin” within the 0–100% range, the cellularity of the input image is estimated to be 20% ± 5%. Instead of using a DCNN trained by this type of regression model, the present invention provides a direct estimate of a particular cellularity score from a DCNN trained according to an ordinal regression model.

[0032] The system of the present invention can be implemented in the form of stand-alone computer software or deployed on an existing clinical CAD (computer-aided diagnosis) system. Application examples of the methods described in this specification include, in addition to NAT monitoring and survival prediction, other image-based evaluation tasks such as cancer grading.

[0033] Although specific embodiments have been described, these specific embodiments are merely illustrative and not limiting.

[0034] For the implementation of the routines of specific embodiments, any suitable programming language including C, C++, Java, assembly language, etc. can be used. Different programming techniques such as procedural or object-oriented can be used. These routines can be executed on a single processing device or multiple processors. Although steps, operations, or calculations may be shown in a specific order, this order can be changed in different specific embodiments. In some specific embodiments, multiple steps shown sequentially herein can also be executed simultaneously.

[0035] Specific embodiments can be implemented on a computer-readable storage medium used by or connected to an instruction execution system, apparatus, system, or device. Specific embodiments can also be implemented in the form of control logic in software or hardware or a combination thereof. The control logic can execute what has been described in specific embodiments when executed by one or more processors.

[0036] Certain embodiments can be implemented by using a programmed general-purpose digital computer, by using application-specific integrated circuits, programmable logic devices, field programmable gate arrays, optical, chemical, biological, quantum or nano engineering systems, components and mechanisms. Generally, the functions of certain embodiments can be realized by any means well known in the art. Distributed networked systems, components and / or circuits can also be used. The communication or transfer of data can be by means of wire, wireless or any other means.

[0037] Also, it will be understood that when useful for a particular application, one or more of the elements shown in the drawings / figures can be implemented in a more separated or integrated form, or in some cases removed or made inoperable. Implementing a program or code storable in a machine-readable medium that enables a computer to execute any of the methods described above is also within the spirit and scope of the present invention.

[0038] "Processor" includes any suitable hardware and / or software system, mechanism, or component that processes data, signals, or other information. The processor can include a general-purpose central processing unit, multiple processing units, a dedicated circuit for realizing functions, or a system having other systems. The processing need not be limited to a geographical location or have a time limit. For example, the processor can execute its functions in "real-time", "offline", "batch mode", etc. Some of the processing can also be executed by different (or the same) processing systems at different times and in different locations. Examples of processing systems can include servers, clients, end-user devices, routers, switches, networked storage, etc. A computer can be any processor that communicates with a memory. The memory can be any suitable processor-readable storage medium such as random access memory (RAM), read-only memory (ROM), magnetic or optical disks, or other non-transitory media suitable for storing instructions executed by the processor.

[0039] As used throughout this specification and the following claims, the indefinite article "a" and the definite article "the" include references to the plural unless the context clearly dictates otherwise. Also, as used throughout this specification and the following claims, the meaning of "in" includes the meanings of "in" and "on" unless the context clearly dictates otherwise.

[0040] Above, specific embodiments have been described in this specification. However, in the above disclosure, modifications, various changes, and substitutions are intended. In some examples, it should be understood that some features of a specific embodiment are used without the use of corresponding other features without departing from the described scope and spirit. Therefore, many modifications can be made to adapt to a specific situation or material to the basic scope and spirit.

Description of Reference Numerals

[0041] 400 Cellularity Estimation Method 410 Input digital pathology images 420 Extract patches from digital pathology images 430 The first DCNN classifies each patch as normal or suspicious based on cellularity 435 Normal? 440 Output a cellularity score of 0% 445 The second DCNN trained by the ordinal regression method estimates the cellularity score of each suspicious patch sent from the first DCNN 450 Output the estimated cellularity score

Claims

1. A method for automatically estimating cellularity in a digital pathology slide image, comprising: extracting a region of interest patch from the digital pathology slide image; for each of the extracted patches, operating using a trained first deep convolutional neural network (DCNN) to classify the patch as either a normal patch having an estimated cellularity of 0% or a suspicious patch having an estimated cellularity higher than 0%; for each of the suspicious patches, operating using a second DCNN trained using a deep ordinal regression model to determine an estimated cellularity score for the suspicious patch; combining the estimated cellularity scores of the region of interest patches to provide an estimated cellularity of the digital pathology slide image at a per-patch level; comprising: the first and second DCNNs are trained using a first and a second plurality of training digital pathology images as inputs respectively; training the second DCNN includes, for each training digital pathology image in the second plurality of training digital pathology images: the DCNN model extracting image features from the training digital pathology image; a fully connected output layer operating on the extracted image features to generate an intermediate level representation of the training digital pathology image; a stack of K binary classifiers operating on the intermediate level representation, each binary classifier predicting whether the cellularity of the training digital pathology image is higher than a rank percentage specific to the binary classifier to generate an ordinal output; fusing the K binary outputs from the stack of binary classifiers in a fusion element that weights the outputs of different classifiers in different ways to provide a fused cellularity score; iteratively adjusting the parameters of the fully connected output layer and the parameters in the DCNN model to minimize a loss function determined by the ground truth cellularity score of the training digital pathology image, such that a final set of training parameters for the fully connected output layer and the DCNN model is established for the ordinal regression model when the loss function is minimized; A method characterized by including the above.

2. Each of the first and second plurality of training digital pathology images is derived at least in part by data augmentation of one or more initial training digital pathology images, The method according to claim 1.

3. The data augmentation includes at least one of flipping, rotation, and color perturbation. The method according to claim 2.

4. The digital pathology slide image is a tissue image stained with hematoxylin and eosin (H&E). The method according to claim 1.

5. The estimated cellularity of the digital pathology slide image is used as a measure of cancer grading. The method according to claim 1.

6. The digital pathology slide image is created using tissue from a patient. The estimated cellularity of the digital pathology slide image is used in predicting the clinical outcome of the patient. The method according to claim 1.

7. A method of training a first deep convolutional neural network (DCNN) and a second deep convolutional neural network (DCNN) to automatically estimate the cellularity of a digital pathology slide image, training the first DCNN to classify each of a first plurality of training digital pathology images input to the first DCNN as either normal having an estimated cellularity of 0% or suspect having an estimated cellularity higher than 0%, training the second DCNN to estimate the cellularity score of each of a second plurality of training digital pathology images using an ordinal regression model, comprising, training the second DCNN to estimate the overall cellularity of each of the second plurality of training digital pathology images includes, for each training digital pathology image, training a DCNN model to estimate the initial cellularity of the training digital pathology image, extracting image features from the training digital pathology image, using a fully connected output layer to operate on the extracted image features to generate an intermediate level representation of the training digital pathology image, using each of a stack of K binary classifiers to compare the estimated initial cellularity with a threshold rank percentage specific to the binary classifier to provide a corresponding binary output, fusing the K binary outputs from the stack of binary classifiers in a fusion element that weights the outputs of different classifiers in different forms to provide a fused cellularity score. Minimize the loss function determined by the ground truth cellularity score of the training digital pathology image for the parameters of the fully connected output layer and the parameters in the DCNN model, and iteratively adjust so that the final training parameter set of the fully connected output layer and the DCNN model is established for the ordinal regression model when the loss function is minimized. A method characterized by including the above. **Claim 8** The loss function includes the sum of the cross-entropies of the outputs of the binary classifiers. The method according to claim 7. **Claim 9** An apparatus for automatically estimating cellularity in a digital pathology slide image, One or more processors, Logic encoded on one or more non-transitory media for execution by the one or more processors, Comprising, when the logic is executed, Extract a region of interest patch from the digital pathology slide image, For each of the extracted patches, operate using a trained first deep convolutional neural network (DCNN) to classify the patch as either a normal patch with an estimated cellularity of 0% or a suspicious patch with an estimated cellularity greater than 0%, For each of the suspicious patches, operate using a second DCNN trained using a deep ordinal regression model to determine the estimated cellularity score of the suspicious patch, Combine the estimated cellularity scores of the region of interest patches to provide the estimated cellularity of the digital pathology slide image at the patch level, Is operable to The first and second DCNNs are trained using a first and second plurality of training digital pathology images as inputs respectively, Training the second DCNN includes, for each training digital pathology image in the second plurality of training digital pathology images, The DCNN model extracts image features from the training digital pathology image, The fully connected output layer operates on the extracted image features to generate an intermediate level representation of the training digital pathology image, A stack of K binary classifiers operates on the intermediate level representation, and each binary classifier predicts whether the cellularity of the training digital pathology image is higher than the rank percentage specific to the binary classifier to generate an ordinal output. fusing K binary outputs from the stack of binary classifiers in a fusion element that weights the outputs of different classifiers differently to provide a fusion cellularity score; iteratively adjusting parameters of the fully-connected output layer and parameters of the DCNN model by minimizing a loss function determined by ground truth cellularity scores of the training digital pathology images, such that upon minimization of the loss function, a final set of training parameters of the fully-connected output layer and the DCNN model is established for the ordinal regression model; Including, An apparatus characterized in that

10. each of the first and second plurality of training digital pathology images is derived at least in part by data augmentation of one or more initial training digital pathology images; 10. The apparatus of claim 9.

11. the data augmentation includes at least one of flipping, rotation, and color perturbation; 11. The apparatus of claim 10.

12. The digital pathology slide image is a hemotoxylin and eosin (H&E) stained tissue image.

10. The apparatus of claim 9.

13. The estimated cellularity of the digital pathology slide image is used as a measure of cancer grading.

10. The apparatus of claim 9.

Citation Information

Patent Citations

  • Multi-task learning for dense object detection

    US20210056361A1

  • Computer supported review of tumors in histology images and post operative tumor margin assessment

    WO2020243545A1