Systems and methods for identifying cancer in pets

Abstract

In one embodiment, the disclosure provides a method comprising: receiving an image of a biopsy slide; executing first programmed instructions to input the image into a first trained machine learning model programmed as a cancer classifier to output a classification; determining based on the classification that the image should be further processed; executing a tissue detection algorithm on the image of the biopsy slide to output a first set of coordinates; executing second programmed instructions to input the image of the biopsy slide and the first set of coordinates into a second trained machine learning model programmed for image segmentation to output a binary pixel mask; executing third programmed instructions to generate an expanded pixel mask using the binary pixel mask; and digitally storing in a computer memory the second set of coordinates indicated by the expanded pixel mask. In one embodiment, ten high power fields can be generated and digitally stored.

Description

Priority

[0001] This application claims the benefit under 35 U.S.C. § 119 of U.S. Provisional Patent Application No. 63 / 233,674, filed August 16, 2021, U.S. Provisional Patent Application No. 63 / 153,308, filed February 24, 2021, and U.S. Provisional Patent Application No. 63 / 125,926, filed December 15, 2020, the entire contents of which are incorporated by reference into this application for all purposes as if fully set forth herein. [Technical field]

[0002] The present disclosure relates generally to machine learning systems for identifying cancer from digital images of animal tissue. [Background technology]

[0003] One way to quantify the aggressiveness of cancer in an animal or pet is to count the number of cells undergoing mitosis in a given area of ​​the body in a given time. Mitotic counts can play a role in traditional grading schemes, such as those for canine mast cell tumors, canine soft tissue sarcomas, canine melanocytic tumors, canine and feline mammary tumors, and the like. An increased density of mitotic cells in a given area of ​​the body may be indicative of a cancer or other disease affecting the animal, for example, it may indicate whether a detected neoplasm is benign or malignant, or may provide valuable prognostic information to a physician. However, the current process for determining an increased density of mitotic cells involves reviewing medical images of a given area of ​​the animal or pet, and manually identifying any areas of concern. This review process can be tedious and time-consuming, and is subject to a variety of inaccuracies or inter-observer variability. Observing the density of mitotic cells can be difficult, at least in part, because the areas of concern may be quite small compared to the overall image being viewed. Further complications can arise from poor resolution of specific mitotic regions of the image. Different observers, even if trained in similar methods, may reach different conclusions when reviewing the same image, resulting in inconsistent application and variability between different observers. The subjective nature of these observations can introduce errors in mitotic counting and reduce overall efficiency. Summary of the Invention [Problem to be solved by the invention]

[0004] As a result, there is a need to automate the process for analyzing medical images of various resolutions and sizes to provide more consistent, more efficient and objective mitotic counts, as well as to make clinically reliable interpretations of mitotic cells. [Means for solving the problem]

[0005] Certain non-limiting embodiments provide systems, methods, media, and combinations thereof for identifying cancer using a machine learning system. For example, digital biopsy images, such as tiled SVS format images often used by medical scanning devices, or any other Digital Imaging and Communications in Medicine (DICOM) images, are processed using one or more machine learning models or tools. The machine learning models or tools can be, for example: K-nearest neighbor (KNN); naive Bayes (NB); decision trees or random forests; support vector machines (SVM); deep learning models such as convolutional neural networks (CNN), region-based CNN (RCNN), one-dimensional (1-D) CNN, recurrent neural networks (RNN); or any other machine learning models or techniques. The machine learning models or tools can be trained to identify relevant images or slides and identify mitotic cells depicted in the slides.

[0006] Certain non-limiting embodiments may be directed to a method that may include one or more of the following steps: receiving an image of a biopsy slide at a computing device; executing first programmed instructions to input the image of the biopsy slide into a first trained machine learning model programmed as a cancer classifier to output a classification of the image of the biopsy slide; determining based on the classification that the image of the biopsy slide should be further processed; executing a tissue detection algorithm on the image of the biopsy slide to output a first set of coordinates; executing second programmed instructions to input the image of the biopsy slide and the first set of coordinates into a second trained machine learning model programmed for image segmentation to output a binary pixel mask; executing third programmed instructions to generate an expanded pixel mask using the binary pixel mask; and digitally storing in a computer memory a second set of coordinates indicated by the expanded pixel mask.

[0007] One embodiment includes: outputting a field of view with the most mitotic figures based on the dilated pixel mask by executing fourth programmed instructions implementing a kd-tree based algorithm; and digitally storing the field of view in a file in the computer memory along with the second set of coordinates.

[0008] In one embodiment, the field of view comprises 10-high-power fields.

[0009] One embodiment includes displaying the field of view on a device display.

[0010] In one embodiment, the tissue detection algorithm includes performing a coarse segmentation of tissue in a low resolution view of the slide image of the biopsy.

[0011] In one embodiment, the third programmed instructions are programmed to: determine, for each connected set of coordinates of the binary pixel mask representing a mitotic figure, a corresponding rotated rectangle, each of the rotated rectangles being of a minimum area that encloses an approximate outline of the mitotic figure; and generate the expanded pixel mask to include only coordinates associated with rotated rectangles each having a width greater than a threshold width digitally stored in the computer memory.

[0012] In one embodiment, the third programmed instructions are further programmed to dilate the binary pixel mask with a number of dilation iterations proportional to the kernel size stored in the computer memory and the inter-pole distance of mitotic cells.

[0013] In one embodiment, the second trained machine learning model includes a convolutional neural network with skip connections trained to minimize at least one of a balanced cross entropy (BCE) soft DiceLoss function or a soft DiceLoss function.

[0014] In one embodiment, the second trained machine learning model further includes a U-Net model pre-trained on ImageNet.

[0015] In one embodiment, the image of the biopsy slide comprises an image of a biopsy of at least one of an animal or pet, the image of the biopsy being generated using a medical imaging device.

[0016] In one embodiment, the image of the biopsy slide includes a whole slide image (WSI).

[0017] One embodiment includes displaying an output image including the image of the biopsy slide and an indication of the classification.

[0018] One embodiment includes a step of programmatically pre-processing the image of the biopsy slide, the pre-processing step including determining that the image of the biopsy slide is sufficiently clear for further processing.

[0019] In one embodiment, the image of the biopsy slide includes a whole slide image, and the pre-processing step includes at least one of determining that a resolution of the image of the biopsy slide exceeds a threshold resolution stored in the computer memory, or determining that the number of slide images included in the whole slide image exceeds a threshold number stored in the computer memory.

[0020] One embodiment includes the steps of: programmatically calculating a mitotic figure count using said second set of coordinates; and digitally storing said mitotic figure count in said computer memory.

[0021] In one embodiment, the computing device comprises a server computer and the image of the biopsy slide is received from a client computing device.

[0022] In one embodiment, the file comprises an extensible markup language (XML) file.

[0023] One embodiment includes transmitting information about the biopsy slide image from the server computer to the client computing device as annotations rendered from the Extensible Markup Language (XML) file.

[0024] One embodiment includes performing the method using a distributed, decentralized, and asynchronous architecture that includes a scalable block, the scalable block including a management program, the management program being implemented using Docker containers.

[0025] Certain non-limiting embodiments may be directed to a computer-readable non-transitory storage medium that includes instructions operable, when executed by one or more processors, to cause a system to perform any of the methods or techniques described herein.

[0026] Certain non-limiting embodiments may be directed to a system comprising: one or more processors; and one or more computer-readable, non-transitory storage media coupled to one or more of the processors and including instructions that, when executed by one or more of the processors, operate to cause the system to perform any of the methods or techniques described herein.

[0027] Certain non-limiting embodiments may be directed to system architectures for use in clinical environments with the systems and methods of the present disclosure. For example, in certain embodiments, one or more pre-processing and / or post-processing steps may be utilized to improve resource efficiency. These steps may be performed by a system architecture that utilizes a pull-based asynchronous architecture that may add new scans to a processing queue as they become available during a designated time period.

[0028] The embodiments disclosed herein are merely examples, and the scope of the disclosure is not limited thereto. Certain non-limiting embodiments may include all, some, or none of the components, elements, features, functions, operations, or steps of the embodiments disclosed herein. Embodiments according to the invention are specifically disclosed in the appended claims directed to methods, storage media, systems, and computer program products, and any feature recited in one claim category, e.g., in the method category, may also be claimed in another claim category, e.g., in the system category. Dependencies or references in the appended claims are selected for formality reasons only. However, subject matter resulting from deliberate reference to previous claims (in specific dependencies) may also be claimed, and therefore any combination of claims and their features is disclosed and may be claimed, regardless of the dependencies selected in the appended claims. Subject matter that may be claimed includes not only the combination of features recited in the appended claims, but also any other combination of features in the claims, and each feature recited in the claims may be combined with any other feature or combination of features in the claims. Furthermore, any of the embodiments and features described or illustrated in this specification may be claimed in a separate claim and / or in any combination with any of the embodiments or features described or illustrated in this specification or with any of the features of the accompanying claims.

[0029] For a more complete understanding of the present disclosure and its features and advantages, reference is now made to the following descriptions, which should be understood in conjunction with the accompanying drawings, in which: [Brief description of the drawings]

[0030] [Figure 1] An exemplary seven-step framework for evaluating mitotic figures to identify cancer [Diagram 2] Exemplary Machine Learning Models According to Certain Embodiments [Diagram 3]Exemplary Set of Biopsy Slide Classification Data and Categories [Figure 4] Exemplary Components of a Machine Learning Model in Accordance with Certain Embodiments [Diagram 5] Exemplary Methods for Using Machine Learning Systems to Identify Cancer [Figure 6] Exemplary Computer Systems or Devices Used to Facilitate Cancer Diagnosis Using Machine Learning Tools, According to Certain Non-Limiting Embodiments [Figure 7] Exemplary Biopsy Slide Classification According to Certain Non-Limiting Embodiments [Figure 8] Example of a Deployed Distributed Architecture According to Certain Non-Limiting Embodiments [Figure 9] Conditional λi of Poisson regression analysis according to certain non-limiting embodiments [Figure 10] Results of an exemplary mitotic count performed without the use of artificial intelligence, an exemplary mitotic count performed with the assistance of artificial intelligence, and an exemplary mitotic count performed solely with artificial intelligence. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] The terms used herein generally have their ordinary meanings in the art, within the context of this disclosure, and in the specific context in which each term is used. Certain terms are discussed below or elsewhere in the specification to provide further guidance in describing the compositions and methods of the present disclosure and how to make and use them.

[0032] As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise.

[0033] As used herein, the terms "comprises," "comprising," or any other variations thereof are intended to cover a non-exclusive inclusion, and thus a process, method, article, system, or apparatus that includes a list of elements may include other elements that are not expressly listed or that are necessarily included in such process, method, article, system, or apparatus, rather than including only those elements.

[0034] As used herein, the term "animal" or "pet" as used in accordance with the present disclosure refers to domestic animals, including but not limited to, domestic dogs, domestic cats, horses, cows, ferrets, rabbits, pigs, rats, mice, gerbils, hamsters, goats, and the like. Domestic dogs and domestic cats are specific non-limiting examples of pets. The term "animal" or "pet" as used in accordance with the present disclosure may also refer to wild animals, as appropriate, including but not limited to, bison, elk, deer, venison, ducks, poultry, fish, and the like.

[0035] As used herein, a "feature" of an image or slide can be determined based on one or more measurable characteristics of the image or slide. For example, a feature can be a detected tissue-dense region in a biopsy image.

[0036] In the Detailed Description of the Invention section of this specification, references to "embodiment," "an embodiment," "one embodiment," "various embodiments," "certain embodiment," "some embodiment," "other embodiment," "certain other embodiment," and the like indicate that one or more of the described embodiments may include a particular feature, structure, or characteristic, but not all embodiments necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in connection with one embodiment, it is understood that such a particular feature, structure, or characteristic may also be realized in connection with other embodiments, whether or not expressly stated. After reading this description, it will become apparent to a person skilled in the art of one or more of the relevant fields how to implement the present disclosure in other embodiments.

[0037] As used herein, the term "device" refers to a computing system or a mobile device. For example, the term "device" may include a smartphone, a tablet computer, or a laptop computer. In particular, a computing system may include a capability to determine its position, direction, or orientation, such as a GPS receiver, a compass, a gyroscope, or an accelerometer. A client device may also include a capability for wireless communication, such as BLUETOOTH communication, near-field communication (NFC), or infrared (IR) communication, or communication with a wireless local area network (WLAN) or a cellular network. Such a device may also include one or more cameras, scanners, touch screens, microphones, or speakers. A client device may also run software applications, such as games, web browsers, or social networking applications. A client device may include, for example, a user equipment, a smartphone, a tablet computer, a laptop computer, a desktop computer, or a smartwatch.

[0038] The exemplary processes and embodiments may be executed or performed by a computing system or client device through a mobile application and associated graphical user interface ("UX" or "GUI"). In a specific, non-limiting embodiment, the computing system or client device may be a mobile computing system, such as, for example, a smartphone, a tablet computer, or a laptop computer. The mobile computing system may include capabilities to determine its location, direction, or orientation, such as a GPS receiver, a compass, a gyroscope, or an accelerometer. Such devices may also include wireless communication capabilities, such as "BLUETOOTH" communication, near field communication (NFC), or infrared (IR) communication, or communication with a wireless local area network (WLAN), 3G, 4G, LTE, LTE-A, 5G, Internet of Things, or a cellular network. Such devices may also include one or more cameras, scanners, touch screens, microphones, or speakers. The mobile computing system may also run software applications, such as games, web browsers, or social networking applications. Using social networking applications, users may connect to and communicate with other users in their social network and share information.

[0039] Generally, mitotic figure counting involves identifying the site of highest mitotic activity across multiple slide sections of a tumor (or similar). Due to the size of the pathology sample (including up to 20 slides per tumor in certain exemplary embodiments), site selection can be a time-consuming process. After site selection, quantitative assessment of the number of mitotic figures requires additional time and effort, especially in areas of high mitotic activity. Inefficiencies caused by inter-observer variability and subjective judgment in both of these steps reduce the accuracy of the resulting mitotic figure counts, wasting valuable diagnostic and treatment time.

[0040] Automation of histopathology analysis tools offers the potential for practice-based efficiencies by allowing the objective determination of mitotic figure counts without the errors that can be caused by the subjective judgment of individual observers.

[0041] FIG. 1 illustrates an exemplary seven-step process framework for evaluating mitotic figures to identify cancer. This identification may include diagnosis or diagnosis. In particular, the process illustrated in FIG. 1 includes seven steps that are used to identify mitotic cells or regions containing mitotic cells within a given image. Although the process illustrated in FIG. 1 includes seven steps, other embodiments may use one or more of the steps identified in FIG. 1, which may be performed in the order listed or in any other order, and in any other steps known to those of skill in the art. In some embodiments, all seven steps are computer-implemented using one or more stored programs of instructions, methods, functions, or other software elements, as further described in other sections herein.

[0042] In a first step 110, a computer system, which may include computer components such as those shown in FIG. 6, as well as other components capable of appropriately receiving and processing images, receives digitally stored biopsy images. In some embodiments, each biopsy image is a whole slide image ("WSI") obtained by digital imaging, scanning, or photography of tissue taken at the biopsy. The biopsy images may include one or more slide images from a medical imaging device. The slide images may have a variety of resolutions, for example, up to 200,000×200,000 pixels. In other non-limiting embodiments, the first step 110 may involve receiving one or more of any other Digital Imaging and Communications in Medicine (DICOM) images. The images may be received from a server, such as a cloud server, or from any other computing or client device on the network, using a database query to an image store, a programmatic call to an API, receiving a network pathname of a folder or directory containing the images and repeatedly opening the file, or other programmatic techniques. The images may be obtained by any known medical imaging device.

[0043] In a second step 120, the method is programmed to classify each slide in the biopsy image. The classification in step 120 can, for example, determine whether a slide should be considered as part of the analysis based on whether the slide has cancer cells or is clear enough to perform an analysis of mitotic figures. The classification in step 120 can allow slides that do not have cancer cells to be filtered out of the analysis at a preliminary stage and not subjected to further analysis. Thus, in one embodiment, step 120 includes determining that a slide is likely to contain cancer based on the classification and sending the slide for further processing based on this determination. Furthermore, the classification in step 120 can help reduce the workload of the system by removing or eliminating slides that do not need to be processed in steps 130, 140, 150, 160, 170. In other non-limiting embodiments, the second step 120 can evaluate any other characteristics of the slide, including the resolution of the slide and / or the number of slide images included in a given WSI. Slides that do not meet basic threshold criteria can be rejected, the values ​​of which may be digitally stored in a configuration file, received by user input, or hard-coded as declared constants or fixed values. In certain embodiments, the classification model can be trained using CNN. For example, a pre-trained residual neural network (ResNet) architecture (e.g., on ImageNet) can be used to extract features of digital biopsy images, followed by a fully connected layer and a softmax activation function that can be used to determine the probability that a slide has cancer cells. In certain embodiments, a ResNet-18 architecture with a depth of 18 layers can be used.

[0044] In a third step 130, the method is programmed to calculate sliding window coordinates of the WSI. As a result of the large size of biopsy images and / or potential memory limitations of a system implementing the process framework of FIG. 1, only a small portion of all data associated with each slide can be examined at a time using a sliding window of data. The sliding window of data can be a subset of the data of a complete slide, programmed to have a size that the system can observe and process in main memory in a given period of time. In some non-limiting embodiments, the sliding window can be determined by first identifying tissues within the slide image. For example, the tissues can be identified by detecting image contrast between the tissue and the overall background, where detecting contrast can distinguish, for example, a bright color background produced by backlit glass. Identifying and / or separating the tissue from the overall background can be referred to as tissue segmentation. Detecting contrast in this manner can improve efficiency by quickly focusing the method on a window of data that is more likely to produce useful and substantive results rather than representing noise.

[0045] In a fourth step 140, the method is programmed to evaluate the data for each identified sliding window. After the method evaluates each sliding window, the method is programmed to identify potential mitotic cells within a particular window. The identified mitotic cells can be highlighted in a copy of the image displayed to the user using a display device of the user computing device and / or sent to step 150. In certain non-limiting embodiments, a deep learning network can be used to perform the data evaluation of step 140. For example, a CNN can be used with architectures and techniques including U-Net and / or skip connections. U-Net can be utilized with other techniques including, but not limited to, BatchNorm, DiceLoss, Tversky loss, Mixup, Architecture Changes, Image Patching, GAN-Training, Bilinear Upsampling, and / or Residual ConvBlocks. Other machine learning models, techniques, or architectures can incorporate one or more of hard negative mining, aggressive augmentation, loss blending, ensembling, masking, resizing, and / or segmentation. The system identifies potential mitotic figures using one or more of the techniques described above.

[0046] In a specific, non-limiting embodiment, the deep learning network may include U-Net followed by CNN. A deep learning network might use the following loss function:

[0047]

number

[0048] can be trained to minimize TIFF0007633406000002.tif4114

[0049] denotes the loss function, α denotes a hyperparameter in the range ∈[0,1], TIFF0007633406000003.tif4114

[0050] represents an image pixelation of 150x150 pixels, TIFF0007633406000004.tif4114

[0051] represents an image pixelation of 600x600 pixels. In some non-limiting examples, TIFF0007633406000005.tif4114

[0052] can be set as the BCE soft Dice loss trained for 50 epochs and / or the soft Dice loss trained for 100 epochs. In a specific non-limiting embodiment, TIFF0007633406000006.tif4114

[0053] can be set as the BCE soft Dice loss for a number of epochs in training the model, followed by the soft Dice loss for additional epochs.

[0054] In a fifth step 150, the method is programmed to perform a post-processing stage that can remove false positives from the initial output data. False positives can be determined using a machine learning model trained with one or more sets of labeled training data. These sets of labeled training data can be a plurality of images of a biopsy slide and a plurality of ground truth labels or annotations associated with each. An exemplary relationship represented in the training data can be that dense tissue indicates mitotic cells. This relationship can be represented in the training data that includes a designated set of digital images that have been labeled manually or using a separate trained machine learning model. The training of the model is now more specifically described. In certain non-limiting embodiments, negative training data in the form of a mask can be used in the method to identify cases where one or more potential clumps of mitotic cells have been falsely identified or counted twice.

[0055] In a sixth step 160, the method is programmed to count mitotic figures by determining the coordinates of all of the pixels that represent mitotic figures. Mitotic figures can be counted as described more specifically herein. The coordinates of all of the pixels of the mitotic figures can be digitally stored. Additionally, the fields with the most mitotic figures, such as a sub-area of ​​10 high power fields, can be determined and digitally stored.

[0056] In a seventh step 170, the results may be sent to a pathologist or the like for further analysis or interpretation. The results may then be displayed on a computing device over a network and / or viewable by a pathologist or other interested party. Additional information regarding the analysis and interpretation may also be included.

[0057] FIG. 2 illustrates a non-limiting embodiment of a machine learning model that can be used to compare sliding windows of images or image data of anatomical structures with different shapes, sizes, and / or orientations. The training dataset used to train the machine learning model can include multiple images with different shapes, sizes, and / or orientations, so that the machine learning model can be trained to flip, rotate, and reshape the images to allow proper evaluation and comparison. In the example of FIG. 2, the machine learning model uses a U-Net architecture 200. The U-Net architecture 200 comprises a specific encoder-decoder scheme that uses multiple layers. The encoder reduces the spatial dimension and increases the channels. The decoder increases the spatial dimension and reduces the channels. Finally, the spatial dimension can be restored to make a prediction for each pixel in the input image.

[0058] The exemplary U-Net 200 shown in FIG. 2 follows a symmetric architecture. The encoder (left) consists of repeated 3×3 convolution blocks 201. Each convolution can be followed by ReLU and batch normalization. The 3×3 convolution blocks 201 can be followed by a 2×2 max pooling block 203 that reduces the spatial dimension. The decoder (right) comprises upsampling and 2×2 transposed convolution blocks 204, which halves the number of feature channels. Each of these upsampling blocks 204 can be followed by repeated 3×3 convolution blocks 201 (each followed by a ReLU). In the last layer, a 1×1 convolution 205 can be used to map the channels to the desired number of classifications. Notably, the illustrated U-Net 200 also includes a “copy and crop” block 202, which may be necessary due to the loss of boundary pixels after each convolution.

[0059] In certain embodiments, U-Net 200 can be utilized in conjunction with other techniques, including but not limited to BatchNorm, DiceLoss, Tversky loss, Mixup, Architecture Changes, Image Patching, GAN-Training, Bilinear Upsampling, and / or Residual ConvBlocks.

[0060] In some non-limiting embodiments, the machine learning framework can include a convolutional neural network (CNN) component that is trained from collected training data from a plurality of images and corresponding quality and classification values. The collected training data can be, for example, one or more images captured by a client device. A CNN is a type of artificial neural network that includes one or more convolutional and subsampling layers with one or more nodes. One or more layers, including one or more hidden layers, can be stacked to form a CNN architecture. In embodiments, a CNN is configured to learn to determine image parameters and subsequent classification and quality of an input image by exposing it to a large amount of labeled data. In some examples, a neural network can train learned weights for each input-output pair, but a CNN can also learn small and primitive features and combine them in complex ways by convolving trainable fixed-length kernels or filters along its inputs. The CNN of FIG. 2 trained on a synthetic dataset of specific labeled images enables accurate object segmentation as well as classification of image features and prediction of features that may be present in real images. A CNN can be supervised or unsupervised.

[0061] In certain non-limiting embodiments, pooling, padding, and / or striding can be used to reduce the size of the output of the CNN of FIG. 2 in the dimension in which the convolution is performed, thereby reducing computational costs and / or the likelihood of overtraining. Striding can describe the size or number of steps over which the filter window slides, and padding can include filling some areas of the data with zeros to buffer the data before or after striding. For example, pooling can include simplifying the information collected by the convolutional layer, or any other layer, and generating a compressed version of the information contained within the layer.

[0062] In some examples, a region-based CNN (RCNN) or one-dimensional (1-D) CNN can be used. RCNN can include using selective search to identify one or more regions of interest in an image and extracting features from each region of interest separately for classification. Types of RCNN employed in one or more embodiments include Fast RCNN, Faster RCNN, or Mask RCNN. In other examples, 1-D CNN can process fixed-length time series segments generated using a sliding window. Such 1-D CNN can be run in a many-to-one configuration, using pooling and striding to concatenate the output of the final CNN layer. A fully connected layer can then be used to generate classification predictions at one or more time steps.

[0063] In contrast to 1-D CNNs, which convolve a fixed-length kernel along the input signal, recurrent neural networks (RNNs) process each time step sequentially, so that the final output of an RNN layer is a function of all previous time steps. In certain embodiments, a variant of an RNN known as a long short-term memory (LSTM) model can be used. An LSTM can include memory cells and / or one or more control gates to model time dependencies in long sequences. In some examples, the LSTM model can be unidirectional, i.e., the model processes the time series in the order in which it was recorded or received. In another example, two parallel LSTM models can be evaluated in opposite directions, both forward and backward in time, when the entire input sequence is available. The results of two parallel LSTM models can be concatenated to form a bidirectional LSTM (bi-LSTM), which can model time dependencies in both directions.

[0064] In some embodiments, one or more CNN models can be combined with one or more LSTM models. The combined model can include a stack of four non-strided CNN layers, followed by two LSTM layers and a softmax classifier. The softmax classifier can normalize a probability distribution that includes a number of probabilities proportional to an exponential function of the input. Since the input signal to the CNN is not padded, for example, each CNN layer shortens the time series by a few samples, even if the layers are not strided. Since the LSTM layers are unidirectional, the softmax classification corresponding to the final LSTM output can be used not only for training and evaluation, but also for reconstructing the output time series from the sliding window segments. However, the combined model can operate in a many-to-one configuration.

[0065] In some embodiments, Transformer-based models can be used to identify relevant features within image segments, which can then be combined with other networks such as CNNs.

[0066] In some embodiments, the machine learning model of FIG. 2 can be trained using pre-validated data through multiple stages. Training can be performed in two stages using two types of training data. In the first stage, slide classification data can be collected and the machine learning model can be trained to classify whether a slide should be counted for mitotic figures. In the second stage, mitotic figure data can be collected automatically or by board-certified anatomic pathologists and other experts.

[0067] FIG. 3 illustrates an exemplary set of biopsy slide classification data and categories. In certain exemplary embodiments, multiple WSIs can be collected and analyzed manually or computerized to determine whether a slide image should be included in the mitotic figure counting process. Each collected slide can be categorized as a count slide 310 or a non-count slide 320, where the count slide 310 can be further analyzed for mitotic figure detection or counting, while the non-count slide 320 is not further analyzed for mitotic figure counting. In addition to being classified as a non-count slide 320, the slide can be further categorized to include the reason for being classified as a non-count, such as "lipoma," "non-neoplastic lesion," "skin punch biopsy," and / or containing in whole or in part "skin and subcutaneous tissue." The analyzed slides can be further divided into two groups, a training group and a validation group, where images in the training group are selected for training a machine learning model and the validation group is used for validation (i.e., testing) of the machine learning model or for validation of the category assignments of the training group images.

[0068] In addition to categorizing slides, data on the labeling of mitotic figures can also be collected for use in training. Similar to the classification of slides, the labeling of mitotic figures can be performed manually or automatically by a computer. Furthermore, these labeled figures can be used to both train machine learning models and / or test previously trained models. In certain exemplary embodiments, a first plurality of mitotic figures can be labeled in and / or across one or more WSIs. In one experiment, 54 mitotic figures were labeled across 11 WSIs. These identified mitotic figures can then be extracted as a number of image patches, each including approximately 150x150 pixels centered on the mitotic figure of each patch. Each of these extracted image patches can then be used to train a first machine learning model, which can then identify a second plurality of mitotic figures in or across the multiple WSIs. In one embodiment, a second plurality of mitotic figures was recognized across 74 WSIs. From this second plurality of mitotic figures, a subset can be sampled (manually or using a machine learning algorithm) to prepare a second plurality of image patches. This process can be repeated until the machine learning model is sufficiently trained. Furthermore, image patches of various sizes can be used, such as 150x150 pixels, 600x600 pixels, and / or 1200x1200 pixels. These patches do not need to be square, and the mitotic figure does not need to be at the center of each patch.

[0069] Certain features of mitotic cells can be identified corresponding to different stages of mitosis that can be identified with appropriate labels. These stages can include: prophase (including circular clusters of chromatin and rods); metaphase (including linear plates, bands, and / or rings, including chromosomes aligned at the equatorial plane with rods protruding from the surface); and anaphase (including two aggregates that are generally linear and have spiked projections, spaced a variable distance apart). Additionally, morphological filters can be used to detect telophase (including two separate aggregates at opposite ends of the cell, with a cleavage furrow between them). As part of training the machine learning algorithm, a model can be trained to identify these different features and appropriately label the features of the input images corresponding to the individual slide frames. A trained machine learning algorithm can also perform this analysis after being trained on previously labeled images (as described above). In addition to automatic identification and labeling by a trained (or being trained) machine learning algorithm, a qualified observer, such as a board-certified anatomical pathologist, can also identify mitotic figures. These manually labeled images and features can also be used to train machine learning algorithms or check the accuracy of previously trained algorithms.

[0070] Additional training data can be created by submitting images that have been annotated to show mitotic figures identified by the model (either trained or being trained). An annotator, which can be a researcher such as an anatomical pathologist or a well-trained model, can then inspect the annotated images and apply appropriate labels. For example, if all of the identified mitotic figures are correct, a label of "correct" can be added to the image. If all of the identified mitotic figures are incorrect, a label of "incorrect" can be added to the image. If the identified mitotic figures are a mixture of correct and incorrect, the image can be annotated at this stage to include the portion of the image that contains mitotic figures. A mask can then be applied to these marked images, as described herein in connection with FIG. 4. After all of the images have been reviewed and appropriately labeled, the data can be collected and added to the training set.

[0071] FIG. 4 illustrates exemplary components of a machine learning model according to certain embodiments. In particular, FIG. 4 illustrates an exemplary method of training data augmentation using a generative adversarial network ("GAN"). In this exemplary method, one or more actual mask features 410 having a certain shape and size can be compared to an input image 420. In other words, in certain embodiments, the machine learning tool can convert the input image 420 into a mask feature 410, which can then be compared to other images of similar size and / or shape stored in a database. In other specific non-limiting embodiments, a single mask feature 430 can be compared to other images 440, even if the images 440 contain dense tissue of different orientations, sizes, and shapes. Thus, the image input by the machine learning tool can be adjusted or manipulated so that the image can be compared to images 440 of different sizes, shapes, and / or orientations. In a further alternative, the mask features 310, 330 can be used to generate synthetic data for training or testing. By applying a mask to the input image, an image having mitotic figures can be generated based on the transformation of one of the masks. As described above, the mask features can be applied manually by a pathologist (e.g., 450) or the annotated sample image (e.g., 460) can be presented to a pathologist for further evaluation and / or verification.

[0072] FIG. 5 illustrates an exemplary method 500 for identifying cancer using a machine learning system. The method may begin with a first step 510, where the system receives an image of a pet from a medical imaging device, the image including one or more features. In a second step 520, the system may determine a corresponding classification for each of the one or more features based on the image, the classification being generated by a machine learning model and the classification being associated with the one or more features. In a third step 530, the system may determine mitotic cells in the image of the pet based on the determined classification. In accordance with the subject matter of the present disclosure, the method may repeat one or more steps of the method of FIG. 5 as necessary. Although the present disclosure has described and illustrated certain steps of the method of FIG. 5 as occurring in a particular order, the present disclosure contemplates that any suitable steps of the method of FIG. 5 may occur in any suitable order. Further, although this disclosure describes and illustrates certain exemplary methods of identifying cancer that include certain steps of the method of Figure 5, this disclosure contemplates any suitable method of identifying cancer that includes any suitable steps, which may include all, some, or none of the steps of the method of Figure 5, as appropriate. Further, although this disclosure describes and illustrates certain components, devices, or systems that perform certain steps of the method of Figure 5, this disclosure contemplates any suitable combination of any suitable components, devices, or systems that perform any suitable steps of the method of Figure 5.

[0073] FIG. 6 illustrates an exemplary computer system 600 used to facilitate identification of mitotic cells using machine learning tools, according to some non-limiting embodiments. In certain non-limiting embodiments, one or more computer systems 600 perform one or more steps of one or more methods described or illustrated herein. In other certain non-limiting embodiments, one or more computer systems 600 provide functionality described or illustrated herein. In certain non-limiting embodiments, software running on one or more computer systems 600 performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein. Some non-limiting embodiments include one or more portions of one or more computer systems 600. In this specification, reference to a computer system encompasses a computing device, where appropriate, and vice versa. Furthermore, reference to a computer system may encompass one or more computer systems, where appropriate.

[0074] The present disclosure contemplates any suitable number of computer systems 600. The present disclosure contemplates computer system 600 taking any suitable physical form. By way of example and not limitation, computer system 600 may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (e.g., a computer-on-module (COM) or a system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile phone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented reality / virtual reality device, or a combination of two or more of these. As desired, computer system 600 may: include one or more computer systems 600; be centralized or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud which may include one or more cloud components in one or more networks. If desired, one or more computer systems 600 may perform one or more steps of one or more methods described or illustrated herein in the substantial absence of spatial or temporal constraints. By way of example and not limitation, one or more computer systems 600 may perform one or more steps of one or more methods described or illustrated herein in real time or in batch mode. If desired, one or more computer systems 600 may perform one or more steps of one or more methods described or illustrated herein at different times or in different locations.

[0075] In a particular, non-limiting embodiment, computer system 600 includes a processor 602, memory 604, storage 606, an input / output (I / O) interface 608, a communication interface 610, and a bus 612. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular configuration, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable configuration.

[0076] In some non-limiting embodiments, the processor 602 includes hardware for executing instructions, such as those that make up a computer program. By way of example and not limitation, to execute instructions, the processor 602 may: retrieve (or fetch) instructions from an internal register, an internal cache, memory 604, or storage 606; decode and execute them; and subsequently write one or more results to an internal register, an internal cache, memory 604, or storage 606. In certain non-limiting embodiments, the processor 602 may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates the processor 602 including any suitable number of any suitable internal caches, as appropriate. By way of example and not limitation, the processor 602 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in an instruction cache may be copies of instructions in memory 604 or storage 606, and the instruction cache may speed up retrieval of these instructions by the processor 602. The data in the data cache may be copies of data in memory 604 or storage 606 for instructions executed by processor 602 to operate on; results of previous instructions executed by processor 602 for access by subsequent instructions executed by processor 602 or for writing to memory 604 or storage 606; or other suitable data. The data cache may speed up read or write operations by processor 602. The TLB may speed up virtual address translations for processor 602. In some non-limiting embodiments, processor 602 may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates processor 602 including any suitable number of any suitable internal registers, as appropriate.Where appropriate, the processor 602 may: include one or more arithmetic logic units (ALUs); may be a multi-core processor; or may include one or more processors 602. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

[0077] In some non-limiting embodiments, memory 604 includes a main memory for storing instructions for processor 602 to execute or data for processor 602 to operate on. By way of example and not limitation, computer system 600 can load instructions into memory 604 from storage 606 or another source (such as another computer system 600). Processor 602 can then load the instructions from memory 604 into an internal register or cache. To execute instructions, processor 602 can retrieve and decode instructions from the internal register or cache. During or after execution of instructions, processor 602 can write one or more results (which may be intermediate or final results) into an internal register or cache. Processor 602 can then write one or more of these results to memory 604. In some non-limiting embodiments, the processor 602 executes instructions only from one or more internal registers or caches or from memory 604 (rather than from locations such as storage 606) and operates only on data in one or more internal registers or caches or from memory 604 (rather than from storage 606). One or more memory buses (each of which may include an address bus and a data bus) may couple the processor 602 to the memory 604. As described below, the bus 612 may include one or more memory buses. In certain non-limiting embodiments, one or more memory management units (MMUs) are present between the processor 602 and the memory 604 to facilitate accesses to the memory 604 requested by the processor 602. In other specific non-limiting embodiments, the memory 604 includes random access memory (RAM). The RAM may be volatile memory, as desired. The RAM may be dynamic RAM (DRAM) or static RAM (SRAM), as desired. The RAM may be single-ported RAM or multi-ported RAM, as desired. This disclosure contemplates any suitable RAM.The memory 604 may, where appropriate, comprise one or more memories 604. Although this disclosure describes and illustrates certain memory components, this disclosure contemplates any suitable memory.

[0078] In some non-limiting embodiments, storage 606 includes mass storage for data or instructions. By way of example and not limitation, storage 606 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Storage 606 may be a removable or non-removable (i.e., fixed) medium, as appropriate. Storage 606 may be internal or external to computer system 600, as appropriate. In certain non-limiting embodiments, storage 606 is a non-volatile solid-state memory. In some non-limiting embodiments, storage 606 includes read-only memory (ROM). Optionally, the ROM may be masked ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these. This disclosure contemplates the mass storage 606 taking any suitable physical form. Optionally, the storage 606 may include one or more storage control units that facilitate communications between the processor 602 and the storage 606. Optionally, the storage 606 may include one or more storages 606. Although this disclosure describes and illustrates a particular storage, this disclosure contemplates any suitable storage.

[0079] In certain non-limiting embodiments, I / O interface 608 includes hardware, software, or both that provide one or more interfaces for communication between computer system 600 and one or more I / O devices. Computer system 600 may include one or more of these I / O devices, as appropriate. One or more of these I / O devices may enable communication between a human and computer system 600. By way of example and not limitation, an I / O device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable IO device, or a combination of two or more of these. An I / O device may include one or more sensors. This disclosure contemplates any suitable I / O device and any suitable I / O interface 608 therefor. As appropriate, I / O interface 608 may include one or more device or software drivers that enable processor 602 to drive one or more of these IO devices. I / O interface 608 may, where appropriate, include one or more I / O interfaces 608. Although this disclosure describes and illustrates a particular I / O interface, this disclosure contemplates any suitable I / O interface.

[0080] In some non-limiting embodiments, the communication interface 610 includes hardware, software, or both that provide one or more interfaces for communication (e.g., packet-based communication, etc.) between the computer system 600 and one or more other computer systems 600 or one or more networks. By way of example and not limitation, the communication interface 610 may include a network interface controller (NIC) or network adapter for communication with an Ethernet or other wired-based network, or a wireless NIC (WNIC) or wireless adapter for communication with a wireless network, such as a Wi-Fi network. This disclosure contemplates any suitable network and any suitable communication interface 610 therefor. By way of example and not limitation, the computer system 600 may communicate with one or more portions of an ad-hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or the Internet, or a combination of two or more of these. One or more, or one or more portions of, these networks may be wired or wireless. By way of example, computer system 600 may communicate with a wireless PAN (WPAN) (such as a "BLUETOOTH" WPAN, a WI-FI network, a WI-MAX network, a cellular network (such as a Global System for Mobile Communications (GSM) network), or other suitable wireless networks, or a combination of two or more of these. Computer system 600 may include any suitable communication interface 610 for any of these networks, as desired.Communications interface 610 may, where appropriate, include one or more communications interfaces 610. Although this disclosure describes and illustrates a particular communications interface, this disclosure contemplates any suitable communications interface.

[0081] In particular, non-limiting embodiments, bus 612 includes hardware, software, or both that couple components of computer system 600 together. By way of example and not limitation, bus 612 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a front side bus (FSB), a HYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or other suitable bus, or a combination of two or more of these. Bus 612 may include one or more buses 612, as appropriate. Although this disclosure describes and illustrates a particular bus, this disclosure contemplates any suitable bus or interconnect.

[0082] Certain non-limiting embodiments are directed to a system architecture for use of the disclosed systems and methods in a clinical environment. In such non-limiting embodiments, one or more deep learning or machine learning models can be used to operate on one or more uploaded scans in real time. If resources in the system architecture are available, one or more WSIs can be downloaded to the device. The WSI can include, for example, a 224x224 pixel thumbnail. In some non-limiting embodiments, a classification of the biopsy slide can be performed. If the classifier returns "no-count", the process can be stopped. If the classifier returns "count", a next step can be performed. The next step can include running a tissue detection algorithm. The tissue detection algorithm can detect the foreground and calculate one or more coordinates for a (model-based) mitotic figure decoder. The above algorithm can be, for example, a non-model-based algorithm. A mitotic figure decoder model can be run on all the calculated coordinates. A pixel mask can then be predicted. The prediction performance can be improved by applying at least one of thresholding and / or morphological filters. These filters can remove any marked pixels that do not contain mitosis by selecting an image that does not contain a mask of the appropriate size, color, or shape.

[0083] In some non-limiting embodiments, a second non-model-based algorithm can be used to calculate the coordinates of the mitotic figures from the dilated pixel mask. A k-dimensional tree (kd-tree), an efficient space-partitioning binary tree data structure for organizing points in k-dimensional space, can be used to find a subregion of the WSI, e.g., a high-power 10 field subregion, that has the identified coordinates of the mitotic cells. This subregion represents the highest concentration of mitotic cells in the WSI. In certain embodiments, a kd-tree in a two-dimensional search space can build a data structure in O(n log n), TIFF0007633406000007.tif6114

[0084] A range report query can be processed by: where n can represent the total number of points and k can represent the number of reported points, each point referring to a unique mitotic cell center coordinate on the 2D plane of the WSI. The coordinates of one or more mitotic figures and the 10 high power fields can be written to an Extensible Markup Language (XML) file, or any other data file, and the results can be written to a database. This particular second non-model-based algorithm presented here is generalized across many shapes and sizes of subregions to be searched, and is not limited to squares, and can be any shape or size. P We can find the subregion of highest density that can be shaped within a sphere of space, where L P The space is a vector space with a defined notion of distance between two given points (e.g., p=2 is a vector space with Euclidean norm). The algorithm can be used to identify subregions with the highest concentration of mitoses bounded by other shapes such as squares [p=infinity], diamonds [p=1], and circles [p=2]. The algorithm presented here generalizes equally well to subregions of any size with the highest concentration of mitoses (i.e., high power fields such as 10, 20, or 50; limited by available memory).

[0085] The models described herein can be integrated into the workflow of a reader. The reader can be, for example, a pathologist and / or any medical professional. The pathologist can, for example, receive information from one or more models as rendered annotations from an XML file. The XML file can be viewed using any known WSI viewer. Once received, the reader can provide additional annotations. The annotations can, for example, be annotated for false positives and / or false negatives. The annotations made by the reader can be stored in a database and used to fine-tune or train a detection model. Fine-tuning of the detection model can be performed based on a large amount of feedback or annotations.

[0086] In certain non-limiting embodiments, non-model-based algorithms can be used at various stages to support mitotic figure counting. For example, tissue detection can be performed by coarse segmentation of tissue in a low-resolution view of the WSI. The low-resolution image can be resized to match at least one of the size of the sliding window of the mitotic figure detector and / or the size of the high-resolution image. The sliding window of the mitotic figure detector can be, for example, an integer number of widths of the resolution image. The low-resolution image can then be converted to a Lab color space image. A binary image threshold can be applied to the L channel of the image using a threshold detection technique such as the Otsu threshold technique. Dilation, erosion, and blurring of the resulting binary image can then be performed on the converted image to remove noise and refine tissue boundaries.

[0087] As mentioned above, the prediction of the pixel mask can be performed using a non-model-based algorithm. For example, geometric and / or morphological procedures can be performed to refine the inference results, reduce false positives, and / or improve the prediction of the high magnification 10 field area. In some non-limiting embodiments, the segmentation of the mitotic figure instances can be performed by computing the connected components of a given binary mask and labeling the connected pixels with a unique integer label. In this way, pixels that belong to a particular candidate mitotic figure instance can receive a unique integer. Mitotic figures, in certain non-limiting embodiments, mitotic figures with abnormally small size can be excluded. A size of about 3 micrometers can be the minimum acceptable size along the width of a mitotic figure. To utilize this minimum acceptable size, the algorithm can determine a rotated rectangle of minimum area that encloses the approximate contour of the mask of the mitotic figure. The width can then be measured as the maximum (height, width) of the rotated rectangle. Then, suitable mitotic figures can be excluded. Then, late stage mitotic figures can be counted. Mitotic figures in the late stages of mitosis may be double-counted, for example, due to the small distance that separates them during the mitotic process. To reduce such double counting, an algorithm can be implemented to dilate the binary pixel mask with a number of dilation iterations proportional to the kernel size and the pole-to-pole distance of the mitotic cells, allowing multiple mitotic figure instances within a given pole-to-pole distance to be counted as a single instance. The connected components can then be assigned unique mitotic figure identifiers.

[0088] In a specific non-limiting embodiment, a kd-tree based algorithm (or other machine learning algorithm) can be used to find the 10 high power fields with the maximum number of mitotic figures. The 10 high power fields (10HPF), which can be a square area for reporting mitotic counts, can be converted to pixel size using the WSI svs format (or other medical imaging file type). The square area can be, for example, 2.37 mm 2The square region of the mitotic figure may be a square region of 0.01 mm. To find the 10 high power fields with the maximum number of mitotic figures, given the coordinates of the mitotic figure center, an algorithm may be implemented in a range search, where in a particular example the range may be two dimensional. The above algorithm is implemented by finding the 10 high power fields with the maximum number of mitotic figures, given the coordinates of the mitotic figure center. TIFF0007633406000008.tif4114

[0089] The system can be configured to find and count all mitotic figures within 10 high-power fields of view of R 2 Let R be the set of N points in R 2 The range may be a family of ranges within, but other ranges are contemplated. TIFF0007633406000009.tif5114

[0090] where N is the total number of mitotic figures detected. The above algorithm can be expressed as, for example, TIFF0007633406000010.tif4114

[0091] This can include constructing a kd-tree for the set of N two-dimensional points (i.e., the centers of all possible detected mitotic figures) using the norm (also known as the Chebyshev distance) of TIFF0007633406000011.tif6114

[0092] The extent of the Euclidean metric in the space bounded by a circle of radius r can be taken as the region bounded by a square of side 2*r in terms of Chebyshev distance. The algorithm then calculates for each point in the set of N two-dimensional points a region of radius R = (HPF side ) / 2 nearest neighbors, where HPF siderepresents the length of one side of the square 10HPF region. The set of vertices of all possible 10HPF regions can be a set derived from a combination of the X and Y coordinates of the MF centers, or a set derived as a Cartesian product between the X and Y coordinates of the MF centers. TIFF0007633406000012.tif4114

[0093] To find points for the kd-tree search in, the vertices of all possible 10HPF regions can be transformed into centers of the regions by adding a 10HPF radius. For each proposed 10HPF center, the kd-tree constructed in step 1 can be searched for all neighbors within a radius of 10HPF. The algorithm can then report the point with the largest number of neighbors.

[0094] FIG. 7 illustrates the results of an exemplary biopsy slide classification. A "count" slide 710 may indicate that the slide is likely to contain cancer based on image features detected during execution of the biopsy slide classification algorithm. A "non-count" slide 720 may indicate that no cancer was detected during execution of the biopsy slide classification algorithm. The figure further illustrates an exemplary slide that was categorized as a "count" slide 710 and edited into a fully annotated slide 730 to indicate mitotic figures 740. This annotation can be performed manually, as described above, or automatically by a machine learning (or artificial intelligence) algorithm. As can be seen in the enlarged portion of the fully annotated slide 730, an exemplary identified mitotic figure 740 is shown.

[0095] FIG. 8 illustrates an example of a deployed distributed architecture for implementing certain non-limiting embodiments. The decentralized and asynchronous structure can include various scalable blocks. These scalable blocks can include a management program, such as a Python queue management program. In a further embodiment, the management program can include Python "worker" programs such as a download worker 860, an inference worker 870, and a post-processing worker 880. The download worker 860 can download multiple slides from a download queue 840 that receives slides from a network pool of slides 810. The download worker 860 can create a local buffer of slides that can be accessible to the inference worker 870 from the inference queue 830. The inference worker 870 can receive the slides and process them to extract mitotic figures. Such actions can be distributed to multiple inference workers 870 based on various factors including, for example, the availability of graphics processing units (GPUs). A post-processing worker 880 can then receive the slides from the post-processing queue 820, perform the post-processing steps as described above, and provide the final results to the pathologist viewer 850. The various workers 860, 870, 880 can have their own queues 820, 830, 840. Each worker can contain one or more Docker containers, each containing a separate bundle of software, libraries, and configuration files specific to the function of that worker.

[0096] According to certain non-limiting embodiments, the architecture shown in Figure 8 can use trained artificial intelligence to obtain more accurate mitotic counts than mitotic counts without trained artificial intelligence. As can be seen in Table 1 below, the grade (i.e., confidence) of the mitotic figure counts can be improved with the use of artificial intelligence beyond what an observer (i.e., pathologist) could achieve without artificial intelligence. Even a pathologist assisted by artificial intelligence can achieve better results than an unassisted pathologist.

[0097] [Table 1]

[0098] For each method i (AI-assisted, AI-only, and unassisted) and for each type j of cancer (see Table 4 below), the mitotic count y ij can be modeled using Poisson regression as follows:

[0099]

number

[0100]

number

[0101] where λ i is a control for the method used to find the mitotic count, and β j is the adjustment for variation across cancer types. The coefficient e in Table 2 λi represents the relative mitotic count after controlling for cancer type:

[0102] [Table 2]

[0103] As shown in Figure 9, the full conditional posterior distributions of the three main effects are displayed in logarithmic scale for ease of reading. These results indicate that a system such as the one described above can be deployed within the workflow and time constraints of a high-throughput multisite veterinary diagnostic pathology service, and that automating mitotic figure counting not only leads to greater laboratory efficiency but also improves the accuracy of mitotic counting. During the testing phase, inference was performed on an average of 3,323 whole-slide images per day, and the scalability of the presented system shows that the workload increases commensurate with the system's capabilities.

[0104] FIG. 10 shows charts illustrating exemplary improvements in accuracy of mitotic counts for canine (C) and feline (F) neoplasms that can be achieved with the use and / or assistance of a trained artificial intelligence model such as those described herein. As can be seen, in most example studies, AI alone counts more detected mitotic figures than AI-assisted and unaided (i.e., performed by an individual) counts. The complete results of these counts are shown in Table 3 below, which shows that the results of several exemplary tests show improvements in certain embodiments depending on a number of factors, including but not limited to, tumor type. The up and down arrows indicate whether the mean MC with AI assistance has increased or decreased compared to the MC without AI assistance by the pathologist.

[0105] [Table 3-1]

[0106] [Table 3-2]

[0107] [Table 3-3]

[0108] [Table 3-4]

[0109] [Table 3-5]

[0110] [Table 3-6]

[0111] [Table 3-7]

[0112] Systems and methods according to exemplary embodiments of the present invention provide improvements over conventional methods. For example, as shown in Table 4 below, the kd-tree method according to certain embodiments shows improvements in efficiency (i.e., time savings) over other brute force techniques. These advantages exist regardless of the number of mitotic figures in an image, and can improve methods of counting mitotic figures.

[0113] [Table 4]

[0114] As used herein, the one or more computer-readable non-transitory storage media may include one or more semiconductor-based or other integrated circuits (ICs) (e.g., field-programmable gate arrays (FPGAs), or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM drives, secure digital cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these, as appropriate. The computer-readable non-transitory storage media may be volatile, non-volatile, or a combination of volatile and non-volatile, as appropriate.

[0115] As used herein, "or" is inclusive and not exclusive, unless expressly indicated otherwise or the context indicates otherwise. Thus, as used herein, "A or B" means "A, B, or both," unless expressly indicated otherwise or the context indicates otherwise. Furthermore, "and" means both jointly and severally, unless expressly indicated otherwise or the context indicates otherwise. Thus, as used herein, "A and B" means "A and B, jointly or severally," unless expressly indicated otherwise or the context indicates otherwise.

[0116] The scope of the present disclosure encompasses any changes, substitutions, variations, changes, and modifications to the exemplary embodiments described or illustrated herein that would be understood by a person of ordinary skill in the art. The scope of the present disclosure is not limited to the exemplary embodiments described or illustrated herein. Furthermore, although the present disclosure describes or illustrates each embodiment herein as including certain components, elements, features, functions, operations, or steps, any of these embodiments can include any combination or permutation of the components, elements, features, functions, operations, or steps described or illustrated anywhere herein that would be understood by a person of ordinary skill in the art. Furthermore, any reference in the appended claims to an apparatus or system, or an apparatus or system component being adapted to, arranged to, capable to, configured to, enabled to, operable to, or operative to perform a particular function, encompasses the apparatus, system, or component, regardless of whether the or that particular function is activated, turned on, or unlocked, so long as the apparatus, system, or component is adapted, arranged, capable, configured, enabled, operable, or operative as described above. Furthermore, although the present disclosure has described or illustrated certain non-limiting embodiments as providing certain advantages, a particular non-limiting embodiment may provide none, some, or all of these advantages.

[0117] Furthermore, the method embodiments presented and described as flow charts in this disclosure are provided as examples to provide a more complete understanding of the technology. The methods of the present disclosure are not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are performed independently.

[0118] Although various embodiments have been described for purposes of this disclosure, these embodiments should not be construed as limiting the teachings of this disclosure to these embodiments. Various changes and modifications can be made to the elements and operations described above to obtain results that are within the scope of the systems and processes described in this disclosure.

[0119] The embodiments disclosed herein are merely examples, and the scope of the disclosure is not limited thereto. Certain non-limiting embodiments may include all, some, or none of the components, elements, features, functions, operations, or steps of the embodiments disclosed above. Embodiments are specifically disclosed in the appended claims directed to methods, storage media, systems, and computer program products, and any feature recited in one claim category, e.g., the method category, may also be claimed in another claim category, e.g., the system category. The dependency or reference in the appended claims is selected for formality reasons only. However, any subject matter resulting from an intentional reference to any preceding claim (in a particular multiple dependency) may also be claimed, and therefore any combination of claims and their features may be disclosed and claimed, regardless of the dependency selected in the appended claims. Claimable subject matter includes not only combinations of features recited in the appended claims, but also any other combination of features in the claims, where each feature recited in a claim may be combined with any other feature or combination of features in the claim. Furthermore, any of the embodiments and features described or illustrated herein may be claimed in a separate claim and / or in any combination with any of the embodiments or features described or illustrated herein, or with any of the features of the appended claims.

[0120] As described above in connection with certain embodiments, certain components, such as computer system 500, may include a single computer or multiple computers, processors, networks, mobile devices, clusters, servers, or other hardware for performing various functions. Furthermore, certain elements of the subject matter of the present disclosure may be embodied in computer readable code that may be stored on a computer readable medium and that, when executed, may cause a processor to perform certain functions described herein. In these embodiments, computers and / or other hardware play an important role in enabling the systems and methods for displaying medical image records. For example, the presence of computers, processors, memory, storage, and networking hardware provides the ability to display medical image records in a more efficient manner. Furthermore, the archiving and storage of digital records cannot be accomplished using pen or paper, as such information is received over a network in electronic form.

[0121] The subject matter and operations described herein can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed herein and structural equivalents thereof, or in any combination of one or more of these.Embodiments of the subject matter described herein can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on a computer storage medium for execution by, or for controlling the operation of, a data processing apparatus.

[0122] A computer storage medium may be or may be included in a computer readable storage device, a computer readable storage substrate, a random or serial access memory array or device, or a combination of one or more of these. Further, a computer storage medium is not a propagating signal, but a computer storage medium may be a source or destination of computer program instructions encoded in an artificially generated propagating signal. A computer storage medium may also be or may be included in one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

[0123] The term "processor" encompasses any kind of apparatus, device, and machine for processing data, including, for example, a programmable processor, a computer, a system on a chip, or a combination or plurality of these. An apparatus may include dedicated logic circuitry (e.g., FPGA or ASIC). In addition to hardware, an apparatus may also include code that creates an execution environment for the computer program in question, such as code that constructs processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of these. The apparatus and execution environment may realize infrastructures for a variety of different computational models, such as web services, distributed computing infrastructures, and grid computing infrastructures.

[0124] A computer program (also known as a program, software, software application, script, or code) can be written in any type of programming language, including compiled or interpreted, declarative or procedural, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program can correspond to a file in a file system, but this does not necessarily have to be the case. A program can be stored: in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document); in a single file dedicated to the program in question; or in several unified files (e.g., several files storing one or more modules, subprograms, or portions of code). A computer program can be deployed to be executed on one computer or on several computers, either at one site or distributed across several sites and interconnected by a communication network.

[0125] The processes and logic flows described herein may be implemented by one or more programmable processors that execute one or more computer programs to perform actions by manipulating input data to generate output. The processes and logic flows may also be implemented by, and apparatus may be implemented as, special purpose logic circuitry, such as an FPGA or ASIC.

[0126] Processors suitable for executing computer programs include, by way of example and not limitation, both general purpose and special purpose microprocessors. Devices suitable for storing computer program instructions and data include, by way of example and not limitation, all types of non-volatile memory, media and memory devices, including semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory can be supplemented by or incorporated in special purpose logic circuitry.

[0127] Additionally, as described above in connection with certain embodiments, certain components may communicate with other certain components, for example, via a network, such as a local area network or the Internet. Unless expressly stated above, the subject matter of the present disclosure is intended to encompass both sides of each transaction, including sending and receiving. Those skilled in the art will readily appreciate that with respect to the above features, if one component sends, transmits, or otherwise makes available something to another component, the other component will receive or get it, whether or not expressly stated.

[0128] In addition to the specific embodiments claimed below, the presently disclosed subject matter is directed to other embodiments having any other possible combinations of the dependent features claimed below and disclosed above. Thus, the specific features presented in the dependent claims and disclosed above can be combined with each other in other possible combinations. Thus, the foregoing description of specific embodiments of the presently disclosed subject matter has been presented for purposes of illustration and description. The foregoing description is not intended to be exhaustive or to limit the presently disclosed subject matter to the embodiments disclosed herein.

[0129] It will be apparent to those skilled in the art that various modifications and variations can be made to the method and system of the presently disclosed subject matter without departing from the spirit or scope of the presently disclosed subject matter. Thus, it is intended that the presently disclosed subject matter cover modifications and variations that come within the scope of the appended claims and their equivalents. Preferred embodiments of the present invention will be described below in detail. EMBODIMENT 1 receiving an image of a biopsy slide at a computing device; executing first programmed instructions to input the image of the biopsy slide into a first trained machine learning model programmed as a cancer classifier to output a classification of the image of the biopsy slide; determining, based on the classification, that the image of the biopsy slide should be further processed; outputting a first set of coordinates by running a tissue detection algorithm on the image of the biopsy slide; inputting the image of the biopsy slide and the first set of coordinates into a second trained machine learning model programmed for image segmentation by executing second programmed instructions to output a binary pixel mask; executing third programmed instructions to generate an expanded pixel mask using the binary pixel mask; and digitally storing in computer memory a second set of coordinates represented by said expanded pixel mask. A computer-implemented method comprising: EMBODIMENT 2 outputting a field of view with the most mitotic figures based on the dilated pixel mask by executing fourth programmed instructions implementing a kd-tree based algorithm; and digitally storing said field of view in said computer memory in a file along with said second set of coordinates. 2. The computer-implemented method of embodiment 1, further comprising: EMBODIMENT 3 3. The computer-implemented method of embodiment 2, wherein the field of view comprises 10 high-power fields. EMBODIMENT 4 3. The computer-implemented method of embodiment 2, further comprising displaying the field of view on a device display. EMBODIMENT 5 2. The computer-implemented method of embodiment 1, wherein the tissue detection algorithm comprises performing a coarse segmentation of tissue in a low-resolution view of the slide image of the biopsy. EMBODIMENT 6 The third programmed instructions include: determining, for each connected set of coordinates of said binary pixel mask indicative of a mitotic figure, a corresponding rotated rectangle, each rotated rectangle being of a minimum area that encloses an approximation of the contour of said mitotic figure; and 2. The computer-implemented method of embodiment 1, programmed to generate the expanded pixel mask to include only coordinates associated with rotated rectangles each having a width greater than a threshold width digitally stored in the computer memory. EMBODIMENT 7 7. The computer-implemented method of embodiment 6, wherein the third programmed instructions are further programmed to dilate the binary pixel mask with a number of dilation iterations proportional to a kernel size stored in the computer memory and a pole-to-pole distance of mitotic cells. EMBODIMENT 8 2. The computer-implemented method of claim 1, wherein the second trained machine learning model comprises a convolutional neural network with skip connections trained to minimize at least one of a balanced cross entropy (BCE) soft DiceLoss function or a soft DiceLoss function. EMBODIMENT 9 9. The computer-implemented method of embodiment 8, wherein the second trained machine learning model further comprises a U-Net model pre-trained on ImageNet. EMBODIMENT 10 2. The computer-implemented method of embodiment 1, wherein the image of the biopsy slide includes an image of a biopsy of at least one of an animal or a pet, and the image of the biopsy was generated using a medical imaging device. EMBODIMENT 11 2. The computer-implemented method of embodiment 1, wherein the image of the biopsy slide comprises a whole slide image (WSI). EMBODIMENT 12 2. The computer-implemented method of embodiment 1, further comprising displaying an output image comprising the image of the biopsy slide and an indication of the classification. EMBODIMENT 13 2. The computer-implemented method of embodiment 1, further comprising a step of programmatically pre-processing the image of the biopsy slide, the pre-processing step including a step of determining that the image of the biopsy slide is sufficiently clear for further processing. EMBODIMENT 14 A computer-implemented method as described in embodiment 13, wherein the image of the biopsy slide includes a whole slide image, and the pre-processing step includes at least one of the steps of determining that the resolution of the image of the biopsy slide exceeds a threshold resolution stored in the computer memory, or determining that the number of slide images contained in the whole slide image exceeds a threshold number stored in the computer memory. EMBODIMENT 15 programmatically calculating the number of mitotic figures using the second set of coordinates; and digitally storing said mitotic figure counts in said computer memory. 2. The computer-implemented method of embodiment 1, further comprising: EMBODIMENT 16 2. The computer-implemented method of embodiment 1, wherein the computing device comprises a server computer and the image of the biopsy slide is received from a client computing device. EMBODIMENT 17 executing fourth programmed instructions implementing a kd-tree based algorithm to output a field of view with the most mitotic figures based on the dilated pixel mask; and digitally storing said field of view in said computer memory in a file along with said second set of coordinates. 17. The computer-implemented method of embodiment 16, further comprising: EMBODIMENT 18 18. The computer-implemented method of embodiment 17, wherein the file comprises an extensible markup language (XML) file. EMBODIMENT 19 20. The computer-implemented method of embodiment 18, further comprising transmitting information about the biopsy slide image from the server computer to the client computing device as annotations rendered from the Extensible Markup Language (XML) file. EMBODIMENT 20 2. The computer-implemented method of embodiment 1, further comprising the step of executing the method using a distributed, decentralized, and asynchronous architecture including a scalable block, the scalable block including a management program, and the management program implemented using a Docker container. EMBODIMENT 21 A system comprising: one or more processors; When coupled to one or more of the processors and executed by one or more of the processors, the method comprises: receiving an image of a biopsy slide; executing first programmed instructions to input the image of the biopsy slide into a first trained machine learning model programmed as a cancer classifier to output a classification of the image of the biopsy slide; determining, based on the classification, that the image of the biopsy slide should be further processed; outputting a first set of coordinates by running a tissue detection algorithm on the image of the biopsy slide; inputting the image of the biopsy slide and the first set of coordinates into a second trained machine learning model programmed for image segmentation by executing second programmed instructions to output a binary pixel mask; executing third programmed instructions to generate an expanded pixel mask using the binary pixel mask; and digitally storing in computer memory a second set of coordinates represented by said expanded pixel mask. one or more computer-readable non-transitory storage media comprising instructions operable to cause the system to perform operations including A system comprising: EMBODIMENT 22 The instructions further include executing fourth programmed instructions that, when executed, implement a kd-tree based algorithm to output a field of view with the most mitotic figures based on the dilated pixel mask; and digitally storing said field of view in said computer memory in a file along with said second set of coordinates. 22. The system of claim 21, further comprising: EMBODIMENT 23 23. The system of embodiment 22, wherein the field of view comprises 10 high power fields of view. EMBODIMENT 24 23. The system of embodiment 22, wherein the instructions, when executed, further operate to cause a step of displaying the field of view on a device display. EMBODIMENT 25 22. The system of embodiment 21, wherein the tissue detection algorithm comprises performing a coarse segmentation of tissue in a low resolution view of the slide image of the biopsy. EMBODIMENT 26 The third programmed instructions include: determining, for each connected set of coordinates of said binary pixel mask indicative of a mitotic figure, a corresponding rotated rectangle, each rotated rectangle being of a minimum area that encloses an approximation of the contour of said mitotic figure; and 22. The system of embodiment 21, programmed to generate the extended pixel mask to include only coordinates associated with rotated rectangles each having a width greater than a threshold width digitally stored in the computer memory. EMBODIMENT 27 27. The system of embodiment 26, wherein the third programmed instructions are further programmed to dilate the binary pixel mask with a number of dilation iterations proportional to the kernel size stored in the computer memory and the interpole distance of mitotic cells. EMBODIMENT 28 22. The system of claim 21, wherein the second trained machine learning model includes a convolutional neural network with skip connections trained to minimize at least one of a balanced cross-entropy (BCE) soft DiceLoss function or a soft DiceLoss function. EMBODIMENT 29 29. The system of embodiment 28, wherein the second trained machine learning model further comprises a U-Net model pre-trained on ImageNet. EMBODIMENT 30 22. The system of embodiment 21, wherein the image of the biopsy slide comprises an image of a biopsy of at least one of an animal or pet, and the image of the biopsy was generated using a medical imaging device. EMBODIMENT 31 22. The system of embodiment 21, wherein the image of the biopsy slide comprises a whole slide image (WSI). EMBODIMENT 32 22. The system of embodiment 21, further comprising displaying an output image comprising the image of the biopsy slide and an indication of the classification. EMBODIMENT 33 The system of embodiment 21, wherein the instructions, when executed, further operate to programmatically pre-process the image of the biopsy slide, the pre-processing step including determining that the image of the biopsy slide is sufficiently clear for further processing. EMBODIMENT 34 The system of embodiment 33, wherein the image of the biopsy slide includes a whole slide image, and the pre-processing step includes at least one of determining that the resolution of the image of the biopsy slide exceeds a threshold resolution stored in the computer memory, or determining that the number of slide images contained in the whole slide image exceeds a threshold number stored in the computer memory. EMBODIMENT 35 The instructions further, when executed: programmatically calculating the number of mitotic figures using the second set of coordinates; and digitally storing said mitotic figure counts in said computer memory. 22. The system of claim 21, further comprising: EMBODIMENT 36 22. The system of embodiment 21, wherein the image of the biopsy slide is received from a client computing device. EMBODIMENT 37 The instructions further, when executed: executing fourth programmed instructions implementing a kd-tree based algorithm to output a field of view with the most mitotic figures based on the dilated pixel mask; and digitally storing said field of view in said computer memory in a file along with said second set of coordinates. 37. The system of embodiment 36, operable to perform the steps: EMBODIMENT 38 38. The system of embodiment 37, wherein the file comprises an Extensible Markup Language (XML) file. EMBODIMENT 39 The system of embodiment 38, wherein the instructions, when executed, further operate to cause the client computing device to perform a step of sending information about the biopsy slide image as annotations rendered from the Extensible Markup Language (XML) file. EMBODIMENT 40 22. The system of embodiment 21, further comprising executing the method using a distributed, decentralized, and asynchronous architecture including a scalable block, the scalable block including a management program, the management program being implemented using Docker containers. [Explanation of symbols]

[0130] 200 U‐Net Architecture, U‐Net 201 3x3 convolution blocks 202 "Copy and Crop" Block 203 2x2 max pooling blocks 204 Upsampling and 2x2 transposed convolution block, upsampling block 205 1x1 convolution 310 Counting Slide 320 Non-counting slides 410 Actual Mask Features 420 Input images 430 Mask Features 440 images 450 Mask Features 460 annotated sample images 600 Computer Systems 602 Processor 604 Memory 606 Storage 608 Input / Output (I / O) Interface 610 Communication Interface 710 "Counting" Slide 720 "Non-counting" slide 730 fully annotated slides 740 Mitotic figures 810 Slide Pool 820 Post-Processing Queue 830 Inference Queue 840 Download Queue 850 Pathologist Viewer 860 Slide Download Workers 870 Inference Workers 880 Post Processing Worker

Claims

1. receiving an image of a biopsy slide at a computing device; executing first programmed instructions to input the image of the biopsy slide into a first trained machine learning model programmed as a cancer classifier to output a classification of the image of the biopsy slide; determining, based on the classification, that the image of the biopsy slide should be further processed; outputting a first set of coordinates by running a tissue detection algorithm on the image of the biopsy slide; executing second programmed instructions to input the image of the biopsy slide and the first set of coordinates into a second trained machine learning model programmed for image segmentation to output a binary pixel mask; executing third programmed instructions to generate an expanded pixel mask using the binary pixel mask; and digitally storing in computer memory a second set of coordinates represented by said expanded pixel mask. A computer-implemented method comprising:

2. outputting a field of view with the most mitotic figures based on the dilated pixel mask by executing fourth programmed instructions implementing a kd-tree based algorithm; and digitally storing said field of view in said computer memory in a file with said second set of coordinates. The computer-implemented method of claim 1 , further comprising:

3. The computer-implemented method of claim 2 , wherein the fields of view include 10-high-power fields.

4. The computer-implemented method of claim 2 , further comprising displaying the field of view on a device display.

5. The computer-implemented method of claim 1 , wherein the tissue detection algorithm comprises performing a coarse segmentation of tissue in a low resolution view of the slide image of the biopsy.

6. The third programmed instructions include: determining, for each connected set of coordinates of said binary pixel mask indicative of a mitotic figure, a corresponding rotated rectangle, each rotated rectangle being of a minimum area that encloses an approximation of the contour of said mitotic figure; and 2. The computer-implemented method of claim 1, further programmed to generate the expanded pixel mask to include only coordinates associated with rotated rectangles each having a width greater than a threshold width digitally stored in the computer memory.

7. 7. The computer-implemented method of claim 6, wherein the third programmed instructions are further programmed to dilate the binary pixel mask with a number of dilation iterations proportional to a kernel size stored in the computer memory and a pole-to-pole distance of a mitotic cell.

8. 2. The computer-implemented method of claim 1, wherein the second trained machine learning model comprises a convolutional neural network with skip connections trained to minimize at least one of a balanced cross entropy (BCE) soft DiceLoss function or a soft DiceLoss function.

9. 10. The computer-implemented method of claim 8, wherein the second trained machine learning model further comprises a U-Net model pre-trained with ImageNet.

10. 2. The computer-implemented method of claim 1, wherein the image of the biopsy slide comprises an image of a biopsy of at least one of an animal or a pet, and the image of the biopsy was generated using a medical imaging device.

11. 10. The computer-implemented method of claim 1, wherein the image of the biopsy slide comprises a whole slide image (WSI).

12. The computer-implemented method of claim 1 , further comprising displaying an output image comprising the image of the biopsy slide and an indication of the classification.

13. 2. The computer-implemented method of claim 1, further comprising programmatically pre-processing the image of the biopsy slide, the pre-processing comprising determining that the image of the biopsy slide is sufficiently clear for further processing.

14. 14. The computer-implemented method of claim 13, wherein the image of the biopsy slide includes a whole slide image, and the pre-processing step includes at least one of determining that a resolution of the image of the biopsy slide exceeds a threshold resolution stored in the computer memory, or determining that a number of slide images included in the whole slide image exceeds a threshold number stored in the computer memory.

15. programmatically calculating the number of mitotic figures using the second set of coordinates; and digitally storing said mitotic figure counts in said computer memory. The computer-implemented method of claim 1 , further comprising:

16. The computer-implemented method of claim 1 , wherein the computing device comprises a server computer and the image of the biopsy slide is received from a client computing device.

17. executing fourth programmed instructions implementing a kd-tree based algorithm to output a field of view with the most mitotic figures based on the dilated pixel mask; and digitally storing said field of view in said computer memory in a file along with said second set of coordinates. The computer-implemented method of claim 16 further comprising:

18. 20. The computer-implemented method of claim 17, wherein the file comprises an extensible markup language (XML) file.

19. 20. The computer-implemented method of claim 18, further comprising transmitting information about the biopsy slide image from the server computer to the client computing device as annotations rendered from the Extensible Markup Language (XML) file.

20. 2. The computer-implemented method of claim 1, further comprising: performing the method using a distributed, decentralized, and asynchronous architecture that includes scalable blocks, the scalable blocks including a management program, the management program being implemented using a Docker container.

21. 13. A system comprising: one or more processors; When coupled to and executed by one or more of the processors, the method comprises: receiving an image of a biopsy slide; executing first programmed instructions to input the image of the biopsy slide into a first trained machine learning model programmed as a cancer classifier to output a classification of the image of the biopsy slide; determining, based on the classification, that the image of the biopsy slide should be further processed; outputting a first set of coordinates by running a tissue detection algorithm on the image of the biopsy slide; executing second programmed instructions to input the image of the biopsy slide and the first set of coordinates into a second trained machine learning model programmed for image segmentation to output a binary pixel mask; executing third programmed instructions to generate an expanded pixel mask using the binary pixel mask; and digitally storing in computer memory a second set of coordinates represented by said expanded pixel mask. one or more computer-readable non-transitory storage media comprising instructions operable to cause the system to perform operations including A system comprising:

22. The instructions further include executing fourth programmed instructions that, when executed, implement a kd-tree based algorithm to output a field of view with the most mitotic figures based on the dilated pixel mask; and digitally storing said field of view in said computer memory in a file with said second set of coordinates.

22. The system of claim 21 , operative to cause the implementation of:

23. 23. The system of claim 22, wherein the fields of view include 10 high power fields of view.

24. 23. The system of claim 22, wherein the instructions, when executed, further operate to cause a step of displaying the field of view on a device display.

25. 22. The system of claim 21, wherein the tissue detection algorithm comprises performing a coarse segmentation of tissue in a low resolution view of the slide image of the biopsy.

26. The third programmed instructions include: determining, for each connected set of coordinates of said binary pixel mask indicative of a mitotic figure, a corresponding rotated rectangle, each rotated rectangle being of a minimum area that encloses an approximation of the contour of said mitotic figure; and 22. The system of claim 21, programmed to generate the expanded pixel mask to include only coordinates associated with rotated rectangles each having a width greater than a threshold width digitally stored in the computer memory.

27. 27. The system of claim 26, wherein the third programmed instructions are further programmed to dilate the binary pixel mask with a number of dilation iterations proportional to a kernel size stored in the computer memory and a pole-to-pole distance of a mitotic cell.

28. 22. The system of claim 21, wherein the second trained machine learning model comprises a convolutional neural network with skip connections trained to minimize at least one of a balanced cross-entropy (BCE) soft DiceLoss function or a soft DiceLoss function.

29. 30. The system of claim 28, wherein the second trained machine learning model further comprises a U-Net model pre-trained with ImageNet.

30. 22. The system of claim 21, wherein the image of the biopsy slide comprises an image of a biopsy of at least one of an animal or pet, the image of the biopsy being generated using a medical imaging device.

31. 22. The system of claim 21, wherein the image of the biopsy slide comprises a whole slide image (WSI).

32. 22. The system of claim 21, further comprising displaying an output image comprising the image of the biopsy slide and an indication of the classification.

33. 22. The system of claim 21, wherein the instructions, when executed, further operate to cause a step of programmatically pre-processing the image of the biopsy slide, the pre-processing step including determining that the image of the biopsy slide is sufficiently clear for further processing.

34. 34. The system of claim 33, wherein the image of the biopsy slide includes a whole slide image, and the pre-processing step includes at least one of determining that a resolution of the image of the biopsy slide exceeds a threshold resolution stored in the computer memory, or determining that a number of slide images included in the whole slide image exceeds a threshold number stored in the computer memory.

35. The instructions further, when executed: programmatically calculating the number of mitotic figures using the second set of coordinates; and digitally storing said mitotic figure counts in said computer memory.

22. The system of claim 21 , operative to cause the implementation of:

36. 22. The system of claim 21, wherein the image of the biopsy slide is received from a client computing device.

37. The instructions further, when executed: executing fourth programmed instructions implementing a kd-tree based algorithm to output a field of view with the most mitotic figures based on the dilated pixel mask; and digitally storing said field of view in said computer memory in a file along with said second set of coordinates.

37. The system of claim 36, operative to cause the implementation of:

38. 40. The system of claim 37, wherein the file comprises an Extensible Markup Language (XML) file.

39. 40. The system of claim 38, wherein the instructions, when executed, further operate to cause the client computing device to perform a step of transmitting information about the biopsy slide image as rendered annotations from the Extensible Markup Language (XML) file.

40. 22. The system of claim 21, further comprising: performing the operations using a distributed, decentralized, and asynchronous architecture that includes scalable blocks, the scalable blocks including a management program, the management program being implemented using a Docker container.

Images