Systems and methods for cytological analysis

Abstract

A computer-implemented method for cytological grading includes receiving (302, 402) first image data (102) representing a biological sample, and parsing (304, 404) the received first image data into a plurality of patches (104). Each of the plurality of patches represents a corresponding portion of the received first image data. The method further includes identifying (306, 406) at least one cytological feature of each of the plurality of patches (108) using a first trained machine learning model (106), determining (308, 408) at least one statistic (110) based on the identified at least one cytological feature of each of the plurality of patches using a second trained machine learning model, and outputting (310, 410) the determined at least one statistic.

Description

[0001] Cross-references to related applications This patent application claims priority to U.S. Application No. 63 / 594,713, filed October 31, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0002] Various embodiments of this disclosure generally relate to systems and methods for cytological analysis. More specifically, this disclosure relates to systems and methods for performing cytological analysis and grading of samples (e.g., samples of skin mast cell tumors) using machine learning. Background Technology

[0003] Mast cell tumors (MCTs) are the most common skin tumors in dogs. Traditionally, MCTs are graded histopathologically. For example, once tissue containing MCTs is surgically removed from a dog, a pathologist can examine and evaluate the tissue. However, due to the availability of effective treatments for MCTs, such as intratumoral injection and electrochemotherapy, the need for cytological grading (e.g., cell evaluation) of MCTs is increasing. To perform cytological grading, MCT samples can be collected from dogs using fine-needle aspiration, and the samples can be scanned using a Hamamatsu scanner to generate digital micrographs of the samples. Pathologists can then grade the samples by examining, for example, the size of cell nuclei and / or the number of mitotic figures depicted in the digital micrographs. However, because digital micrographs can depict a large number of cells, a detailed examination of each depicted cell (or the entire digital micrograph) can be very time-consuming for pathologists. Furthermore, cytological grading of MCTs can be prone to inter-observer variability. Therefore, a more efficient and accurate method is needed for cytological analysis and grading of MCTs, such as those in canine skin.

[0004] This disclosure aims to address the challenges described above. The background description provided herein is intended to provide a general context for this disclosure. Unless otherwise indicated herein, the materials described in this section are not prior art for the purposes of the claims in this application, and are not admitted as prior art or suggestions of prior art simply because they are included in this section. Summary of the Invention

[0005] According to certain aspects of this disclosure, implementation schemes for using machine learning to perform cytological analysis and grading of samples are disclosed.

[0006] A computer-implemented method for cytological grading is disclosed. The method may include receiving first image data representing a biological sample and parsing the received first image data into a plurality of patches. Each of the plurality of patches may represent a corresponding portion of the received first image data. The method may further include identifying at least one cytological feature of each of the plurality of patches using a first trained machine learning model. Additionally, the method may include determining at least one statistic based on the identified at least one cytological feature of each of the plurality of patches using a second trained machine learning model and outputting the determined at least one statistic.

[0007] A computer system for cytological grading is disclosed. The system may include at least one memory storing instructions, and at least one processor configured to execute the instructions to perform operations. Execution of the instructions may cause the at least one processor to receive first image data representing a biological sample and parse the received first image data into a plurality of patches. Each of the plurality of patches may represent a corresponding portion of the received first image data. Execution of the instructions may also cause the at least one processor to identify at least one cytological feature of each of the plurality of patches using a first trained machine learning model. Furthermore, execution of the instructions may cause the at least one processor to determine at least one statistic based on the identified at least one cytological feature of each of the plurality of patches using a second trained machine learning model, and output the determined at least one statistic.

[0008] A non-transitory computer-readable medium storing instructions is disclosed. When executed by at least one processor, the instructions cause the at least one processor to perform operations for cytological grading. The operations may include receiving first image data representing a biological sample and parsing the received first image data into a plurality of patches. Each of the plurality of patches may represent a corresponding portion of the received first image data. The operations may further include identifying at least one cytological feature of each of the plurality of patches using a first trained machine learning model. Additionally, the operations may include determining at least one statistic based on the identified at least one cytological feature of each of the plurality of patches using a second trained machine learning model and outputting the determined at least one statistic.

[0009] It should be understood that the foregoing general description and the following detailed description are exemplary and illustrative only, and do not limit the claimed disclosed embodiments. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various exemplary embodiments and, together with the specification, serve to explain the principles of the disclosed embodiments.

[0011] Figure 1 A flowchart illustrating an exemplary process for cytological analysis and grading of skin MCT samples according to one or more embodiments is depicted.

[0012] Figure 2 Exemplary block diagrams of systems and networks for performing cytological analysis and grading of samples, according to one or more embodiments, are depicted.

[0013] Figure 3 A flowchart illustrating an exemplary process for cytological grading according to one or more embodiments is depicted.

[0014] Figure 4 A flowchart illustrating an exemplary process for cytological grading according to one or more embodiments is depicted.

[0015] Figure 5A Example blocks are depicted among a plurality of blocks according to one or more implementation schemes.

[0016] Figure 5B Example blocks are depicted among a plurality of blocks according to one or more implementation schemes.

[0017] Figure 6 An example system that can perform the techniques presented in this article is described. Detailed Implementation

[0018] The terminology used below may be interpreted in its broadest and most reasonable manner, even when used in conjunction with the detailed description of certain specific examples of this disclosure. In fact, some terms may even be emphasized below; however, any term intended to be interpreted in any limiting manner will be disclosed and specifically defined in this Detailed Description section. Both the foregoing general description and the following detailed description are merely exemplary and explanatory and do not limit the claimed features.

[0019] In this disclosure, the term “based on” means “at least partially based on”. Unless the context otherwise specifies, the singular forms “a,” “an,” and “the” include plural referents. The term “exemplary” is used in the sense of “example” rather than “ideal.” The terms “comprises,” “comprising,” “includes,” “including,” or other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or product that comprises a list of elements does not necessarily include only those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. The term “or” is used separately, such that “at least one of A or B” includes (A), (B), (A and A), (A and B), etc. Related terms such as “substantially” and “usually” are used to indicate possible variations of ±10% in the specified or understood value.

[0020] It should also be understood that although the terms first, second, third, etc., are used in some cases to describe various elements herein, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first contact may be named a second contact, and similarly, a second contact may be named a first contact, without departing from the scope of the various described embodiments. Both first and second contacts are contacts, but they are not the same contact.

[0021] As used herein, depending on the context, the term "if" may optionally be interpreted as "when," "at," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if [the stated condition or event] is detected" may optionally be interpreted as "when determination," "in response to determination," "when [the stated condition or event] is detected," or "in response to detection of [the stated condition or event]."

[0022] In the detailed description herein, references to "implementation," "an implementation," "a non-limiting implementation," "in various implementations," etc., indicate that the described implementation may include specific features, structures, or characteristics, but not every implementation necessarily includes specific features, structures, or characteristics. Furthermore, such phrases do not necessarily refer to the same implementation. Further, when a specific feature, structure, or characteristic is described in connection with an implementation, whether explicitly stated or not, it is assumed that the influence of such feature, structure, or characteristic on other implementations is within the knowledge of those skilled in the art. After reading this specification, those skilled in the art will understand how to implement this disclosure in alternative implementations.

[0023] Generally speaking, terms can be understood at least in part from their use in context. For example, terms such as “and,” “or,” or “and / or” as used herein can have multiple meanings, which can depend at least in part on the context in which such terms are used. Typically, “or,” when used with an associative list such as A, B, or C, is intended to mean A, B, and C, used here in an inclusive sense, and A, B, or C, used here in an exclusive sense. Additionally, depending at least in part on the context, the term “one or more” as used herein can be used to describe any feature, structure, or characteristic in a singular sense, or a combination of features, structures, or characteristics in a plural sense. Similarly, terms such as “a,” “an,” or “the” can also be understood to convey a singular or plural usage, depending at least in part on the context. Furthermore, the term “based on” can be understood to not necessarily convey a set of exclusive factors, but rather to allow for additional factors that are not necessarily explicitly described, which also depends at least in part on the context.

[0024] As used herein, the terms “comprises,” “comprising,” “includes,” “including,” or any other variation thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but may also include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0025] The term "veterinarian" can include, for example, but not limited to, any person, organization, and / or group of people who provide medical care to non-human animals. For example, "veterinarian" can include veterinary technicians, veterinary personnel, and veterinary practitioners.

[0026] The terms "canine" and "dog" can include, for example, but are not limited to, recognized dog breeds (some of which can be further subdivided). For example, recognized dog breeds can include Afghan Hound, Irish Terrier, Akita, Alaskan Malamute, Basset Hound, Beagle, Belgian Shepherd, Bloodhound, Border Collie, Border Terrier, Russian Wolfhound, Boxer, Bulldog, Bull Terrier, Cairn Terrier, Chihuahua, Chow Chow, Cocker Spaniel, Collie, Corgi, Dachshund, Dalmatian, Doberman Pinscher, English Setter, Fox Terrier, German Shepherd, Golden Retriever, Great Dane, Greyhound, and others. Brussels Griffon, Irish Setter, Irish Wolfhound, Cavalier King Charles Spaniel, Labrador Retriever, Lhasa Apso, Mastiff, Newfoundland, Old England Shepherd, Papillon, Pekingese, Pointer, Pomeranian, Poodle, Pug, Rottweiler, Saint Bernard, Saluki, Samoyed, Schnauzer, Scottish Terrier, Shetland Sheepdog, Shih Tzu, Siberian Husky, Skye Terrier, Springer Spaniel, West Highland White Terrier, Yorkshire Terrier, etc.

[0027] As used herein, "machine learning model" generally encompasses instructions, data, and / or models configured to receive input and apply one or more of weights, biases, classifications, or analyses to that input to generate output. Output may include, for example, a classification of the input, an analysis based on the input, a design, processing, prediction, or recommendation associated with the input, or any other suitable type of output. Machine learning models are typically trained using training data (e.g., empirical data and / or samples of input data) that is fed into the model to establish, tune, or modify one or more aspects of the model, such as weights, biases, criteria used to form classifications or clusters, etc. Aspects of a machine learning model may operate on the input linearly and in parallel via a network (e.g., a neural network) or via any suitable configuration.

[0028] The execution of a machine learning model may include deploying one or more machine learning techniques, such as neural networks, convolutional neural networks, region convolutional neural networks, masked region convolutional neural networks, deformable detection transformers, linear regression, logistic regression, random forests, gradient boosting machines (GBM), deep learning, and / or deep neural networks. Supervised and / or unsupervised training may be employed. For example, supervised learning may include providing training data and corresponding labels (e.g., as ground truth) to the training data. Unsupervised methods may include clustering, classification, etc. Any suitable type of training may be used, such as stochastic, gradient boosting, randomly seeded, recursive, epoch-based, or batch-based training, etc.

[0029] The term "gradation" can include, for example, a measure or assessment of how abnormal cancer cells appear when viewed in digital microscopic images or under a microscope. In cytology, gradation can be determined by evaluating the appearance and / or behavior of cells. In histopathology, gradation can be determined by evaluating the structure and function of tissue.

[0030] The following description uses block diagrams and operational examples of methods, processes, apparatus, and devices to describe certain non-limiting embodiments. It should be understood that each block in the block diagrams or operational examples, and combinations of blocks in the block diagrams or operational examples, can be implemented by analog or digital hardware and computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, ASIC, or other programmable data processing apparatus to modify its functionality as described in detail herein, such that instructions executed via the processor of the computer or other programmable data processing apparatus implement the functions / actions specified in the block diagram or one or more operational blocks. In some alternative embodiments, the functions / actions marked in the blocks may not occur in the order indicated in the operational examples. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functions / actions involved.

[0031] In the following description, embodiments will be described with reference to the accompanying drawings. Various embodiments, methods, systems, and computer-readable media for cytological analysis and grading of samples (or specimens) are disclosed below, as will be discussed in more detail.

[0032] In an exemplary use case, a fine-needle aspiration method can be used to collect a canine skin MCT sample from a dog. The sample can be stained and then scanned using a digital scanner (e.g., a Hamamatsu scanner) to generate a digital microscopic image of the stained sample. In some embodiments, the digital microscopic image can be resolved (or divided) into patches, where each patch represents a corresponding portion of the digital microscopic image. The patches can be processed by a first trained machine learning model comprising one or more trained sub-models, where each trained sub-model is configured to recognize (or detect) a corresponding cytological feature depicted in the patch. For example, in some embodiments, a first trained machine learning sub-model can be used to identify mast cells in each patch, a second trained machine learning sub-model can be used to identify mitotic figures in each patch, a third trained machine learning sub-model can be used to identify mast cell nuclei in each patch, and a fourth trained machine learning sub-model can be used to identify multinucleated cells in each patch. Furthermore, in some embodiments, a second trained machine learning model can be used to perform statistical analysis and grading on each detected cytological feature. For example, a second-trained machine learning model can determine a first ratio of the total number of mitotic figures identified in a patch to the total number of mast cells identified in the patch. Alternatively, the second-trained machine learning model can determine a second ratio of the total number of multinucleated cells identified in a patch to the total number of mast cells identified in the patch. The second-trained machine learning model can also determine the cytological grade (or assessment) of the sample based on the first and / or second ratios. Furthermore, the second-trained machine learning model can determine whether a correlation exists between the cytological grade and the histopathological grade of the sample.

[0033] As described above, existing techniques for examining whole digital micrographs are generally inefficient, and traditional cytological grading systems can be prone to inter-observer variability. However, aspects of this disclosure provide procedures for more effectively analyzing whole digital micrographs and more accurately grading samples depicted in the images. Furthermore, the embodiments described herein provide a more detailed and complete cytological analysis of the samples, which can thereby support improved clinical decision-making.

[0034] While the above examples relate to canine skin MCT samples, it should be understood that the techniques of this disclosure are applicable to any suitable sample (e.g., cells associated with cancer, infectious agents, or other diseases or conditions) taken from canine or non-canine (e.g., human, cat, bird, etc.) patients. Furthermore, it should be understood that the examples above are merely illustrative. The techniques and methods of this disclosure can be applied to any suitable application.

[0035] Figure 1A flowchart illustrating an exemplary process 100 for cytological analysis and grading of stained skin MCT samples according to one or more embodiments is depicted. In some aspects, samples can be extracted from canines using fine-needle aspiration, then placed on a glass slide and stained. Figure 1 As shown, process 100 may include using a digital scanner (such as Hamamatsu NanoZoomer) at step 102. ® The S360 digital slide scanner generates whole-slide images (or digital micrographs) of stained samples. All aspects of this disclosure recognize that whole-slide images represent Hamamatsu NanoZoomer Digital Pathology Image (NDPI) files with high pixel resolution (e.g., 200,000 pixels x 100,000 pixels). However, in some embodiments, whole-slide images may be files of a different (or non-NDPI) type. In some aspects, whole-slide images may depict a large number of cells.

[0036] In some implementations, process 100 may include, at step 104, using a computing device (e.g., a workstation, desktop computer, laptop computer, etc.) to parse (or divide) the whole slice image into multiple tiles. In some aspects, each tile may depict a corresponding (or different) portion of the whole slice image. Although for simplicity, Figure 1 Only six tiles are shown, but a full-slice image can be resolved into any number of tiles (e.g., thousands) suitable for analysis by one or more machine learning models. In some respects, dividing a full-slice image into multiple tiles can facilitate the training of machine learning models.

[0037] Process 100 may further include processing each patch of the whole-slice image at step 106 using a first trained machine learning model. In some embodiments, the first trained machine learning model may include one or more trained machine learning sub-models (also referred to herein as “sub-models”), wherein each of the one or more sub-models is configured to identify (or detect) a corresponding cytological feature. For example, the first trained machine learning model may include a first sub-model (e.g., a masked region convolutional neural network model) configured to detect the number and / or location of each mast cell (if any) depicted in each patch. The first trained machine learning model may also include a second sub-model (e.g., a masked region convolutional neural network model) configured to detect the number, location, and / or size of each mast cell nucleus (if any) depicted in each patch. The first trained machine learning model may also include a third sub-model (e.g., a masked region convolutional neural network model) configured to detect the number and / or location of mitotic figures (if any) depicted in each patch. Additionally, the first trained machine learning model may include a fourth sub-model (e.g., a deformable detection transformer model) configured to detect the number and / or location of multinucleated cells (if any) in each tile. In some aspects, each of the first, second, third, and fourth sub-models may be configured not only to identify the corresponding cytological features but also to annotate (e.g., using bounding boxes and / or various colors) each identified feature in the tile. Furthermore, using one or more of the first, second, third, and fourth sub-models (or the corresponding masked region convolutional neural network model to identify each of mast cells, mast cell nuclei, and mitotic figures, and the deformable detection model to identify multinucleated cells) can facilitate accurate cytological grading of MCT samples (e.g., cytological grading associated with a known histopathological grade of the MCT sample, as further described below). In some implementations, in addition to (or as an alternative to) the first, second, third, and fourth sub-models, the first trained machine learning model may include one or more sub-models configured to identify and annotate other cytological features, such as reactive fibroblasts and polymorphism.

[0038] In some implementations, the first trained machine learning model can output a digital image (or digital image data) annotated with one or more cytological features among the identified cytological features, as shown in step 106. More specifically, the first trained machine learning model can output a digital image in which bounding boxes and / or various colors and corresponding legends are used to indicate identified mast cells, nuclei, mitotic figures, multinucleated cells, and / or other cytological features. In some aspects, a whole-section image (or one or more patches of a whole-section image) can correspond to or map to the digital image output from the first trained machine learning model. In some implementations, the first trained machine learning model can also output numerical data representing, for example, the number, size, and / or location (within a given patch and / or whole-section image) of one or more cytological features among the identified cytological features. For example, the first trained machine learning model can output the number, size, and location (within a given patch and whole-section image) of identified mast cells, mast cell nuclei, mitotic figures, and multinucleated cells.

[0039] In some implementations, a second trained machine learning model (e.g., a logistic regression model) may receive numerical data (and optionally annotated digital image data) as input from the output of a first trained machine learning model. The second trained machine learning model may be configured to determine one or more statistics based on the received numerical data. For example, the second trained machine learning model may be configured to determine the total number of mast cells, the total number of mitotic figures, the total number of mast cell nuclei, and the total number of multinucleated cells identified by the first machine learning model in the whole-slice image. The second trained machine learning model may also be configured to determine a first ratio of the total number of mitotic figures identified in the whole-slice image to the total number of mast cells identified in the whole-slice image. The second trained machine learning model may also be configured to determine a second ratio of the total number of multinucleated cells identified in the whole-slice image to the total number of mast cells identified in the whole-slice image. Furthermore, the second trained machine learning model may be configured to determine the size distribution of multinucleation in the whole-slice image. Additionally, the second trained machine learning model may be configured to determine the cytological grade of skin MCT based on, for example, the first and / or the second ratio.

[0040] In some implementations, a second trained machine learning model can be configured to determine whether there is a correlation between the cytological grade of skin MCTs in a sample and the known histopathological grade of skin MCTs. The known histopathological grade can be based on, for example, the Kiupel grading system and / or the Patnaik grading system. In some implementations, the second trained machine learning model can be configured to determine whether there is a correlation between the cytological grade of skin MCTs in a sample and known clinical outcomes.

[0041] like Figure 1 As shown, process 100 may include outputting a report at step 110. The report may include, for example, cytological grades and / or one or more statistics output by a second trained machine learning model. The report may also, or alternatively, include annotated images and / or numerical data output by a first trained machine learning model. In some embodiments, one or more of the cytological grades, statistics, numerical data, and / or annotated images may be input into a third trained machine learning model (e.g., a generative machine learning model) configured to generate a customized report based on the received cytological grades, statistics, numerical data, annotated images, and / or other received data (e.g., data representing the intended audience of the report). For example, the third trained machine learning model may be configured to generate a report customized for the intended audience (such as pathologists, veterinarians, or laypeople). In some embodiments, the third trained machine learning model may be configured to generate a customized report using one or more templates. Although not explicitly stated in Figure 1 As shown, but in some embodiments, process 100 may include inputting data representing, for example, blood test results and / or X-rays into one or more machine learning models. The one or more machine learning models may be configured to identify features in the input data, perform statistical analysis on the identified features, and determine a rank in a manner similar to that described above.

[0042] Figure 2 Exemplary block diagrams of systems and networks for cytological analysis and grading of samples, according to one or more embodiments, are depicted. Specifically, Figure 2An electronic network 220 is illustrated that can be connected to servers in locations such as hospitals (e.g., veterinary hospitals), laboratories, and / or doctors' offices (e.g., veterinary offices). For example, each of the following—veterinary server 221, veterinary hospital server 222, clinical trial server 223, research laboratory server 224, and / or laboratory information system 225—can be connected to the electronic network 220, such as the Internet, via one or more computers, servers, and / or handheld mobile devices. According to an exemplary embodiment of this application, the electronic network 220 can also be connected to a server system 210, which may include one or more storage devices 209 for storing images (e.g., whole slide images, etc.) and data (e.g., histopathological grades, etc.) received from at least one of the following: veterinary server 221, veterinary hospital server 222, clinical trial server 223, research laboratory server 224, and / or laboratory information system 225. The server system 210 may also include processing devices such as workstations, computers, laptops, or other electronic devices. The processing device can be configured to process images and data stored in storage device 209 using cytological imaging, analysis and grading platform 200 (also referred to herein as "platform 200").

[0043] like Figure 2 As shown, platform 200 may include imaging tool 201, analysis tool 202, and feature detection, analysis, and classification tool 203. In some embodiments, imaging tool 201 may include at least one digital scanner (e.g., Hamamatsu NanoZoomer). ® The S360 digital slide scanner (at least one of which is configured to generate whole slide images, as described above) is used to... Figure 1 As described in step 102. Furthermore, in some embodiments, one or more of the veterinary hospitals, laboratories, veterinary offices, etc., associated with servers 221-225 may include imaging tool 201. Analysis tool 202 may be a software module configured to analyze (or segment) the whole-slice images generated by imaging tool 201, as described above regarding... Figure 1 As described in step 104. Feature detection, analysis, and classification tools 203 may include, for example, those referenced above. Figure 1 The discussion focuses on the first trained machine learning model and the second trained machine learning model, and how they are configured to execute... Figure 1 Steps 106, 108, and 110.

[0044] In some embodiments, one or more of the veterinary hospitals, laboratories, veterinary offices, and other entities associated with servers 221-225 may use imaging tool 201 to generate whole-section images of cytological samples (e.g., skin MCT samples collected from canine patients). Furthermore, in some embodiments, the whole-section images may be transmitted from one of servers 221-225 to analysis tool 202 via electronic network 220. In some other embodiments, the whole-section images may be transmitted directly from imaging tool 201 to analysis tool 202. In some aspects, each of servers 221-225 may be configured to transmit the histopathological grade corresponding to the sample depicted in the whole-section image to feature detection, analysis, and grading tool 203 via electronic network 220. Additionally, each of servers 221-225 may be configured to receive reports from feature detection, analysis, and grading tool 203 and via electronic network 220, including, for example, analysis and grading associated with the whole-section images. The reports may be read by pathologists, veterinarians, laypeople, or other individuals.

[0045] Figure 3 A flowchart illustrating an exemplary process 300 for cytological grading according to one or more embodiments is depicted. In some aspects, process 300 may use the references above. Figure 2 The discussion focuses on parsing tool 202 and feature detection, analysis, and classification tool 203 for execution. For example... Figure 3 As shown, process 300 may include receiving first image data representing a biological sample (step 302). In some embodiments, the first image data may be a digital micrograph (e.g., a whole-section image), and the biological sample may be, for example, a stained biological sample of MCT collected from a dog. Process 300 may include parsing the received first image data into a plurality of patches, wherein each of the plurality of patches represents a corresponding portion of the received first image data (step 304). Method 300 may also include identifying at least one cytological feature of each of the plurality of patches using a first trained machine learning model (step 306). Method 300 may also include determining at least one statistic based on the identified at least one cytological feature of each of the plurality of patches using a second trained machine learning model (308). In some embodiments, the at least one statistic may be associated with a tumor (e.g., MCT) or an infectious agent (e.g., bacteria, virus, or fungus).

[0046] Method 300 may include outputting at least one determined statistic (310). In some embodiments, the output of step 310 may include outputting a report comprising at least one statistic. Furthermore, in some embodiments, process 300 may include generating a report using a third trained machine learning model based on the determined at least one statistic. The third trained machine learning model may include a generative machine learning model configured to generate the report based on data representing the intended audience of the report. In some embodiments, process 300 may include outputting second image data representing annotated images from a first trained machine learning model and / or a second trained machine learning model.

[0047] Figure 4 A flowchart illustrating an exemplary process 400 for cytological grading according to one or more embodiments is depicted. Process 400 may be... Figure 3 The implementation plan for process 300. In some respects, process 400 can be referenced from the above. Figure 2 The discussion covers the parsing tool 202 and the feature detection, analysis, and classification tool 203.

[0048] like Figure 4 As shown, process 400 may include receiving first image data representing a biological sample (step 402). In some embodiments, the first image data may represent a digital micrograph (e.g., referenced above). Figure 1 Step 102 discusses the whole-slice image), and the biological sample can be, for example, a stained biological sample of MCT collected from a dog. Process 400 may include parsing the received first image data into a plurality of patches, wherein each of the plurality of patches represents a corresponding portion of the received first image data (step 404). Figure 5A The first example tile of a plurality of tiles is depicted (where the first example tile shows low-grade cytological features of a skin mast cell tumor from a dog). Figure 5B A second example patch from a plurality of patches was depicted (where the second example patch shows high-grade cytological features from a canine skin mast cell tumor). In some respects, the received first image data can be parsed into any number of patches suitable for processing using one or more machine learning models.

[0049] Process 400 may further include using at least one of a trained convolutional neural network model and a trained deformable detection transformer to identify at least one cytological feature for each of a plurality of patches, the at least one cytological feature including one or more of the following: number of mast cells, number of mast cell nuclei, number of mitotic figures, and number of multinucleated cells (step 406). In some embodiments, at least one cytological feature may also, or alternatively, include the position of one or more mast cells, mast cell nuclei, mitotic figures, and multinucleated cells relative to the patch and / or first image data (e.g., a whole-slice image). At least one cytological feature may also, or alternatively, include the size of one or more mast cells, mast cell nuclei, mitotic figures, and multinucleated cells.

[0050] In some implementations, the trained convolutional neural network model may include a first trained masked region convolutional neural network sub-model configured to identify the total number of mast cells in each of a plurality of patches. The trained convolutional neural network model may also include, or alternatively includes, a second trained masked region convolutional neural network sub-model configured to identify the total number of mitotic figures in each of the plurality of patches. The trained convolutional neural network model may also include, or alternatively includes, a third trained masked region convolutional neural network sub-model configured to identify the total number of mast cell nuclei in each of the plurality of patches. A trained deformable detection transformer may be configured to identify the total number of multinucleated cells in each of the plurality of patches.

[0051] In some embodiments, process 400 may further include using a trained logistic regression model and based on at least one cytological feature identified for each of the multiple patches, determining a ratio of the total number of mitotic figures identified for the multiple patches to the total number of mast cells identified for the multiple patches, and at least in part based on the correlation between this ratio and histopathological grade (step 408). In some embodiments, process 400 may further include, or alternatively includes using a trained logistic regression model and based on at least one cytological feature identified for each of the multiple patches, determining a ratio of the total number of multinucleated cells identified for the multiple patches to the total number of mast cells identified for the multiple patches, and at least in part based on the correlation between this ratio and histopathological grade. In some embodiments, process 400 may further include outputting the determined ratio and the determined correlation (410).

[0052] Generally speaking, the terms discussed in this disclosure are understood to refer to any computer-implementable process or operation (such as...). Figure 3 or Figure 4The illustrated process can be executed by one or more processors of a computer system. A process or process step executed by one or more processors can also be referred to as an operation. One or more processors can be configured to execute such a process by accessing instructions (e.g., software or computer-readable code) that, when executed by one or more processors, cause the one or more processors to perform the process. Instructions can be stored in the memory of the computer system. A processor can be a central processing unit (CPU), a graphics processing unit (GPU), or any suitable type of processing unit.

[0053] A computer system (such as a system or apparatus that implements the processes or operations in the examples above) may include one or more computing devices. One or more processors of the computer system may be located in a single computing device or distributed across multiple computing devices. The memory of the computer system may include the respective memory of each of the multiple computing devices.

[0054] Figure 6 The exemplary embodiments according to this disclosure can be configured to perform separately. Figure 3 and Figure 4 A simplified functional block diagram of the apparatus 600 of methods 300 and / or 400. In other words, according to an exemplary embodiment of this disclosure, apparatus 600 can be configured to Figure 2 Parsing tools 202 and / or feature detection, analysis, and classification tools 203. For example... Figure 6 As shown, device 600 may include a central processing unit (CPU) 620. CPU 620 can be any type of processor device, including, for example, any type of dedicated or general-purpose microprocessor device. As those skilled in the art will understand, CPU 620 can also be a single processor in a multi-core / multi-processor system that operates independently, or in a cluster of computing devices operating in a cluster or group of servers. CPU 620 can be connected to data communication infrastructure 610, such as a bus, message queue, network, or multi-core messaging scheme.

[0055] The device 600 may also include main memory 640, such as random access memory (RAM), and may also include auxiliary memory 630. Auxiliary memory 630 (e.g., read-only memory (ROM)) may be, for example, a hard disk drive or a removable storage drive. Such a removable storage drive may include, for example, a floppy disk drive, a magnetic tape drive, an optical disk drive, flash memory, etc. The removable storage drive in this example reads from and / or writes to the removable storage unit in a well-known manner. The removable storage unit may include floppy disks, magnetic tapes, optical disks, etc., read and written by the removable storage drive. As those skilled in the art will understand, such a removable storage unit typically includes a computer-usable storage medium storing computer software and / or data.

[0056] In alternative embodiments, auxiliary memory 630 may include other similar means for allowing computer programs or other instructions to be loaded into device 600. Examples of such means may include a program box and box interface (such as those present in video game devices), a removable memory chip (such as EPROM or PROM) and associated socket, and other removable storage units and interfaces that allow software and data to be transferred from the removable storage units to device 600.

[0057] Device 600 may also include a communication interface (“COM”) 660. Communication interface 660 allows the transfer of software and data between device 600 and external devices. Communication interface 660 may include a modem, network interface (such as an Ethernet card), communication port, PCMCIA slot, and card, etc. The software and data transferred via communication interface 660 may be in the form of signals, which may be electronic, electromagnetic, optical, or other signals that can be received by communication interface 660. These signals may be provided to communication interface 660 via a communication path of device 600, which may be implemented using, for example, wires or cables, optical fibers, telephone lines, cellular telephone links, RF links, or other communication channels.

[0058] The hardware components, operating system, and programming language of such a device are essentially conventional and are assumed to be sufficiently familiar to those skilled in the art. Device 600 may also include input and output ports 650 for connection to input and output devices such as a keyboard, mouse, touchscreen, monitor, display, etc. Of course, various server functions can be implemented in a distributed manner on multiple similar platforms to distribute the workload. Alternatively, the server can be implemented through appropriate programming of a single computer hardware platform.

[0059] Throughout this disclosure, references to components or modules generally refer to items that can be logically combined to perform a function or a group of related functions. Similar reference numerals are generally intended to denote the same or similar components. Components and modules may be implemented in software, hardware, or a combination of software and hardware.

[0060] The aforementioned tools, modules, and functions can be executed by one or more processors. "Storage" media can include any or all tangible memory of a computer, processor, etc., or related modules thereof, such as various semiconductor memories, tape drives, disk drives, etc., which can provide non-transitory storage for software programming at any time.

[0061] Software can communicate via the Internet, cloud service providers, or other telecommunications networks. For example, communication can enable the loading of software from one computer or processor into another. As used herein, unless limited to non-transitory tangible "storage" media, the term "computer or machine-readable medium" refers to any medium involved in providing instructions to a processor for execution.

[0062] The foregoing general description is exemplary and illustrative only, and not intended to limit this disclosure. Other embodiments of the invention will be apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. The specification and examples are intended to be considered exemplary only.

Claims

1. A computer-implemented method for cytological grading, the computer-implemented method comprising: Receive first image data representing a biological sample; The received first image data is parsed into multiple blocks, each of which represents a corresponding part of the received first image data; At least one cytological feature of each of the plurality of tiles is identified using a first trained machine learning model; At least one statistic is determined using a second trained machine learning model based on at least one identified cytological feature of each of the plurality of tiles; as well as Output at least one of the determined statistics.

2. The computer-implemented method of claim 1, wherein identifying the at least one cytological feature of each of the plurality of patches includes identifying at least one of the number of mast cells, the number of mast cell nuclei, the number of mitotic figures, and the number of multinucleated cells.

3. The computer-implemented method of claim 1, wherein the determined at least one statistic includes the ratio of the total number of mitotic figures identified for the plurality of patches to the total number of mast cells identified for the plurality of patches.

4. The computer-implemented method of claim 3, wherein the determined at least one statistic further includes at least in part based on the correlation between the ratio and the histopathological grade.

5. The computer-implemented method of claim 1, wherein the determined at least one statistic includes the ratio of the total number of multinucleated cells identified for the plurality of tiles to the total number of mast cells identified for the plurality of tiles.

6. The computer-implemented method of claim 5, wherein the determined at least one statistic further includes at least in part based on the correlation between the ratio and the histopathological grade.

7. The computer-implemented method of claim 1, wherein the first trained machine learning model comprises at least one of a convolutional neural network sub-model and a deformable detection transformer sub-model, and wherein the second trained machine learning model comprises a logistic regression model.

8. The computer-implemented method of claim 1, wherein the first trained machine learning model includes a mask region convolutional neural network sub-model, the mask region convolutional neural network sub-model being configured to identify the total number of mast cells in each of the plurality of tiles.

9. The computer-implemented method of claim 1, wherein the first trained machine learning model includes a mask region convolutional neural network sub-model, the mask region convolutional neural network sub-model being configured to identify the total number of mast cell nuclei in each of the plurality of tiles.

10. The computer-implemented method of claim 1, wherein the first trained machine learning model includes a mask region convolutional neural network sub-model, the mask region convolutional neural network sub-model being configured to identify the total number of mitotic figures in each of the plurality of tiles.

11. The computer-implemented method of claim 1, wherein the first trained machine learning model includes a deformable detection transformer sub-model configured to identify the total number of multinucleated cells in each of the plurality of tiles.

12. The computer-implemented method of claim 1, wherein at least one statistic determined is associated with a tumor.

13. The computer-implemented method of claim 1, wherein at least one statistic determined is related to the infectious agent.

14. The computer-implemented method of claim 1, further comprising: Output the second image data representing the annotated image.

15. The computer-implemented method of claim 1, wherein the output of at least one determined statistic includes an output report.

16. The computer-implemented method of claim 1, further comprising: A report is generated using a third trained machine learning model based on at least one determined statistic, wherein outputting the at least one determined statistic includes outputting the report.

17. The computer-implemented method of claim 16, wherein the third trained machine learning model comprises a generative machine learning model configured to generate the report based on data representing the intended audience of the report.

18. A computer system for cytological grading, the computer system comprising: At least one memory, wherein the at least one memory stores instructions; as well as At least one processor, configured to execute the instructions to perform an operation, the operation including: Receive first image data representing a biological sample; The received first image data is parsed into multiple blocks, each of which represents a corresponding part of the received first image data; At least one cytological feature of each of the plurality of tiles is identified using a first trained machine learning model; A second trained machine learning model is used to determine at least one statistic based on at least one identified cytological feature of each of the plurality of tiles; and Output at least one of the determined statistics.

19. The computer system of claim 18, wherein identifying the at least one cytological feature of each of the plurality of patches includes identifying at least one of the number of mast cells, the number of mast cell nuclei, the number of mitotic figures, and the number of multinucleated cells.

20. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the processor to perform operations for cytological grading, the operations comprising: Receive first image data representing a biological sample; The received first image data is parsed into multiple blocks, each of which represents a corresponding part of the received first image data; At least one cytological feature of each of the plurality of tiles is identified using a first trained machine learning model; At least one statistic is determined using a second trained machine learning model based on at least one identified cytological feature of each of the plurality of tiles; as well as Output at least one of the determined statistics.

Images