Multilevel image classifier for blood cell images

Abstract

Various embodiments of a blood cell classification system are disclosed herein, which may include multiple general classifiers and specialized classifiers. The system is configured to determine predicted cell labels for imaged blood cells using the multiple general classifiers. The multiple predicted cell labels are then provided to a decision aggregator, which determines whether the predicted cell labels satisfy one or more suitability criteria. In response to determining that the predicted cell labels satisfy the one or more suitability criteria, the decision aggregator assigns a fixed cell label based on one or more of the predicted cell labels provided by the multiple classifiers. Alternatively, in response to determining that the predicted cell labels do not satisfy the one or more suitability criteria, the decision aggregator provides the cell label to a specialized classifier configured to assign a fixed cell label.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to, but is not a provisional application for, Provisional Patent Application No. 63 / 434,798, entitled "Multi-Level Image Classifier for Blood Cell Images," filed with the U.S. Patent and Trademark Office on December 22, 2022, which is incorporated herein by reference in its entirety. [Background technology]

[0002] Blood cell analysis is one of the most commonly performed medical tests to provide an overview of a patient's health status. A blood sample is collected from a patient's body and stored in a test tube containing an anticoagulant to prevent clotting. A whole blood sample typically contains three major classes of blood cells, including red blood cells, white blood cells, and platelets. Each class may be further divided into subclasses of members. For example, the five major types or subclasses of white blood cells (WBCs) have different shapes and functions. White blood cells may include neutrophils, lymphocytes, monocytes, eosinophils, and basophils. There are also subclasses of red blood cell types. The appearance of particles in a sample may vary depending on pathological conditions, cell maturity, and other factors. Subclasses of red blood cells may include reticulocytes and nucleated red blood cells. As a result, classifying different cell types and subtypes can be challenging, and there is a need for improved techniques for this purpose. Summary of the Invention

[0003] Embodiments of the present disclosure can use multiple classifiers to classify one or more blood cells in a sample.

[0004] One embodiment may provide a computer-implemented cell classification system having a processor and a non-transitory computer-readable medium storing instructions that cause the processor to perform a set of operations. These operations may include determining a predicted cell label for a cell using a plurality of classifiers, each of the plurality of classifiers providing a predicted cell label for the cell, and providing the predicted cell labels to a decision aggregator configured to assign a fixed cell label.

[0005] In a further embodiment, there may be a computer-implemented method for classifying a cell. In the method, an image of the cell is obtained. The method can then identify a plurality of predicted cell labels for the cell using a plurality of classifiers and the image of the cell, each predicted cell label of the plurality of predicted cell labels being obtained using a respective classifier of the plurality of classifiers. The method can also assign a fixed cell label to the cell using a decision aggregator and one or more of the plurality of predicted cell labels.

[0006] In another embodiment, there can be a computer-implemented learning method for classifying cells, in which an image of the cell is received. The method can then provide the image of the cell to a plurality of classifiers, training each of the plurality of classifiers to provide a predicted cell label based on the image of the cell. The method can also train a decision aggregator to identify a fixed cell label of the cell based on the predicted cell label.

[0007] While the specification concludes with claims that particularly point out and distinctly claim the invention, it is believed the present invention will be better understood from the following description of specific examples when read in conjunction with the accompanying drawings, in which like reference numerals identify identical elements and in which: [Brief explanation of the drawings]

[0008] [Figure 1]FIG. 1 is a schematic diagram, partially in cross section and not to scale, illustrating the operation of an exemplary flow cell, autofocus system, and high optical resolution imaging device for sample image analysis using digital image processing. [Figure 2] FIG. 2 shows a slide-type vision inspection system according to an embodiment. [Figure 3] FIG. 3 illustrates an exemplary classification system having multiple general and specialized classifiers, according to an embodiment. [Figure 4] FIG. 4 shows a flow chart illustrating a method that may be used in an architecture according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] The drawings are not intended to be limiting in any manner, and it is understood that various embodiments of the invention may be embodied in a variety of other forms, including forms not necessarily depicted in the drawings. The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate several aspects of the invention and, together with the detailed description, serve to explain the principles of the invention. It is understood, however, that the invention is not limited to the precise configurations shown.

[0010] The present disclosure relates to devices, systems, and methods for analyzing blood samples containing blood cells. In one embodiment, the disclosed technology may be used in the context of an automated imaging system comprising an analyzer, which may be, for example, a visual analyzer. In some embodiments, the visual analyzer may further comprise a processor that facilitates automated conversion and / or analysis of images.

[0011] According to some aspects of the present disclosure, a system including a visual analyzer may be provided for acquiring images of a sample containing particles (e.g., blood cells) suspended in a liquid. This system may be useful in characterizing particles in biological fluids, such as detecting and quantifying red blood cells, reticulocytes, nucleated red blood cells, platelets, and white blood cells (including differential count, categorization and subcategorization, and analysis). Other similar uses are also contemplated, such as characterizing blood cells in other fluids.

[0012] While the classification of blood cells in a blood sample is an exemplary application for which the subject matter is particularly well suited, other types of bodily fluid samples may be used. For example, aspects of the disclosed technology may be used to analyze non-blood bodily fluid samples containing blood cells (e.g., white blood cells and / or red blood cells), such as serum, bone marrow, lavage fluid, exudate, effusion, cerebrospinal fluid, pleural fluid, peritoneal fluid, and amniotic fluid. The sample may also be a solid tissue sample, e.g., a biopsy sample processed to produce a cell suspension. The sample may also be a suspension obtained by processing a stool sample. The sample may also be a laboratory or manufacturing line sample containing particles, such as a cell culture sample. The term sample may be used to refer to a sample obtained from a patient or laboratory, or any fraction, portion, or aliquot thereof. The sample may also be diluted, divided into portions, or stained in some processing. Furthermore, the technology discussed herein can be used with a variety of biological cell types to enable appropriate classification of the cells. The embodiments presented herein are primarily described with respect to blood cells, but may also be used for other cell types and with other types of samples, such as urine samples or other biological particles / cells, and thus the techniques discussed herein are aimed at suitable classification of biological materials.

[0013] In some embodiments, samples are presented, imaged, and analyzed in an automated manner. In the case of blood samples, the sample may be significantly diluted with an appropriate diluent or saline solution, thereby reducing the extent to which the appearance of some cells is obscured by other cells in undiluted or less diluted samples. Cells can be treated with agents that enhance the contrast of some cellular aspects, e.g., using permeabilizing agents to permeabilize cell membranes and using histological stains to attach to and reveal features such as granules and nuclei. In some cases, it may be desirable to stain an aliquot of the sample to count and characterize particles including reticulocytes, nucleated red blood cells, and platelets, as well as for the differentiation, characterization, and analysis of white blood cells. In other cases, samples containing red blood cells may be diluted prior to introduction into the flow cell and / or imaging in the flow cell or otherwise.

[0014] Details of sample preparation devices and methods for sample dilution, permeabilization, and histological staining can generally be achieved using precision pumps and valves operated by one or more programmable controllers, as exemplified in patents such as U.S. Patent No. 7,319,907. Similarly, techniques for differentiation of specific cell categories and / or subcategories according to their attributes, such as relative size and color, are known from U.S. Patent No. 5,436,978 in the context of white blood cells, the disclosures of which are incorporated herein by reference in their entireties.

[0015] Referring now to the drawings, Figure 1 schematically illustrates an exemplary flow cell 22 for conveying a sample fluid through a viewing region 23 of a high optical resolution imaging device 24 in a configuration for imaging microparticles in a sample flow stream 32 using digital image processing. The flow cell 22 is coupled to a source 25 of sample fluid, which may have been subjected to treatment such as contact with a particle contrast agent composition and heating. The flow cell 22 is also coupled to one or more sources 27 of particle and / or organelle alignment liquid (PIOAL), such as a clear glycerol solution having a viscosity higher than that of the sample fluid. Examples are disclosed in U.S. Patent Nos. 9,316,635 and 10,451,612, the disclosures of which are incorporated herein by reference in their entireties.

[0016] Sample fluid is injected into the flow cell 22 through a flat opening at the distal end 28 of the sample supply tube 29 at a point where PIOAL flow is substantially established, resulting in a steady, symmetric laminar flow of PIOAL above and below (or on opposite sides of) the ribbon-shaped sample stream. The sample and PIOAL streams may be supplied by a precision metering pump that moves the PIOAL with the injected sample fluid along a significantly narrowing flow path. The PIOAL encapsulates and compresses the sample fluid within the narrowing region 21. The reduced flow path thickness in region 21 may therefore contribute to the geometric focusing of the sample flow stream 32. The sample flow stream 32 is encapsulated and transported along with the PIOAL downstream of the narrowing region 21, for example, in front of or through the viewing region 23 of the high-resolution imaging device 24, where images are collected using a CCD 48. The processor 18 may receive pixel data from the CCD 48 as an input. The sample fluid ribbon, along with the PIOAL, flows to the outlet 33.

[0017] As shown, narrowed region 21 can have a proximal flow path portion 21a having a proximal thickness PT and a distal flow path portion 21b having a distal thickness DT, such that distal thickness DT is less than proximal thickness PT. Thus, sample fluid can be injected through distal end 28 of sample tube 29 at a location distal to proximal portion 21a and proximal to distal portion 21b. Thus, as the PIOAL stream is compressed by region 21, the sample fluid will enter the PIOAL envelope. The sample fluid injection tube has a distal exit port through which sample fluid is injected into the flowing sheath liquid, the distal exit port being restricted by the reduced flow path size of the flow cell.

[0018] A digital high optical resolution imager 24 having an objective lens 46 is oriented along an optical axis that intersects the ribbon-shaped sample flow stream 32. The relative distance between the objective lens 46 and the flow cell 22 is varied by operation of a motor drive 54 to resolve and collect a focused digital image on a photosensor array. Further information regarding the construction and operation of an exemplary flow cell such as that shown in FIG. 1 is provided in U.S. Patent No. 9,322,752, filed March 17, 2014, entitled "Flow Cell Systems and Methods for Particle Analysis in Blood Samples," the disclosure of which is incorporated herein by reference in its entirety.

[0019] Aspects of the disclosed technology may be applied in contexts other than flow cell systems such as that shown in FIG. 1. For example, FIG. 2 illustrates a slide-based vision inspection system 200 in which aspects of the disclosed technology may be used. In the system illustrated in FIG. 2, a slide 202 containing a sample, such as a blood sample, is placed in a slide holder 204. The slide holder 204 may be adapted to hold multiple slides or only one slide, as shown in FIG. 2. An image capture device 206, including an optical system 208 and an image sensor 210, is adapted to acquire image data depicting the sample within the slide 202. A light emitting device (not shown) may also be used to control the light environment and thereby acquire image data that is easier to analyze.

[0020] Image data acquired by the image capture device 206 can be transferred to an image processing device 212. The image processing device 212 can be an external device, such as a personal computer, connected to the image capture device 206. Alternatively, the image processing device 212 can be incorporated into the image capture device 206. The image processing device 212 can include a processor 214 associated with a memory 216 configured to determine the amount of change required to determine the difference between the actual focus and the correct focus for the image capture device 206. Once the difference is determined, instructions can be transferred to a steering motor system 218. The steering motor system 218 can change the distance z between the slide 202 and the optical system 208 based on instructions from the image processing device 212. Descriptions of approaches that can be used to achieve focus using this type of setup are provided in U.S. Patent Nos. 9,857,361, 10,794,900, 10,705,008, and 10,705,011, the disclosures of each of which are incorporated herein by reference in their entirety.

[0021] In the systems shown in FIG. 1 or FIG. 2, a process such as that shown in FIG. 3 can be used to classify cells imaged by a camera, such as the high optical resolution imager 24 of FIG. 1 or the image sensor 210 of FIG. 2. Initially, the process of FIG. 3 involves receiving an image / representation 301 of a plurality of cells. This may include, for example, a processor receiving one or more images, each including a representation of a plurality of cells (e.g., as may be imaged in a slide-type system as shown in FIG. 2). Alternatively, receiving a representation 301 of a plurality of cells may include a processor receiving multiple images, each including a representation of only a single cell (e.g., as may be imaged by a flow cell-type flow imaging system as shown in FIG. 1). Note that the term "representation" refers to communicating an analytical representation of a cell, such as an image, frequency domain, pixel analysis associated with a cell image, or numerical data associated with analyzing a representation (e.g., an image) of a cell. An image is a subset of a representation; in other words, an image is a type of representation.

[0022] These representations can then be isolated (e.g., using a cell isolation algorithm, such as an algorithm that thresholds images captured by a flow cell-based system, to identify portions of the image that represent cells and portions of the image that do not), and various imaging parameters can be determined for each of the represented cells. This may be done, for example, by applying an image analysis algorithm that can process the representations of the cells to identify values ​​for various parameters associated with their classification. Exemplary types of algorithms that can be applied include those described in U.S. Pat. No. 4,538,299, issued August 27, 1985, for "Method and Apparatus for Locating the Boundary of an Object," the disclosure of which is incorporated herein by reference in its entirety. However, in some embodiments, there may be multiple types of cells to classify, including some cells that are extremely difficult to distinguish due to their biological and morphological properties. In some cases, multiple classifiers can be utilized to address this. Furthermore, multiple classifiers can be used to increase the accuracy of any final label given and reduce the risk of mislabeling cells.

[0023] An example of how multiple classifiers may be used in some cases is provided in FIG. 3. As shown in FIG. 3, classification system 300 may include multiple general classifiers 310 (e.g., 311, 312, 313, 314). The general classifiers may be configured to receive cell representations 301 and generate classification labels. However, as noted herein, there are typically multiple cell types that need to be classified for a flow image analyzer. For example, in some embodiments, the system may need to classify cells into 11 different cell types, such as neutrophils, immature granulocytes, lymphocytes, monocytes, eosinophils, basophils, nucleated red blood cells (NRBCs), blasts, other WBCs, non-WBCs, and indistinguishable. In some cases, each of the general classifiers 310 may be trained to identify all of the cell types of interest, such as each of the cell types listed above. While a general classifier is outlined herein using artificial intelligence (e.g., a convolutional neural network (CNN)), it should be understood that other classification methods may be utilized, such as population-based (PB) classifiers that utilize imaging data and corresponding cluster-based population segmentation to identify cell types, as disclosed in U.S. Provisional Patent Application No. 63 / 434,658, filed December 22, 2022, which is incorporated herein by reference in its entirety. In some examples, a classifier (e.g., a CNN) may utilize artificial intelligence or machine learning concepts to analyze images of cells and classify cell types by providing associated labels. In some examples, a classifier (e.g., a PB classifier) ​​may utilize a population distribution, whereby imaging features become parameters for organizing the population distribution of multiple cell types. Cell types are distinguished within this distribution, for example, by exploring clusters of data and assigning similar labels to cells within a particular cluster. In other words, certain classifiers, such as the PB classifier, may utilize pixel analysis and rules that involve thresholding in conjunction with pixel analysis (e.g., generating a mask to analyze the pixels of a cell according to various pixel analysis parameters), creating a distribution, and assigning cell types based on clustering analysis rather than deep learning or machine learning techniques. In an example where cell classification is performed using a CNN, the cell classifier can be organized into the following layers: 1st layer: Input layer that takes in a 128x128x3 RGB image Second layer: Convolutional layer (filter size = 5, stride = (2, 2), number of filters = 64, and ReLU activation) + batch normalization layer. Output is a 64x64x64 tensor. Layer 3: Convolutional layer (filter size = 5, stride = (2, 2), number of filters = 128, and ReLU activation) + batch normalization layer. Output is a 32x32x128 tensor. Layer 4: Convolutional layer (filter size = 5, stride = (2, 2), number of filters = 256, and ReLU activation) + batch normalization layer. Output is a 16x16x256 tensor. Layer 5: Convolutional layer (filter size = 5, stride = (2, 2), number of filters = 512, and ReLU activation) + batch normalization layer. Output is an 8x8x512 tensor. Layer 6: Convolutional layer (filter size = 5, stride = (2, 2), number of filters = 512, and ReLU activation) + batch normalization layer. The output is a 4x4x512 tensor. 7th layer: flattening layer Layer 8: Fully connected layer with 100 outputs Layer 9: Fully connected layer with 11 outputs

[0024] It should be understood that the above nine-layer model is only one example of how a CNN can be used in a classification system (e.g., to provide the functionality of a general classifier). Accordingly, additional or alternative embodiments are possible, in which the CNN-based classifier may have a different structure than that described above, or may have the same structure but be trained on different data. For example, in some embodiments, the input image size may be specified as N x N x 3, where N varies from 10 to 1000. In further embodiments, filters (e.g., filters used in layers 2 through 6) may vary in size from 3 to 9. Further embodiments may also exist in which the number of convolutional layers can vary significantly (e.g., from 3 to 100).

[0025] 3 , a cell representation 301 may be received and analyzed by each of general classifiers 311, 312, 313, 314, etc. As a non-limiting example, "General Classifier 1" 311 may independently analyze cell representation 301 based on its own learning, while "General Classifier 2" 312 and any subsequent "General Classifier N" 314 may also independently analyze cell representation 301 based on their own learning (e.g., based on the training data set used to create those classifiers). In some embodiments, and as represented by "..." 313, the number of general classifiers may vary depending on the needs of the system. Thus, it should be understood that any number of general classifiers may be utilized.

[0026] Once the general classifiers 310 receive and analyze a cell representation 301 (e.g., a cell image), the system 300 may evaluate 302 the classification of each of the general classifiers 310 to determine whether an accurate cell label was generated. In some embodiments, evaluating 302 the generated cell labels may involve identifying a “majority vote” among all of the general classifiers 310. As a non-limiting example, in an embodiment in which “General Classifier 1” 311 categorizes cell representation 301 as a neutrophil, “General Classifier 2” 312 classifies the cell representation as a monocyte, and “General Classifier 3” (not shown) and “General Classifier 4” (not shown) classify the cell representation as a neutrophil, the system may determine that the cell is likely a neutrophil because three-quarters (i.e., 75%) of the general classifiers determined that the cell representation is a neutrophil. Continuing with the non-limiting example, once the system determines 302 that an accurate label was generated, it may apply 303 to the cell representation 301, which in this example may be a neutrophil. In various examples, the majority vote may require a threshold of at least 50% or a threshold of 50% or greater (by way of example, the number of classifiers used, such as odd or even, may affect the thresholding).

[0027] It should be understood that the above non-limiting examples are for illustrative purposes only and that there may be various methods 302 for assessing label accuracy. For example, in some embodiments, the system may require 100% overall agreement by all general classifiers 310 to determine that a label is accurate. In further embodiments, the system 300 may require a specific percentage of the general classifiers 310 to agree on a cell type (e.g., 60%, 66%, 75%, etc.). Other embodiments may require different threshold percentages to be assigned to different cell types. For example, in some embodiments, 60% agreement between the general classifiers 310 may be required to determine that a cell representation 301 is an NRBC, and 75% agreement between the general classifiers may be required to determine that a cell representation 301 is a monocyte. In other words, a particular cell type (e.g., a first cell type) may require a higher percentage of agreement between classifiers than another cell type (e.g., a second cell type that is different from the first cell type), e.g., a rare cell type or a cell type that is more difficult to classify.

[0028] In addition to cell type, the output of the general classifier 310 may be weighted based on the type of classification system used. For example, if general classifier 1 311 uses a population-based classification system and general classifier 2 312 uses a CNN-based classification system, then general classifier 2 may have a greater weight (e.g., greater influence) during the evaluation of label accuracy 302. For example, rather than simple voting where each vote is equal, votes may be weighted (e.g., general classifier 2 may have a weight of 1.1 compared to a weight of 1.0 for general classifier 1). For example, in a situation where general classifier 1 is better at identifying certain cell types (e.g., red blood cell types) and therefore is weighted more heavily for other cell types, and general classifier 2 is better at identifying certain cell types (e.g., white blood cell types) and therefore is weighted more heavily for certain cell types, different cell types may have different weights for each classifier. It is also possible that the output of a classifier may be weighted based on other information, such as its accuracy, precision, and reproducibility in classifying certain types of cells.

[0029] Alternatively, if it is determined (302) that an accurate label was not generated, the system 300 may invoke an appropriate specialized classifier 304 (e.g., a specialized classifier trained to distinguish between only the cell types provided by the general classifier with maximum confidence). In some embodiments, the specialized classifier 304 may be an image classifier trained to evaluate a small number (e.g., two) of cell types. This is in contrast to the general classifier 310, which is trained to identify all desired cell types, including, but not limited to, neutrophils, immature granulocytes, lymphocytes, monocytes, eosinophils, basophils, nucleated red blood cells (NRBCs), blasts, other WBCs, non-WBCs, and indistinguishables. The specialized classifier may be any appropriate type of classifier, such as a CNN-based classifier or a population-based classifier.

[0030] Thus, the specialized classifier 304 is more accurate and reliable when making decisions because it is trained to distinguish only a few (e.g., two) cell types. Thus, because there are multiple cell types of interest, a specific / unique specialized classifier 304 may be trained for each pair of cell types. Thus, by way of non-limiting example, multiple specialized classifiers 304 may be utilized, each specialized classifier trained on a unique or custom decision between a pair of cells, such as, for example, determining whether the cell representation 301 indicates one of the following: 1. Neutrophils vs. Immature Granulocytes 2. Neutrophils vs. Lymphocytes 3. Neutrophils vs. Monocytes 4. Neutrophils vs. Eosinophils 5. Neutrophils vs. Basophils 6. Neutrophils vs. NRBCs 7. Neutrophils vs. Blasts 8. Neutrophils vs. Others 9. Immature granulocytes vs. lymphocytes 10. Immature granulocytes vs. monocytes 11. Or any other combination of cell types

[0031] Thus, if the general classifiers 310 disagree on the cell type, the system 300 will determine (302) that the generated label is inaccurate, which may require the specialized classifier 304 to make the final decision (e.g., identify the fixed label). For example, if 50% of the general classifiers 310 classify the cell representation 301 as a lymphocyte and the other 50% of the general classifiers classify the cell representation as a monocyte, the system 300 can utilize the specialized classifier 304, trained to specifically distinguish between lymphocytes and monocytes, to make the final decision between the two cell types. Once the specialized classifier 304 has completed its decision, the system may apply the final label 303.

[0032] In one example embodiment, system 300 may include two general classifiers 310 (e.g., 311 and 312), each generating a classification label for a given input image (e.g., cell representation 301). If the classification labels from the two general classifiers match, the system may determine (302) that the label is accurate and apply a final classification label 303. Alternatively, if the classification labels from the two general classifiers do not match (e.g., lymphocytes vs. monocytes), a specialized classifier 304 trained to evaluate these two cell types (e.g., lymphocytes and monocytes) is invoked to generate the final label 303.

[0033] In a further example embodiment, system 300 may include three general classifiers 310, each of which generates a classification label and a confidence score (e.g., an indication of the likelihood that the classification label is accurate) for a given input image, and this confidence score may be used for classification. For example, in some cases, if a general classifier has particularly high confidence in a particular label (e.g., 98% confidence that a cell is a lymphocyte), that label may be treated as the final label without considering specialized classifiers, even if other general classifiers may provide different labels. This may only be done in certain cases. For example, if a general classifier is identified as a preferred classifier for a particular type of cell (e.g., is particularly accurate in identifying that cell type), and if that general classifier identifies a representation of that cell type with greater than a threshold confidence, then the representation will be classified as that type of cell without recourse to specialized classifiers, even if other general classifiers may disagree. It should be noted that the above high confidence scores that provide a final label without consulting specialized classifiers are exemplary, and different scores can be used for different cell types (e.g., a score of 98% or higher for lymphocytes would result in lymphocyte classification being used without consulting specialized classifiers, while a score of 95% for neutrophils would result in neutrophil classification being used without consulting specialized classifiers). It should be noted that the above confidence scores are exemplary, and any range of scores, e.g., 90%-100%, can be used to trigger this automatic final classification.

[0034] Confidence scores may be applied in other ways as well. For example, if two or three of the classification labels of the general classifiers match, in some embodiments (e.g., with a 65% majority threshold), the matching classification labels of the general classifiers become the final label 303. On the other hand, if all three classification labels of the general classifiers are different, the two classification labels with the highest confidence scores may be provided to the specialized classifier 304 to generate the final label. As a non-limiting example, assume that the labels and confidence scores of the three general classifiers are (lymphocyte, 0.80), (monocyte, 0.65), and (NRBC, 0.71). Therefore, because the classification labels of the three general classifiers do not match, the specialized classifier 304 configured to perform binary classification between lymphocytes and NRBCs will be invoked to generate the final label because it has the highest confidence scores of 0.8 and 0.71, respectively.

[0035] In further illustrative embodiments, the system (e.g., 300) may include more classifiers and / or more complex operations. Specifically, in some embodiments, the system may include a population-based (PB) general classifier, a first convolutional neural network (CNN) general classifier (CNN1), a second CNN general classifier (CNN2), and multiple specialized classifiers. In some embodiments, the PB general classifier may evaluate cell representations (e.g., 301 in FIG. 3 ) to determine cell labels (e.g., PB_Label), and the general classifier CNN1 may evaluate the same cell representations to determine cell labels (e.g., CNN1_label).

[0036] As described herein, a general classifier (e.g., 310) can assign a confidence score to any determined classification (e.g., label). Thus, in some embodiments, the system will evaluate and / or determine a confidence level for CNN1_Label, which is then compared to a particular threshold. If CNN1_Label has an insufficient confidence level (e.g., does not meet the threshold), the system immediately assigns an indistinguishable label. On the other hand, if the confidence level for CNN1_Label meets or exceeds the threshold, then the determined cell labels (e.g., PB_Label and CNN1_Label) are compared to determine whether they match.

[0037] If a match is determined between PB_Label and CNN1_Label, the system will assign CNN1_Label, which is also PB_Label based on the match. In a further embodiment, the system may again evaluate the confidence level of CNN1 to determine whether it meets or exceeds a predetermined threshold (e.g., 0.6, 0.7, 0.8, etc.). If the system determines that CNN1_Label meets or exceeds the threshold, CNN1_Label will be assigned to the cell representation. Alternatively, if the system determines that CNN1_Label is equal to or less than the threshold, CNN1_Label is not assignable and the system will assign an "unidentifiable" label.

[0038] Alternatively, in this type of system, it may be determined that PB_Label and CNN1_Label do not match. In this case, the system would evaluate whether the confidence level of CNN1 meets or exceeds a predetermined threshold (e.g., 0.6, 0.7, 0.8, etc.). If the system determines that CNN1_Label meets or exceeds the threshold, CNN1_Label may be compared to CNN2_Label (e.g., the cell label determined by the general classifier CNN2) to determine whether they match. If CNN1_Label and CNN2_Label match (e.g., a majority of matches), the system would assign CNN1_Label, which is also CNN2_Label based on the match, to the cell representation. In other embodiments, the system would determine that CNN1_Label is below the threshold and therefore not assignable. If the identified label does not meet or exceed the threshold, the system would assign an "unidentifiable" label.

[0039] There may also be scenarios in which a specialized classifier is required to generate a final label in this type of system. As described herein, the specialized classifier may be used to identify a particular cell type based on an existing classification generated by a general classifier (e.g., 310). Thus, in some cases (e.g., when (1) PB_Label and CNN1_Label do not match and the confidence score of CNN1_Label is below a threshold, (2) PB_Label and CNN1_Label do not match and the confidence score of CNN1_Label is above a threshold, or (3) CNN1_Label and CNN2_Label do not match, or in other scenarios in which the specialized classifier is not able to generate a cell type or indistinguishable label), the specialized classifier may be used to determine which label (e.g., PB_Label, CNN1_Label, CNN2_Label, etc.) is accurate and assign that label as SC_Label. SC_Label will then be evaluated to determine whether it has a confidence score above a given threshold. If the SC_Label meets or exceeds the threshold, the cell representation will be assigned the SC_Label, but if the confidence score is below the threshold, the system will assign an "indistinguishable" label.

[0040] Further illustration of how multiple types of systems can be combined to provide cell classification is shown in the logic maps of Tables 1 and 2. In some examples, each classifier is of a similar general type (e.g., each is a CNN). In some examples, at least some of the classifiers are of different general types (e.g., Classifier 1 is a PB, Classifier 2 is a CNN, and Classifier 3 is a CNN). In the example in the table below, Classifier 2 is a general first-order classifier (configured to generally classify a variety of cells, and its labels carry a higher weight), Classifier 1 is a general second-order classifier (configured to generally classify a variety of cells, but its labels carry a lower weight than Classifier 2), and Classifier 3 is a specialized classifier (configured to classify only one specific cell type), and the specialized classifier is configured to select from these two different labels provided by the two different classifiers, just as selecting from two different labels, as described above.

[0041] In the table below, the first specific cell type and the second specific cell type may be specific types of cells that are difficult to classify and may require the use of specialized rules. For example, those labeled as the first specific cell type may be certain cells that are difficult to categorize and may require a second threshold different from the first threshold. Those labeled as the second specific cell type may be certain cells that are difficult to categorize and may require confirmation by a third classifier (Classifier 3) that is specifically / solely configured to analyze only the second specific cell type.

[0042] [Table 1]

[0043] [Table 2]

[0044] It should be understood that in some cases, any threshold may be updated or modified (e.g., automatically or manually) based on a number of factors, such as, for example, changing user preferences, changing evaluation requirements, changes in the imaging system, based on an AI or machine learning algorithm, or otherwise. It should also be understood that thresholds may be equal across the system (e.g., a confidence score of 0.7 is required for each threshold test above) or alternatively, thresholds may be customized for each step of the processing (e.g., the threshold for CNN1_Label in a first determination may be lower than the threshold in a subsequent determination step).

[0045] 4, a flow diagram of an illustrative method 400 is shown in accordance with various embodiments disclosed herein. In some embodiments, and as shown, method 400 will begin with step 401, in which a plurality of predicted cell labels are determined using a plurality of general classifiers (e.g., 310). The predicted cell labels are then provided 402 to a decision aggregator, which is configured to evaluate the predicted cell labels and determine 403 whether one of the predicted cell labels meets criteria for an accurate label (e.g., whether the label represents at least a threshold level of match, or whether the label is given a sufficiently high confidence, etc.).

[0046] In response to determining that one of the plurality of predicted labels meets the criteria for an accurate label, method 400 will assign a final cell label as the label that meets the criteria for an accurate label (404). Alternatively, in some embodiments, in response to determining that none of the plurality of classifiers meets the criteria for an accurate label, method 400 will continue at step 405 with providing a labeled cell representation to a specialized classifier trained to classify cells into two or more categories of the predicted cell labels.

[0047] As described herein above, various metrics may be used, selected, or modified (e.g., automatically or manually) to determine fitness criteria (e.g., thresholds). Thus, in some embodiments, one or more fitness criteria may be derived from or based on a majority consensus among multiple classifiers of predicted cell labels and / or a unanimous consensus among multiple classifiers of predicted cell labels. In additional embodiments, one or more fitness criteria may be derived from or based on one of multiple classifiers meeting or exceeding a confidence score threshold.

[0048] As further illustrations of potential implementations and applications of the disclosed technology, the following examples are provided in a non-exclusive manner in which the teachings herein may be combined or applied. It should be understood that the following examples are not intended to limit the scope of protection of any claims that may be presented at any time in this application or a subsequent application related to this application. No disclaimer is intended. The following examples are provided for illustrative purposes only. It is contemplated that the various teachings herein may be configured and applied in numerous other ways. It is also contemplated that some variations may omit specific features mentioned in the following examples. Therefore, none of the aspects or features mentioned below should be considered essential unless expressly stated otherwise, such as at a later date, by the inventor or the inventor's successor in interest. If any claims are presented in this application or in a subsequent application related to this application that include additional features other than those mentioned below, those additional features should not be presumed to have been added for any reason regarding patentability.

[0049] Example 1 A computer-implemented method for blood cell classification, comprising: a. acquiring an image of a blood cell; b. identifying a plurality of predicted cell labels for the blood cell using a plurality of classifiers and the image of the blood cell, each predicted cell label of the plurality of predicted cell labels being acquired using a respective classifier of the plurality of classifiers; and c. assigning a fixed cell label to the blood cell using a decision aggregator and one or more of the predicted cell labels of the plurality of predicted cell labels.

[0050] Example 2 The computer-implemented method of Example 1, wherein the decision aggregator is further configured to provide the image of the blood cell to a specialized classifier configured to: a. determine whether one of the predicted cell labels provided by the plurality of classifiers meets the criteria for an accurate label; b. in response to determining that one of the predicted cell labels meets the criteria for an accurate label, assign a fixed cell label based on the cell label that meets the criteria for an accurate label; and c. in response to determining that none of the predicted cell labels meets the criteria for an accurate label.

[0051] Example 3 The computer-implemented method of Example 2, wherein the criteria for correct labeling are selected from the group consisting of: a. majority consensus of predicted cell labels among the multiple classifiers; and b. unanimous consensus of predicted cell labels among the multiple classifiers.

[0052] Example 4 3. The computer-implemented method of Example 2, wherein the criteria for a correct label are selected from the group consisting of: a. one of the predicted cell labels of the plurality of classifiers that meets a confidence score threshold; and b. one of the predicted cell labels of the plurality of classifiers that exceeds a confidence score threshold.

[0053] Example 5 5. The computer-implemented method of any of Examples 2 to 4, wherein the specialized classifier is one of a plurality of specialized classifiers, each of the plurality of specialized classifiers configured to classify the cell images into one of two classes.

[0054] Example 6 6. The computer-implemented method of any of Examples 2 to 5, wherein each of the plurality of classifiers provides a confidence score associated with the predicted cell label provided by the classifier, and the processor is configured to select a specialized classifier based on the two predicted cell labels having the two highest confidence scores.

[0055] Example 7 7. The computer-implemented method of any of Examples 2 to 6, wherein the specialized classifier comprises a convolutional neural network.

[0056] Example 8 8. The computer-implemented method of any of Examples 1 to 7, wherein one of the plurality of classifiers comprises a convolutional neural network.

[0057] Example 9 9. The computer-implemented method of any of Examples 1 to 8, wherein for each image in the set of images, each of the plurality of classifiers is configured to utilize image analysis of blood cells depicted in the image to provide a predicted cell label thereof.

[0058] Example 10 The computer-implemented method of any of Examples 1 to 9, wherein the step of acquiring images of blood cells comprises: a. passing a blood sample through a flow cell; and b. capturing images of the blood cells using a camera as the blood cells pass through a viewing area of ​​the flow cell.

[0059] Example 11 1. A computer-implemented system for classifying blood cells, comprising: a. a processor; and b. a non-transitory computer-readable medium storing instructions causing the processor to perform a set of operations, the set of operations comprising: i. receiving a set of images, each image in the set of images depicting a blood cell; and ii. for each image in the set of images, A. determining a set of predicted cell labels for blood cells depicted in the image using a plurality of classifiers, each of the plurality of classifiers providing a respective predicted cell label for blood cells depicted in the image; and B. providing the set of predicted cell labels to a decision aggregator configured to assign fixed cell labels.

[0060] Example 12 The computer-implemented cell classification system of Example 11, wherein the decision aggregator is further configured to: a. determine whether one of the predicted cell labels in the set of predicted cell labels satisfies the criteria for an accurate label; b. in response to determining that one of the predicted cell labels satisfies the criteria for an accurate label, assign a fixed cell label based on the predicted cell label that satisfies the criteria for an accurate label; and c. in response to determining that none of the predicted cell labels satisfies the criteria for an accurate label, provide the cell label to a specialized classifier configured to assign a fixed cell label.

[0061] Example 13 13. The computer-implemented cell classification system of Example 12, wherein the criteria for correct labeling are selected from the group consisting of: a. majority consensus of predicted cell labels among the multiple classifiers; and b. unanimous consensus of predicted cell labels among the multiple classifiers.

[0062] Example 14 13. The computer-implemented cell classification system of Example 12, wherein the criteria for an accurate label are selected from the group consisting of: a. one of the predicted cell labels of the plurality of classifiers that meets a confidence score threshold; and b. one of the predicted cell labels of the plurality of classifiers that exceeds a confidence score threshold.

[0063] Example 15 15. The computer-implemented cell classification system of any of Examples 12 to 14, wherein the specialized classifier is one of a plurality of specialized classifiers, each of the plurality of specialized classifiers configured to classify the cell image into one of a set of two possible cell classes.

[0064] Example 16 16. The computer-implemented cell classification system of any of Examples 12 to 15, wherein each of the plurality of classifiers provides a confidence score associated with the predicted cell label provided by the classifier, and the processor is configured to select a specialized classifier based on the two predicted cell labels having the two highest confidence scores.

[0065] Example 17 17. The computer-implemented cell classification system of any of Examples 12 to 16, wherein the specialized classifier comprises a convolutional neural network.

[0066] Example 18 18. The computer-implemented cell classification system of any of Examples 11 to 17, wherein one of the plurality of classifiers comprises a convolutional neural network.

[0067] Example 19 19. The computer-implemented cell classification system of any of Examples 11 to 18, wherein for each image in the set of images, each of the plurality of classifiers is configured to utilize image analysis of blood cells depicted in the image to provide a predicted cell label thereof.

[0068] Example 20 20. The computer-implemented cell sorting system of any of Examples 11 to 19, wherein: a. the system comprises: i. a camera; and ii. a flow cell having a viewing area; and b. instructions stored on a non-transitory computer-readable medium that, when executed, cause a processor to capture a set of images by imaging a blood sample as it flows through the viewing area of ​​the flow cell.

[0069] Example 21 1. A computer-implemented training method for blood cell classification, comprising: a. receiving a plurality of blood cell images; b. training a plurality of general classifiers using the plurality of blood cell images, each classifier of the plurality of general classifiers being trained to classify cell images into a set of classes corresponding to the general classifier; and c. training a plurality of specialized classifiers using the plurality of blood cell images, each classifier of the plurality of specialized classifiers being trained to classify cell images into a set of classes corresponding to the specialized classifier, wherein A. the plurality of general classifiers are A. a corresponding set of classes comprises a general classifier having a minimum general cardinality, the minimum general cardinality being less than or equal to any other cardinality of the set of classes corresponding to any classifier in the set of general classifiers; B. a plurality of specialized classifiers comprises specialized classifiers whose corresponding sets of classes have a maximum specialized cardinality, the maximum specialized cardinality being greater than or equal to any other cardinality of the set of classes corresponding to any classifier in the set of specialized classifiers; and C. the minimum general cardinality is greater than the maximum specialized cardinality.

[0070] Example 22 22. The computer-implemented learning method of example 21, wherein at least one of the set of general classifiers comprises a convolutional neural network.

[0071] Example 23 23. The computer-implemented learning method of any of Examples 21 or 22, wherein each of the plurality of specialized classifiers is configured to provide a corresponding confidence value when providing a label for the cell image.

[0072] Example 24 24. The computer-implemented learning method of any of Examples 21 to 23, wherein the maximum specialization cardinality is 2.

[0073] Example 25 25. The computer-implemented learning method of any of Examples 21 to 24, wherein the minimum general cardinality is 11.

[0074] Each of the calculations or operations / operations described herein may be performed using a computer or other processor having hardware, software, and / or firmware. Various method steps may be performed by modules, which may comprise any of a variety of digital and / or analog data processing hardware and / or software configured to perform the method steps described herein. A module optionally comprises data processing hardware adapted to perform one or more of these steps by having appropriate machine programming code associated therewith, where modules for two or more steps (or portions of two or more steps) are integrated on a single processor board or separated onto different processor boards in any of a variety of integrated and / or distributed processing architectures. These methods and systems often employ tangible media embodying machine-readable code of instructions for performing the method steps described above. Suitable tangible media may comprise memory (including volatile and / or nonvolatile memory), storage media (magnetic recording on floppy disks, hard disks, tapes, etc., optical memory such as CDs, CD-R / Ws, CD-ROMs, DVDs, or any other digital or analog storage medium), etc.

[0075] All patents, patent publications, patent applications, journal articles, books, technical literature, and the like mentioned in this disclosure are hereby incorporated by reference in their entirety for all purposes.

[0076] Different configurations of the components depicted in the drawings or described above, as well as components and steps not shown or described, are possible. Similarly, some features and subcombinations are useful and may be employed without reference to other features and subcombinations. The embodiments of the invention have been described for purposes of illustration and not limitation, and alternative embodiments will become apparent to readers of this patent. In certain cases, method steps or actions may be performed in a different order, or actions may be added, deleted, or modified. It will be appreciated that in certain aspects of the invention, a single component may be substituted for multiple components, or multiple components may be substituted for a single component, to provide an element or structure or to perform a given function or functions. Except to the extent that such a substitution would render the particular embodiment of the invention inoperable, it is deemed to be within the scope of the invention. Accordingly, the claims should not be treated as limited to the examples, drawings, embodiments, and descriptions provided above, but should be understood as having the scope to be given when those terms are given their broadest reasonable interpretation as afforded by a commonly used dictionary. except that where a term or phrase is set forth in the subheading "Expressly Provided" as having a particular meaning, it is to be understood to have that meaning when used in the claims.

[0077] Explicit provision It should be understood that in the above examples and claims, a statement that something is "based on" something, etc., should be understood to mean that it is at least partially specified by the thing that is indicated as being based on. To indicate that something must be completely specified based on something else, whatever that thing must be completely specified by, it is described as "based exclusively on."

[0078] It should be understood that in the above examples and claims, the term "a set" should be understood as one or more grouped together.

Claims

1. 1. A computer-implemented method for blood cell differentiation, comprising: a. acquiring an image of a blood cell; b. identifying a plurality of predicted cell labels for the blood cell using a plurality of classifiers and the image of the blood cell, each predicted cell label of the plurality of predicted cell labels being obtained using a respective classifier of the plurality of classifiers; c. assigning a fixed cell label to the blood cell using a decision aggregator and one or more of the predicted cell labels of the plurality of predicted cell labels; A computer-implemented method comprising:

2. The decision aggregator further comprises: a. determining whether one of the predicted cell labels provided by the plurality of classifiers meets an accurate label criterion; b. responsive to determining that one of the predicted cell labels meets the correct label criteria, assigning the fixed cell label based on the cell labels that meet the correct label criteria; c. providing the image of the blood cell to a specialized classifier configured to assign the fixed cell label in response to determining that none of the predicted cell labels meets the criteria for the correct label.

2. The computer-implemented method of claim 1, configured to:

3. The criteria for accurate labeling are: a. A majority consensus of the predicted cell labels among the plurality of classifiers; and b. A consensus of the predicted cell labels among the plurality of classifiers.

3. The computer-implemented method of claim 2, wherein the method is selected from the group consisting of:

4. The criteria for accurate labeling are: a. one of the predicted cell labels of the plurality of classifiers that meets a confidence score threshold; and b. one of the predicted cell labels of the plurality of classifiers that exceeds a confidence score threshold 3. The computer-implemented method of claim 2, wherein the method is selected from the group consisting of:

5. 3. The computer-implemented method of claim 2, wherein the specialized classifier is one of a plurality of specialized classifiers, each of the plurality of specialized classifiers configured to classify cell images into one of two classes.

6. 3. The computer-implemented method of claim 2, wherein each of the plurality of classifiers provides a confidence score associated with the predicted cell label provided by that classifier, and the processor is configured to select the specialized classifier based on the two predicted cell labels having the two highest confidence scores.

7. The computer-implemented method of claim 2 , wherein the specialized classifier comprises a convolutional neural network.

8. The computer-implemented method of claim 1 , wherein one of the plurality of classifiers comprises a convolutional neural network.

9. 2. The computer-implemented method of claim 1, wherein for each image in the set of images, each of the plurality of classifiers is configured to utilize image analysis of the blood cells depicted in that image to provide a predicted cell label thereof.

10. The step of acquiring the image of the blood cell includes: a. passing a blood sample through a flow cell; b) capturing images of the blood cells using a camera as they flow through a viewing area of ​​the flow cell; The computer-implemented method of claim 1 , comprising:

11. 1. A computer-implemented blood cell classification system, comprising: a. a processor; b. A non-transitory computer-readable medium storing instructions that cause said processor to perform a set of operations, said set of operations comprising: i. receiving a set of images, each image in the set depicting a blood cell; ii. for each image in the set of images: A. Determining a set of predicted cell labels for the blood cells depicted in the image using a plurality of classifiers, each of the plurality of classifiers providing a respective predicted cell label for the blood cells depicted in the image; B. Providing the set of predicted cell labels to a decision aggregator configured to assign fixed cell labels; a non-transitory computer-readable medium, A computer-implemented blood cell classification system comprising:

12. The decision aggregator further comprises: a. determining whether one of the predicted cell labels in the set of predicted cell labels meets criteria for an accurate label; b. responsive to determining that one of the predicted cell labels meets the correct label criteria, assigning the fixed cell label based on the predicted cell labels that meet the correct label criteria; c. In response to determining that none of the predicted cell labels meet the criteria for the correct label, providing the cell label to a specialized classifier configured to assign the fixed cell label.

12. The computer-implemented cell classification system of claim 11, configured to:

13. The criteria for accurate labeling are: a. A majority consensus of the predicted cell labels among the plurality of classifiers; and b. A consensus of the predicted cell labels among the plurality of classifiers.

13. The computer-implemented cell classification system of claim 12, selected from the group consisting of:

14. The criteria for accurate labeling are: a. one of the predicted cell labels of the plurality of classifiers that meets a confidence score threshold; and b. one of the predicted cell labels of the plurality of classifiers that exceeds a confidence score threshold 13. The computer-implemented cell classification system of claim 12, selected from the group consisting of:

15. 13. The computer-implemented cell classification system of claim 12, wherein the specialized classifier is one of a plurality of specialized classifiers, each of the plurality of specialized classifiers configured to classify a cell image into one of a set of two possible cell classes.

16. 13. The computer-implemented cell classification system of claim 12, wherein each of the plurality of classifiers provides a confidence score associated with the predicted cell label it provides, and the processor is configured to select the specialized classifier based on the two predicted cell labels having the two highest confidence scores.

17. 13. The computer-implemented cell classification system of claim 12, wherein the specialized classifier comprises a convolutional neural network.

18. 12. The computer-implemented cell classification system of claim 11, wherein one of the plurality of classifiers comprises a convolutional neural network.

19. 12. The computer-implemented cell classification system of claim 11 , wherein for each image in the set of images, each of the plurality of classifiers is configured to utilize image analysis of the blood cells depicted in that image to provide a predicted cell label thereof.

20. a. the system comprises: i. a camera; ii. a flow cell having a viewing area; Equipped with 12. The computer-implemented cell sorting system of claim 11, wherein the instructions stored on the non-transitory computer-readable medium comprise instructions that, when executed, cause the processor to capture the set of images by imaging a blood sample as it flows through a viewing area of ​​the flow cell.

21. 1. A computer-implemented learning method for blood cell classification, comprising: a. receiving a plurality of blood cell images; b. training a plurality of generic classifiers using the plurality of blood cell images, each of the plurality of generic classifiers being trained to classify cell images into a set of classes corresponding to the generic classifier; c) training a plurality of specialized classifiers using the plurality of blood cell images, each of the plurality of specialized classifiers being trained to classify cell images into a set of classes corresponding to the specialized classifier; Equipped with A. the plurality of general classifiers comprise general classifiers whose corresponding set of classes has a minimum general cardinality, the minimum general cardinality being less than or equal to any other cardinality of the set of classes corresponding to any classifier in the set of general classifiers; B. the plurality of specialized classifiers comprise specialized classifiers whose corresponding sets of classes have a maximum specialized cardinality, the maximum specialized cardinality being greater than or equal to any other cardinality of the set of classes corresponding to any classifier in the set of specialized classifiers; C. The computer-implemented learning method, wherein the minimum general cardinality is greater than the maximum specialized cardinality.

22. 22. The computer-implemented learning method of claim 21, wherein at least one of the set of general classifiers comprises a convolutional neural network.

23. 22. The computer-implemented learning method of claim 21, wherein each of the plurality of specialized classifiers is configured to, when providing a label for a cell image, provide a corresponding confidence value.

24. 22. The computer-implemented learning method of claim 21, wherein the maximum specialized cardinality is two.

25. 22. The computer-implemented learning method of claim 21, wherein the minimum general cardinality is 11.

Images