Morphology-Based Verifiable Screening

Abstract

The system comprises a plurality of user-adjustable rules that allow a user to customize when a sample, such as a peripheral blood smear, is flagged for human morphological review and when automated review of the sample is sufficient. The user-adjustable rules allow a user to set rules appropriate for a healthcare provider to refine workflow and allow the user better control of the balance between automated and manual review of sample review of samples, such as peripheral blood smears. The user-adjustable rules may comprise a set of values, such as thresholds or ranges for cell types, that may result in a sample either being flagged or not flagged for morphological review by an expert, such as a pathologist.

Description

[Technical field]

[0001] (Related Applications) This application claims the benefit under 35 U.S.C. § 119(e) of U.S. Provisional Patent Application No. 63 / 268,437, filed February 24, 2022, which is incorporated by reference in its entirety herein. [Background technology]

[0002] Previous approaches to analyzing cells and cell morphology from samples such as blood samples may not be ideal in at least some respects. Previous approaches to analyzing blood samples using microscopy include manual review of microscope slides by qualified personnel. This manual approach can be time-consuming and somewhat limited by the number of slides and cells on each slide that are manually viewed. Moreover, manual review of slides can be somewhat slower than would be ideal and, at least in some cases, somewhat limited by a lack of qualified personnel. Research related to the present disclosure suggests that manual review of slides can be slower than would be ideal, which can lead to delayed or erroneous diagnoses. For example, previous clinical standards for review and analysis of blood samples such as peripheral blood smears may be based on a compromise between what would be ideal and what could be achieved by a person manually reviewing the slides. This can lead to failure to detect rare cell types and morphological structures, which, at least in some cases, can lead to flawed diagnoses. Moreover, the statistical sampling of previous approaches may not be ideal due to the limited number of cells that can be analyzed, and in at least some cases, a diagnosis is made without statistical significance.

[0003] While efforts are being made to digitize the analysis of samples such as peripheral blood smears, previous approaches may, at least in some cases, be less than ideal in reducing the manual aspects of slide review under less than ideal conditions. Approaches such as artificial intelligence have been proposed to review slides, but these approaches may not adequately detect potential problems with sample or cell type identification, which may in many cases result in further review by a human, often a human in a laboratory. Moreover, some healthcare providers may, at least in some cases, be somewhat hesitant to rely on fully automated systems to analyze samples such as peripheral blood smears. Thus, there is a need to combine automated analysis of samples with manual review by a human. Research related to the present disclosure suggests that previous approaches may not ideally balance the tradeoff between automated and manual review of samples, which may, at least in some cases, result in a less than ideal workflow. Moreover, previous approaches may not adequately address the allocation between automated and manual review that may be specific to different healthcare providers and countries.

[0004] In light of the above, it would be beneficial to have improved methods, systems, devices, and microscopes that ameliorate at least some of the aforementioned limitations of previous approaches. For example, it would be useful to allow a user of an at least partially automated system to have more control over the circumstances in which a sample is reviewed by a human and in which an automated process for reviewing the sample is sufficient. Summary of the Invention [Means for solving the problem]

[0005] In some embodiments, the disclosed systems, methods, and devices comprise a plurality of user-adjustable rules that allow a user to customize when a sample, such as a peripheral blood smear, is flagged for human morphological review and when automated review of the sample is sufficient. The user-adjustable rules allow a user to set rules that are appropriate for a healthcare provider to improve workflow and allow the user better control of the balance between automated and manual review of sample review of samples, such as peripheral blood smears.

[0006] The user-adjustable rules may comprise a set of values, such as thresholds or ranges for cell types, that may result in a sample being either flagged or not flagged for morphological review by a specialist, such as a pathologist. In some embodiments, a sample is analyzed for a first plurality of cell types and compared to a set of user-adjustable rules to determine whether to trigger a flag for the morphology of the sample by a specialist, such as a pathologist.

[0007] In some embodiments, the sample is analyzed for a second plurality of cell types that are not compared to a set of user-adjustable rules that would trigger a human review of the sample's morphology by a specialist, such as a pathologist. For the second plurality of cell types, the counts for each of the second plurality of cell types can be compared to rules that may be user-adjustable to flag the results for review by a clinician, rather than morphological review of the sample. In some embodiments, the counts for each of the second plurality of cell types can trigger a flag for further review of the results, rather than further review of the sample's morphology. For the second plurality of cell types, the values ​​for the counts of cells of each cell type can be reported to the patient record, even if the values ​​are outside of the normal range, without triggering a flag for morphological review of the sample by a specialist, such as a pathologist.

[0008] In some embodiments, the user interface is configured to receive user input to switch cell types between the first and second plurality of cell types, which can further refine the workflow. In some embodiments, the user interface is configured to allow a user to identify a type of cell as belonging to either a first set of cell types corresponding to the first plurality of cell types or a second set of cell types corresponding to the second plurality of cell types, which can provide additional customization of the workflow and potentially reduce manual review of samples by experts such as pathologists.

[0009] In some embodiments, a computer-implemented method for processing microscopic image data includes receiving microscopic image data from a slide of a patient sample, the microscopic image data comprising at least 100 white blood cells, at least 1,000 red blood cells, and at least 50,000 μm 2 and platelet image data from an area of ​​the sample. The microscopic image data is processed with one or more classifiers to identify a plurality of cell types, the plurality of cell types comprising a first plurality of cell types and a second plurality of cell types. A number of cells is determined for each of the first plurality of cell types and a number of cells is determined for each of the second plurality of cell types. The number of cells of each of the first plurality of cell types is compared to a corresponding user-adjustable rule, the corresponding user-adjustable rule comprising one or more values ​​for flagging or not flagging the sample for visual review by a professional, such as a pathologist. The sample is either flagged or not flagged for review in response to a comparison of the number of cells of each of the first plurality of cell types to the corresponding user-adjustable rule.

[0010] Although the set of user-adjustable rules for the first plurality of cell types can be configured in many ways, in some embodiments the set of rules comprises a plurality of rules for the first plurality of cell types. Alternatively, or in combination, the set of rules may comprise a data structure, such as a look-up table, configured to compare the counts for each of the first plurality of cell types. Although a plurality of user-adjustable rules for each of the first plurality of cell types is mentioned, the set of rules may comprise a rule with a sub-rule for determining whether to flag the sample for further morphological review by an expert in response to the counts of the first plurality of cell types.

[0011] Although the cell types can be flagged in many ways, in some embodiments, the sample is not flagged for further morphological review by an expert in response to the number of cells of each of the second plurality of cell types. In some embodiments, the number of cells of each of the first plurality of cell types and the second plurality of cell types is reported to the patient record, depending on when the sample was flagged. In some embodiments, the number of cells is automatically reported to the patient record in response to the sample not being flagged for further morphological review by an expert, such as a pathologist or other person trained in morphology. In some embodiments, the number of cell types of the first and second plurality of cell types is not reported to the patient record until the sample is reviewed by an expert, such as a pathologist, and the reported value can be adjusted by the expert. (Incorporated by reference)

[0012] All patents, applications, and publications referenced and identified in this specification are incorporated herein by reference in their entirety and shall be considered to be incorporated by reference in their entirety even if referenced elsewhere in this application. [Brief description of the drawings]

[0013] A better understanding of the features, advantages, and principles of the present disclosure will be obtained by reference to the following detailed description that sets forth illustrative embodiments and the accompanying drawings.

[0014] [Figure 1] FIG. 1 shows a schematic diagram of an exemplary microscope according to some embodiments.

[0015] [Diagram 2] FIG. 2 illustrates a method of processing a sample according to some embodiments.

[0016] [Diagram 3] FIG. 3 illustrates an exemplary computing system according to some embodiments.

[0017] [Figure 4] FIG. 4 illustrates an example network architecture according to some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] The following detailed description provides a deeper understanding of the features and advantages of the invention described in this disclosure through the embodiments disclosed herein. Although the detailed description includes many specific embodiments, these are provided by way of example only and should not be construed as limiting the scope of the invention disclosed herein.

[0019] The disclosed systems, methods, and devices are well suited for combination with previous approaches to analyzing samples such as blood samples and blood smears. For example, the optical scanning device may comprise one or more components of a conventional microscope with sufficient numerical aperture or a computational microscope as described in U.S. Patent Application No. 15 / 775,389, filed November 10, 2016, entitled "Computational microscopes and methods for generating an image under different illumination conditions" and published as US20190235224. The system may comprise one or more components of an autofocus system as described, for example, in U.S. Patent No. 10,705,326, entitled "Autofocus system for a computational microscope". While the system may comprise any suitable user interface and data storage device, in some embodiments, the system comprises one or more components for data storage and user interaction as described in U.S. Patent No. 10,935,779, entitled "Digital microscope which operates as a server". The system may include one or more components of an automated loader for loading slides, for example, as described in U.S. patent application Ser. No. 16 / 875,665, filed May 15, 2020, entitled "Multi / parallel scanner." The system may include one or more components for selectively scanning areas of a sample, for example, as described in U.S. patent application Ser. No. 16 / 875,721, filed May 15, 2020, entitled "Accelerating digital microscopy scans using empty / dirty area detection," published as US20200278530.The system may include a grid with a known pattern to facilitate image reconstruction, for example, as described in U.S. Patent No. 10,558,029, entitled "System for image reconstruction using a known pattern." The system may include one or more classifiers, such as those described in U.S. Ser. No. 17 / 755,356, entitled "Method and apparatus for visualization of bone marrow cell populations," published December 29, 2022 as US20220415480, which may be modified by one of skill in the art in accordance with the present disclosure. Each of the foregoing patents and applications is incorporated herein by reference.

[0020] The disclosed systems and methods are highly suitable for use with automated methods and classifiers for analyzing cell morphology, such as those described in PCT / IL2021 / 051329, filed on November 9, 2021, entitled "FULL FIELD MORPHOLOGY - PRECISE QUANTIFICATION OF CELLULAR AND SUB-CELLULAR MORPHOLOGICAL EVENTS IN RED / WHITE BLOOD CELLS," and published on December 1, 2022 as WO / 2022 / 097155.

[0021] FIG. 1 is a diagrammatic representation of a microscope 100 consistent with exemplary disclosed embodiments. The term "microscope" as used herein generally refers to any device or instrument for magnifying objects smaller than those readily observable by the naked eye (e.g., capable of creating an image of the object for a user, the image being larger than the object). One type of microscope may be an "optical microscope" that uses light in combination with an optical system to magnify the object. An optical microscope may be a single microscope with one or more magnifying lenses, or an optical microscope where an image is constructed from a hologram, for example, using a digital holographic microscope. Another type of microscope may be a "computational microscope" that includes an image sensor and image processing algorithms to enhance or magnify the size or other characteristics of an object. A computational microscope may be a dedicated device or may be created by incorporating software and / or hardware into an existing optical microscope to generate a high-resolution digital image. As shown in FIG. 1, the microscope 100 includes an image capture device 102, a focus actuator 104, a controller 106 connected to a memory 108, an illumination assembly 110, and a user interface 112. An exemplary use of the microscope 100 can be to capture images of a sample 114 mounted on a stage 116 located within a field of view (FOV) of the image capture device 102, process the captured images, and present a magnified image of the sample 114 on the user interface 112. Although presenting a magnified image on the user interface is mentioned, in some embodiments, the processor is configured to automatically obtain the image data, for example, without displaying a magnified image of the sample 114 on the user interface. Alternatively, or in combination, the magnified image can be viewed by a user for morphological review of the sample, for example, when the sample has been flagged for morphological review by a specialist, such as a pathologist, as described herein.

[0022] The image capture device 102 may be used to capture an image of the sample 114. As used herein, the term "image capture device" generally refers to a device that records an optical signal incident on a lens as an image or a sequence of images. The optical signal may be in the near infrared, infrared, visible, and ultraviolet spectrum. Examples of image capture devices include CCD cameras, CMOS cameras, color cameras, photosensor arrays, video cameras, mobile phones equipped with cameras, web cameras, preview cameras, microscope objective lenses and detectors, and the like. Some embodiments may include only a single image capture device 102, while other embodiments may include two, three, or even four or more image capture devices 102. In some embodiments, the image capture device 102 may be configured to capture an image within a defined field of view (FOV). Moreover, when the microscope 100 includes several image capture devices 102, the image capture devices 102 may have overlapping areas within their respective FOVs. Image capture device 102 may have one or more image sensors (not shown in FIG. 1 ) for capturing image data of sample 114. In other embodiments, image capture device 102 may be configured to capture images at an image resolution greater than VGA, greater than 1 megapixel, greater than 2 megapixel, greater than 5 megapixel, greater than 10 megapixel, greater than 12 megapixel, greater than 15 megapixel, or greater than 20 megapixel. Additionally, image capture device 102 may be configured to have a pixel size less than 15 micrometers, less than 10 micrometers, less than 5 micrometers, less than 3 micrometers, or less than 1.6 micrometers.

[0023] In some embodiments, the microscope 100 includes a focus actuator 104. The term "focus actuator" as used herein generally refers to any device capable of converting an input signal into a physical motion for adjusting the relative distance between the sample 114 and the image capture device 102. A variety of focus actuators may be used, including, for example, linear motors, electrostrictive actuators, electrostatic motors, capacitive motors, voice coil actuators, magnetostrictive actuators, and the like. In some embodiments, the focus actuator 104 may include an analog position feedback sensor and / or a digital position feedback element. The focus actuator 104 is configured to receive commands from the controller 106 to focus the light beam and form a clear and sharply defined image of the sample 114. In the example illustrated in FIG. 1, the focus actuator 104 may be configured to adjust the distance by moving the image capture device 102.

[0024] However, in other embodiments, the focus actuator 104 may be configured to adjust the distance by moving the stage 116 or by moving both the image capture device 102 and the stage 116. The microscope 100 may also include a controller 106 for controlling the operation of the microscope 100 according to disclosed embodiments. The controller 106 may include various types of devices for performing logical operations based on one or more inputs of image data and other data according to stored or accessible software instructions that provide the desired functionality. For example, the controller 106 may include a central processing unit (CPU), support circuits, digital signal processors, integrated circuits, cache memory, or any other type of device for image processing and analysis, such as a graphics processing unit (GPU). The CPU may include any number of microcontrollers or microprocessors configured to process imagery from the image sensor. For example, the CPU may include any type of single or multi-core processor, mobile device microcontroller, etc. A variety of processors may be used, including, for example, processors available from manufacturers such as Intel®, AMD®, etc., and may include various architectures (e.g., x86 processor, ARM®, etc.). The support circuits may be any number of circuits commonly known in the art, including cache, power supplies, clocks, and input / output circuits. The controller 106 may be in a remote location, such as a computing device that is communicatively coupled to the microscope 100.

[0025] In some embodiments, the controller 106 may be associated with a memory 108 that is used to store software that, when executed by the controller 106, controls the operation of the microscope 100. In addition, the memory 108 may also store electronic data associated with the operation of the microscope 100, such as, for example, captured or generated images of the sample 114. In one case, the memory 108 may be integrated into the controller 106. In another case, the memory 108 may be separate from the controller 106.

[0026] Specifically, memory 108 may refer to multiple structures or computer-readable storage media located at the controller 106 or at a remote location, such as a cloud server. Memory 108 may comprise any number of random access memory, read-only memory, flash memory, disk drives, optical storage devices, tape storage devices, removable storage devices, and other types of storage devices.

[0027] The microscope 100 may include an illumination assembly 110. As used herein, the term “illumination assembly” generally refers to any device or system capable of projecting light to illuminate a sample 114.

[0028] The illumination assembly 110 may include any number of light sources, such as light emitting diodes (LEDs), LED arrays, lasers, and lamps configured to emit light, such as halogen lamps, incandescent lamps, or sodium lamps. For example, the illumination assembly 110 may include a Kohler illumination source. The illumination assembly 110 may be configured to emit polychromatic light. For example, the polychromatic light may include white light.

[0029] In some embodiments, the illumination assembly 110 may include only a single light source. Alternatively, the illumination assembly 110 may include 4, 16, or even more than 100 light sources organized in an array or matrix. In some embodiments, the illumination assembly 110 may use one or more light sources located on a parallel surface to illuminate the sample 114. In other embodiments, the illumination assembly 110 may use one or more light sources located on a surface perpendicular to the sample 114 or at an angle thereto.

[0030] In addition, the illumination assembly 110 may be configured to illuminate the sample 114 in a series of different illumination conditions. In one example, the illumination assembly 110 may include multiple light sources arranged at different illumination angles, such as a two-dimensional arrangement of light sources. In this case, the different illumination conditions may include different illumination angles. For example, FIG. 1 depicts a beam 118 projected from a first illumination angle α1 and a beam 120 projected from a second illumination angle α2. In some embodiments, the first illumination angle α1 and the second illumination angle α2 may have the same value, but of opposite signs. In other embodiments, the first illumination angle α1 may be distinct from the second illumination angle α2. However, both angles originate from a point within the acceptance angle of the optical system. In another example, the illumination assembly 110 may include multiple light sources configured to emit light at different wavelengths. In this case, the different illumination conditions may include different wavelengths. For example, each light source may be configured to emit light with a full width at half maximum bandwidth of 50 nm or less, so as to emit substantially monochromatic light. In yet another example, the illumination assembly 110 may be configured to use several light sources at a given time. In this case, the different illumination conditions may comprise different illumination patterns. For example, the light sources may be arranged to sequentially illuminate the sample at different angles to provide one or more of digital refocusing, aberration correction, or resolution enhancement. Thus, consistent with the present disclosure, the different illumination conditions may be selected from a group including different durations, different intensities, different positions, different illumination angles, different illumination patterns, different wavelengths, or any combination thereof.

[0031] Although computational microscopy is mentioned, the disclosed systems and methods are well suited for use with many types of microscopy and microscopes, such as one or more of a high resolution microscope, a digital microscope, a scanning digital microscope, a 3D microscope, a phase imaging microscope, a phase contrast microscope, a dark field microscope, a differential interference contrast microscope, a light sheet microscope, a confocal microscope, a holographic microscope, or a fluorescence-based microscope.

[0032] In some embodiments, the image capture device 102 may have an effective numerical aperture ("NA") of at least 0.8, although any effective NA may be used. In some embodiments, the effective NA corresponds to the resolving power of a microscope that has the same resolving power as an objective lens with that NA under relevant illumination conditions. The image capture device 102 may also have an objective lens with a suitable NA to provide an effective NA, although the NA of the objective lens may be less than the effective NA of the microscope. For example, the imaging apparatus may include a computational microscope for reconstructing an image from multiple images captured using different illumination angles as described herein, where the reconstructed image corresponds to an effective NA that is higher than the NA of the objective lens of the image capture device. In some embodiments involving a conventional microscope, the NA of the microscope objective lens corresponds to the effective NA of the image. The lens may include any suitable lens, such as an oil immersion lens or a non-oil immersion lens.

[0033] Consistent with the disclosed embodiments, the microscope 100 may comprise, be connected to, or communicate with (e.g., via a network or wirelessly, e.g., via Bluetooth®) a user interface 112. The term "user interface" as used herein generally refers to any device suitable for presenting data, such as numerical results, microscope image data, or a magnified image of the sample 114, or for receiving input from one or more users of data related to the microscope 100, e.g., a remote user. FIG. 1 illustrates two examples of a user interface 112. The first example is a smartphone or tablet that wirelessly communicates with the controller 106 via Bluetooth®, a cellular connection, or a Wi-Fi connection, directly or through a remote server. The second example is a PC display physically connected to the controller 106. In some embodiments, the user interface 112 may comprise a user output device including, for example, a display, a haptic device, a speaker, etc. In other embodiments, the user interface 112 may comprise user input devices including, for example, a touch screen, a microphone, a keyboard, a pointer device, a camera, knobs, buttons, etc. With such input devices, a user may be able to provide information input or commands to the microscope 100 by typing instructions or information, providing voice commands, selecting on-screen menu options using buttons, pointers, or eye tracking capabilities, or through any other suitable technique for communicating information to the microscope 100. The user interface 112 may be coupled (physically or wirelessly) with one or more processing devices, such as the controller 106, to provide and receive information to or from a user and process that information. In some embodiments, such processing devices may execute instructions to respond to keyboard strokes or menu selections, recognize and interpret touches and / or gestures made on a touch screen, recognize and track eye movements, receive and interpret voice commands, etc.

[0034] The microscope 100 may also include (or be connected to) a stage 116. The stage 116 may include any horizontal, rigid surface on which the sample 114 may be mounted for inspection. The stage 116 may include a mechanical connector for holding the slide containing the sample 114 in a fixed position. The mechanical connector may use one or more of the following: a mount, a mounting member, a holding arm, a clamp, a clip, an adjustable frame, a locking mechanism, a spring, or any combination thereof. In some embodiments, the stage 116 may include a translucent portion or opening to allow light to illuminate the sample 114. For example, light transmitted from the illumination assembly 110 may pass through the sample 114 toward the image capture device 102. In some embodiments, the stage 116 and / or the sample 114 may be moved using a motor or manual control in the XY plane to allow imaging of multiple areas of the sample.

[0035] Figure 2 is a flow diagram of an exemplary computer-implemented method 200 for processing microscopy image data according to some embodiments. The steps illustrated in Figure 2 may be performed by a microscope system, such as the systems illustrated in Figures 1, 3, and / or 4. In some embodiments, each of the steps illustrated in Figure 2 may represent an algorithm whose structure includes and / or is represented by multiple sub-steps, examples of which will be provided in more detail below.

[0036] 2, in step 205, a sample is prepared. The sample may comprise any suitable sample, such as, for example, a peripheral blood smear on a slide.

[0037] The slide may comprise a slide, such as a glass or plastic slide, having a substantially two-dimensional surface on which a smear of sample is placed. Although a smear of sample on a slide is referred to, the sample may comprise, for example, a sample within a microfluidic chamber.

[0038] In step 210, the slide with the sample is placed in the cassette, although this step is optional, as would be understood by one of ordinary skill in the art. For example, the slide may be manually placed in the microscope. Alternatively, or in combination, the slide may be provided to the microscope using a coverslipping process and apparatus as described in PCT / IL2022 / 050565, filed May 26, 2022, entitled "Systems and methods for coverslipping," and published December 1, 2022 as WO2022249191.

[0039] In step 215, the slide is loaded into the microscope using a slide loader, although this step is optional, as will be understood by those skilled in the art.

[0040] At step 220, the sample is imaged to generate microscopic image data, which may comprise one or more images of the sample. In some embodiments, the microscopic image data comprises a plurality of cells, the plurality of cells comprising at least 100 white blood cells, at least 1,000 red blood cells, and at least 50,000 μm 2 and platelet image data from the area.

[0041] The microscopic image data may comprise any suitable number of cells, for example, data from two or more slides relating to a patient. In some embodiments, the sample comprises one or more of at least 200 white blood cells, at least 500 white blood cells, or at least 1,000 white blood cells. In some embodiments, the sample comprises one or more of at least 2,000 red blood cells, at least 5,000 red blood cells, or at least 10,000 white blood cells. In some embodiments, the sample comprises at least 100,000 μm 2 , 200,000μm 2 , or at least 500,000 μm2 The platelet image data is provided from one or more of:

[0042] The one or more images may have a suitable aperture numerical aperture, such as, for example, at least one or more of 0.8, 0.9, or 1. In some embodiments, the aperture numerical aperture may be achieved by using a high numerical aperture lens in air. In some embodiments, the high numerical aperture may be achieved by using a refractive index matching material, such as immersion oil or water. In some embodiments, the high numerical aperture may be achieved by using a lower numerical aperture refractive index lens in conjunction with a computational method that leads to a higher aperture numerical aperture as described herein. In some embodiments, the high numerical aperture may be achieved, for example, by using a lensless computational architecture.

[0043] In some embodiments, the sample area may be divided among multiple areas on a single slide or among multiple slides.

[0044] In some embodiments, the one or more images generated in step 220 may be generated using one or more imaging techniques. In some embodiments, the imaging techniques may include computational photography, computational imaging, digital holographic microscopy, computational microscopy, ptychography, or Fourier ptychography, as discussed herein with respect to FIG. 1, for example. In some embodiments, the images may be generated from multiple imaging conditions. For example, in some embodiments, the multiple imaging conditions may include one or more of illuminating the sample at different illumination angles, illuminating the sample with different illumination patterns, or illuminating the sample with different wavelengths of light, as described herein.

[0045] In step 225, the slide is removed from the microscope. In some embodiments, the slide is removed using a slide loader and a new slide is placed into the microscope, although this step is optional and may be performed, for example, manually. Although reference is made to removing the slide from the slide loader, in some embodiments the slide constitutes a surface of a microfluidic chamber configured, for example, for viewing cells within the chamber. In some embodiments, multiple slides are loaded and unloaded from the microscope using a slide loader, and each of the multiple slides is imaged using the microscope.

[0046] In step 230, the microscopic image data is processed. In some embodiments, the microscopic image data is processed using one or more classifiers to identify a plurality of cell types, the plurality of cell types comprising a first plurality of cell types and a second plurality of cell types. The classifier may comprise any suitable classifier, such as one or more of a machine learning classifier, a neural network, or a convolutional neural network. One skilled in the art can train the classifier according to the present disclosure. Step 230 can be performed after, during, or overlapping with one or more of the steps of method 200 described herein (e.g., overlapping with one or more of step 220 of imaging the sample or step 225 of removing the sample, etc.).

[0047] The combination of high resolution and field of view provided by imaging discussed herein with analysis and classification provides for analyzing and classifying a large number of cells simultaneously. For example, an image of a sample at 100x magnification may contain hundreds, thousands, tens of thousands, or even hundreds of thousands of cells that are input to the classifier. The one or more images processed with the classifier may comprise, for example, multiple images, a single image, or several images combined with a conventional microscope scan across several fields of view, or multiple images captured with a computational microscope as described herein and combined into an image with improved resolution. In some embodiments, the one or more images input to the classifier may comprise, for example, a scan of an area that generates several images that are combined and input into the classifier. The analysis and classification may be performed, for example, on the entire image or a portion thereof. Such analysis and classification may, for example, simultaneously classify hundreds, thousands, tens of thousands, or hundreds of thousands of cells from an image of a sample. In some embodiments, at least 1,000 cells, at least 10,000 cells, or at least 100,000 cells of a sample are imaged, processed, and / or sorted simultaneously.

[0048] Sensitivity and specificity are measures of the ability of a test, such as the ability of a classifier to correctly identify and classify cell types in an image. In some embodiments, sensitivity refers to the ability of a classifier to accurately designate cell types in an image. In some embodiments, a highly sensitive test means fewer false negative results, and therefore fewer instances where a cell type is misidentified. In some embodiments, a highly specific test means fewer false positive results with respect to identifying cell types.

[0049] The studies relevant to this disclosure have demonstrated that at least 100 white blood cells, at least 1,000 red blood cells, and at least 50,000 μm 2and platelet image data from an area of ​​100 mm, the image data from a slide from a patient sample comprising a plurality of cells is highly suitable for cell type identification and classification according to the present disclosure with high sensitivity and specificity, where the first plurality of cell types comprises one or more of blasts, abnormal lymphocytes, intracellular microorganisms, nucleated red blood cells (nRBCs), immature myeloids, smudge cells, or schistocytes, and the second plurality of cell types comprises one or more of monocytes, toxic neutrophils, large granular lymphocytes (LGLs), anisocytosis, or platelet clumps. Additional numbers of cells and types from the sample can be identified and classified, which can increase sensitivity and can be beneficial according to some embodiments.

[0050] In some embodiments, the resolution of the image used in the classifier may exceed the resolution of the image acquired in the imaging step. For example, an imaging process such as computational photography, computational imaging, computational microscopy, ptychography, or Fourier ptychography may combine multiple images taken under different conditions into an image with a higher resolution than the constituent images used to generate the combined image. Similarly, in some embodiments, the images may be obtained using an imaging sensor coupled to a microscope objective lens having a certain numerical aperture (NA). In some embodiments, after processing the images through one or more of computational photography, computational imaging, computational microscopy, ptychography, or Fourier ptychography, the resolution of one or more images processed using the classifier corresponds to an effective NA that exceeds the NA of the microscope objective lens.

[0051] In some embodiments, the classification processing step 230 is performed using multiple separate processes, such as multiple threads, allowing the processing with the classifier to be performed more efficiently. In some embodiments, the processing classification step 230 comprises additional steps, such as cell detection, cell segmentation, and cell classification, some of which may comprise sub-steps or tasks of the classification step 230. For example, the processing with the classifier may comprise sub-tasks that are performed in parallel, for example, on the same or different processors. In some embodiments, the separate classifier processes are allocated to different processor resources, such as separate cores of a processor, or are arranged in sub-tasks that perform processes in parallel on the same processor or different processors. The classifier process may comprise sub-tasks that may be arranged in a queue of sub-tasks to be performed on the same processor or different processors, and combinations thereof.

[0052] The number of cells for each cell type is determined, in step 235. In some embodiments, the number of cells is determined for each of the first plurality of cell types and the number of cells is determined for each of the second plurality of cell types.

[0053] At step 240, patient data is received. The patient data may comprise any suitable patient data, such as one or more of the patient's age or gender. The patient data may comprise additional information, such as results from a previous test, such as a blood cell count, e.g., a complete blood count (CBC) from a CBC analyzer. The patient data may comprise additional or alternative data, such as, for example, the patient's weight or height, or medical history or possible illnesses. The patient data can be received at any time or at different times, such as prior to imaging the sample or during any step or steps described herein. In some embodiments, blood cell count data is received prior to imaging the sample, for example, to affect the area of ​​the sample that is imaged.

[0054] In step 245, the number of cells is compared to a rule for each cell type of the first plurality of cells to flag or not flag the sample for further scrutiny. In some embodiments, the number of cells of each of the first plurality of cell types is compared to a corresponding user-adjustable rule, the corresponding user-adjustable rule comprising one or more values ​​for flagging or not flagging the sample for further scrutiny. In some embodiments, demographic data regarding the patient is received. The patient data may comprise, for example, one or more of the patient's age or sex, and the comparison may be performed in response to the patient's age.

[0055] In some embodiments, the first plurality of cell types comprises one or more of blasts, abnormal lymphocytes, intracellular microorganisms, nucleated red blood cells (nRBCs), immature myeloids, smudge cells, or schistocytes, and the second plurality of cell types comprises one or more of monocytes, toxic neutrophils, large granular lymphocytes (LGLs), anisocytosis, or platelet clumps. In some embodiments, each of the first plurality of cell types is compared to user-adjustable rules to flag or not flag the sample for morphological review by a specialist, such as a pathologist, and the second plurality of cell types are not compared to rules to flag or not flag the sample for further review by a specialist, such as a pathologist, but a number of the second plurality of cell types that are outside of normal ranges may be flagged for consideration by the treating physician.

[0056] The adjustable rule for cell types in the first plurality of cell types can be related to the age of the patient. In some embodiments, the patient is over 1 month old and the first plurality of cell types comprises nRBCs. However, when the patient is less than 1 month old, nRBCs are typically more prevalent and nRBCs are considered in the second plurality of cell types, and elevated nRBCs will not flag the sample for further scrutiny by a specialist, such as a pathologist. Thus, the classification of cell types as being in the first plurality of cell types or the second plurality of cell types can depend on the age of the patient.

[0057] In some embodiments, the intracellular microbe comprises a pathogen, such as, for example, malaria.

[0058] In some embodiments, the immature myeloids comprise, for example, one or more of metamyelocytes, myelocytes, or promyelocytes.

[0059] At step 250, the sample is either flagged or not flagged for review by an expert, such as a pathologist. In some embodiments, the sample is either flagged or not flagged for further review in response to a comparison of the number of cells of each of the first plurality of cell types to a corresponding user-adjustable rule.

[0060] In some embodiments, the sample is not flagged for further review of the microscopic image data in response to the number of cells of each of the second plurality of cell types.

[0061] In step 255, the cell counts for the multiple cell types are reported to the patient record.

[0062] In some embodiments, in step 255, the method 200 may include generating a report of the sample. Values ​​or characteristics in the report may be auto-populated according to values ​​from the number of cell types.

[0063] In step 257, a morphological review of the sample is performed by a human, such as a pathologist. In some embodiments, this review is triggered in response to a comparison of the number of cells of the first plurality of cell types to a user-adjustable rule in step 245. Alternatively, or in combination, this review is triggered in response to step 250 either flagging or not flagging the sample for morphological review by a specialist, such as a pathologist. In some embodiments, this review is performed by a specialist viewing an image of the sample in response to a review flag being set in step 250. In some embodiments, the flag triggers a notification or alert or other indication for morphological review by a specialist. In some embodiments, the number of cells of the first plurality is reported to the patient record after review is completed in step 257 if the sample is flagged for review.

[0064] Although it is noted that review is performed by an expert in response to a flag for an expert to review the sample, in some embodiments the system includes a decision support system configured for a user to review the number of each of a plurality of cell types and corresponding images.

[0065] In some embodiments, the system presents supporting data from the analysis to aid in the decision or to allow for further analysis or modification of the analysis, which may be useful when step 257 is performed.

[0066] In some embodiments, the method may include using a decision support system for analysis by a user, such as a remote user. The decision support system may compare the values ​​of the detected objects to predefined values ​​(which may be default values ​​or may be determined by the user or a center and adjusted based on data analysis) and suggest whether the values ​​are within normal or abnormal ranges. The user may be given the option to override the suggestions and adjust the values. The decision support system may make its recommendations based on characteristics, for example, as described herein.

[0067] The decision support system may include a graphical presentation of a point of data. For example: presenting a scan with annotations around the detected object of interest; laying out the number / density of organisms by type, formation, context (e.g., inside or outside the cell), color, or any other characteristic. A decision support system suitable for incorporation according to the present disclosure is described in PCT Application No. PCT / IL2021 / 051329, published May 12, 2022 as WO2022 / 097155, entitled "Full field morphology - precise quantification of cellular and sub-cellular morphological events in red / white blood cells."

[0068] In some embodiments, the decision support system provides a portion of the one or more images corresponding to a location of a cell of the first plurality of cells detected using the classifier, which can facilitate expert review. In some embodiments, the decision support system compares the value of the detected cells to a reference value and indicates a comparison of the number of cells to a corresponding user-adjustable rule. In some embodiments, the decision support system presents a portion of the one or more images with annotations around the object of interest, providing one or more of the number and density of the object of interest by type, the formation of the object of interest, the context of the object of interest, the inner context of the cell, the outer context of the cell, or the color of the object of interest.

[0069] In step 260, a user input is received to adjust a rule for a cell type of the first plurality of cell types. In some embodiments, a user input is received to adjust at least one user-adjustable rule, and the sample is flagged or unflagged in response to the user input. In some embodiments, the user interface is configured for a user to adjust one or more of the thresholds or ranges of the user-adjustable rules for each of the first plurality of cell types.

[0070] In some embodiments, user input is received prior to step 245, e.g., where the number of cells is compared to the user-adjustable rules. In some embodiments, the sample is flagged or unflagged in response to, e.g., the user-adjusted rules for each of the first plurality of cell types.

[0071] While the rules can be configured and applied in many ways, in some embodiments, the counts of each of the second plurality of cell types are not compared to user-adjustable rules for flagging or not flagging the sample for morphological review by a specialist, such as a pathologist, but rather the counts of the second plurality of cell types are compared to rules for flagging or not flagging a count for review by a clinician for each of the second plurality of cell types. This may occur when a cell type of the second plurality of cell types is outside of the normal range, and morphological review by a specialist may not be useful, but flagging the result (e.g., cell count) is useful for the treating physician.

[0072] In some embodiments, user input is received to adjust a corresponding user-adjustable rule for each of the first plurality of cell types.

[0073] Any suitable number of steps of method 200 can be automated. In some embodiments, receiving the microscopic image data, processing the microscopic image data, determining the number of cells, comparing the number of cells, and flagging or not flagging the sample are automated. In some embodiments, the method is fully automated, from loading and unloading the slides to flagging or not flagging the sample. In some embodiments, if the sample is not flagged for further review, such as morphological review by a human, the first plurality of cell types and the second plurality of cell types are reported to the patient record, and if the sample is flagged for further review by a human, the number of the first plurality of cell types is not reported to the patient record until user input is received indicating that a human has reviewed the morphology of the sample. Alternatively, or in combination, if the sample is flagged for review, in some embodiments, the results for the second plurality of cell types are not reported to the patient record until user input is received indicating that the sample has been reviewed and the sample morphology has been reviewed by an appropriate human. In some embodiments, if a sample is flagged for review, no results will be sent to the patient record until the sample has been reviewed and the appropriate input has been received.

[0074] Although the order of the steps of method 200 may be varied, in some embodiments, the steps of receiving microscopic image data, processing the microscopic image data, determining the number of cells, comparing the number of cells, and flagging or not flagging the sample are performed in the order shown in FIG. 2 for a sample from a patient.

[0075] 2, in step 265, a user input is received to switch the cell type between the first plurality of cell types and the second plurality of cell types. In some embodiments, the user-adjustable rules are configured to classify the cell type as being in either the first plurality of cells or the second plurality of cells.

[0076] In some embodiments, the first plurality of cell types initially comprises one or more of blasts, abnormal lymphocytes, intracellular microorganisms, nucleated red blood cells (nRBCs), immature myeloids, smudge cells, or schistocytes, and the second plurality of cell types initially comprises one or more of monocytes, toxic neutrophils, large granular lymphocytes (LGLs), anisocytosis, or platelet clumps. A user interface is configured for a user to move one or more of the cell types from the first plurality of cell types to the second plurality of cell types. Alternatively, or in combination, the user interface can be configured for, for example, a user to move one or more of the cell types from the second plurality of cell types to the first plurality of cell types. In some embodiments, the user interface is configured for, for example, classifying the cell types as belonging to either the first plurality of cell types or the second plurality of cell types.

[0077] Those skilled in the art will recognize that the method 200 for processing microscopic image data can be modified in many ways. The process parameters and sequence of steps described and / or illustrated herein are given by way of example only and can be varied as desired. For example, although the steps illustrated and / or described herein may be shown or discussed in a particular order, these steps need not necessarily be performed in the order shown or discussed. Moreover, some of the steps can be performed, for example, sequentially or at least partially simultaneously. Furthermore, some of the steps can be omitted, some of the steps can be repeated, and some of the steps can comprise sub-steps of other steps. Any of the steps can be automated or combined with input from a user, for example, a remote user or a user operating the system.

[0078] Below is an exemplary set of rules for the first plurality of cell types, providing values, thresholds, and ranges that would trigger manual morphological review of the sample by an expert in response to the number of cells of each of the first plurality of cell types according to some embodiments.

[0079] detection of two or more blasts;

[0080] Detection of one or more abnormal lymphocytes;

[0081] Detection of one or more intracellular microorganisms, such as malaria;

[0082] detection of two or more nRBCs in patients older than 1 month (below that age, more nRBCs would be expected);

[0083] detection of >50 nRBCs for patients aged <1 month;

[0084] detection of immature myeloids (metamelocytes, myelocytes, promyelocytes) with joint counts of >10%;

[0085] Detection of schistocyte levels of ++ or +++;

[0086] Detection of >200 smudge cells.

[0087] The following examples are provided with respect to a second plurality of cell types that do not trigger manual morphological review of images in response to cell counting according to some embodiments, for example.

[0088] monocyte count;

[0089] platelet mass;

[0090] Detection of anisocytosis of red blood cells;

[0091] toxic neutrophils;

[0092] LGL count.

[0093] The rule for the first plurality of cell types can be adjusted as described herein.For example, a sample can be flagged for manual morphological review by an expert, for example, when only one blast cell is detected, or when only one nRBC is detected in a patient who is older than 1 month.

[0094] The rules can be adjusted to allow the user to change the classification of cell types as belonging to, for example, a first plurality of cell types or a second plurality of cell types as described herein. Thus, the above list of the first plurality of cell types and the second plurality of cell types is provided merely as an example and can be modified according to the present disclosure. By classifying cell types as belonging to a first plurality of cell types that may flag the sample for manual review by an expert or as belonging to a second plurality of cell types that do not flag the sample for manual review, the cases of unnecessary manual review of samples by experts can be reduced and the overall workflow can be improved.

[0095] One of skill in the art can develop additional rules and modify values ​​as appropriate, for example with reference to clinically accepted standards such as the International Council for Standardization in Hematology ("ICSH") guidelines.

[0096] Although the number of cells for each of the first plurality of cell types and the second plurality of cell types are referred to, these numbers may comprise any suitable parameter, such as an approximation, a scored value such as + or ++, or any suitable metric, as would be understood by one of ordinary skill in the art of hematology.

[0097] According to some embodiments, Table 1 shows an exemplary data structure, such as a lookup table, that comprises a set of rules that may be used to determine which cell types are used to trigger morphological review of a sample by an expert and which cell types are not used to trigger morphological review of a sample by an expert. In some embodiments, a first category of cell types, e.g., category 1, comprises a first plurality of cell types, and a second category of cell types, e.g., category 2, comprises a second plurality of cell types as described herein. [Table 1]

[0098] In some embodiments, the number of cells for a cell type identified as a member of category 1 may trigger a flag for morphological review of the sample by an expert and comprises a first plurality of cell types as described herein. In some embodiments, the number of cells for a cell type identified as a member of category 2 may not trigger a flag for morphological review of the sample by an expert and comprises a second plurality of cell types as described herein.

[0099] In some embodiments, the rules comprise a first rule for determining whether the sample should be flagged for expert morphological review and a second rule for flagging the number of cells for clinician review, and these rules can be applied independently to each of the multiple cell types.For example, the first rule for flagging the sample for expert morphological review can correspond to the presence of two or more blasts, and the second rule for flagging the number of cells for clinician review can comprise, for example, one or more blasts.Alternatively or in combination, the first rule for flagging the sample for expert morphological review can comprise two or more blast values, and the second value for flagging the number of cells for clinician review can comprise, for example, one or more nRBCs.

[0100] The user interface can be configured, for example, for a user to change the category of a cell type from a first category to a second category (or vice versa). In some embodiments, a data structure, such as a table, is presented to the user with the user interface, and the user is configured to change values ​​in the data structure with user input. In some embodiments, the user interface is configured, for example, to change a cell type from category 1 to category 2, or from category 2 to category 1. When a cell type switches from category 2 to category 1, rules for that cell type are presented on the user interface, pre-populated with values, and the user interface can be configured to receive input for the user to accept the rules. Alternatively, or in combination, the user can, for example, enter rules directly into the user interface.

[0101] The data structure shown in Table 1 can be configured in many ways. In some embodiments, the flagging rule includes two or more cell types. In some embodiments, the category of cell types comprises a category parameter and the cell type comprises a cell type parameter associated with a corresponding user-adjustable rule. The value of the corresponding user-adjustable rule shown in Table 1 may comprise, for example, the value of the rule parameter. The data structure can be arranged, for example, to comprise a vector or matrix. Any suitable number of rules can be included. For example, the data structure may comprise a rule for flagging a number of cell types in category 1 for further review by a clinician, as described herein, that does not trigger morphological review of the sample by an expert, such as a pathologist. In some embodiments, a data structure, such as a lookup table, is used as an input to a process that compares the number of cells for each of a plurality of cell types to one or more rules, as described herein.

[0102] Table 1 is provided as an example, and one of ordinary skill in the art will recognize numerous modifications and variations in accordance with this disclosure.

[0103] 3 is a block diagram of an exemplary computing system 810 capable of implementing one or more of the embodiments described and / or illustrated herein. For example, all or a portion of the computing system 810 may be a means for performing and / or implementing one or more of the steps described herein (such as one or more of those illustrated in FIG. 2), either alone or in combination with other elements. All or a portion of the computing system 810 may be a means for performing and / or implementing any other steps, methods, or processes described and / or illustrated herein. All or a portion of the computing system 810 may correspond to or be otherwise integrated with the microscope 100 (e.g., one or more of the controller 106, the memory 108, and / or the user interface 112).

[0104] Computing system 810 broadly represents any single or multi-processor computing device or system capable of executing computer-readable instructions. Examples of computing system 810 include, but are not limited to, a workstation, a laptop, a client-side terminal, a server, a distributed computing system, a handheld device, or any other computing system or device. In its most basic configuration, computing system 810 may include at least one processor 814 and a system memory 816.

[0105] Processor 814 generally represents any type or form of physical processing unit (e.g., a hardware-implemented central processing unit) capable of processing data or interpreting and executing instructions. In an embodiment, processor 814 may receive instructions from a software application or module. These instructions may cause processor 814 to perform one or more functions of the exemplary embodiments described and / or illustrated herein.

[0106] In some embodiments, the classification is performed using multiple separate processes, such as multiple threads, which allows processing with the classifier to be performed more efficiently on the processor 814, which may comprise, for example, a single-core processor, a multi-core processor, or multiple processors. In some embodiments, the separate classifier processes are allocated to different processor resources, such as separate cores of the processor 814, or are arranged in subtasks that perform the processes in parallel on the same processor or different processors. The classifier process may comprise subtasks that may be arranged in a queue of subtasks to be performed on the same processor or different processors, and combinations thereof.

[0107] System memory 816 generally represents any type or form of volatile or non-volatile storage device or medium capable of storing data and / or other computer-readable instructions. Examples of system memory 816 include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, or any other suitable memory device. Although not required, in some embodiments, computing system 810 may include both volatile memory units (such as system memory 816) and non-volatile storage devices (such as primary storage device 832, described in more detail below). In one example, one or more of the steps from FIG. 2 may be computer instructions that may be loaded into system memory 816.

[0108] In some examples, system memory 816 may store and / or load an operating system 840 for execution by processor 814. In one example, operating system 840 may include and / or represent software that manages computer hardware and software resources and / or provides common services to computer programs and / or applications on computing system 810. Examples of operating systems 840 include, but are not limited to, LINUX®, JUNOS, MICROSOFT WINDOWS®, WINDOWS® MOBILE, MAC OS®, APPLE'S IOS, UNIX®, GOOGLE CHROME OS, GOOGLE'S ANDROID®, SOLARIS®, one or more variations thereof, and / or any other suitable operating system.

[0109] In an embodiment, an exemplary computing system 810 may include one or more components or elements in addition to a processor 814 and a system memory 816. For example, as illustrated in FIG. 3, the computing system 810 may include a memory controller 818, an input / output (I / O) controller 820, and a communication interface 822, each of which may be interconnected via a communication infrastructure 812. The communication infrastructure 812 generally represents any type or form of infrastructure capable of facilitating communication between one or more components of a computing device. Examples of the communication infrastructure 812 include, but are not limited to, a communication bus (such as an Industry Standard Architecture (ISA), Peripheral Component Interconnect (PCI), PCI Express (PCIe), or similar bus) and a network.

[0110] Memory controller 818 generally represents any type or form of device capable of handling memory or data or controlling communications between one or more components of computing system 810. For example, in one embodiment, memory controller 818 may control communications between processor 814, system memory 816, and I / O controller 820 via communications infrastructure 812.

[0111] I / O controller 820 generally represents any type or form of module capable of coordinating and / or controlling the input and output functions of a computing device. For example, in one embodiment, I / O controller 820 may control or facilitate the transfer of data between one or more elements of computing system 810, such as processor 814, system memory 816, communications interface 822, display adapter 826, input interface 830, and storage interface 834.

[0112] 3, computing system 810 may also include at least one display device 824 (which may correspond to user interface 112) coupled to I / O controller 820 via a display adapter 826. Display device 824 generally represents any type or form of device capable of visually displaying information transferred by display adapter 826. Similarly, display adapter 826 generally represents any type or form of device configured to transfer graphics, text, and other data from communications infrastructure 812 (or from a frame buffer as known in the art) for display on display device 824.

[0113] 3, the exemplary computing system 810 may also include at least one input device 828 (which may correspond to user interface 112) coupled to the I / O controller 820 via an input interface 830. The input device 828 generally represents any type or form of input device capable of providing either computer or human generated input to the exemplary computing system 810. Examples of the input device 828 include, but are not limited to, a keyboard, a pointing device, a voice recognition device, one or more variations or combinations thereof, and / or any other input device.

[0114] Additionally or alternatively, the exemplary computing system 810 may include additional I / O devices. For example, the exemplary computing system 810 may include I / O devices 836. In this example, the I / O devices 836 may include and / or represent a user interface that facilitates human interaction with the computing system 810. Examples of the I / O devices 836 include, but are not limited to, a computer mouse, a keyboard, a monitor, a printer, a modem, a camera, a scanning device, a microphone, a touch screen device, one or more variations or combinations thereof, and / or any other I / O device.

[0115] The communication interface 822 broadly represents any type or form of communication device or adapter capable of facilitating communication between the exemplary computing system 810 and one or more additional devices. For example, in some embodiments, the communication interface 822 may facilitate communication between the computing system 810 and a private or public network including the additional computing system. Examples of the communication interface 822 include, but are not limited to, a wired network interface (such as a network interface card), a wireless network interface (such as a wireless network interface card), a modem, and any other suitable interface. In at least one embodiment, the communication interface 822 may provide a direct connection to a remote server via a direct link to a network such as the Internet. The communication interface 822 may also provide such a connection indirectly, for example, through a local area network (such as an Ethernet network), a personal area network, a telephone or cable network, a cellular telephone connection, a satellite data connection, or any other suitable connection.

[0116] In some embodiments, communications interface 822 may also represent a host adapter configured to facilitate communication between computing system 810 and one or more additional network or storage devices over an external bus or communication channel. Examples of host adapters include, but are not limited to, Small Computer System Interface (SCSI) host adapters, Universal Serial Bus (USB) host adapters, Institute of Electrical and Electronics Engineers (IEEE) 1394 host adapters, Advanced Technology Attachment (ATA), Parallel ATA (PATA), Serial ATA (SATA), and External SATA (eSATA) host adapters, Fibre Channel interface adapters, Ethernet adapters, and the like. Communications interface 822 may also enable computing system 810 to engage in distributed or remote computing. For example, communications interface 822 may receive instructions from a remote device or send instructions to a remote device for execution.

[0117] In some examples, the system memory 816 may store and / or load a network communication program 838 for execution by the processor 814. In one example, the network communication program 838 may include and / or represent software that enables the computing system 810 to establish a network connection 842 with another computing system (not shown in FIG. 3 ) and / or communicate with the other computing systems using the communication interface 822. In this example, the network communication program 838 may manage the flow of downstream traffic transmitted to the other computing systems via the network connection 842. Additionally or alternatively, the network communication program 838 may manage the processing of upstream traffic received from the other computing systems via the network connection 842 associated with the processor 814.

[0118] 3, the network communications program 838 may alternatively be stored and / or loaded within the communications interface 822. For example, the network communications program 838 may include and / or represent at least a portion of the software and / or firmware executed by a processor and / or application specific integrated circuit (ASIC) incorporated within the communications interface 822.

[0119] As illustrated in FIG. 3, the exemplary computing system 810 may also include a primary storage device 832 and a backup storage device 833 coupled to the communications infrastructure 812 via a storage interface 834. The storage devices 832 and 833 generally represent any type or form of storage device or medium capable of storing data and / or other computer-readable instructions. For example, the storage devices 832 and 833 may be magnetic disk drives (e.g., so-called hard drives), solid-state drives, floppy disk drives, magnetic tape drives, optical disk drives, flash drives, etc. The storage interface 834 generally represents any type or form of interface or device for transferring data between the storage devices 832 and 833 and other components of the computing system 810. In one example, data 835 (which may correspond to captured images described herein) may be stored and / or loaded into the primary storage device 832.

[0120] In an embodiment, the storage devices 832 and 833 may be configured to read from and / or write to removable storage units configured to store computer software, data, or other computer-readable information. Examples of suitable removable storage units include, but are not limited to, floppy disks, magnetic tapes, optical disks, flash memory devices, and the like. The storage devices 832 and 833 may also include other similar structures or devices for allowing computer software, data, or other computer-readable instructions to be loaded into the computing system 810. For example, the storage devices 832 and 833 may be configured to read and write software, data, or other computer-readable information. The storage devices 832 and 833 may be part of the computing system 810 or may be separate devices accessed through other interface systems.

[0121] Many other devices or subsystems may be connected to the computing system 810. Conversely, not all of the components and devices illustrated in FIG. 3 need to be present to practice the embodiments described and / or illustrated herein. The above-mentioned devices and subsystems may also be interconnected in different ways than those illustrated in FIG. 3. The computing system 810 may also employ any number of software, firmware, and / or hardware configurations. For example, one or more of the exemplary embodiments disclosed herein may be encoded as a computer program (also referred to as computer software, software application, computer readable instructions, or computer control logic) on a computer-readable medium. The term "computer-readable medium" as used herein generally refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transmission-type media such as carrier waves, and non-transitory type media such as magnetic storage media (e.g., hard disk drives, tape drives, and floppy disks), optical storage media (e.g., compact discs (CDs), digital video discs (DVDs), and BLU-RAY discs), electronic storage media (e.g., solid-state drives and flash media), and other distribution systems.

[0122] A computer readable medium containing a computer program may be loaded into the computing system 810. All or part of the computer program stored on the computer readable medium may then be stored in various portions of the system memory 816 and / or the storage devices 832 and 833. When executed by the processor 814, the computer program loaded into the computing system 810 may cause the processor 814 to perform and / or be a means for performing one or more functions of the exemplary embodiments described and / or illustrated herein. Additionally or alternatively, one or more of the exemplary embodiments described and / or illustrated herein may be implemented in firmware and / or hardware. For example, the computing system 810 may be configured as an application specific integrated circuit (ASIC) adapted to implement one or more of the exemplary embodiments disclosed herein.

[0123] 4 is a block diagram of an example network architecture 900 in which client systems 910, 920, and 930 and servers 940 and 945 may be coupled to a network 950. As detailed above, all or a portion of network architecture 900, either alone or in combination with other elements, may perform and / or be a means for performing one or more of the steps disclosed herein (such as one or more of those illustrated in FIG. 2). All or a portion of network architecture 900 may be used to perform and / or be a means for performing other steps and features described in this disclosure.

[0124] The client systems 910, 920, and 930 generally represent any type or form of computing device or system, such as the exemplary computing system 810 of FIG. 3. Similarly, the servers 940 and 945 generally represent computing devices or systems, such as application servers or database servers configured to provide various database services and / or execute certain software applications. The network 950 generally represents any telecommunications or computer network, including, for example, an intranet, a WAN, a LAN, a PAN, or the Internet. In one example, the client systems 910, 920, and / or 930, and / or the servers 940 and / or 945 may include all or a portion of the microscope 100 from FIG. 1.

[0125] 4, one or more storage devices 960(1)-(N) may be attached directly to server 940. Similarly, one or more storage devices 970(1)-(N) may be attached directly to server 945. Storage devices 960(1)-(N) and storage devices 970(1)-(N) generally represent any type or form of storage device or medium capable of storing data and / or other computer-readable instructions. In one embodiment, storage devices 960(1)-(N) and storage devices 970(1)-(N) may represent network-attached storage (NAS) devices configured to communicate with servers 940 and 945 using a variety of protocols, such as network file system (NFS), server message block (SMB), or common internet file system (CIFS).

[0126] The servers 940 and 945 may also be connected to a storage area network (SAN) fabric 980. The SAN fabric 980 generally represents any type or form of computer network or architecture capable of facilitating communication between multiple storage devices. The SAN fabric 980 may facilitate communication between the servers 940 and 945 and multiple storage devices 990(1)-(N) and / or intelligent storage array 995. The SAN fabric 980 may also facilitate communication between the client systems 910, 920, and 930 and the storage devices 990(1)-(N) and / or intelligent storage array 995 via the network 950 and the servers 940 and 945 in a manner such that the devices 990(1)-(N) and array 995 appear as locally attached devices to the client systems 910, 920, and 930. Like storage devices 960(1)-(N) and storage devices 970(1)-(N), storage devices 990(1)-(N) and intelligent storage array 995 generally represent any type or form of storage device or medium capable of storing data and / or other computer-readable instructions.

[0127] In one embodiment, with reference to the exemplary computing system 810 of FIG. 3, a communications interface, such as communications interface 822 of FIG. 3, may be used to provide connectivity between each client system 910, 920, and 930 and the network 950. The client systems 910, 920, and 930 may be able to access information on the server 940 or 945 using, for example, a web browser or other client software. Such software may enable the client systems 910, 920, and 930 to access data hosted by the server 940, the server 945, the storage devices 960(1)-(N), the storage devices 970(1)-(N), the storage devices 990(1)-(N), or the intelligent storage array 995. Although FIG. 4 depicts the use of a network (such as the Internet) to exchange data, the embodiments described and / or illustrated herein are not limited to the Internet or any particular network-based environment.

[0128] In at least one embodiment, all or a portion of one or more of the exemplary embodiments disclosed herein may be encoded as a computer program and loaded onto and executed by server 940, server 945, storage devices 960(1)-(N), storage devices 970(1)-(N), storage devices 990(1)-(N), intelligent storage array 995, or any combination thereof. All or a portion of one or more of the exemplary embodiments disclosed herein may also be encoded as a computer program and stored within server 940, executed by server 945, and distributed to client systems 910, 920, and 930 via network 950.

[0129] As described in detail above, one or more components of computing system 810 and / or network architecture 900 may perform and / or be a means for performing one or more steps of method 200, either alone or in combination with other elements.

[0130] As described herein, the computing devices and systems described and / or illustrated herein broadly represent any type or form of computing device or system capable of executing computer-readable instructions such as those contained within the modules described herein. In their most basic configurations, these computing devices may each include at least one memory device and at least one physical processor.

[0131] The term "memory" or "memory device" as used herein generally refers to any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, a memory device may store, load, and / or maintain one or more of the modules described herein. Examples of memory devices include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD), optical disk drive, cache, one or more variations or combinations thereof, or any other suitable storage memory.

[0132] Additionally, the term "processor" or "physical processor" as used herein generally refers to any type or form of hardware-implemented processing unit capable of interpreting and / or executing computer-readable instructions. In one example, a physical processor may access and / or modify one or more modules stored in a memory device described above. Examples of physical processors include, but are not limited to, a microprocessor, a microcontroller, a central processing unit (CPU), a field programmable gate array (FPGA) implementing a soft-core processor, an application specific integrated circuit (ASIC), one or more portions thereof, one or more variations or combinations thereof, or any other suitable physical processor. A processor may include, for example, a distributed processor system running parallel processors, or a remote processor such as a server, and combinations thereof.

[0133] Although illustrated as separate elements, the method steps described and / or illustrated herein may represent parts of a single application. Additionally, in some embodiments, one or more of these steps may represent or correspond to one or more software applications or programs that, when executed by a computing device, may cause the computing device to perform one or more tasks, such as method steps.

[0134] In addition, one or more of the devices described herein may transform data, physical devices, and / or representations of physical devices from one form to another. Additionally or alternatively, one or more of the modules listed herein may transform a processor, volatile memory, non-volatile memory, and / or any other portion of a physical computing device from one form of computing device to another form of computing device by executing on the computing device, storing data on the computing device, and / or otherwise interacting with the computing device.

[0135] The term "computer-readable medium" as used herein generally refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transmission-type media such as carrier waves, and non-transitory-type media such as magnetic storage media (e.g., hard disk drives, tape drives, and floppy disks), optical storage media (e.g., compact discs (CDs), digital video discs (DVDs), and BLU-RAY discs), electronic storage media (e.g., solid-state drives and flash media), and other distributed systems.

[0136] Those skilled in the art will recognize that any process or method disclosed herein can be modified in many ways. The process parameters and sequence of steps described and / or illustrated herein are given by way of example only and can be varied as desired. For example, although the steps illustrated and / or described herein may be shown or discussed in a particular order, these steps do not necessarily have to be performed in the order shown or discussed.

[0137] The various exemplary methods described and / or illustrated herein may omit one or more of the steps described or illustrated herein or may include additional steps in addition to those disclosed. Furthermore, the steps of any method as disclosed herein may be combined with any one or more steps of any other method as disclosed herein.

[0138] A processor as described herein may be configured to perform one or more steps of any of the methods disclosed herein. Alternatively, or in combination, a processor may be configured to combine one or more steps of one or more methods as disclosed herein.

[0139] Unless otherwise stated, the terms "connected to" and "coupled to" (and their derivatives) as used in this specification and the claims shall be interpreted as allowing both direct and indirect (i.e., via other elements or components) connections. Additionally, the terms "a" or "an" as used in this specification and the claims shall be interpreted as meaning "at least one of." Finally, for ease of use, the terms "including" and "having" (and their derivatives) as used in this specification and the claims shall be synonymous with and have the same meaning as the word "comprising."

[0140] A processor as disclosed herein may be configured with instructions to perform any one or more of the methods as disclosed herein.

[0141] It should be understood that the terms "first," "second," "third," etc. may be used herein to describe various layers, elements, components, regions, or sections without referring to any particular order or sequence of events. These terms are merely used to distinguish one layer, element, component, region, or section from another layer, element, component, region, or section. A first layer, element, component, region, or section as described herein may be referred to as a second layer, element, component, region, or section without departing from the teachings of the present disclosure.

[0142] As used herein, the term "or" is used inclusively to refer to items as alternatives and in combination, unless otherwise indicated. In some embodiments, the term "or" is used exclusively to refer to items that are only alternatives, for example, in describing a workflow of either flagging or not flagging a sample for further scrutiny.

[0143] As used herein, letters such as numbers refer to similar elements.

[0144] As used herein, the terms "comprise" and "include" are synonymous.

[0145] As used herein, the terms "in response to" and "based on" are synonymous.

[0146] As used herein, the term resolution corresponds to the minimum distance by which images of lines on a resolution target can be separated.

[0147] As used herein, mathematical symbols have their ordinary meaning as would be understood by one of ordinary skill in the art, unless indicated to the contrary. For example, the symbols "<" and ">" have their standard meaning of "less than" and "greater than," respectively.

[0148] This disclosure includes the following numbered appendices:

[0149] Appendix 1. A computer-implemented method for processing microscopy image data, the method comprising: receiving microscopy image data from a slide from a patient sample, the microscopy image data comprising a plurality of cells, the plurality of cells comprising at least 100 white blood cells, at least 1,000 red blood cells, and at least 50,000 μm 2 and platelet image data from an area of ​​a patient's body; processing the microscopic image data with one or more classifiers to identify a plurality of cell types, the plurality of cell types comprising a first plurality of cell types and a second plurality of cell types; determining a number of cells for each of the first plurality of cell types and a number of cells for each of the second plurality of cell types; comparing the number of cells of each of the first plurality of cell types to a corresponding user-adjustable rule, the corresponding user-adjustable rule comprising one or more values ​​for flagging or not flagging the sample for further scrutiny; in response to comparing the number of cells of each of the first plurality of cell types to the corresponding user-adjustable rule, either flagging or not flagging the sample for further scrutiny; and reporting the number of cells of each of the first plurality of cell types and the number of cells of each of the second plurality of cell types to a patient record.

[0150] Clause 2. The method of clause 1, wherein the sample is not flagged for further scrutiny of the microscopic image data in response to the number of cells of each of the second plurality of cell types.

[0151] Addendum 3. The method of any of Addendums 1-2, further comprising receiving a user input for adjusting at least one user-adjustable rule, wherein the sample is flagged or unflagged in response to the user input.

[0152] Clause 4. The method of any of Clauses 1-3, wherein user input is received to adjust a corresponding user-adjustable rule for each of the first plurality of cell types.

[0153] Clause 5. The method of any of Clauses 1-4, wherein the sample is flagged or unflagged in response to a user-adjusted rule for each of the first plurality of cell types.

[0154] Appendix 6. The method of any of Appendixes 1-5, wherein the counts for each of the second plurality of cell types are not compared to user-adjustable rules for flagging or not flagging the sample for morphological review, and the counts are compared to rules for flagging or not flagging each of the second plurality of cell types for review by a clinician.

[0155] Clause 7. The method of any of Clauses 1-6, further comprising imaging the slide and generating microscopic image data.

[0156] Appendix 8. The method of any of Appendixes 1-7, wherein the steps of receiving the microscopic image data, processing the microscopic image data, determining the number of cells, comparing the number of cells, and flagging or unflaggling the sample are automated.

[0157] Addendum 9. The method of any of Addendums 1-8, wherein the steps of receiving microscopic image data, processing the microscopic image data, determining the number of cells, comparing the number of cells, and flagging or unflaggling the sample are performed in sequence.

[0158] Clause 10. The method of any of clauses 1-9, further comprising loading and unloading a plurality of slides to and from the microscope using a slide loader, each of the plurality of slides being imaged using the microscope.

[0159] Clause 11. The method of any of clauses 1-10, wherein the method is fully automated, from loading and unloading multiple slides to flagging or unflaggling samples.

[0160] Appendix 12. The method of any of Appendixes 1-11, wherein if the sample is not flagged for further human review, the first plurality of cell types and the second plurality of cell types are reported to the patient record, and if the sample is flagged for further human review, the number of the first plurality of cell types is not reported to the patient record until user input is received indicating that a human has reviewed the sample.

[0161] Addendum 13. The method of any of Addendums 1-12, wherein the user-adjustable rules are configured for a user to classify a cell type as being in either the first plurality of cells or the second plurality of cells, and optionally, the user interface is configured for a user to switch the cell type between the first plurality of cell types and the second plurality of cell types.

[0162] Addendum 14. The method of any of Addendums 1-13, wherein the user interface is configured for a user to adjust one or more of the thresholds or ranges of the user-adjustable rules of each of the first plurality of cell types.

[0163] Clause 15. The method of any of clauses 1-14, wherein the first plurality of cell types comprises one or more of blasts, abnormal lymphocytes, intracellular microorganisms, nucleated red blood cells (nRBCs), immature myeloids, smudge cells, or schistocytes, and the second plurality of cell types comprises one or more of monocytes, toxic neutrophils, large granular lymphocytes (LGLs), anisocytosis, or platelet clumps.

[0164] Clause 16. The method of any of Clauses 1-15, further comprising receiving data regarding a patient, the sample being taken from the patient, and the patient data comprising one or more of the patient's age or gender.

[0165] Appendix 17. The method of any of appendices 1-16, wherein the comparing is performed in response to one or more of patient age or sex.

[0166] Clause 18. The method of any of clauses 1-17, wherein the age is greater than 1 month and the first plurality of cell types comprises nRBCs.

[0167] Addendum 19. The method of any of Addendums 1-18, wherein the intracellular microorganism comprises a pathogen, and optionally, the pathogen comprises malaria.

[0168] Clause 20. The method of any of clauses 1-19, wherein the immature myeloids comprise one or more of metamyelocytes, myelocytes, or promyelocytes.

[0169] Addendum 21. The method of any of Addendums 1-20, wherein the sample comprises one or more of at least 200 white blood cells, at least 500 white blood cells, or at least 1,000 white blood cells.

[0170] Addendum 22. The method of any of Addendums 1-21, wherein the sample comprises one or more of at least 2,000 red blood cells, at least 5,000 red blood cells, or at least 10,000 white blood cells.

[0171] Appendix 23. Samples must be at least 100,000 μm 2 , 200,000μm 2 , or at least 500,000 μm 2 23. The method of any of claims 1-22, further comprising providing platelet image data from one or more of:

[0172] Clause 24. An apparatus comprising a processor configured to carry out a method according to any of the preceding clauses.

[0173] The embodiments of the present disclosure are shown and described herein and are provided only as examples. Those skilled in the art will recognize numerous adaptations, changes, modifications, and substitutions without departing from the scope of the present disclosure. Several alternatives and combinations of the embodiments disclosed herein can be utilized without departing from the scope of the present disclosure and the invention disclosed herein. Therefore, the scope of the present disclosure is to be defined only by the scope of the appended claims and their equivalents.

Claims

1. 1. A computer-implemented method for processing microscopy image data, the method comprising: receiving microscopic image data from a slide from a patient sample, the microscopic image data comprising a plurality of cells, the plurality of cells comprising at least 100 white blood cells, at least 1,000 red blood cells, and at least 50,000 μm ２ and platelet image data from an area of processing the microscopy image data with one or more classifiers to identify a plurality of cell types, the plurality of cell types comprising a first plurality of cell types and a second plurality of cell types; determining a number of cells for each of the first plurality of cell types and a number of cells for each of the second plurality of cell types; comparing the number of cells of each of the first plurality of cell types to a corresponding user-adjustable rule, the corresponding user-adjustable rule comprising one or more values ​​for flagging or not flagging the sample for further scrutiny; either flagging or not flagging the sample for further scrutiny in response to comparing the number of each cell of the first plurality of cell types to the corresponding user-adjustable rule; reporting said number of each cell of said first plurality of cell types and said number of each cell of said second plurality of cell types to a patient record. A method comprising:

2. 10. The method of claim 1, wherein the sample is not flagged for further review of the microscopic image data in response to the number of cells of each of the second plurality of cell types.

3. The method of claim 1 , further comprising receiving a user input for adjusting at least one user-adjustable rule, wherein the sample is flagged or unflagged in response to the user input.

4. The method of claim 3 , wherein user input is received to adjust the corresponding user-adjustable rule for each of the first plurality of cell types.

5. 5. The method of claim 4, wherein the sample is flagged or unflagged in response to a user-adjusted rule for each of the first plurality of cell types.

6. 2. The method of claim 1, wherein the counts of each of the second plurality of cell types are not compared to user-adjustable rules for flagging or not flagging the sample for morphological review, and the counts are compared to rules for flagging or not flagging each of the second plurality of cell types for review by a clinician.

7. The method of claim 1 , further comprising imaging the slide and generating the microscopic image data.

8. 2. The method of claim 1, wherein the steps of receiving the microscopic image data, processing the microscopic image data, determining the number of cells, comparing the number of cells, and flagging or unflaggling the sample are automated.

9. 2. The method of claim 1, wherein the steps of receiving the microscopic image data, processing the microscopic image data, determining the number of cells, comparing the number of cells, and flagging or unflaggling the sample are performed in sequence.

10. 10. The method of claim 1, further comprising loading and unloading a plurality of slides to and from a microscope using a slide loader, each of the plurality of slides being imaged using the microscope.

11. 11. The method of claim 10, wherein the method is fully automated, from loading and unloading the plurality of slides to flagging or unflaggling the samples.

12. 12. The method of claim 11, wherein if the sample is not flagged for further review by a human, the first plurality of cell types and the second plurality of cell types are reported to the patient record, and if the sample is flagged for further review by a human, the number of the first plurality of cell types is not reported to the patient record until user input is received indicating that the human has reviewed the sample.

13. 2. The method of claim 1, wherein the user-adjustable rules are configured for the user to classify cell types as being in either the first plurality of cells or the second plurality of cells, and optionally a user interface is configured for the user to switch cell types between the first plurality of cell types and the second plurality of cell types.

14. The method of claim 1 , wherein a user interface is configured for a user to adjust one or more of the thresholds or ranges of the user-adjustable rules for each of the first plurality of cell types.

15. 2. The method of claim 1, wherein the first plurality of cell types comprises one or more of blasts, abnormal lymphocytes, intracellular microorganisms, nucleated red blood cells (nRBCs), immature myeloids, smudge cells, or schistocytes, and the second plurality of cell types comprises one or more of monocytes, toxic neutrophils, large granular lymphocytes (LGLs), anisocytosis, or platelet clumps.

16. 16. The method of claim 15, further comprising receiving data regarding a patient, the sample being taken from the patient, and the patient data comprising one or more of an age or a gender of the patient.

17. 17. The method of claim 16, wherein the comparing is performed in response to one or more of the patient's age or sex.

18. 20. The method of claim 17, wherein the age is greater than 1 month and the first plurality of cell types comprises nRBCs.

19. 16. The method of claim 15, wherein the intracellular microorganism comprises a pathogen, and optionally, the pathogen comprises malaria.

20. 16. The method of claim 15, wherein the immature myeloids comprise one or more of metamyelocytes, myelocytes, or promyelocytes.

21. 10. The method of claim 1, wherein the sample comprises one or more of at least 200 white blood cells, at least 500 white blood cells, or at least 1,000 white blood cells.

22. 10. The method of claim 1, wherein the sample comprises one or more of at least 2,000 red blood cells, at least 5,000 red blood cells, or at least 10,000 white blood cells.

23. The sample is at least 100,000 μm ２ , 200,000 μm ２ , or at least 500,000 μm ２ The method of claim 1 , further comprising:

24. 23. An apparatus comprising a processor configured to perform a method according to any preceding claim.

Images