Focus quality determination through multi-layer processing

Abstract

The focusing distance in a visual analytics system (e.g., the distance between the focal plane of the camera used when the image was captured and the actual focal plane for the in-focus image) may be determined by subjecting one or more images captured by such a system to a multi-layer analysis. In such an analysis, the input may be subjected to one or more convolution filters, and the final result of such convolution may be fed to a dense layer that can yield the focusing distance. This focusing distance may then be used to (re)focus the camera or for other purposes (such as generating an alert).

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to and is a nonprovisional application of U.S. Provisional Patent Application No. 63 / 291,044, entitled “Autofocusing through multi-layer processing,” filed in the U.S. Patent and Trademark Office on December 17, 2021, which is incorporated herein by reference in its entirety. [Background technology]

[0002] Blood cell analysis is one of the most commonly performed medical tests to provide an overview of a patient's health status. A blood sample may be drawn from a patient's body and stored in a test tube containing an anticoagulant to prevent clotting. A whole blood sample usually comprises three major classes of blood cells, including red blood cells (erythrocytes), white blood cells (leukocytes), and platelets (thrombocytes). Each class may be further divided into subclass components. For example, the five major types or subclasses of white blood cells (WBCs) have different shapes and functions. White blood cells may include neutrophils, lymphocytes, monocytes, eosinophils, and basophils. There are also subclasses of red blood cell types. The appearance of particles in a sample may vary according to disease states, cell maturation, and other causes. Subclasses of red blood cells may include reticulocytes and nucleated red blood cells.

[0003] This analysis may require capturing images of the sample comprising blood cells, and the higher the quality of these images, the more suitable they are for analysis. However, capturing high quality images presents many problems. For example, ensuring that images are in focus can be complicated by the fact that changes in temperature or other factors related to the operation of the analyzer may cause a previously focused optical system to require refocusing. Furthermore, some types of focusing are not effective for all types of blood cells that may be found in the sample (e.g., a focusing method based on feature extraction may be suitable for red blood cells but not for white blood cells). Thus, there is a need for improvements in the field related to focusing of the analyzer optical system, such as by automatically refocusing during image acquisition or by detecting focusing challenges so that appropriate responsive steps can be taken. Such improvements may have utility, for example, in stationary imaging systems (e.g., the sample is stationary or fixed at the imaging location) as well as flow imaging systems (e.g., the sample is flowed through the imaging location). Summary of the Invention

[0004] Described herein are apparatus, systems, and methods for automatically determining the focusing quality of an optical analysis system, such as a bioquantification system that captures images of blood cells from a blood sample.

[0005] An exemplary implementation of such a technique relates to a system comprising a camera adapted to capture an image of a blood sample and a non-transitory computer-readable medium. Such a medium may store a plurality of sets of convolution filters and a fully connected layer. Such a medium may also store instructions operable, when executed by a processor, to generate an output for each set of convolution filters, such output having a depth equal to the cardinality of the convolution filters of the set. This may be done by performing, for each set of convolution filters, a step including convolving an input with each filter in the set. In such a case, the input for the first set of convolution filters may be an image captured by the camera, and the input for each other set of convolution filters may be the output of a different set of convolution filters. In such a case, the medium may also store instructions to generate a focal length by performing acts including providing the output generated for the set of convolution filters as an input to a fully connected layer and focusing the camera based on the focal length.

[0006] While numerous examples have been described herein, still other examples of the described subject matter will be apparent to those skilled in the art from the following detailed description and drawings, which show and describe illustrative examples of the disclosed subject matter. As will be understood, the disclosed subject matter can be modified in various aspects without departing from the spirit and scope of the described subject matter. Accordingly, the drawings and detailed description are to be regarded as illustrative in nature and not intended to be limiting.

[0007] While this specification concludes with claims which in part point out and distinctly claim the invention, it is believed the invention will be better understood from the following description of several specific examples taken in conjunction with the drawings in which like reference numerals identify the same elements. [Brief description of the drawings]

[0008] [Figure 1]FIG. 1 is a partially cross-sectional, non-dimensional schematic diagram showing operational aspects of an exemplary flow cell autofocus system and high optical resolution imaging device for sample image analysis using digital image processing. [Diagram 2] FIG. 1 is a diagram of a sliding vision inspection system in which aspects of the disclosed technology may be used. [Diagram 3] FIG. 1 is an illustration of a process that may be used to refocus an imaging device. [Figure 4] FIG. 1 is a diagram of an architecture that may be used in analyzing an image. [Diagram 5] FIG. 5 is a diagram of layers that may be included in an architecture such as that shown in FIG. [Figure 6] 5 is a flow chart illustrating a method that may be used in an architecture such as that shown in FIG. 4. [Figure 7] FIG. 13 is a diagram of the determination of the focusing position on a blood sample run used to generate a white blood cell (WBC) image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The drawings are not intended to be limiting in any manner, and it is contemplated that various embodiments of the invention may be practiced in a variety of other ways, including those not necessarily depicted in the drawings. The accompanying drawings, which are incorporated in and form a part of this specification, illustrate several aspects of the invention and, together with the description, serve to explain the principles of the invention. It will be understood, however, that the invention is not limited to the precise configurations shown.

[0010] The present disclosure relates to instruments, systems, compositions, and methods for analyzing samples containing particles. In one embodiment, the present invention relates to an automated particle imaging system comprising an analyzer, which may be, for example, a visual analyzer. In some embodiments, the visual analyzer further comprises a processor for facilitating the automated analysis of the image.

[0011] According to some aspects of the present disclosure, a system with a visual analyzer may be provided for obtaining images of a sample containing particles suspended in a liquid. Such a system may be useful in characterizing particles in biological fluids, such as, for example, detection and quantification, as well as classification, subclassification, and analysis of red blood cells, reticulocytes, nucleated red blood cells, platelets, and white blood cells, including differential white blood cell percentages. Other similar uses, such as characterizing blood cells from other fluids, are also contemplated.

[0012] Although the identification of blood cells in a blood sample is an exemplary application for which the subject matter is particularly well suited, other types of bodily fluid samples may be used. For example, aspects of the disclosed technology may be used in the analysis of non-blood bodily fluid samples that contain blood cells (e.g., white blood cells and / or red blood cells), such as serum, bone marrow, lavage fluid, effusion, exudate, cerebrospinal fluid, pleural fluid, peritoneal fluid, amniotic fluid, etc. The sample may also be a solid tissue sample, such as a biopsy sample that has been processed to produce a cell suspension. The sample may also be a suspension obtained from processing a stool sample. The sample may also be a laboratory or production line sample that contains particles, such as a cell culture sample. The term sample may be used to refer to a sample obtained from a patient or laboratory, or any fragment, portion, or aliquot thereof. The sample may be diluted, divided into portions, or stained in several steps.

[0013] In some embodiments, the sample is presented, imaged, and analyzed in an automated manner. In the case of blood samples, the sample may be substantially diluted with a suitable diluent or saline solution, which reduces the degree of visibility of some cells that may be hidden by other cells in undiluted or less diluted samples. Cells may be treated with agents that increase the contrast of some cellular aspects, for example, with permeabilizing agents to render cell membranes permeable, and histological stains that attach to and reveal features such as granules and nuclei. In some cases, it may be desirable to stain an aliquot of the sample to count and characterize particles including reticulocytes, nucleated red blood cells, and platelets, as well as for white blood cell differential, characterization, and analysis. In other cases, samples containing red blood cells may be diluted prior to introduction into the flow cell and / or imaging in the flow cell or otherwise.

[0014] Details of sample preparation equipment and methods for sample dilution, permeabilization, and histological staining can generally be accomplished using precision pumps and valves operated by one or more programmable controllers. Examples can be found in patents such as U.S. Pat. No. 7,319,907. Similarly, techniques for differentiation between several specific cell classes and / or subclassifications by attributes such as relative size and color can be found in U.S. Pat. No. 5,436,978 for white blood cells. The disclosures of these patents are incorporated herein by reference in their entirety.

[0015] Referring now to the drawings, Figure 1 illustrates generally an exemplary flow cell 22 for conveying a sample fluid through a viewing area 23 of a high optical resolution imaging device 24 in a configuration for imaging microscopic particles in a sample flow stream 32 using digital image processing. The flow cell 22 is coupled to a source 25 of sample fluid that may have undergone processing such as contact with particle contrast agents, constructs, and heating. The flow cell 22 is also coupled to one or more sources 27 of particle and / or intracellular organelle alignment liquid (PIOAL), such as a clear glycerol solution having a viscosity greater than that of the sample fluid.

[0016] Sample fluid is injected into the interior of the flow cell 22 through a flat opening at the distal end 28 of the sample delivery tube 29, at which point a PIOAL flow is substantially established resulting in a steady, symmetric, laminar flow of the PIOAL above and below (or on either side of) the ribbon-shaped sample stream. The sample and PIOAL streams may be supplied by precision metering pumps that move the PIOAL with the injected sample fluid along a substantially narrowing flow path. The PIOAL surrounds and compresses the sample fluid in the narrowing region 21. Thus, the reduction in the flow path thickness in the region 21 contributes to the geometric focusing of the sample stream 32. The sample fluid ribbon 32 is surrounded and conveyed along the narrowing region 21 downstream of the PIOAL to pass in front of or otherwise pass through the viewing region 23 of the high optical resolution imaging device 24, where an image is collected, for example, using a CCD 48. The processor 18 may receive pixel data from the CCD 48 as an input. The sample fluid ribbon flows with the PIOAL to the exhaust 33 .

[0017] As shown here, narrowing region 21 can have a proximal flow path portion 21a having a proximal thickness PT and a distal flow path portion 21b having a distal thickness DT, where the distal thickness DT is less than the proximal thickness PT. Thus, sample fluid can be injected through a distal end 28 of sample tube 29 at a point distal to proximal portion 21a and proximal to distal portion 21b. Thus, sample fluid can enter the PIOAL enclosure as the PIOAL stream is compressed by region 21. The sample fluid injection tube has a distal exit port through which the sample fluid is injected into the sheath fluid through which it is flowing, the distal exit port being bounded by the reduction in flow path size of the flow cell.

[0018] A digital high optical resolution imager 24 with objective lens 46 is oriented along an optical axis that intersects with the ribbon-shaped sample stream 32. The relative distance between the objective 46 and the flow cell 33 is variable by operation of a motor drive 54 to resolve and collect a focused digitized image on a photosensor array. Additional information regarding the structure and operation of an exemplary flow cell as shown in FIG. 1 is provided in U.S. Patent No. 9,322,752, filed March 17, 2014, and entitled "Flowcell Systems and Methods for Particle Analysis in Blood Samples," the disclosure of which is incorporated herein by reference in its entirety.

[0019] Aspects of the disclosed technology may be applied in contexts other than flow cell systems as shown in FIG. 1. For example, FIG. 2 illustrates a slide-based visual inspection system 200 in which aspects of the disclosed technology may be used. In the system illustrated in FIG. 2, a slide 202 with a sample, such as a blood sample, is placed in a slide holder 204. The slide holder 204 may be adapted to hold multiple slides or only one, as shown in FIG. 2. An image capture device 206, comprising an optical system 208 and an image sensor 210, is adapted to capture image data depicting the sample in the slide 202. Additionally, a light emitting device (not shown) may be used to control the light environment and thus obtain image data that is easier to analyze.

[0020] Image data captured by the image capture device 206 may be transmitted to an image processor 212. The image processor 112 may be an external device, such as a personal computer, connected to the image capture device 206. Alternatively, the image processor 212 may be incorporated into the image capture device 206. The image processor 212 may include a processor 214 associated with a memory 216 and configured to determine a change required to address a difference between the actual focus and the correct focus for the image capture device 206. Once the difference is determined, an instruction may be transmitted to a steering motor system 218. The steering motor system 218 varies the distance z between the slide 202 and the optical system 208 in response to an instruction from the image processor 212.

[0021] In a system such as that shown in FIG. 1 or FIG. 2, components such as the motor drive 54 or steering motor system 218 may be used in a process as shown in FIG. 3 to continuously refocus an imager (e.g., digital high optical resolution imager 24 or image capture device 206) to maximize the quality of the resulting images for subsequent analysis. In a process as shown in FIG. 3, once an image is captured 301, it may be analyzed 302 to identify an offset between the expected focusing plane (i.e., the plane on which the imager was focused when the image was captured) and the correct focusing plane (i.e., the plane on which the imager needed to be focused to capture an in-focus image). In the event that the offset was zero (i.e., the plane on which the imager was focused when the image was captured was the correct focusing plane), the process may continue capturing images until all of the images required for analysis have been captured. Alternatively, if the offset was not zero (e.g., if there was a 2 μm separation between the expected focusing plane and the correct focusing plane), then the focusing plane of the imager may be adjusted 303 to account for the offset. For example, if the correct focusing plane was 2 μm below the expected focusing plane, then components such as the motor drive 54 or steering motor system 218 may be used to move the imager 2 μm closer to the sample, or move the sample 2 μm closer to the imager (e.g., by moving the flow cell holding the sample), or move the lens of the imager 2 μm closer to the imager (e.g., by holding the imager stationary while zooming the lens closer to the flow cell) so that the expected focusing plane and the correct focusing plane are the same for the next image. This process may then be performed repeatedly until image capture is complete 304, thereby ensuring that any deviations caused by factors such as temperature changes are detected and addressed.

[0022] Referring now to FIG. 4, FIG. 4 illustrates an architecture that may be used to analyze an image, such as during focus offset identification 302 in the process of FIG. 3. In the architecture of FIG. 4, an input image 401 is analyzed in a series of stages 402a-402n, each of which may be referred to as a "layer" and is illustrated in more detail in FIG. 5. As shown in FIG. 5, an input 501 (which in the first layer 502a of FIG. 4 is the input image 401, otherwise the output of a previous layer) is provided to layer 502, where it is processed to generate one or more transformed images 503a-503n. This processing may include convolving the input 501 with a set of filters 504a-504n, each of which then identifies a type of feature from the underlying image that is captured in that filter's corresponding transformed image. For example, as a simplified example, convolving an image with the filters shown in Table 1 may generate a transformed image that captures the edges of the input 501.

[0023] [Table 1] More complex filters are also possible. For example, if the input 501 is an NxN 3-channel image (e.g., an RGB image with red, blue, and green channels), the input 501 may be convolved with the 5x5x3 filter shown in Table 2 to produce a transformed image.

[0024] [Table 2]

[0025] As shown in FIG. 5, in addition to generating transformed images 503a-503n, the layer may also generate a pooled image 505a-505n for each of the transformed images 503a-503n. This is done, for example, by organizing the appropriate transformed images into a set of regions and then replacing the values ​​in the region with a single value, such as the maximum value of the region or the average of the values ​​of the region. The result is a pooled image that has a reduced resolution relative to its corresponding transformed image based on the size the regions are divided into (e.g., if the transformed image has NxN dimensions and it is divided into 2x2 regions, then the pooled image has a size of (N / 2)x(N / 2)). These pooled images 505a-505n are then combined into a single output image 506, with each of the pooled images 505a-505n being treated as a separate channel in the output image 506. This output image 506 may then be provided as an input to the next layer as shown in FIG. 4.

[0026] Returning to the discussion of Figure 4, after the final output image 403 has been created through various stages of processing 402a-402n, the final output image 403 may be provided as an input to a neural network 404. This may be done, for example, by providing the values ​​of each channel of each pixel in the output image 403 to input nodes of a tightly connected single layer network. The output of the neural network 404 may then be taken to be indicative of characteristics of the input image 401 that may be used in improving the quality of future images captured by the imaging system (e.g., the distance of the input image's focusing plane relative to the optical focusing plane for capturing an in-focus image).

[0027] Reference is now made to Figure 6, which shows a flow chart illustrating a method that may be used in an architecture such as that shown in Figure 4. Initially, in the method of Figure 6, an image is captured 301. This image may be treated as an input image 401 as shown in Figure 4 and convolved 602 with a filter for the current (e.g., first) processing layer. A check 603 may then be performed to determine if the convolution 602 should be repeated with a different filter. For example, if the current layer was configured to generate n transformed images 503a-503n, then the convolution 602 may be performed n times, with a different filter being used each time until n transformed images have been created.

[0028] In the method as shown in FIG. 6, after all the filters of a layer have been applied, a further check 604 may be performed to determine if pooling should be applied to the transformed image. As previously described in the context of FIG. 5, a pooled image may be created from the transformed image by dividing the transformed image into regions and replacing the values ​​for the pixels in the regions with a single pixel in the pooled image. However, this type of pooling may not be found in all layers, so the check 604 determines if pooling should be applied. If pooling should be applied, then the transformed image created by the convolution may be pooled 605 as previously described in the context of FIG. 5. If not, pooling is not performed and a check 506 is made whether more convolutions are needed (e.g., whether there are more layers). If more convolutions are needed, then the method as shown in FIG. 6 proceeds to the next layer and begins convolving 602 the output of the previous layer with the filters of the next layer. If not, the fully processed image resulting from the various convolution 602 and pooling 605 steps may be provided to a neural network 607. The method of Figure 6 then ends 608 with the output of the neural network taken as the result of the method itself. For example, if the method of Figure 6 was implemented as a function for identifying 302 the focus offset in the method of Figure 3, then the method of Figure 6 ends 608 by providing the result of applying 607 the neural network as a return value for identifying 302 the focus offset.

[0029] Architectures such as those shown and discussed in the context of Figures 4-6 may be implemented in a variety of ways. For example, such architectures may be implemented as a convolutional neural network (CNN), where each transformed image 502a-502n is represented by an output layer with one output node for each pixel in the transformed image, each output node being coupled to an input node representing a pixel in the input image 501 that has been convolved to generate a value for that output node's pixel in the transformed image, with the weighting of the connection between the input node and the output node being equal to the value of the filter used in the convolution. In this type of implementation, the specific values ​​used to analyze the image and generate the output may be optimized by training the neural network with blood cell images with known focusing positions to minimize regression errors between known and calculated focusing positions. For example, a CNN implementation with the layers listed in Tables 3 and 4 below has been found to be effective in determining focus for WBC images.

[0030] [Table 3]

[0031] [Table 4]

[0032] An architecture having the configuration described in Tables 3 and 4 can be trained to perform focus analysis using blood cell images with known focus positions to minimize regression errors. Such blood cell images can be obtained through manual annotation of images generated during normal operation of the analyzer (e.g., manually inspecting the images and then labeling the images with the difference, if any, between the actual focal plane and the optical focal plane by identifying focused and out-of-focus cell images based on one's experience and training), but the images can also be obtained in other ways. For example, the analyzer can be used to capture an in-focus image, and then capture an out-of-focus image by a known amount by intentionally changing the relationship between the image and the sample being imaged after the in-focus image is captured.

[0033] Table 5 shows examples of ways to purposefully offset the ideal focal length that can be used as part of a training procedure. In various examples, the camera or camera lens is set to a first ideal focus position to capture in-focus blood cells. The camera or camera lens is then offset in either direction to establish a training set for out-of-focus data. For example, the camera or camera lens starts at a position X (e.g., offset zero) that correlates with the ideal focus quality position. The camera or camera lens can then be offset in both directions at fixed intervals (e.g., 0.1 micron, 0.2 micron, 0.3 micron, 0.4 micron, or 0.5 micron intervals), for example, between -1 and +1 micron in either direction, between -2 and +2 micron in either direction, between -3 and +3 micron in either direction, between -4 and +4 micron in either direction, or between -5 and +5 micron in either direction. In the context of Table 5, X indicates the starting position and n indicates the offset increment (e.g., 0.3 micron) that defines the fixed interval the camera is offset in each sample run. Other approaches are possible, such as moving in variable increments, moving in different increments in different directions (e.g., moving 0.3 micron increments away from the flow cell and 0.2 micron increments closer to the flow cell), obtaining images from a different number of positions than those shown in Table 5 (e.g., moving 6n closer to the flow cell and 4n away from the flow cell), etc. Different types of training data creation are also possible, such as providing a set of images to human raters and asking them to specify an offset distance for each image. Thus, the description of how to intentionally offset the ideal focal length that may be used as part of the training procedure should be understood as exemplary only and should not be considered to imply a limitation on the protection afforded by this document or any related document.

[0034] In some examples, this training step is performed on separate groups of blood cells, for example red blood cells in a first sample and white blood cells in another sample, so that the system is trained to discriminate focus quality from smaller cells (e.g. red blood cells) and larger cells (e.g. white blood cells). The different types of cells used to train the system can include red blood cells, platelets, and different groups of white blood cells (neutrophils, lymphocytes, monocytes, eosinophils, and basophils). In other examples, the system is trained only on a particular cell type (e.g. only red blood cells, only white blood cells, or only a specific type of white blood cell, such as only neutrophils).

[0035] [Table 5]

[0036] After training as described above was performed on the architectures shown in Tables 3 and 4, the network implemented with those configurations was found to produce the results shown in FIG. 7 for nine blood sample runs used to generate WBC images. In those runs, each run was set up with a predefined focusing plane that ranged from -2 μm to 2 μm offset from the correct focusing plane, and hundreds of WBC images were taken with that focus. The CNN then calculated the focusing position for each image in the run, and the median of all the calculated focusing positions was taken as the calculated focusing position for that sample run. As shown in FIG. 7, this provided a good correlation between the original focusing position and the calculated focusing position, validating this type of multi-layer analysis approach for auto-focusing on blood sample images. Furthermore, while the tests that are shown in FIG. 7 were done with WBC images, it will be clear to one skilled in the art that the same type of approach (e.g., training a CNN with the type of image to be evaluated and using the trained CNN for the auto-focusing method) can also be applied to red blood cell (RBC) images, or both RBC and WBC images.

[0037] Other variations and implementations of the disclosed techniques are also possible. For example, while the above examples have given particular dimensions for the filters, those dimensions are not intended to be limiting, and the inventors contemplate that other sizes (e.g., dimensions from 3x3 to 9x9) may be utilized when performing convolutions as described in the context of Figures 4-6 and Tables 1-2. Similarly, while the above examples have included particular inputs, those inputs should also be understood to be non-limiting, and the inventors contemplate that other inputs are possible. For example, additional input channels may be used in some cases in addition to or as an alternative to the three-channel inputs described in the context of Tables 1 and 2. Furthermore, even when the input channels are held constant, the input dimensions may be varied. For example, configurations such as those described in Tables 1 and 2 may be varied to have inputs of the form NxNx3, where N can range from tens to hundreds. Variations in the physical apparatus used to implement the disclosed techniques are also possible. For example, while Figures 1 and 2 show systems in which the disclosed techniques may be used for focusing, the focusing techniques described herein may be applied to other types of systems. Indeed, the inventors contemplate that autofocusing as described herein may be used to improve image quality in any type of device that captures still or flow images of blood cells. Accordingly, the above figures and tables, and their associated discussion, should not be used to imply any limitation on the scope of protection afforded by this document or by any other document that claims the benefit of this document and is otherwise related to this document.

[0038] Variations in methods that may utilize focusing techniques as described herein may also be possible. For example, an autofocus process as described herein may be implemented to run a series of samples to determine how the camera should be focused and adjust the focus for the run by run bias, but not for the image by image bias. Similarly, rather than automatically refocusing the camera, the focus position may be used to generate a warning (e.g., if the difference between the expected focusing plane and the correct focusing plane exceeds a threshold or indicates a trend toward being out of focus), after which the user decides whether to refocus the analyzer or continue with the then current imaging task. Automatic focusing as described herein may also / alternatively be included in periodic (e.g., daily) quality control processes. Data gathered with automatic focusing may then be used to improve the operation of the system. For example, if adjustments made during automatic focusing are found to be consistently in one direction, this may be used as a diagnostic indicator that there was a system malfunction in the mechanical or optical components of the analyzer that, if corrected, would reduce the need for automatic refocusing. As another example of how automatic focusing as described herein may be applied, consider that in some cases, even if the focus is acceptable, different focusing positions within the acceptable range may result in different features that are perceptibly sharper or less sharp in the image. In such cases, the focusing information may be used to characterize the image captured by the system (e.g., closer to the sample or further away from the sample while still being within the acceptable range), so that downstream processes may be optimized as necessary (e.g., by applying a sharpening kernel if a particular feature is more difficult to identify based on the characterization) depending on what features are detected. Thus, the images from the image autofocus previously described should be understood as exemplary only and should not be considered to imply any limitations on the protection afforded by this document or any related document.

[0039] Variations in the manner in which the focusing method as described herein may be implemented are also possible. For example, in some cases, the method as shown in FIG. 6 may include additional steps to facilitate image processing and utilization. An example of such additional steps may be pre-processing or reformatting the image before subjecting it to analysis. In an example where processing is performed using a CNN trained to take a certain type of image as input (e.g., 128x128x3 RGB images), this may include resizing the image to match the training input before subjecting it to convolution and / or pooling. Thus, specific implementations may vary from case to case, as may the potential contexts and applications in which the disclosed technology may be used, and the examples, figures, and descriptions set forth herein should be understood as illustrative only and should not be considered limiting.

[0040] As further illustration of potential implementations and applications of the disclosed technology, the following non-exhaustive examples of ways in which the teachings herein may be combined or applied are given. It should be understood that the following examples are not intended to limit the subject matter of any claims that may be presented at any time in this application or any subsequent application of this application. No disclaimer is intended. The following examples are not provided in any way other than for illustrative purposes. It is contemplated that the various teachings herein may be configured and applied in numerous other ways. It is also contemplated that some variations may omit some specific features referenced in the following examples. Thus, any of the aspects or features referenced below should not be considered conclusive unless expressly indicated otherwise, such as at a later date by the inventor or a successor in interest of the inventor. If any claims are presented in this application or any subsequent application related to this application that include additional features that deviate from those referenced below, those additional features are not presumed to have been added for any reason related to patentability. EXAMPLES

[0041] [Example 1] 1. A system for assessing focus quality of a blood cell image, comprising: a) a camera adapted to capture an image of a blood sample; and b) a non-transitory computer-readable medium having stored thereon: i) a plurality of sets of convolution filters; ii) a fully-connected layer; and iii) instructions that, when executed by a processor, are operable to: A) for each set of convolution filters, generate an output having a depth equal to a cardinality of the set of convolution filters by performing acts including, for each convolution filter in the set of convolution filters, convolving an input with the convolution filter, where I) the input for the first set of convolution filters is an image captured by the camera; and II) for each set of convolution filters other than the first set of convolution filters, the input for the set of convolution filters is an output generated by a different set of convolution filters; and B) generate a focal length by performing acts including, for each set of convolution filters other than the first set of convolution filters,

[0042] [Example 2] 2. The system of claim 1, wherein, for each set of convolution filters, a) an input to the set of convolution filters has a resolution, b) an output generated for the set of convolution filters has a resolution, and c) instructions stored on a non-transitory computer-readable medium are operable to reduce the resolution of the output generated for the set of convolution filters from the resolution of the input to the set of convolution filters by pooling pixels in a plurality of regions in the output generated for the set of convolution filters, where each region includes a single value for the region.

[0043] [Example 3] 3. The system of example 2, wherein for each region for each set of convolution filters, the single value included for that region in the output generated for that set of convolution filters is the maximum value for that region.

[0044] [Example 4] 2. The system of example 1, wherein the non-transitory computer-readable medium stores a plurality of sets of convolution filters as weights in a convolutional neural network.

[0045] [Example 5] The system of Example 1, wherein a) the fully connected layer includes a plurality of inputs and a plurality of outputs, and b) a non-transitory computer-readable medium stores instructions that, when executed by a processor, are operable to generate a focal length by performing acts including: i) providing each output of the fully connected layer to an input of a second fully connected layer; and ii) obtaining a focal length as an output of the second fully connected layer.

[0046] [Example 6] The system described in Example 1, wherein: a) the system comprises a flow cell, the flow cell including a viewing area and connected to a source of sample fluid and a source of alignment fluid, the alignment fluid being transparent and having a higher viscosity than the sample fluid; and b) the camera is adapted to capture an image of the blood sample by capturing an image of particles constituted by the stream of sample fluid through a layer of the alignment fluid disposed between the stream of sample fluid and the camera as the particles constituted by the stream of sample fluid flow through the viewing area of ​​the flow cell.

[0047] [Example 7] The system of Example 6, wherein a) the system comprises a motor; and b) the instructions stored in the non-transitory computer readable medium, when executed by a processor, comprise instructions operable to focus the camera by using the motor to move the position of the camera relative to the viewing area of ​​the flow cell based on a focal length.

[0048] [Example 8] The system of Example 7, wherein a) the non-transitory computer readable medium, when executed by a processor, is operable to obtain a plurality of focal lengths, where i) each focal length from the plurality of focal lengths corresponds to a different image captured by the camera, and ii) the focal lengths are comprised of the plurality of focal lengths, and b) focusing the camera by using a motor to move a position of the camera relative to a viewing area of ​​the flow cell based on the focal length includes: i) determining a median of the plurality of focal lengths; and ii) translating the camera a distance equal to the median of the plurality of focal lengths.

[0049] [Example 9] 2. The system of claim 1, wherein a) the first set of convolution filters is adapted to receive as an input an image having a predetermined size; and b) the non-transitory computer-readable medium, when executed by a processor, is operable to: i) resize a raw image captured by a camera to match the predetermined size; and ii) provide the resized image as an input to the first set of convolution filters.

[0050] [Example 10] 2. The system of example 1, wherein the first set of convolution filters is adapted to receive as input an image depicting one or more white blood cells.

[0051] [Example 11] A method for assessing focus quality of a blood cell image, comprising: a) for each of a plurality of convolution filters, performing acts including, for each convolution filter in a set of convolution filters, convolving an input with the convolution filter to generate an output having a depth equal to a cardinality of the set of convolution filters, where i) the input for a first set of convolution filters is an image captured by a camera having a set of blood cells, and ii) for each set of convolution filters other than the first set of convolution filters, the input for the set of convolution filters is an output generated by a different set of convolution filters; and b) generating a focal length by performing acts including, providing the output generated for the set of convolution filters as an input to a fully connected neural network layer.

[0052] [Example 12] 12. The method of claim 11, wherein for each set of convolution filters, a) an input to the set of convolution filters has a resolution, b) an output generated for the set of convolution filters has a resolution, and c) the method includes reducing the resolution of the output generated for the set of convolution filters from the resolution of the input to the set of convolution filters by pooling pixels in a plurality of regions in the output generated for the set of convolution filters, where each region includes a single value for the region.

[0053] [Example 13] 13. The method of claim 12, wherein for each region for each set of convolution filters, the single value included for that region in the output generated for that set of convolution filters is the maximum value for that region.

[0054] [Example 14] 12. The method of example 11, wherein a) the fully connected neural network layer includes a plurality of inputs and a plurality of outputs, and b) the method includes generating the focal length by performing acts including: i) providing each output of the fully connected neural network layer to an input of a second fully connected neural network layer; and ii) obtaining the focal length as an output of the second fully connected neural network layer.

[0055] [Example 15] The method of example 11, wherein the method comprises capturing an image provided as an input to a first set of convolution filters by performing the acts of: a) flowing a stream of sample fluid through a flow cell, the flow cell including a viewing area and connected to a source of sample fluid and a source of alignment fluid, the alignment fluid being transparent and having a higher viscosity than the sample fluid; and b) imaging particles constituted by the stream of sample fluid through a layer of alignment fluid disposed between the stream of sample fluid and a camera as the particles constituted by the stream of sample fluid flow through the viewing area of ​​the flow cell.

[0056] [Example 16] The method of Example 15, comprising focusing the camera by using a motor to move the position of the camera relative to the viewing area of ​​the flow cell based on the focal length.

[0057] [Example 17] The method of Example 16, wherein a) the method includes obtaining a plurality of focal lengths, where i) each focal length from the plurality of focal lengths corresponds to a different image captured by the camera, and ii) the focal lengths are comprised of the plurality of focal lengths; and b) focusing the camera by using a motor to move a position of the camera relative to a viewing area of ​​the flow cell based on the focal length includes i) determining a median of the plurality of focal lengths, and ii) translating the camera a distance equal to the median of the plurality of focal lengths.

[0058] [Example 18] 12. The method of claim 11, wherein a) the first set of convolution filters is adapted to receive as an input an image having a predetermined size, and b) the method includes: i) resizing a raw image captured by the camera to match the predetermined size; and ii) providing the resized image as an input to the first set of convolution filters.

[0059] [Example 19] 12. The method of embodiment 11, wherein the first set of convolution filters is adapted to receive as input an image depicting one or more white blood cells.

[0060] [Example 20] A blood cell analysis machine comprising: a) a camera; and b) means for determining a focal length for the camera based on an image depicting one or more blood cells.

[0061] Each of the calculations or operations described herein may be implemented using a computer or other processor having hardware, software, and / or firmware. Various method steps may be implemented by modules, which may comprise any of a wide variety of digital and / or analog data processing hardware and / or software configured to perform the method steps described herein. Modules optionally comprise data processing hardware adapted to perform one or more of these steps by having appropriate machine programming code associated therewith, with modules for two or more steps (or portions of two or more steps) being integrated on a single processor board or separated on separate processor boards in any of a wide variety of integrated and / or distributed processing architectures. These methods and systems will often employ tangible media embodying machine-readable code with instructions for performing the method steps described above. Suitable tangible media may comprise memory (including volatile and / or non-volatile memory), or storage media (such as magnetic recording on floppy disks, hard disks, tapes, etc., optical memory such as CDs, CD-R / W, CD-ROMs, DVDs, etc., or any other digital or analog storage media), etc.

[0062] All patents, patent publications, patent applications, journal articles, books, technical references, and the like discussed in this disclosure are hereby incorporated by reference in their entirety for all purposes.

[0063] Different arrangements of the components depicted in the figures and described above, as well as components and steps not shown or described, are possible. Similarly, some features and subcombinations are useful and may be employed without reference to other features and subcombinations. The embodiments of the present invention have been described for illustrative and not limiting purposes, and alternative embodiments will be apparent to the reader of this patent. In some specific cases, method steps or operations may be performed in a different order, or operations may be added, deleted, or modified. It will be apparent that in some specific aspects of the present invention, a single component may be replaced by multiple components, and multiple components may be replaced by a single component, to provide an element or configuration or to perform a given function or functions. Such subsituations are considered within the scope of the present invention, except where such subsituations are not operative to practice some specific embodiments of the present invention. Thus, the claims should not be considered limited to the examples, figures, embodiments, and illustrations given above, but should instead be understood to have the scope given when the terms in the claims are given their broad and reasonable interpretation given by a common dictionary. However, if a term or phrase is indicated to have a particular meaning under the heading "Explicit Definition," the term or phrase is to be understood to have that meaning when used in the claims.

[0064] Explicit definition It will be appreciated that in the above examples and claims, the phrase "based on" something else should be understood to mean at least partially determined by the thing it is shown to be based on. To indicate that something must be determined entirely based on something else, it is described as being "based solely on" whatever it is that must be entirely determined by.

[0065] In the above examples and claims, it will be understood that the phrase "means for determining a focal length for the camera based on an image depicting one or more blood cells" is a means-plus-function limitation provided in 35 U.S.C. § 112(f), where the function is "determining a focal length for the camera based on an image depicting one or more blood cells," and the corresponding structure is a computer configured to use the algorithms shown in Figures 4-6 and Tables 3-4 and described in the accompanying description.

[0066] It will be appreciated that in the above examples and in the claims, the term "set" is to be understood as one or more items that are grouped together. [Explanation of symbols]

[0067] 18 Processors 21 areas 21a Proximal flow path section 21b Distal flow channel portion 22 Flow Cell 23 Viewing area 24 High optical resolution imaging device 25 Source of sample fluid 27 Sources of PIOAL 28 Distal end 29 Sample delivery tube 32 Sample Streams 33 Discharge section 46 Objective Lens 48 CCD 50 Working Distance 52 Displacement distance 54 Motor drive unit 200 Slide Type Visual Inspection System 202 Slides 204 Slide Holder 206 Image capture device 208 Optical System 210 Image Sensor 212 Image Processing Device 214 processor 216 Memory 218 Steering Motor System 301 Capture 302 identification 303 Adjustment 304 Completed 401 Input Image 402a~n stage 403 Final output image 404 Neural Networks 501 Input 502 layers 503a~n Converted images 504a~n Filter 505a~n Pooled images 506 Output Images 602 Convolution 603 Confirmation 604 Confirmation 605 Pooling 606 Confirmation 607 Application of Neural Networks 608 End DT Distal thickness PT proximal thickness z distance

Claims

1. 1. A system for assessing focus quality of a blood cell image, comprising: a) a camera adapted to capture an image of the blood sample; b) a non-transitory computer-readable medium, comprising: i) a plurality of sets of convolution filters; ii) a fully connected layer; and iii) instructions that, when executed by a processor, A) for each set of convolution filters, performing acts including, for each convolution filter in the set of convolution filters, convolving an input with that convolution filter, thereby generating an output having a depth equal to the cardinality of the convolution filter in the set; I) the input for a first set of convolution filters is an image captured by the camera; II) for each set of convolution filters other than the first set of convolution filters, the input for that set of convolution filters is an output produced by a different set of convolution filters; B) generating a focal length by performing an act including providing the outputs generated for a set of convolution filters as inputs to the fully connected layer; and A non-transitory computer-readable medium storing the A system comprising:

2. For each set of convolution filters, a) the input to the set of convolution filters has a resolution, b) the output generated for the set of convolution filters has a resolution; c) the instructions stored on the non-transitory computer readable medium are operable to reduce the resolution of the output generated for the set of convolution filters from the resolution of the input to the set of convolution filters by pooling pixels in multiple regions in the output generated for the set of convolution filters, each region including a single value for that region. The system of claim 1 .

3. 3. The system of claim 2, wherein for each region for each set of convolution filters, the single value included for that region in the output generated for that set of convolution filters is the maximum value for that region.

4. 2. The system of claim 1, wherein the non-transitory computer readable medium stores the sets of convolution filters as weights in a convolutional neural network.

5. a) the fully connected layer includes a plurality of inputs and a plurality of outputs; b) the non-transitory computer-readable medium, when executed by the processor, i) providing each output of the fully connected layer to an input of a second fully connected layer; ii) obtaining the focal length as an output of the second fully connected layer; and storing instructions operable to generate said focal length by performing acts including: The system of claim 1 .

6. a) the system comprises a flow cell, the flow cell including a viewing area and coupled to a source of sample fluid and a source of alignment fluid, the alignment fluid being transparent and having a higher viscosity than the sample fluid; b) the camera is adapted to capture an image of the blood sample by capturing an image of the particles constituted by the stream of sample fluid through a layer of alignment fluid disposed between the stream of sample fluid and the camera as the particles constituted by the stream of sample fluid flow through the viewing area of ​​the flow cell; The system of claim 1 .

7. a) the system comprises a motor; b) the instructions stored on the non-transitory computer readable medium include instructions that, when executed by the processor, are operable to focus the camera by using the motor to move a position of the camera relative to the viewing area of ​​the flow cell based on the focal length. The system of claim 6.

8. a) the non-transitory computer readable medium, when executed by the processor, is operable to obtain a plurality of focal lengths; i) each focal length from the plurality of focal lengths corresponds to a different image captured by the camera; ii) the focal length is made up of the plurality of focal lengths; b) focusing the camera by using the motor to move the position of the camera relative to the viewing area of ​​the flow cell based on the focal length; i) determining a median of the plurality of focal lengths; ii) translating the camera a distance equal to the median of the plurality of focal lengths; Including, The system of claim 7.

9. a) the first set of convolution filters is adapted to receive as input an image having a predetermined size; b) the non-transitory computer-readable medium, when executed by the processor, i) resizing a raw image captured by said camera to match said predetermined size; ii) providing the resized image as the input for the first set of convolution filters. It is operable as follows: The system of claim 1 .

10. The system of claim 1 , wherein the first set of convolution filters is adapted to receive as input an image depicting one or more white blood cells.

11. 1. A method for assessing focus quality of a blood cell image, comprising: a) for each of a plurality of convolution filters, performing acts including, for each convolution filter in the set of convolution filters, convolving an input with that convolution filter, thereby generating an output having a depth equal to a cardinality of the set of convolution filters; i) the input for a first set of convolution filters is an image captured by a camera, the image having a set of blood cells; ii) for each set of convolution filters other than the first set of convolution filters, the input for that set of convolution filters is an output produced by a different set of convolution filters; Producing an output; and b) generating a focal length by performing acts including providing the generated outputs of the set of convolution filters as inputs to a fully connected neural network layer; The method includes:

12. For each set of convolution filters, a) the input to the set of convolution filters has a resolution, b) the output generated for the set of convolution filters has a resolution; c) the method includes reducing the resolution of the output generated for the set of convolution filters from the resolution of the input to the set of convolution filters by pooling pixels in a plurality of regions in the output generated for the set of convolution filters, each region including a single value for that region. The method of claim 11.

13. 13. The method of claim 12, wherein for each region for each set of convolution filters, the single value included for that region in the output generated for that set of convolution filters is the maximum value for that region.

14. a) the fully connected neural network layer includes a plurality of inputs and a plurality of outputs; b) the method comprises: i) providing each output of said fully-connected neural network layer to an input of a second fully-connected neural network layer; ii) obtaining the focal length as an output of the second fully connected neural network layer; and generating said focal length by performing acts including: The method of claim 11.

15. The method further comprising: a) flowing a stream of a sample fluid through a flow cell, the flow cell including a viewing area and coupled to a source of sample fluid and a source of alignment fluid, the alignment fluid being transparent and having a higher viscosity than the sample fluid; b) imaging particles constituted by the sample fluid stream through a layer of alignment fluid disposed between the sample fluid stream and the camera as the particles constituted by the sample fluid stream flow through the viewing area of ​​the flow cell; capturing the image provided as an input to the first set of convolution filters by performing acts including: The method of claim 11.

16. 16. The method of claim 15, comprising focusing the camera by using a motor to move a position of the camera relative to the viewing area of ​​the flow cell based on the focal length.

17. a) the method includes obtaining a plurality of focal lengths, i) each focal length from the plurality of focal lengths corresponds to a different image captured by the camera; ii) the focal length is composed of the plurality of focal lengths; Including obtaining b) focusing the camera by using the motor to move the position of the camera relative to the viewing area of ​​the flow cell based on the focal length; i) determining a median of the plurality of focal lengths; ii) translating the camera a distance equal to the median of the plurality of focal lengths; Including, 17. The method of claim 16.

18. a) the first set of convolution filters is adapted to receive as input an image having a predetermined size; b) the method comprises: i) resizing a raw image captured by said camera to match said predetermined size; ii) providing the resized image as the input for the first set of convolution filters; Including, The method of claim 11.

19. The method of claim 11 , wherein the first set of convolution filters is adapted to receive as input an image depicting one or more white blood cells.

20. a) a camera; and b) means for determining a focal length for the camera based on an image depicting one or more blood cells; A blood cell analysis machine comprising:

Images