Systems, devices and methods for non-invasive hematological measurement

Abstract

To solve the problem in which tests to derive the WBC count are invasive and there can be a certain bias of some parameters caused by inherent differences between measurements and true physiological properties.SOLUTION: A system for non-invasive hematological measurement includes a platform to receive a body portion of a user, and an imaging device to acquire a set of images of a capillary bed in the body portion. For each image, a controller detects one or more capillaries in the body portion of a finger so as to identify a first set of capillaries by estimating one or more attributes of each capillary (e.g., structural attributes, flow attributes, imaging attributes, or combinations thereof), where at least one attribute of each capillary meets a predetermined criterion. The controller also identifies a second set of capillaries from the first set of capillaries, so that each capillary of the second set of capillaries is visible in a predetermined number of images of the set of images.SELECTED DRAWING: Figure 23

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 62 / 572,738, entitled "DEVICE AND METHODS FOR NON-INVASIVE HEMATOLOGICAL MEASUREMENTS," filed October 16, 2017, the entire disclosure of which is incorporated herein by reference.

[0002] Declaration of support This invention was made with Government support under Grant No. U54EB015403 awarded by the National Institutes of Health. The Government has certain rights in this invention.

[0003] The present disclosure relates generally to systems, devices, and methods for analyzing blood cell and cell population dynamics, and more particularly, to systems, devices, and methods for extracting white blood cell information from non-invasive, in vivo, and / or time-lapse images. [Background technology]

[0004] White blood cells (WBCs, also called leukocytes) are cells of the immune system involved in protecting the body against both infectious diseases and foreign invaders. WBCs can be present in the blood as well as in the lymphatic system and tissues. Some conditions can trigger a response in the immune system, causing an increase in the number of WBCs (also called WBC count). Other conditions can affect the production of WBCs by the bone marrow, or the survival of existing WBCs in the circulation. Either way, these conditions can cause a change (either an increase or decrease) in the number of circulating WBCs. Thus, WBC count can be a relevant physiological parameter for the diagnosis, monitoring, and treatment of various conditions, including, but not limited to, bacterial and viral infections (e.g., pneumonia or meningitis), bone marrow functionality associated with chemotherapy toxicity, and hematological proliferative processes such as leukemia.

[0005] In current clinical practice, most of the tests that derive WBC counts are performed using large-scale equipment in central clinical laboratories. Generally, these ex vivo tests are still invasive because blood samples (usually one vial of blood is required for each test) are collected from the patient. These blood samples are then transported, queued, and analyzed in a clinical laboratory, which may take several days to receive any results. This procedure can be burdensome for patients who require regular WBC counts or for patients with conditions as well as their caregivers. Furthermore, due to the ex vivo nature of traditional blood tests, there may be certain biases of some parameters due to the inherent differences between the measurements and the true physiological characteristics. Summary of the Invention

[0006] The system includes a platform that receives a portion of a user's body during use, and an imaging device coupled to the platform for acquiring a set of images of at least a capillary bed of the portion of the body. The system also includes a controller communicatively coupled to the imaging device for detecting one or more capillaries of the portion of the body in each image of the set of images to identify a first set of capillaries across the set of images. The detecting includes estimating one or more attributes of each capillary of the first set of capillaries, the one or more attributes being one or more structural attributes, one or more flow attributes, one or more image attributes, or the like. A first attribute of the one or more attributes of each capillary of the first set of capillaries satisfies a predetermined criterion for a first attribute. The controller also identifies a second set of capillaries from the first set of capillaries, such that each capillary of the second set of capillaries is visible in a predetermined number of images of the set of images.

[0007] A method comprising: acquiring a set of images of at least a capillary bed of a portion of a user's body; and detecting one or more capillaries of the portion of the body in each image of the image set to identify a first capillary set across the image set. The detecting includes estimating one or more attributes of each capillary of the first capillary set, the one or more attributes including one or more structural attributes, one or more flow attributes, one or more image attributes, or a combination thereof. A first attribute of the one or more attributes of each capillary of the first capillary set satisfies a predetermined criterion for the first attribute. The method also includes identifying a second capillary set from the first capillary set, such that each capillary of the second capillary set is visible in a predetermined number of images of the image set.

[0008] An apparatus comprising a controller for receiving a set of images of a capillary bed of a body portion of a user and detecting one or more capillaries of a capillary bed of the body portion of the finger in each image of the image set to identify a first capillary set across the image set. The detecting includes estimating one or more attributes of each capillary of the first capillary set, the one or more attributes including one or more structural attributes, one or more flow attributes, one or more image attributes, or a combination thereof. A first attribute of the one or more attributes of each capillary of the first capillary set satisfies a predetermined criterion for the first attribute. The controller also identifies a second capillary set from the first capillary set, such that each capillary of the second capillary set is visible in a predetermined number of images of the image set. The controller also detects a set of cellular events for the image set and in the second capillary set, each cellular event of the set of cellular events being associated with the passage of a white blood cell in a capillary of the second capillary set. The controller also estimates an event count for the second set of capillaries based on the set of cellular events.

[0009] It should be understood that all combinations of the foregoing concepts and additional concepts discussed in more detail below (provided that such concepts are not mutually inconsistent) are considered to be part of the inventive subject matter disclosed herein. In particular, all combinations of subject matter recited in the claims appearing at the end of this disclosure are considered to be part of the inventive subject matter disclosed herein. It should also be understood that the terms explicitly utilized in any disclosure incorporated by reference should be given the meaning most consistent with the particular concepts disclosed herein. [Brief description of the drawings]

[0010] The patent or application file contains at least one drawing in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0011] Those skilled in the art will appreciate that the drawings are primarily for illustrative purposes and are not intended to limit the scope of the inventive subject matter described herein. The drawings are not necessarily to scale, and in some instances, various aspects of the inventive subject matter disclosed herein may be shown exaggerated or enlarged in the drawings to facilitate an understanding of different features. In the drawings, like reference characters generally refer to like features (e.g., functionally similar and / or structurally similar elements).

[0012] [Figure 1] FIG. 1 is a flow chart illustrating a method for analyzing blood cell dynamics according to some embodiments. [Diagram 2] FIG. 2 is a flow chart illustrating a method for analyzing blood cell dynamics using spatiotemporal profiles and Radon transform according to some embodiments. [Diagram 3] FIG. 3 is an example of a nail fold image that can be used in the method illustrated in FIGS. 1-2, according to some embodiments. [Figure 4]4A-4B are plots of cubic spline interpolation that can be used in the method illustrated in FIGS. 1-2 to resample user-specified contour points of capillaries, according to some embodiments. [Diagram 5] FIG. 5 is an image of a nail fold including two capillaries, user-specified contour points, and a resampled contour curve, according to some embodiments. [Figure 6] 6A-6B are representations of spatiotemporal profiles extracted from the two capillaries shown in FIG. 5, according to some embodiments. [Figure 7] FIG. 7 is a representation of the Radon transform performed on the spatiotemporal profile according to some embodiments. [Figure 8] 8A-8B are representations of spatiotemporal profiles in the Radon domain with local maxima highlighted to indicate WBC events, according to some embodiments. [Figure 9] 9A-9B are plots depicting experimental results of WBC events occurring within the two capillaries shown in FIG. 5, according to some embodiments. [Figure 10] FIG. 10 is a plot comparing WBC events obtained from the method illustrated in FIG. 2 according to some embodiments with WBC events identified by trained human assessors. [Figure 11-1] 11A-11C are schematic diagrams of a system for analyzing blood cell dynamics using a finger holder and an imager, according to some embodiments. [Figure 11-2] Same as above. [Figure 11-3] Same as above. [Figure 12-1] 12A-12B are schematic diagrams of a system for analyzing blood cell dynamics using an imager in a vertical configuration, according to some embodiments. [Figure 12-2] Same as above. [Figure 13] FIG. 13 is a schematic diagram of a system for analyzing blood cell dynamics analysis using a smartphone, according to some embodiments. [Figure 14-1] 14A-14D are images of a system and device for analyzing blood cell dynamics analysis, according to some embodiments. [Figure 14-2] Same as above. [Figure 15-1] 15A-15J are diagrams of an adapter for capturing nailfold images using a smartphone camera for blood cell dynamics analysis, according to some embodiments. [Figure 15-2] Same as above. [Figure 16] FIG. 16 is a schematic diagram of a system including a smartphone and an adapter for capturing nailfold images for blood cell dynamics analysis, according to some embodiments. [Figure 17] 17A-17B are images of a system including a smartphone and an adapter for capturing images of nail folds for blood cell dynamics analysis, according to some embodiments. [Figure 18] FIG. 18 is an example of an image captured by the system shown in FIGS. 17A-17B, according to some embodiments. [Figure 19] FIG. 19 is a schematic illustration of a clamping device for performing blood cell kinetic analysis from nailfold images, according to some embodiments. [Figure 20] FIG. 20 is an image of the clamping device used to capture images of the nail fold and analyze blood cell dynamics. [Figure 21-1] 21A-21F and 21G-21O are images and wireframes, respectively, of a smartphone adapter for capturing nailfold images using a smartphone camera for capillaroscopy and hematological analysis, according to some embodiments. [Figure 21-2] Same as above. [Figure 21-3] Same as above. [Figure 22] 22A-22D are images illustrating a method of using a smartphone adapter to capture images of a nail fold using a smartphone camera for capillaroscopy and hematology analysis, according to some embodiments. [Diagram 23] FIG. 23 illustrates a system for non-invasive hematology measurements, according to an embodiment. [Figure 24] Figures 24A-B illustrate the capillary tracking criteria for selecting suitable capillaries. Each capillary was tracked with a given identifier (id) during 3600 frames of the video. In Figure 24A, black bars = capillary segments used to calculate cell count / white blood cell index. An id is selected if its occurrence exceeds a confidence value of ≠=600 (green case) or discarded if it has more than 3 selected capillaries (red case). If it has less than 3 capillaries, the value is ≠=1. In Figure 24B, capillary segments used to calculate white blood cell index are shown in green, otherwise in red. [Diagram 25] 25A-B illustrate the detection of capillaries in raw videos using the neural network-based method / approach described herein compared to human expert performance: results from two raters (green) vs. combined neural network analysis (red, yellow when overlapping). [Figure 26]26A-C illustrate the average time signal. FIG. 26A illustrates an example of an actual average time signal generated from one of the analyzed capillary videos, where the first 20 seconds (frames 1-1200; see left) including a peak of positive values ​​associated with the detected event is shown, along with a 2 second zoom around the single event (frames 650-769; see right). The zoomed time signal around this exemplary event displays a brightness peak, followed by a slight "dip" of some duration. FIG. 26B illustrates the expected profile of the average time signal around the passage of a single event, which is associated with the passage of a white blood cell in a capillary (see right). According to the zoomed example in (a), the "dip" in intensity is expected to always occur to the right after the maximum intensity peak, due to a higher accumulation of red blood cells upstream of the white blood cells. FIG. 26C illustrates the expected profile of the average time signal around the passage of the plasma gap in a capillary (see right side), where no dip would occur in the absence of white blood cells and therefore no accumulation of red blood cells upstream. [Figure 27] Figures 27A-B illustrate the detection of cellular events by an automated approach vs. a manual rater. Figure 27A shows an example of an event detected on a capillary movie by one human rater (top row) vs. a neural network (bottom row) with blue and white event marks, respectively. Figure 27B illustrates how true positives (TP), false positives (FP), and false negatives (FN) were evaluated for the detection of the event relative to the reference. On the right are the average F1 scores obtained for the three raters (red) and the algorithm (blue) across the 26 analyzed capillary movies, proposing the equivalent performance of the neural network according to the above metric (F1 isocurve shown as black dashed line). The expected behavior of the neural network when adjusting the threshold for event detection is shown as a red dashed line. [Figure 28]28A-D illustrate an example of a method / approach for the detection of events. FIG. 28A - initial capillary movie with selected frames showing a plasma gap flowing through the capillary, with the flow direction highlighted by an arrow. FIG. 28B - preprocessed movie with events appearing as prominent features in a dark background in another scheme. The reference pixel Pref is shown in red, and additional pixels P1 and P2 belonging to the capillary are shown in blue. FIG. 28C - Luminance time signal at the reference pixel positions, P1 and P2, revealing as peaks the effect of the event on the measured luminance (highlighted in green). FIG. 28D - final time signal obtained as the average between the time signals at the individual pixel positions after being aligned with the reference time signal of Pref, with the threshold level used for counting shown as a vertical blue line. [Figure 29] Figures 29A-B illustrate the classification results generated by the fully automated approach disclosed herein for 116 data points. Figure 29A illustrates a box plot showing the classification of raw videos associated with a baseline condition, i.e., ANC>500 (blue) vs. a severe neutropenic condition (ANC<500). Figure 29B shows the ROC curve associated with the classification (AUC=0.96). [Diagram 30] FIG. 30 is a flowchart of a method for non-invasive hematology measurements, according to some embodiments. [Diagram 31] 31A-31F illustrate a method for estimating blood volume based on an analysis of pixel intensity values. [Diagram 32] 32A-32C illustrate a method for volumetric resampling of capillary profiles. [Diagram 33] FIG. 33 illustrates the sub-selection of capillaries based on observed size. [Diagram 34] 34A and 34B illustrate the sub-selection of capillaries based on the distribution of arrival times between observed gaps. [Diagram 35]35A-35C illustrate a device for detecting severe neutropenia based on imaging of nailfold capillaries. [Diagram 36] FIG. 36 illustrates two different timing points for acquiring images from a patient in a clinical trial using the apparatus shown in FIG. 35A. [Figure 37] 37A and 37B show examples of raw images acquired by the apparatus shown in FIG. 35A. [Figure 38] 38A-38E illustrate an example of an optical gap flowing through a capillary blood vessel. [Figure 39] 39A-39E show the results of blinded event assessment using images acquired by the device shown in FIG. 35A. [Diagram 40] FIG. 40 shows the number of confirmed events per minute in all capillary pairs tested. [Diagram 41] FIG. 41 shows the differentiation between baseline and severe neutropenia observed in clinical trials. [Diagram 42] FIG. 42 illustrates a clinical trial pre-processing workflow for processing images acquired by the device shown in FIG. 35A. [Diagram 43] FIG. 43 shows the number of events labeled by one single rater in a clinical trial. [Diagram 44] Figures 44A and 44B show the differentiation between baseline and severe neutropenia using capillary populations. [Diagram 45] 45A-45C show an example of capillary segmentation. [Figure 46] FIG. 46 shows the expected event rate per minute in a capillary under shot noise. [Figure 47] FIG. 47 shows the distribution of capillary diameters at event locations. [Figure 48] 48A and 48B show ST maps of capillaries with high vs. low rates of confirmed events. [Figure 49]FIG. 49 shows capillary selections obtained from both experts in the raw video pair for patient 01, region 1. [Figure 50] FIG. 50 shows capillary selections obtained from both experts in the raw video pair for patient 01, region 2. [Figure 51] FIG. 51 shows capillary selections obtained from both experts in the raw video pair for patient 02, region 1. [Figure 52] FIG. 52 shows capillary selections obtained from both experts in the raw video pair for patient 02, region 2. [Diagram 53] FIG. 53 shows capillary selections obtained from both experts in the raw video pair for patient 03. [Figure 54] FIG. 54 shows capillary selections obtained from both experts in the raw video pair for patient 04. [Figure 55] FIG. 55 shows capillary selections obtained from both experts in the raw video pair for patient 05. [Figure 56] FIG. 56 shows capillary selections obtained from both experts in the raw video pair for patient 06. [Figure 57] FIG. 57 shows capillary selections obtained from both experts in the raw video pair for patient 07. [Figure 58] FIG. 58 shows capillary selections obtained from both experts in the raw video pair for patient 08. [Figure 59] FIG. 59 shows capillary selections obtained from both experts in the raw video pair for patient 09. [Figure 60] FIG. 60 shows capillary selections obtained from both experts in a pair of raw videos for patient 10. [Figure 61]61A-61B illustrate the classification results for video segments of a particular duration. FIG. 61A is the area under the curve (AUC) of classification as a function of video segment duration, where the video segment starts at the very beginning of the full video. It can be seen that the classification performance remains optimal and equal to that based on the full one minute video for at least 29 seconds of duration, i.e., nearly half of the original one minute duration. FIG. 61B illustrates the classification AUC when using different 29 second video segments when varying the start time of each video segment in the full one minute video, and the resulting AUC values ​​are nearly constant, demonstrating the stability of the classification results. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Presented below is a more detailed description of various concepts and implementations related to systems, devices, and methods for non-invasive hematology measurements. It should be understood that the various concepts introduced above and discussed in more detail below can be implemented in numerous ways. Examples of specific implementations and applications are provided primarily for illustrative purposes to enable those skilled in the art to practice implementations and alternatives that are apparent to them.

[0014] The figures and implementation examples described below are not meant to limit the scope of the implementation of the present application to a single embodiment. Other implementations are possible by replacing some or all of the elements described or illustrated. Furthermore, when a particular element of the disclosed exemplary implementation can be partially or completely implemented using known components, in some cases, only some of such known components necessary for understanding the implementation of the present application are described, and detailed descriptions of other parts of such known components are omitted so as not to obscure the implementation of the present application.

[0015] In view of the challenges associated with conventional blood testing as discussed above, the present inventors have recognized and appreciated the various advantages of non-invasive, in vivo techniques for deriving WBC counts that would allow for more frequent monitoring, fewer clinic visits, and immediate access to testing in areas lacking adequate laboratory facilities or reagent supplies.

[0016] A non-invasive method for counting WBCs that utilizes the optical properties of WBCs is to count WBCs in the nailfold capillaries. , as gaps in retinal capillaries, or as moving particles in oral mucosal capillaries. However, specialized and / or non-portable devices (e.g., adaptive optics and confocal microscopes) may be required to derive WBC counts. An optical device called a capillary microscope can be used to obtain optical images of the morphology of nailfold capillaries to diagnose rheumatic diseases, but the images obtained require time-consuming analysis by trained human assessors.

[0017] Overview of Non-Invasive In Vivo Analysis of Blood Cell Dynamics 1 is a flow chart illustrating a non-invasive in vivo method of analyzing blood cell dynamics according to some embodiments, including but not limited to detecting the number of WBCs passing through a given capillary over a period of time, one or more velocities of WBCs flowing through said capillary, and the total number of WBC events per μL. In some embodiments, some or all aspects of method 100 may be implemented by one or more of the systems, apparatus, and devices as described herein, such as system 2300 and / or device 2340 as described in more detail with respect to FIG.

[0018] More specifically, the method 100 illustrated in FIG. 1 includes a step 110 of normalizing and / or registering a source image of an object containing capillaries. Normalizing a source image (also called contrast widening, histogram widening, or dynamic alignment widening) may change the range of pixel intensity values, for example to increase the contrast of the image. Registering the source images may align the source images such that the same pixel locations on different images correspond to the same physical locations on the object. In some embodiments, the source images are taken within a finite range of time during which the camera, the object, or both may be moving. Image registration may account for such movement. Image registration may use one of the images (e.g., the first image in the sequence, the last image in the sequence, or an image that may contain a desired field of view) as a reference image, and other images in the sequence may be corrected relative to the reference image. Correction may be performed to maximize a particular image similarity metric that quantifies the similarity between the reference image and other images. Examples of image similarity metrics include, for example, cross-correlation, mutual information, sum of squared intensity differences, and image uniformity ratio.

[0019] In step 120 of the method 100, capillary segmentation and / or profile extraction is performed. Capillary segmentation may be used to identify capillary segments in either the original image or the registered image (e.g., by identifying the boundaries of the capillary segments). Profile extraction may be used to extract pixel information within the capillary segments for further analysis. Since WBC information is usually only contained in the capillaries, it may be useful to extract pixel information within the capillary segments and exclude information in other regions of the image. The extracted pixel information may include the position and value (also called intensity) of the pixel within the capillary segment. The position of the pixel in each image may be represented in 2D Cartesian coordinates, and the capillaries may be curvilinear. Therefore, the position may be useful to transform the image from the 2D Cartesian coordinate system to a different coordinate system, where the same point in the same capillary but on different images in the image sequence has the same coordinates. One example of such a coordinate system is a curvilinear coordinate system that uses a point in a curved capillary as the origin, and any other point has a one-dimensional (1D) coordinate that is the distance between that point and the origin.

[0020] In step 130 of method 100, the profiles extracted from the sequence of images are compiled into a single image (also called a spatiotemporal profile) for analyzing WBC events. The profile extracted from each image is used to identify WBC events or other events of interest. The profiles extracted from different images in an image sequence, taken at different time points, may contain information about the spatial distribution (i.e., location) of WBC events at different time points. Thus, a spatiotemporal profile compiled from all of these extracted profiles from the entire image sequence may provide a wealth of information related to both the spatial distribution and the temporal evolution of WBC events. For example, the trace of a particular WBC may be visualized from said spatiotemporal profile. Also, the velocity of a particular WBC flowing through a capillary may be derived, for example, by considering the time difference between two or more images in the image sequence.

[0021] In step 140 of method 100, the spatiotemporal profile is processed to detect and / or analyze WBC events. In some embodiments, the processing is manual. A user may identify WBC events by inspecting the spatiotemporal profile and, for example, detecting visual gaps in the spatiotemporal profile (e.g., pixels having high or low pixel values ​​compared to surrounding pixels). A user may also derive motion traces and flow velocities from the spatiotemporal profile. In some embodiments, the processing is automatic or semi-automatic. For example, the spatiotemporal profile may be transformed, for example, into the Radon domain. WBC events may then be detected based on local maxima in the Radon domain. Based on the detected WBC events, additional analyses such as WBC counts and WBC flow velocities may be derived. Furthermore, the WBC counts, flow velocities, etc. may be used for subsequent diagnosis, monitoring, and / or treatment of a disease or condition.

[0022] According to some embodiments, the systems, devices, and methods for analysis of blood cell dynamics are based on in vivo images of capillaries without blood sampling or other invasion of the subject's body and without removing the WBCs from their natural environment in the subject's body. Moreover, these systems, devices, and methods can be used for real-time or substantially real-time results. For example, processing of images (from source images to detection of WBC events and calculation of WBC counts) can be performed while taking new images. WBC events may be monitored to examine the subject's body's response to medical treatment, thereby providing feedback on the efficiency of treatment. Furthermore, the systems, devices, and methods herein can identify WBCs from relatively low-resolution, low-frame-rate, noisy source images. For example, the source images can be frames of video clips taken by off-the-shelf cameras, mobile phones, or other image-taking devices.

[0023] Methods for non-invasive in vivo analysis of blood cell dynamics 2 is a flow chart illustrating a method for analyzing blood cell dynamics using Radon transform of a spatiotemporal profile of a capillary image, according to some embodiments. Method 200 is used to detect WBC events from in vivo capillary data associated with a subject. In some embodiments, some or all aspects of method 200 can be implemented by one or more of the systems, apparatus, and devices as described herein, such as system 2300 and / or device 2340 as described in more detail with respect to FIG.

[0024] At step 210 of method 200, capillary data is non-invasively obtained. The capillary data may include a plurality of images of one or more capillary segments captured over a first period of time.

[0025] At step 220 of method 200, contours of one or more capillary vessel segments in the images are designated. In particular, for each image of the plurality of images, a first set of two-dimensional (2D) coordinates may be designated that correspond to interior contour points of capillary vessel segments that are visible in the image. Step 220 may also include specifying a second set of 2D coordinates corresponding to outer contour points of the capillary segments. These sets of 2D coordinates for each image may define the boundaries of the capillary segments in the images.

[0026] In step 230 of method 200, each of the first and second 2D coordinate sets are interpolated to generate a first and second set of resampled coordinates, respectively. This interpolation may fill in potential gaps between adjacent contour points specified in step 220, and therefore define smoother boundaries of one or more capillary segments.

[0027] In step 240 of method 200, a plurality of intermediate curves are generated based on the first set of resampled coordinates and the second set of resampled coordinates. The intermediate curves may be within a capillary segment defined by the interior and exterior contour points. The plurality of intermediate curves may include a median curve, which may be used to define a plurality of curve distances, as in step 250 of method 200.

[0028] In step 260 of method 200, a plurality of intensity values ​​are extracted from a plurality of images. Each extracted intensity value corresponds to one of the plurality of images, one of the plurality of intermediate curves, and one of the plurality of curve distances. That is, each extracted intensity value may be indexed by a vector that includes three values ​​representing (1) a particular image, (2) a particular intermediate curve, and (3) a particular curve distance.

[0029] In step 270 of method 200, the extracted intensity values ​​are transformed into the Radon domain. In step 280 of method 200, multiple maximum locations in the Radon domain correspond to flow trajectories in the capillaries, such that visual gaps in the flow trajectories in the capillaries indicate WBC events.

[0030] Capillary data collection According to some embodiments, capillary data (e.g., source images) for analysis of blood cell dynamics may be obtained from a variety of subjects, including, but not limited to, humans and other mammals. In some embodiments, capillary data is collected or captured from one or more locations on or within the subject's body. For example, capillary data may be collected and / or captured from nailfold capillaries, retinal capillaries, and / or oral mucosal capillaries.

[0031] The source images may be captured using a variety of methods. For example, the source images may include a sequence of images extracted from a video clip. Each image in the sequence of images may be captured at a different time such that analysis of blood cell dynamics may include, for example, the flow velocity of WBCs.

[0032] In some embodiments, a location on or within the subject's body is illuminated by a pulsed light source and a source image is captured by a camera synchronized with the pulsed light source. For example, the location may be illuminated by a pulsed laser of a particular wavelength (e.g., blue light at about 430 nm) at which the WBC (or other object of interest) has good reflectivity, to improve the contrast of the resulting image.

[0033] In some embodiments, the source image includes one or more color images, such that each pixel of the source image includes three values, e.g., corresponding to red, green, and blue component values, respectively. In some embodiments, each pixel of the source image includes three values, e.g., 2, 4, 6, 7, 8, or 1024 gray levels, corresponding to 4, 16, 64, 128, 256, and 1024 gray levels, respectively. The source image may include one or more grayscale images, such as having a bit depth of 10 or 11.

[0034] In some embodiments, the capillary data (e.g., obtained in step 210 of FIG. 2) comprises a video acquired from a given subject in 24-bit RGB format. h ×N v ×N f The image is taken as a three-dimensional stack I of a sequence of image frames, represented as h and N v are the horizontal and vertical sizes of each image frame (also called the number of pixels), and N f is the total number of frames in the video data. Each pixel in the video data may be represented as I[k,l], which corresponds to an RGB vector (R[k;l], G[k;l], B[k;l]). The index k=(k1,k2) refers to the location of the pixel in a particular frame, and l refers to the frame index among multiple frames contained in the video data. R[k;l], G[k;l], and B[k;l] correspond to the red, green, and blue component values ​​of pixel I[k,l], respectively. For example, I[(15,20),5] refers to the pixel located at the 15th row and 20th column of the 5th frame of the video data.

[0035] FIG. 3 is an example of an image of a human nailfold that can be used in the methods described above for the analysis of blood cell dynamics in one or more nailfold capillaries. The images are frames taken from a movie of the nailfold capillaries, with a frame rate of r frames per second and S p In Fig. 3, despite the presence of white saturated regions 310 associated with non-ideal acquisition conditions, the relatively dark U-shaped capillary blood vessel 300 profile is easily discernable in the image.

[0036] Preprocessing of capillary data Some pre-processing steps of the images in the capillary data may be performed to facilitate subsequent processing and analysis. These pre-processing steps may include image normalization and / or image registration. In some embodiments, some or all aspects of these pre-processing steps may be performed by one or more of the systems, apparatus, and devices as described herein, such as system 2300 and / or device 2340 as described in further detail with respect to FIG.

[0037] In some embodiments, the capillary data includes one or more grayscale images. Normalization may be used to compress and / or expand the gray levels to a desired range. In some embodiments, the capillary data includes one or more color images. Normalization may be used to convert the color images to grayscale images before compressing and / or expanding the gray levels. The conversion from color images to grayscale images may be accomplished through various methods. In one example, the resulting gray level of a color pixel is the sum of the red, green, and blue values ​​in the pixel. In another example, the resulting gray level is the difference between the red and green component values ​​(RG) to highlight the capillary structures. More advanced methods may be employed to calculate the gray level to highlight specific objects of interest (e.g., WBCs or red blood cells). For example, a specific weighted average of the red, green, and blue components may be calculated, or other types of nonlinear channel conversion (e.g., RGB to HSV (Hue, Saturation, Value) conversion).

[0038] In some embodiments, registration of the images includes one or more intensity-based methods that compare intensity patterns of the images via a correlation metric. The intensity-based methods can register entire images or sub-images. When sub-images are registered, the centers of corresponding sub-images can be treated as corresponding feature points.

[0039] In some embodiments, image registration includes one or more feature-based methods that find correspondences between image features such as points, lines, contours, etc. Feature-based methods may establish correspondences between one or more distinct points of the images. Knowing the correspondences between these distinct points in the images, a geometric transformation may be determined to map the target image to the other image, thereby establishing point-by-point correspondences between the reference image and the other image.

[0040] In some embodiments, the capillary data includes a 24-bit RGB format movie that includes frames such as the images in FIG. 3. In these embodiments, stack I includes a single channel version I n This version can be appropriately exploited for further profile segmentation and analysis. n [k;l] may be obtained by first averaging the RGB channels of I for each pixel point ("pointwise") and then normalizing the resulting intensity. This normalization reduces the intensity of I over any given frame l. n may be implemented so that the mean and standard deviation of the intensity values ​​of I are 0 and 1. n The frames that make up I can be registered, for example, to compensate for camera movement. n This can be achieved by applying the respective correction shifts to each frame l>1 of the matrix i, i.e., using the initial image (l=1) as a reference.

[0041] More specifically and first, a point-wise operation is performed to obtain a temporary stack.

number

[0042] Second, a threshold τ is specified for each separate frame of the sequence. c Binary thresholding stack using I t ' can be obtained, which can be determined experimentally, or using a priori information present in the image (e.g., total luminance, contrast, grayscale variance, distortion, etc.), or through feature learning obtained from a training dataset. In this step, the pixel value at position [k;l] of the temporal stack I'[k;l] is calculated by multiplying I'[k;l] by τ c If it is less than τ, it is set to zero and I'[k;l] is c If it is greater than or equal to, it is set to 1.

number

number

[0043] Third, the applied correction shift is I for l>1. t '[k;1] and I t '[k;1], we calculate the correlation (i.e., the normalized scalar product) between the 2D registration stack I r is calculated.

[0044] Segmentation of capillary profiles Step 220 of method 200 of FIG. 2 is performed according to some embodiments to achieve segmentation of the capillary profile, i.e., to identify segments of capillaries in the image. In some embodiments, the segmentation is performed on the source image (with or without pre-processing). In some embodiments, the segmentation is performed on the source image (with or without pre-processing). The segmentation is performed on the normalized and / or registered images. In some embodiments, the segmentation is performed on the source images after some other processing, such as noise filtering and / or contrast enhancement intended to highlight capillary structures.

[0045] In some embodiments, the segmentation of the capillaries is manual. The user may draw a line along the boundary of the capillaries that are visible in the image. For example, step 220 may be performed by the user. The user may mark some internal and external contour points on the image, and then the computer may call up the 2D coordinates of these marked internal and external contour points for further processing (e.g., resampling in step 230).

[0046] In some embodiments, the segmentation of capillaries is automated or semi-automated using methods such as thresholding and / or edge detection. Artificial intelligence methods may be used to train a computer-based pattern recognition model. In one example, the pattern recognition model may be based on historical images of capillaries so that the pattern recognition model can recognize similar capillary structures when new images are provided. In another example, the pattern recognition model may be based on one or more capillary structures in a portion of an image so that the pattern recognition model can recognize similar capillary structures in other portions of the image.

[0047] In some embodiments, capillary segmentation may involve a hybrid approach, where a user creates some initial and approximate indications of the capillary boundaries, and a computer-based program refines the initial indications using interpolation (e.g., linear, polynomial, or spline interpolation), curve fitting (e.g., using linear or nonlinear regression), and / or extrapolation (e.g., linear, polynomial, or conical extrapolation, or French extrapolation).

[0048] In some embodiments, the segmentation of each capillary is semi-automated with a minimum of user-defined 2D coordinates. For example, the user may define two sets of 2D coordinates P that follow the internal and external capillary boundaries defined for the first frame of stack I. int [j]P ext [j], j = 1, 2, …, N p where j is the index of the jth point on the contour, and N p is the total number of points specified by the user on each contour (interior or exterior). p The total number of points N may depend on several factors, including but not limited to the complexity of the capillary boundary and the length of the capillary. Fewer points may be specified for straight capillaries or portions of straight capillaries compared to capillaries with sharp turns. In some embodiments, the number of specified points N p can range from about 3 points to about 20 points. In some embodiments, the number of specified points N p can range from about 5 to about 10 points.

[0049] point P int [j] or P ext The value of [j] is the 3D stack I or I r Contains a vector that indicates the location of this point in the int [5] refers to the fifth point on the internal contour of the capillary. int The value of [5] may be, for example, (20, 30), which marks the 2D location of the point on the image, i.e., the 20th row and 30th column. After obtaining the 2D location of this point on the curve, we then compute the pixel value (or intensity) of the pixel in image stack I or I r In this way, the index on the curve (j), the pixel position (k), and the pixel value are one-to-one related to each other.

[0050] Resampling of capillary contour points The designated contour points, e.g., in step 220 of method 200, are typically sparse and may be, for example, specific characteristic points (e.g., start points, end points) on the inner and outer boundaries of the capillaries. , or bending points). Thus, in step 230 of method 200, the designated interior and exterior contour points are resampled to further refine the boundary for further processing by some embodiments.

[0051] At least one of three methods and / or a combination thereof may be used to resample the specified contour points. In some embodiments, interpolation (e.g., linear, polynomial, or spline interpolation) is used to generate new data points between two adjacent specified points to fill the gap. In some embodiments, curve fitting (e.g., using linear or nonlinear regression) is used to fit the specified contour points to a curve, which generally has an analytical expression and may include any number of data points for further processing. In some embodiments, extrapolation (e.g., linear, polynomial, conic, or French extrapolation) is employed to generate projected points beyond the curve section defined by the specified contour points. In further embodiments, a combination of two or more of these methods is used.

[0052] In some embodiments, the denser points P' int and P' ext The set of contour points is generated using cubic spline interpolation of the original points with a resampling factor α. Therefore, the total number of contour points after resampling is α(N p −1)+1, where N pis the number of specified contour points. The resampling factor α may depend at least in part on the desired resolution of the resampled contour, i.e., the distance between two adjacent resampled contour points (also called point spacing). For example, when WBC events are detected, it is preferable to set the point spacing smaller than the size of the WBC in order to resolve each WBC in subsequent processing. In some embodiments, the point spacing may be substantially similar to the pixel size in the source image of the capillary data.

[0053] In some embodiments, resampling is performed directly on the 2D coordinates specified in step 220. For example, a 2D spline (also called bicubic interpolation) may be used to resample the specified contour points. In some embodiments, the point resampling operation in step 230 includes separate 1D resampling of corresponding horizontal and vertical coordinate sequences (also called row and column sequences, respectively). FIG. 4A is a plot of cubic spline interpolation of x-coordinates, and FIG. 4B is a plot of cubic spline interpolation of y-coordinates. In FIGS. 4A-4B, the contour points specified in step 220 are marked as crosses (e.g., cross 400), and the resampled contour points connect the crosses. Note that the resampled data points appear as a continuous curve (e.g., apparent curve 410) due to their large number.

[0054] Defining intermediate curves and curve distances In step 240 of method 200, a set of intermediate curves is defined between the inner and outer contours of each identified capillary by some embodiments. In some embodiments, the set of intermediate curves is evenly distributed in the space defined by the inner and outer contours. In some embodiments, the distribution of intermediate curves is non-uniform. For example, a central portion of a capillary may have a higher density of intermediate curves compared to an edge portion of the capillary.

[0055] In some embodiments, a series of N cIntermediate curves P' m [j] is generated by a linear combination of the following form:

number

[0056] Total number of intermediate curves N c may depend on the desired resolution of the collection of intermediate curves, i.e., the distance between two adjacent intermediate curves (also called curve interval). For example, when a WBC event is detected, the curve interval may be smaller than the size of the WBC in order to resolve each WBC in subsequent processing. In some embodiments, the curve interval may be substantially similar to the pixel size in the source image of the capillary data.

[0057] Resampling the contour points in step 230 can be viewed as improving the longitudinal resolution (i.e., the resolution along the direction of the capillary) from the designated contour points in step 220, while creating the intermediate curve can be viewed as improving the lateral resolution (i.e., the resolution along the cross section of the capillary) based on the two boundaries of the capillary (i.e., the inner and outer contours).

[0058] In some embodiments, step 230 is performed before step 240, so that intermediate curves can be created based on the resampled contour points obtained in step 230. In some embodiments, step 240 is performed before step 230. For example, a series of intermediate points can be created between a user-specified point on the inner contour and a user-specified point on the outer contour to fill gaps along the cross-section of the capillary. The same step may be performed for each pair of user-specified points, thereby generating a set of intermediate contour points between the inner and outer contour points. A resampling step as in step 230 may then be performed for each set of intermediate contour points, and the intermediate contour points may be stitched together, thereby generating a set of intermediate curves.

[0059] In step 240, the central curve P' mid can be defined as follows:

number

[0060] FIG. 5 shows normalized and registered image frames I, II, II, III, and IB taken from a video clip containing capillaries 500 and 510. r Both capillaries are highlighted in the processed image along with the corresponding user-defined contour points (shown as crosses) and the interpolation points (shown as curves) that follow the inner and outer curves of each profile. The central curve P' from which the curve distance is calculated midNote that the user specified only six points, and the resampled curve already appears to match the actual boundaries of the two capillaries quite well.

[0061] Extracting the intensity profile In step 260 of the method 200, the intensities (i.e., pixel values) of pixels within the capillary segment are extracted as a function of curve distance to form a spatiotemporal profile f[j;m;l], where j is the curve index (jth point on a curve), m is the curve index (mth curve among multiple curves including the inner boundary, the outer boundary, and intermediate curves generated in between), and l is the frame index (lth image in a sequence of images). More specifically, I r Each P' above m [j] is associated with the intensity value. In some embodiments, bilinear interpolation is used to find the sub-picture at a given curve index j, curve m, and frame l. Calculate the resulting intensities f[j;m;l] with xcel precision.

[0062] In some embodiments, given f[j;m;l], the spatiotemporal profile average f′[j;l] (i.e., from 1 to N c The average of the capillary intensities of all curves can be calculated for every frame and curve distance to produce a single curve (averaged over all m up to m). As an example, this averaging step can be thought of as collapsing all capillary curves (inner contour, outer contour, and intermediate curves) into a single curve.

[0063] In some embodiments, the spatiotemporal profile average f'[j;l] is resampled to generate a resampled spatiotemporal profile g[j;l] that contains the same number of samples as f'[j;l] but has equidistant curve intervals. This resampling may be performed using linear interpolation.

[0064] Spatiotemporal profile analysis The profile g[j;l] may include an intensity profile that evolves as a function of time for each point j along the curvilinear distance of each capillary. This profile may exhibit correlations in space and time that may be extracted to overcome possible challenges in analyzing noisy source images taken at relatively low frame rates and / or with low resolution.

[0065] In some embodiments, a median-normalized version of g is calculated to highlight the intensity variations created by WBC events. More specifically, the median of the time-varying intensity line of g[j;l] may first be set to zero at every curvilinear coordinate j. The same operation may then be performed column-wise at every frame l. To reduce noise, the contrast-normalized profile g' may be filtered along the time axis with a symmetric unit-sum rectangular function of size σ, which yields g". The filtering operation may be performed through convolution with said rectangular function.

[0066] FIG. 6A shows an example of a spatiotemporal profile g″ of the capillary 500 obtained from FIG. 5. The spatiotemporal profile g″ is a function of the original spatiotemporal profile f[j;m;l], which is a function of the spatiotemporal profile g″ ranging from 1 to N cThe spatiotemporal profiles were generated after averaging over all m up to m (to achieve f'[j;l]), resampling to have equidistant curve intervals (to achieve g[j;l]), normalizing by the median (to achieve g'[j;l]), and filtering (to achieve g"[j;l]). On the spatiotemporal profile shown in FIG. 6A, spatial trajectories corresponding to WBC events can be identified. FIG. 6B illustrates these trajectories by dashed lines spanning the 2D intensity plot. The slope of each line can be related to the average velocity of the corresponding WBC events in the capillary. Therefore, spatiotemporal profiles such as those shown in FIG. 6A-B already provide important information about the WBC events in the capillaries, such as the number of events and the flow rate. This information can be extracted manually, automatically, or semi-automatically using the methods described above.

[0067] Radon transform In step 270 of method 200, some embodiments perform a Radon transform on the spatiotemporal profile generated in step 260. The Radon transform converts 2D lines into peaks located at specific locations to identify events and their associated parameters. Event detection using the Radon transform is convenient as well as robust to noise.

[0068] Without being bound to any particular theory or mode of operation, Given f(x), where x is the 2D Cartesian coordinate defined relative to the center of the image f(x),

number

number

[0069] In some embodiments, f(x) is a discrete image, in this case a discrete domain Radon transform R, which includes a finite amount of radial coordinate values ​​and angles in [0;π]. 0 The maximum radial coordinate value may correspond to half the length of the image diagonal. Hence, the profile in the Radon domain

number

number

[0070] FIG. 7 illustrates the application of the Radon transform on the spatiotemporal profile shown in FIG. 6A-6B according to some embodiments. The projection angle θ and radial coordinate R are shown in FIG. 7. The solid arrow illustrates the direction of trajectory line detection. Given a given set of angles θ and radial coordinate R, the Radon transform calculates the projection of the spatiotemporal profile intensity (i.e., the intensity in the non-transformed domain as shown in FIG. 7). More specifically, the Radon transform domain value (see FIG. 8A) for a given angle θ and radial coordinate R corresponds to the integral of all values ​​(i.e., the intensity values ​​in the non-transformed domain as shown in FIG. 7) found on a line with counterclockwise angular tilt θ with respect to the horizontal axis and radial coordinate R.

[0071] FIG. 8A illustrates a Radon transform of the map of FIG. 7 according to some embodiments.

number

[0072] FIG. 8B shows the space-time distribution in the Radon domain with the identifying peaks highlighted in circles. Figure 1 shows a spatiotemporal profile of the WBC event in the capillary. The original physical time and velocity parameters of the WBC events in the capillary can be analytically estimated from the maximum position based on the known frame rate and pixel size via elementary trigonometric relations. More specifically, the velocity is related to the tangent of the projection angle θ in the Radon domain and is therefore proportional to the longitudinal gradient of the corresponding trajectory line in the aforementioned spatiotemporal profile g". The original physical time is related to the intersection between said trajectory line and the time axis of said spatiotemporal profile. In some embodiments, the time of each WBC event corresponds to the first frame where a visual gap appears in the capillary.

[0073] 9A shows experimental results of WBC events occurring in the capillary 500 shown in FIG. 5, and FIG. 9B shows experimental results of events occurring in the capillary 510 shown in FIG. 5, according to some embodiments. The experimental results are compared to manual counts performed by four trained human assessors. In FIG. 9A-9B, WBC events detected by the method described above are marked by crosses, whereas event times estimated by the trained human assessors are indicated by dashed vertical lines. The horizontal lines represent the limits associated with maximum and minimum blood flow.

[0074] The source images are taken from video clips of human nail folds. The acquisition parameters are N h =1280,N v =960, N f = 450, r = 15, and S p = 0.65 μm and the duration of the video clip is 30 s. c is heuristically set to 3 / 4 of the maximum intensity value for each frame l in I'. The amount of user-specified curve points is N p = 6, which is the amount that can produce an accurate segmentation. The number of interpolated curves is N c = 10. The size of the filter used for noise reduction is set to σ = 3. Finally, the window size and threshold used for finding the maximum in the Radon domain are S w = 11 and τ m = 7, and the number of Radon angles is N θ =400.

[0075] 9A-9B, over 80% of the WBC events detected by the methods described above are consistent with those of trained human assessors. Missed events may represent temporally adjacent visual gaps that may be difficult for automated methods of interpretation due to poor frame rates.

[0076] Figure 10 shows a further comparison between the total WBC event counts obtained from the method described above according to some embodiments and those obtained from four trained human assessors. Figure 10 also shows that the variability of manual counts between observers is significant. Each capillary is associated with a box whose center mark is the median. The edges of the box are the 25th and 75th percentiles, and the whiskers extend to the data endpoints. The crosses and circles indicate the counts completed by each of the four trained human assessors and by the approach described herein, respectively. The results obtained from the described method are shown in these box plots to be within the range of human inter-rater variability.

[0077] The slope of the visual gap trajectory in Figures 6A-7B can be employed to calculate the velocity associated with the WBC events for both capillary 500 and capillary 510. The results depicted in Figures 9A-9B are within the range of human nailfold capillary blood flow values ​​previously reported in the literature, i.e., from about 450 μm / sec to about 1200 μm / sec.

[0078] Interpolated inner and outer capillary curves P int [j], P ext [j] can be employed to derive the vessel radius r, which corresponds to approximately 7.5 μm for both capillary 500 and capillary 510, a value consistent with previously published data.

[0079] Without being bound to any particular theory or mode of operation, capillaries may be assumed to have a circular cross section. Considering the average velocity v derived above to be approximately 600 μm / sec, the total blood volume sampled per second V may be determined as follows:

number

[0080] Considering a healthy WBC range of about 3500 WBC / μL to about 9000 WBC / μL, and assuming a video clip duration of 30 seconds, the number of WBC counts c can be determined as follows:

number

[0081] Apparatus and system for non-invasive in vivo analysis of blood cell dynamics - Patents.com 11A-11B are schematic diagrams of a system for detecting white blood cell (WBC) events in a non-invasive manner in vivo, according to some embodiments. In some embodiments, some or all aspects of the system of FIGs. 11A-11B may be structurally and / or functionally similar to one or more of the systems, apparatus, and devices as described herein, such as system 2300 and / or device 2340 as described in more detail with respect to FIG.

[0082] FIG. 11A is a side view of the system, and FIG. 11B is a top view of the system. For illustrative purposes, the embodiments described herein analyze WBC events in a human nailfold, but these embodiments may be modified for analysis in other subjects and / or other locations in or on a subject's body. In FIGS. 11A-11B, the system 1100 includes a finger holder 1110 having a finger hole 1112 for receiving at least a portion of a human finger (e.g., a portion of the finger nailfold). The finger holder 1110 is also configured to receive an imager 1120, which is in optical communication through a transparent window 1114 with the finger received by the finger hole 1112 to capture an image or video of the finger. The imager further includes an optical device 1122 for collecting light reflected or scattered from the finger and a detector 1124 to receive the reflected or scattered light to form an image of the finger. System 1100 further includes a processor 1130 operatively coupled to processor 1120, and a memory 1140 operatively coupled to processor 1130. Memory 1140 is coded with processor-executable instructions that, when executed by processor 1130, may perform the methods described above to analyze images received from imager 1120. System 1100 also includes a display 1150, which may display images or videos captured by imager 1120 and / or data related to WBC events detected by processor 1130.

[0083] In some embodiments, the finger holder 1110 has an igloo shape (as shown in FIGS. 11A-11B) so that a hand can be placed on the dome of the finger holder while the fingers of the hand are received in the finger holes 1112 for imaging. The finger holder 1110 may have other configurations, such as a flat top, a handle configuration (where the hand can grasp the handle while the fingers can be imaged), or any other configuration known in the art.

[0084] In some embodiments, the system 1100 also includes an illumination source 1160 (shown in FIG. 11B ) that illuminates the finger to facilitate capturing an image by the imager 120. In some embodiments, the illumination source 1160 includes a pair of light emitting diodes (LEDs), each of which is disposed on one side of the finger hole 1112. In some embodiments, the illumination source 1160 is configured to emit monochromatic light. The capillary structure of the finger may have a high reflectivity at the wavelength of the monochromatic light.

[0085] 11C shows a schematic diagram of a system 1100 substantially similar to the system shown in FIGURES 11A-11B, according to some embodiments. The system 1100 includes an adjustable transparent window 1115 disposed between a finger hole 1112 and an imager 1120. By adjusting the height of the window 1115, images of the finger may be captured using imagers 1120 having different sizes.

[0086] 12A-12B are schematic diagrams of a system for capturing in vivo images for non-invasive analysis of WBC events, according to some embodiments. In some embodiments, some or all aspects of the system of FIGs. 12A-12B may be structurally and / or functionally similar to one or more of the systems, apparatus, and devices as described herein, such as system 2300 and / or device 2340 as described in more detail with respect to FIG.

[0087] FIG. 12A is a side view of the system 1200, and FIG. 12B is a top view of the system 1200. The system 1200 includes a finger holder 1210 having a finger hole 1212 for receiving at least a portion of a human finger (e.g., a portion of the finger nail crease). The finger holder 1210 is also configured to receive an imager 1220, which is in optical communication with the finger received by the finger hole 1212 through a transparent window 1214 and a mirror 1226 to capture an image or video of the finger. The imager further includes an optical device 1222 for collecting light reflected or scattered from the finger and a detector 1224 to receive the reflected or scattered light to form an image of the finger. In the system 1200, the imager 1220 is disposed in a vertical configuration. The mirror 1226 is configured to reflect an image of the finger toward the imager 1220. In some embodiments, the mirror 1226 is coupled to an actuator (not shown) that can tilt the mirror 1226 in two directions. This tilt can be useful to aim the imager 1220 at a desired portion of the finger for imaging. Similar to system 1100, system 1200 can also include an illumination source 1260 to facilitate image capture by the imager 1220.

[0088] 13 is a schematic diagram of a system for performing in vivo non-invasive analysis of blood cell dynamics using a camera-mounted device (e.g., a smartphone), according to some embodiments. In some embodiments, some or all aspects of the system of FIG. 13 may be structurally and / or functionally similar to one or more of the systems, apparatus, and devices as described herein, such as system 2300 and / or device 2340 as described in more detail with respect to FIG.

[0089] The system 1300 includes a finger holder 1310 having a finger hole 1312 for receiving at least a portion of a finger for imaging. The finger holder 1310 is also configured to receive a smartphone 1320 such that a camera within the smartphone 1320 is in optical communication with the finger in the finger hole 1312 via a transparent window 1315 and a mirror 1326. The system 1300 may also include modified optics 1322, which may modify the optical system of the smartphone 1320 camera to better accommodate image capture. For example, correction optics 1322 may include a lens for adjusting the focal length (optical zoom) of a camera in smartphone 1322. Because many smartphone cameras do not have optical zoom, lens 1322 may increase the optical versatility of the camera. The focal length of a camera is also related to the resolution of the image captured by the camera. Thus, lens 1322 may also be used to adjust the resolution of the image for further processing.

[0090] The smartphone 1320 generally includes its own memory and processor. The methods described in the earlier sections of this application can be coded in the memory of the smartphone 1320 as processor executable instructions. During operation, images captured by the camera can be transmitted to the processor for analysis of blood cell dynamics. In some embodiments, images captured by the camera are wirelessly transmitted (e.g., via Bluetooth, WiFi, 3G network, 4G network, or any other wireless communication protocol known in the art) to another processor for processing. In some embodiments, images captured by the camera are saved locally in the memory of the smartphone 1320.

[0091] 14A-14D are images of systems and devices for in vivo non-invasive analysis of blood cell dynamics, according to some embodiments. In some embodiments, some or all aspects of the systems and devices of FIGs. 14A-14D can be structurally and / or functionally similar to one or more of the systems, devices, and devices as described herein, such as system 2300 and / or device 2340 as described in more detail with respect to FIG.

[0092] FIG. 14A shows a finger holder coupled to an imager. The imager may be a commercially available capillary microscope, such as a Dino-Lite digital microscope available from BigC.com, Torrance, Calif. The imager may include its own illumination source so that when a finger is placed in the finger hole, the finger is sufficiently illuminated for imaging. FIG. 14B shows the imager and finger holder uncoupled. FIG. 14C shows the finger holder when a finger is placed in the finger hole. FIG. 14D shows a system including a finger holder, an imager, and a computer connected to the imager, for example, via a USB connection. The computer may display images or videos captured by the imager in real time. The methods described in the earlier sections of this application may be saved as processor executable instructions in the memory of the computer, and the processor may then perform analysis of the images to detect WBC events.

[0093] 15A-15G are diagrams of an adapter that can be attached to a smartphone for capturing images of a nailfold with a camera-mounted device for blood cell dynamics analysis, according to some embodiments. FIG. 15A is a perspective view of the adapter 1500, including a window section 1510 and a number of suction cups 1520. The window section 1510 can be aligned with a camera typically available on a smartphone. The suction cups 1520 can secure the adapter 1500, for example, to the smartphone. Although four suction cups are shown in FIG. 15A, the number of suction cups can be any number that is applicable. Alternatively or additionally, the adapter can be attached or secured to the smartphone or other device using clips, adhesives, or the like.

[0094] 15B shows adapter 1500 from an oblique angle to illustrate the structure of window section 1510, which further includes window 1512 and finger receptor 1514. Window 1512 is substantially transparent, or at least transparent at certain wavelengths, to allow a camera to capture an image of the finger. Finger receptor 1514 has an internal contour that conforms to the general shape of a finger to securely hold the finger in place with adapter 1500, thereby facilitating subsequent processing. This reduces the burden of image registration (e.g., due to movement) in

[0095] FIG. 15C is a top view of the adapter 1500, FIG. 15D is a first side view of the adapter 1500, and FIG. 15F is a second side view of the adapter 1500. FIG. 15E is a rear view of the adapter 1500 illustrating the illumination channel 1516 and the window 1512. FIG. 15G is a front view of the adapter 1500 (the side that receives the finger). As shown in FIG. 15E and FIG. 15G, the illumination channel 1516 generally does not extend through the entire depth of the adapter 1500. Instead, the illumination channel 1516 may be in optical communication with a flashlight commonly available in smartphones and other devices attached to the camera other than the camera lens. The illumination channel 1516 may receive light emitted by a flashlight and reflect it toward the window 1512 to illuminate at least a portion of the finger being imaged.

[0096] 15H-15J illustrate illumination channel 1516. FIG. 15H is a perspective view of adapter 1500. FIG. 151 is a perspective cross-sectional view of a portion of adapter 1500 to illustrate window 1512 and illumination channel 1516. FIG. 15J is a cross-sectional view of adapter 1500 to illustrate the structure of illumination channel 1516. In particular, illumination channel 1516 may include a curved surface 1517 that reflects received light toward window 1512 to illuminate a finger that is typically placed in front of window 1512 for imaging.

[0097] 16 is a schematic diagram of a system including a smartphone and an adapter for capturing nailfold images for blood cell dynamics analysis, according to some embodiments. In some embodiments, some or all aspects of the system of FIG. 16 may be structurally and / or functionally similar to one or more of the systems, apparatus, and devices as described herein, such as system 2300 and / or device 2340 as described in further detail with respect to FIG.

[0098] The adapter 1600 includes a window 1612 in optical communication with a camera 1632 of a smartphone 1630. The adapter 1600 also includes an illumination channel 1616 in optical communication with an LED light 1636 of the smartphone 1630. The illumination channel 1616 receives light emitted by the LED light 1636, and a curved surface 1617 within the illumination channel 1616 reflects the received light toward the window 1612 to illuminate a finger 1640 (more specifically, a nail crease 1642) that is in intimate contact with the window 1612.

[0099] In some embodiments, the window 1612 may include two lenses to change the focal length of the camera 1632 in the smartphone 1630. By changing the focal length, the resolution of the image captured by the camera 1632 may also be changed. In some examples, an index-matching fluid 1618 may be placed between the window 1612 and the claw fold 1642. The index-matching fluid 1618 may have a refractive index similar to the material of the claw fold or window 1612 to reduce specular reflection at the surface of the window 1612 or the surface of the claw fold 1642. In some embodiments, the index-matching fluid 1618 has a high viscosity. In some embodiments, the index-matching fluid 1618 may be replaced by a suitable reusable clear plastic.

[0100] FIG 17A shows a smartphone with an adapter attached to the smartphone to enable image capture of a nail fold according to some embodiments. FIG 17B shows the smartphone and adapter when a finger is placed in front of the adapter for imaging. FIG 18 shows an image captured by the smartphone using the adapter shown in FIG 17A-B. Multiple capillary structures are visible in the image for further analysis of blood cell dynamics according to some embodiments.

[0101] FIG. 19 is a schematic diagram of a clamping device for capturing images and performing blood cell dynamics analysis, according to some embodiments. The clamping device 1900 includes a spring-loaded casing 1910 for securely receiving a finger 1950. When the finger 1950 is placed in the clamping device 1900, the finger 1950 is illuminated by a pair of LED lights 1940a and 1940b, and the camera 1930 can capture images of the finger 1950. In some embodiments, the LED lights 1940a and 1940b emit broad spectrum white light. In some embodiments, the LED lights 1940a and 1940b emit green light that can be effectively reflected by WBCs. The clamping device may also include a battery and / or processing unit 1920 for processing images captured by the camera 1930 according to the methods described above. FIG. 20 shows the clamping device coupled to a finger for blood cell dynamics analysis. As can be seen from FIG. 20, the spring loaded casing helps secure the finger to the clamping device so as to reduce or eliminate relative movement between the finger and the camera.

[0102] 21A-21O are diagrams of an adapter for capturing images of a nail fold with a smartphone camera for capillaroscopy and hematology analysis, according to some embodiments. Specifically, FIGS. 21A-21F are images of a perspective view of the smartphone adapter. In FIGS. 21A-21E, the adapter is attached to a smartphone such that the smartphone camera is utilized. In FIG. 21F, the adapter is shown without being attached. FIGS. 21G-21O are wireframes showing side and perspective views of the adapter. FIGS. 22A-22D illustrate a method of using a smartphone adapter attached to a smartphone such that the smartphone camera is used for capillaroscopy and hematology analysis, according to some embodiments. In FIG. 22A, an index matching fluid is placed on the surface of a user's nail fold. In FIGS. 22B-22D, the smartphone is placed on a finger, such that the adapter is placed on the finger and the smartphone camera is aligned with the nail fold.

[0103] 23 is a schematic diagram of an environment / system 2300 in which non-invasive hematology measurements may be implemented and / or performed. In some embodiments, aspects of the system 2300 may be structurally and / or functionally similar to the systems, apparatus, and / or devices described herein with respect to FIGS. 1-22 and 31-60 and / or may perform the methods described in FIGS. 1-2.

[0104] The system 2300 includes a platform 2310, an illumination source 2330, an imaging device 2320, and a computing device 2340. In some embodiments, all components of the system 2300 can be included in a common casing, such as, for example, a single housing that presents the system 2300 as an integrated, all-in-one device for a user. In other embodiments, at least some components of the system 2300 can be in separate locations, enclosures, and / or devices. For example, in some embodiments, the computing device 2340 can be a smartphone that communicates with the illumination source 2320 and / or the imaging device 2330 via one or more networks, each of which can be any type of network implemented as a wired network and / or a wireless network, such as, for example, a local area network (LAN), a wide area network (WAN), a virtual network, a telecommunications network, and / or the Internet. As is known in the art, any or all communications can be heavily secured (e.g., encrypted) or unsecured. The system 2300 and / or the computing device 2340 may be or may include a personal computer, a server, a workstation, a tablet, a mobile device, a cloud computing environment, an application, or a module running on these platforms, and the like.

[0105] While described herein as a system for hematological measurements in a nail fold for ease of explanation, it is understood that aspects of the systems, devices, and methods disclosed herein are useful for hematological measurements in any tissue having capillary structures that can be imaged as described herein (e.g., capillary beds, superficial capillaries, peripheral capillaries, capillaries in other parts of a user's finger, etc.). Non-limiting examples include capillaries in the retina, earlobe, lips, gums, etc. As one example, the platform 2310 can be applied to press against a user's gum line during use for hematological measurements in the capillary bed of the gum line. As another example, the platform 2310 can include a tonometer-like instrument for pressing against the retina during use for hematological measurements in the capillaries of the retina.

[0106] In some embodiments, the platform 2310 receives a user's finger during use. In some embodiments, the platform 2310 can be structurally and / or functionally similar to the finger holders described in Figures 11-14 and 19. In some embodiments, the platform 2310 is shaped to guide the placement of a user's finger to position a portion of the finger's nail frame at a predetermined location within the platform.

[0107] The illumination source may be any suitable source of substantially monochromatic light, including, but not limited to, a light emitting diode (LED), laser light, filtered light, etc. The illumination source may be positioned relative to the platform 2310 to illuminate a portion of a user's fingernail fold during use.

[0108] The imaging device 2330 may include any suitable imager disclosed herein, including a smartphone camera, a capillaroscope, etc. In some embodiments, the imaging device 2330 captures an image of a portion of the nail fold of a user's finger during use, e.g., in response to illumination of the portion of the nail fold by the illumination source 2330, e.g., based on a synchronization / timing signal from the device 2340. In some embodiments, the imaging device 2330 captures a set of images per acquisition, e.g., a series of time-lapse images, etc. In some embodiments, the imaging device 2330 captures the set of images as a 60 second movie / movie file, e.g., at a capture rate of 60 frames / second.

[0109] In some embodiments, the platform 2310 is optically coupled to the illumination source 2330 and the imaging device 2330 via any suitable independent means, such as, for example, beam shaping and / or beam steering optics, one or more optical conduits such as optical fibers, direct / optics-free coupling, etc.

[0110] The computing device 2340 includes at least a controller 2350 and a memory 2360. FIG. 23 also illustrates a database 2370, although it will be understood that in some embodiments, the database 2370 and the memory 2360 may be a common data store. In some embodiments, the database 2370 comprises one or more databases. Additionally, in other embodiments (not shown), at least one database may be external to the device 2340 and / or the system 2300. The computing device 2340 may also include one or more input / output (I / O) interfaces (not shown) implemented in software and / or hardware with other components of the system 2300 and / or external to the system 2300 for interacting with the device 2340.

[0111] The memory 2360 and / or the database 2370 may be independently configured, for example, as a random access memory (RAM), a memory buffer, a hard drive, a database, an erasable and may be a programmable read only memory (EPROM), an electrically erasable read only memory (EEPROM), a read only memory (ROM), a flash memory, etc. The memory 2360 and / or the database 2370 may store instructions to cause the controller 2350 to perform processes and / or functions associated with the system 2300.

[0112] The controller 2350 may be any suitable processing device configured to run and / or execute an instruction set or code associated with the device 2340. The controller 2350 may be, for example, a general purpose processor, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a digital signal processor (DSP), etc.

[0113] The controller 2350 receives a set of image / video data from the imaging device 2330. The controller 2350 can execute computer-executable instructions (e.g., stored in the memory 2360 and / or the database 2370) to process the image set in any suitable manner. In some embodiments, given a raw input video / image set, e.g., with a resolution of 1280x1024 pixels and a frame rate of 60 frames / images per second, and given an initial frame index of the input, the controller 2350 can extract two uncompressed videos, limited in duration to one minute (3600 frames) starting from the initial frame index. In some embodiments, the first video at the native pixel resolution is downsampled, e.g., by half the pixel resolution (640x512), to accelerate processing during registration and capillary detection, as described herein. And the uncompressed image is extracted as the first frame / image of the second video.

[0114] In some embodiments, the controller 2350 performs a global registration process on the image set to remove motion that occurs between frames / images. For example, in some embodiments, the controller 2350 aligns all images with respect to the first image of the image set. In some embodiments, the alignment includes correcting for horizontal and / or vertical translation of the view capture of the image set. In some embodiments, the controller 2350 corrects the horizontal and / or vertical translation for each image in a manner that maximizes its correlation with the first image.

[0115] In some embodiments, prior to such an alignment process, the controller 2350 normalizes the intensity of each image in the image set based on predetermined characteristics (e.g., a square filter of size 75x75 pixels) and / or any other suitable image flattening technique, e.g., by a spatial high-pass filter. In this manner, the resulting processed image set (sometimes simply referred to as the "image set") is cropped to a viewing region that remains in view throughout the image set.

[0116] The controller 2350 can execute computer-executable instructions (e.g., stored in memory 2360 and / or database 2370) to detect one or more capillaries in a portion of the nail fold in each image of the processed image set and identify a first capillary set across the image set. In some embodiments, detecting the first capillary set includes detecting each capillary of the first capillary set on at least one image of the image set.

[0117] In some embodiments, the controller 2350 detects each capillary of the first set of capillaries as follows: The controller 2350 determines whether or not the capillary is at least 100% full for one or more attributes of the identified capillary (e.g., whether or not the capillary is at least 100% full for at least a first attribute of the one or more attributes). ), and receive training data including images of capillaries that meet one or more predetermined criteria. In some embodiments, the one or more attributes include structural attributes of the capillaries themselves, such as, but not limited to, capillary length (e.g., long enough that a cellular event moving at a typical or maximum blood flow velocity would appear in more than one frame, given the frame rate of the imager / imaging device), capillary width (e.g., capillary width from about 10 μm to about 20 μm, including all values ​​and subranges therebetween), capillary depth (e.g., capillary depth from about 10 μm to about 350 μm, including all values ​​and subranges therebetween), average capillary diameter, capillary transverse diameter, capillary longitudinal diameter, capillary shape (e.g., capillaries should exhibit distinct arterial and venous limbs), etc.

[0118] The average capillary diameter can generally be characterized as any average of multiple diameter measurements of a capillary at a single cross section or at multiple cross sections along the length of the capillary. In some embodiments, the structural attribute is the average capillary diameter, and the predetermined criteria is that the average capillary diameter should be between about 10 μm and about 20 μm.

[0119] In some embodiments, the one or more attributes include flow attributes, such as, but not limited to, blood flow velocity in the capillaries, transit time of cells in the visible portion of the capillaries, volumetric flow rate, mass flow rate, flow directionality, blood flow velocity stability, etc.

[0120] In some embodiments, the one or more attributes include imaging attributes of the capillary images, such as, but not limited to, contrast (e.g., brightness contrast, root mean square (RMS) contrast, Weber contrast, Michelson contrast, histogram-based methods, etc.), focus / detail (e.g., as measured by gradient-based operators, Laplace operators, wavelet operators, discrete cosine transform (DCT), frequency domain analysis, phase coherence, intensity maps, singular value decomposition, learning algorithms, etc.), signal-to-noise ratio, image stability (e.g., as measured by image registration methods, optical flow, etc.). In some embodiments, a combination of structural attributes and imaging characteristics may be employed. In some embodiments, the training data is data generated by human experts that also accounts for the requirement that capillaries are generally distinct (e.g., free of air bubbles blocking the capillaries) and have distinct morphology (e.g., have distinct arterial and venous limbs). The controller 2350 trains a deep learning neural network (e.g., a fully convolutional neural network such as the deep learning YOLO method / technique as generally disclosed in Joseph Redmon et al. 2016, The IEEE Conference on Computer Vision and Pattern Recognition, the entire disclosure of which is incorporated herein by reference) based on the training data to recognize the first set of capillaries in the image set.

[0121] In some embodiments, the controller 2350 generates for each detected capillary in each image of the image set a bounding box around the capillary and a confidence value associated with the likelihood that the detection corresponds to a capillary and not to another structure / artifact. In some embodiments, the first capillary set includes those detected capillaries that have corresponding confidence thresholds that meet or exceed a predefined confidence threshold.

[0122] In some embodiments, the controller 2350 identifies a second set of capillaries from the first set of capillaries. In some embodiments, the second set of capillaries includes capillaries that are detectable in a threshold number of images of the image set. For example, if the image set includes 60 images, the second set of capillaries may include capillaries that are visible in at least 40 of the 60 images. In some embodiments, the second set of capillaries includes capillaries that are detectable in a threshold number of images of the image set. and with confidence values ​​above a confidence threshold in a minimum number or all of those images. In some embodiments, the controller 2350 generates an indication of a threshold number of images based on the training data.

[0123] In an exemplary embodiment, the controller 2350 feeds the neural network, as described herein, with 130 training images with 795 corresponding bounding boxes (around capillaries) created manually. The training images were extracted as the first frames / images of 130 distinct capillary movie / image sets from 43 distinct patients, thus ensuring sufficient data diversity. Bounding boxes were manually defined by one human expert around each capillary that met the set of criteria described herein. The image dataset was divided into a first set and a second set based on the temporal order of acquisition, and a confidence threshold was set at ≠0.45 to avoid detection of inappropriate capillaries or artifacts. This condition detected 66 images from 23 patients with a total of 416 annotated bounding boxes around identified capillaries in the first set as the first capillary, and 64 images from 19 patients with a total of 379 annotated bounding boxes around identified capillaries in the second set. The neural network is fed with the first and second sets each time for a total of 900 iterations, and the corresponding learned weights W s1 and W s2 As described with reference to the exemplary YOLO technique, in some embodiments, these learned weights W s1and W s2 is determined as a free parameter of the convolutional neural network structure and is optimized such that the capillaries detected by the neural network when run on images from the training set best match those pre-labeled by human raters on these same images as reference.

[0124] After the above single frame detection step / identification of the first set of capillaries, each capillary was tracked with a given identifier. Capillary tracking is based on overlap and Kalman filtering in the event of missed detection between frames, although it is understood that any suitable technique may be used to track capillaries, including but not limited to mathematical forms, cross-correlation, mutual information, optical flow, machine learning, etc. Capillaries are tracked for at least t f images (threshold number of images), i.e., t f If capillaries in the second set are associated with bounding boxes that exceed the confidence parameter t in the first set of images, then they are selected. f empirically set to 600. In the latter case, t f was set.

[0125] To test the selection of the first capillary set, the steps described herein (for detecting the first capillary set) are performed using weights W s2 The first set has weights W s1To validate the detection of the first set of capillaries, a comparison is made against the reference of 24 first images taken from a separate 24 sets of reference images / reference videos, where the capillary boxes have been annotated by two human experts who selected the capillaries according to the following set of criteria: (A) Illumination. Capillaries must be visible with sufficient contrast for the observer; (B) Focus. Detailed capillary structure / dynamics must be visible without blurring; (C) Flow. Blood flow must be present to allow identifying and counting potential events; (D) Stability. Capillaries must remain entirely within the video FOV in all frames; (E) There are no objects (e.g. air bubbles) that may occlude visible capillaries. (F) Morphology. Capillaries must exhibit clear arterial and venous limbs. In the test video / image set, for example, 95% of the videos have 3 or more detected capillaries, which is associated with improved classification results. An example of capillary detection performed as detailed herein is shown below. , as illustrated in FIG.

[0126] 23, in some embodiments, the controller 23 detects a set of cellular events (i.e., one or more) in the second set of capillaries. In some embodiments, each cellular event of the set of cellular events is associated with the passage of a white blood cell in a capillary of the second set of capillaries. As described in more detail herein, red blood cells exhibit greater optical absorption than white blood cells at the wavelengths described herein, and therefore the passage of white blood cells in the capillaries results in an "absorption gap" due to the presence of the white blood cells.

[0127] Thus, the term "cellular event", as used herein, and sometimes also referred to as a "gap" or simply an "event", may refer to one or more of the following: a) detection of a relatively high absorption area (indicating the possible presence of red blood cells) adjacent to a relatively low absorption area compared to a first area in a capillary (indicating the possible presence of one or more white blood cells); b) detection of a first area of ​​relatively high absorption (indicating the possible presence of red blood cells) adjacent to a second area of ​​relatively low absorption compared to the first area (indicating the possible presence of one or more white blood cells downstream of the first area), where the second area is adjacent to a third area of ​​different absorption from the second area in the capillary (indicating either the possible presence of red blood cells downstream of the second area or an area containing substantially no cells).

[0128] While disclosed herein for detection of white blood cells in capillaries, it will be understood that the embodiments of the present disclosure are useful for detecting any other suitable cells so long as they exhibit an absorption contrast to red blood cells. As one example, circulating tumor cells (CTCs) may be detectable by selecting capillaries having a diameter similar to CTCs (i.e., by selecting the first set of capillaries or the second set of capillaries). As another example, one or more white blood cell types (e.g., neutrophils, lymphocytes, monocytes, eosinophils, and / or basophils) may be detectable by selection of appropriate capillaries as described herein.

[0129] In some embodiments, for every capillary of the second set of capillaries, the controller 2350 detects one or more cellular events flowing through that capillary as follows: In some embodiments, the controller 2350 loads all images showing the capillaries as a 3D matrix into the memory 2360 and / or database 2370, along with the x pixel location, y pixel location, and matrix dimensions associated with the images. Each matrix value corresponds to the brightness level of that pixel. In some embodiments, the controller normalizes the brightness levels of the 3D matrix by a) spatially normalizing the brightness levels with respect to the first image showing the capillaries to correct for potential brightness variations associated with the illumination source 2320. In some embodiments, this is achieved by a) assigning pixel brightness (for each pixel) by the local brightness average estimated by a Gaussian filter with a standard deviation of 25 pixels, and b) adjusting the average brightness of every image showing said capillaries such that the brightness value remains closest to that of the first image (in terms of mean square error).

[0130] Each pixel of this pre-processed normalized 3D matrix has a time signal associated with it that will be similar across all pixels passed by the same cell event, with the main difference between these pixels being the pixel-specific time shift. Another difference is that not all time signals are equally strong / precise amplitude. As a first approximation, the controller 2350 selects a pixel away from the edge of the image with the "strongest" time signal to be the reference pixel. In general, a strong / desirable signal is one that has a large contrast between bright gaps (indicating possible presence of white blood cells) and dark red blood cells, e.g., a large contrast between the bright gaps (indicating possible presence of white blood cells) and the ... Such a signal may have a contrast that exceeds the noise level in the signal. This signal may be a candidate for a reference signal for temporal alignment. In some embodiments, the controller estimates the intensity of this candidate reference signal based on the ratio between its sum of squares and its sum of absolute values. This ratio may then be interpreted as an estimate of the intensity of its peak relative to its nominal variation. In some embodiments, the estimation of the reference signal may employ the following pre- and post-normalization steps: (a) subtracting a local time average from the time signal based on an averaging window (e.g., window size 200 ms); (b) estimating the aforementioned ratio as the median of 10 estimates derived from 10 consecutive time chunks of the same size for robustness; (c) forming a 2D map of the image set showing the capillaries; and further spatially filtering the resulting estimate of the above ratio with a Gaussian filter with a standard deviation of 1 pixel. In some embodiments, the controller 2350 ignores and / or otherwise does not consider other capillaries close to the edge of the region of interest due to possible intensity variations due to imperfect registration by introducing a weighting factor to penalize reference pixel candidates close to the frame border. The controller 2350 then selects the pixel that maximizes the peak intensity of the preprocessed temporal signal and its distance from the edge of the region of interest.

[0131] In some embodiments, the controller 2350 further sifts out other structures that are unlikely to be cellular events through spatial filtering of every separate frame. In some embodiments, this is accomplished by applying a bandpass filter in the separate Fourier domain for every frame to remove all features of inappropriate size, with low and high frequency bandpass parameters adjusted to fit the expected cellular event size / expected white blood cell size range.

[0132] In some embodiments, the controller 2350 further sets the intensity of capillaries filled with red blood cells to zero. The purpose of this step is to ensure that event-free flows should display the correct contrast for the passage of events, while the latter are then ideally associated with a (significant) non-zero brightness value after removal of the bias. In some embodiments, the controller achieves this by subtracting the temporal (frame-wise) median value from all corresponding pixels in all images showing that capillary. The pixel-wise operation is performed separately and indiscriminately on all pixels, i.e., independently of whether the pixel is inside or outside the capillary. The assumption of this step is that the capillary is filled with red blood cells for most of its length and for most of the time.

[0133] The controller 2350 then compares the time signal associated with every pixel location of the image showing the capillaries of the second set of capillaries to the reference time signal at the reference pixel estimated as disclosed herein. First, the controller 2350 suppresses long-term intensity fluctuations by applying a high-pass filter to every separate time signal along the time dimension. The controller 2350 then estimates the correlation between every time signal and the reference time signal, resulting in amplitude and phase values ​​for each pixel. A large positive amplitude indicates a strong correlation, and the phase indicates a time shift with respect to the signal at the reference pixel. This per-pixel amplitude and phase information is used as described below to estimate an average signal that is more robust to noise. The controller 2350 performs all operations by duplicating the first and last frames to avoid false detections at the beginning or end of the image sequence showing the capillaries.

[0134] Based on the reference time signal and other time signals with associated phase and amplitude correlation information, the controller 2350 generates an average time signal that is relatively more robust to noise. This average time signal combines all the time signals that are captured as part of the capillary. In some embodiments, the controller 2350 retains any time signal if a) the correlation between the any time signal and the reference signal exceeds a threshold autocorrelation time of the reference signal, and b) the time delay (i.e., phase) (positive or negative) of the time signal with respect to the reference signal is on the order of magnitude of a relatively fixed parameter, the maximum time it takes for a gap / cell event to flow through a capillary.

[0135] The controller 2350 then calculates the upper reference time signal as follows: (a) align all time signals in time with respect to the reference time signal based on phase information, (b) average the aligned time signals, and (c) high-pass filter the resulting average signal in time based on a fixed minimum frequency parameter. Keeping in mind that the time at which a cell event is detected is related to a reference pixel location, the resulting filtered one-dimensional signal is useful for cell event detection. Figures 26A-C illustrate an example of a real-time signal resulting from analyzing a set of images for one capillary, and also show the type of time signal profile that can be expected around a single cell event vs. plasma gap. The gap containing white blood cells is assumed to have a high concentration of red blood cells upstream, corresponding to a dark area, as best illustrated in Figure 26B.

[0136] Each gap / cell event is associated with a non-continuous stream of values ​​where the mean time signal, reflecting the spatial average normalized intensity in the capillary, exceeds a threshold. Due to noise, the threshold can be set as a non-zero positive value, whereas all values ​​other than zero should reflect a passing event. In some embodiments, the threshold is optimized to achieve the best possible separation between signal and noise. For example, in some embodiments, the threshold is set to capture most peak signal information (associated with values ​​above the threshold) while rejecting most noise artifacts (associated with values ​​below the threshold). In some embodiments, the controller 2350 sets this threshold as a multiple of the standard deviation of the noise, where the noise is estimated according to a positive exponential model (i.e., a decaying exponential function is fitted to the positive part of the signal distribution). In some embodiments, the standard deviation of the signal itself may serve as an estimate of the noise. In some embodiments, before thresholding, the controller checks the average temporal signal; specifically, if the ratio between the median filtered version of the signal (e.g., with a filter size of 240 frames) and its first quartile exceeds 1, the controller locally divides the signal by this ratio. For each detected gap / cell event, the associated temporal information is defined with respect to a reference pixel. Then, if the signal exceeds the threshold for more than one image showing the capillary, the controller selects the central image in the series of images showing the capillary.

[0137] In some embodiments, the controller maps the cell event back to a series of x, y, and t in a 3D matrix that is the maximum per image of the capillary. Each cell event may occur in multiple images a number of times that depends on the flow velocity, the capillary length, and the frame rate of acquisition. The ability to detect a cell event is limited by the maximum crossing time of a gap around the reference frame, in which the gap passes through the reference pixel. For each image, the controller 2350 marks the brightest pixel that is deemed to be inside the capillary.

[0138] For each capillary of the second capillary set, the controller 2350 then generates an associated event count. As described in further detail herein, in some embodiments, after averaging the event counts across all capillaries of the second capillary set, each set of images can be used for classification of the user.

[0139] 27A-B illustrate the results obtained from the detection of cellular events on the 26 raw video / image sets when compared to human raters. The F1 score is used as a metric (defined as the harmonic mean of precision P and recall R, where P=TP / (TP+FP) and R=TP / (TP+FN)) to measure the consistency of detected events compared to one of the individual raters. For each individual expert (3 in total), the F1 score was evaluated against the cellular events consistently detected by the remaining experts (i.e., the group of 2). For the embodiments disclosed herein, the F1 score was evaluated as the average between the results for the three groups of these two experts, thereby ensuring an unbiased comparison. The F1 score takes into account true positives, false positives, and false negatives.

[0140] 28A-28D show an example of an overall approach to cellular event detection described herein with respect to controller 2350.

[0141] Referring again to FIG. 23, in some embodiments, the controller 2350 classifies the user into a user type in the user type set based on the number of cellular events detected in the second capillary set. In some embodiments, at least one user type is associated with a diagnosis of neutropenia (absolute neutrophil count <500 cells / uL) and at least one other user type is associated with non-neutropenic users. In some embodiments, the controller 2350 implements a weighting approach (i.e., taking a weighted average of the event counts across all capillaries) to improve the quality of the classification. In some embodiments, the weighting approach may be based on an estimated quality of the image set.

[0142] In some embodiments, the controller 2350 generates a single event metric value (which may sometimes be referred to as a "white blood cell metric value") that summarizes all event counts. In some embodiments, the controller compares the event metric value to an event threshold μ, which (in some embodiments) can be a learning parameter as described in more detail herein. Classification of a user as a user type associated with severe neutropenia (e.g., a first user type) is when the event index value < μ, and another user type (e.g., a second user type). Although two user types are described herein, it should be understood that the controller 2350 can classify a user into three, four, or more user types depending on the threshold parameter.

[0143] As described herein, the controller 2350 can assign a weight to each capillary of the second capillary set based on its estimated quality. As an example, in some embodiments, the weight of a capillary can be set to 1 / 10 when the threshold used for event detection is greater than a fixed level relative to the standard deviation of the signal (indicating lower quality) and otherwise 1.

[0144] As described above, in some embodiments, the event threshold can be a learning parameter. For example, in some embodiments, the event threshold was determined based on a total of 116 raw video / image sets. Specifically, 116 imaging sessions from 42 patients (mostly each associated with one raw video / video set) were employed. Of these sessions, 60 corresponded to reference blood draw values of ANC < 500 and 56 corresponded to ANC > 500 (15 of which were 500 < ANC < 1,500). In cases where consecutive daily blood draws showed a transition from ANC < 500 to ANC > 500, the imaging session was performed within 8 hours of the first blood draw.

[0145] 29A-29B show results obtained from classification of the aforementioned data using the systems, devices, and methods disclosed herein. With a detection rate of severe neutropenia of 96% (AUC=0.96), there were 9% false alarms (i.e., 9% of cases were erroneously classified as severe neutropenia). These results show that 20% of the sessions were deemed irrelevant during the analysis and therefore discarded. The corresponding conditions are: (a) no sessions with <3 capillaries detected, (b) no sessions with a white blood cell index of zero, with individual capillaries (i.e., individual capillaries with no detected events) also being discarded as part of (b). Figures 61A-61B illustrate how classification can be affected by the length of the video / number of images in the employed image set.

[0146] 30 illustrates a method 3000 for non-invasive hematological measurements, according to some embodiments. Method 3000 may be performed by system 2300 or similar structural and / or functional variations thereof. Method 3000 includes, at 3100, acquiring a set of images of at least a portion of a nail fold of a user's finger. In some embodiments, step 3100 further includes acquiring the set of images as a set of frames of a motion picture. In some embodiments, method 3000 further includes illuminating the portion of the nail fold, and acquiring the set of images is responsive to illuminating the portion of the nail fold.

[0147] The method 3000 further includes, at 3200, detecting one or more capillaries within a portion of the fingernail fold in each image of the image set to identify a first set of capillaries across the image set. The detecting may include estimating one or more attributes of each capillary of the first set of capillaries. The one or more attributes may include one or more structural attributes, one or more flow attributes, one or more imaging attributes, or a combination thereof, such that a first attribute of the one or more attributes of each capillary of the first set of capillaries meets a predetermined criterion for the first attribute. In some embodiments, the one or more structural attributes are selected from the group consisting of average capillary diameter, capillary transverse diameter, capillary longitudinal diameter, capillary length, capillary shape, and the like. In some embodiments, the one or more flow attributes are selected from the group consisting of blood flow velocity in the capillaries, transit time of cells in the visible portion of the capillaries, volumetric flow rate, mass flow rate, and the like. In some embodiments, the one or more imaging attributes are selected from the group consisting of contrast, focus, signal to noise ratio, image stability, and the like. In some embodiments, the first attribute is an average capillary diameter, and each capillary of the second set of capillaries has an estimated average capillary diameter of from about 10 μm to about 20 μm.

[0148] In some embodiments, the method 3000 may further include generating a confidence value associated with an image of each capillary of the first capillary set in the image set. The first capillary set may include capillaries whose confidence value exceeds a confidence threshold for each image in which the capillary is detected.

[0149] In some embodiments, the method 3000 may further include receiving a training image set including a specification of one or more capillaries visible in each image of the training image set. The method 3000 may further include training a neural network on the training image set and applying the image set to the neural network to detect the first set of capillaries.

[0150] In some embodiments, detecting the first set of capillaries further includes applying the image set to a neural network, the neural network being trained on a training image set including specifications of one or more capillaries visible in each image of the training image set.

[0151] The method 3000 further includes, at 3300, identifying a second set of capillaries from the first set of capillaries, such that each capillary of the second set of capillaries is included in a predetermined number of images of the set of images. The method further includes making the image visible at

[0152] In some embodiments, the method 3000 further includes detecting a set of cellular events for the set of images and in the second set of capillaries. Each cellular event of the set of cellular events is associated with the passage of a white blood cell in a capillary of the second set of capillaries. The method 3000 may further include estimating an event count for the second set of capillaries based on the set of cellular events.

[0153] In some embodiments, the method 3000 further includes estimating a quality factor for each capillary of the second capillary set. The method 3000 may further include estimating an event count based on the cell event set and the quality factor associated with each capillary of the second capillary set.

[0154] In some embodiments, the method 3000 further includes receiving a training image set associated with a portion of the nail folds of a set of training users. The method 3000 may further include generating an event count threshold based on the training image set via supervised learning. The method 3000 may further include classifying the user into a first user type of a set of user types based on the event count and the event count threshold, at least one user type of the set of user types being associated with a diagnosis of neutropenia. The method 3000 may further include transmitting an indication of the first user type to the user (e.g., via an interface of the system 2300 of the device 2340).

[0155] Additional details regarding various embodiments of the systems, devices, and methods described herein are provided below.

[0156] Volume estimation and capillary subselection to improve counting accuracy As described above (see, e.g., Equation (8)), the WBC concentration can be estimated based on an estimate of the number of WBC cells (e.g., estimated by the number of cell events) and the volume of blood passing through the capillary of interest during the total movie acquisition time. The total blood volume passing through a given capillary compartment typically depends on the local diameter of the capillary compartment and the average local velocity of the blood. Furthermore, conservation of volume (or conservation of blood mass) indicates that the total blood volume of interest can be maintained at a constant value during propagation within the capillary. In other words, the total blood volume can be made constant independent of the capillary compartment considered. Thus, an estimate of the volume of the capillary compartment can be taken as an estimate of the blood volume of interest.

[0157] Based on this observation, methods for robust volume estimation are based on cross-correlation or optical flow techniques.

[0158] At least two approaches can be used to estimate the volume of a capillary compartment. In one approach, the capillary compartment in question can be assumed to have a perfect cylindrical shape. In this case, the width of the capillary, obtained from the video or image, can also be used as the capillary depth.

[0159] In another approach, a method for estimating the volume of the capillary compartment is performed based on the analysis of pixel intensity values. In particular, the capillary depth at a given position in the image plane is expected to be inversely related to the measured light intensity due to light absorption. The longitudinal and transverse diameters of the capillary may not be equal, and the diameter of the visible projection may not represent the longitudinal diameter. In other words, a low pixel value indicates a high value of the capillary depth, and vice versa. This resolved depth estimate can therefore be used to more accurately calculate the cross-sectional area of ​​the capillary compartment.

[0160] 31A-31F illustrate a method for estimating capillary compartment volume based on analysis of pixel intensities. In some embodiments, some or all aspects of the method of FIG. 31A-31F may be performed by one or more of the systems, apparatus, and devices as described herein, such as system 2300 and / or device 2340.

[0161] 31A and 31B show two images of a capillary compartment, where the pixel values ​​in FIG. 31A are larger than those in FIG. 31B (i.e., FIG. 31 is brighter). The two capillary compartments have the same width of about 15 μm at a given position marked in FIG. 31A and FIG. 31B. In a conventional method, this width is also used as the depth of the capillary compartment, and the reconstructed capillary cross-section is perfectly circular as shown in FIG. 31C and FIG. 31D. In contrast, when pixel intensity is considered, different pixel values ​​indicate that the depths of the two capillary compartments are different. FIG. 31E and FIG. 31F show the reconstructed cross-sections corresponding to the capillary compartments shown in FIG. 31A and FIG. 31B, respectively, using a method based on pixel intensity analysis. As can be seen in FIG. 31E and FIG. 31F, a larger pixel value in FIG. 31A indicates a smaller depth, whereas a lower pixel value in FIG. 31B indicates a larger depth. The cross-sectional shape of the two capillary compartments is elliptical instead of circular.

[0162] The same analysis can be performed along the entire capillary shown in Figures 31A and 31B to obtain the cross-sectional area A as a function of curvilinear coordinate (e.g., length l). The total volume V of the capillary can be calculated by integrating A(l) with respect to l, i.e.,

number

[0163] The volumes calculated from the above method can also be used to resample the volume of the capillary profile. This method can analyze the flow velocity in a given capillary using a one-dimensional curvilinear coordinate system. In one example, the coordinates of the one-dimensional curvilinear coordinate system are typically a function of the length along the capillary (see, e.g., Figures 6A and 6B and above). In another example, the coordinates of the one-dimensional curvilinear coordinate system can be a function of the cumulative capillary volume. One advantage of resampling the coordinates by the cumulative capillary volume is that particle / event velocities can be constant within a particular coordinate system, facilitating accurate estimation using correlation or similar techniques.

[0164] 32A-32C illustrate volume resampling of capillary profiles. FIG. 32A shows a representation of a capillary with a constant blood flow velocity but varying diameter along the capillary length. Due to light absorption, red blood cells are represented in black, whereas single white blood cells pushing through the capillary are represented in white. FIG. 32B shows a spatiotemporal map (color map from blue-dark-to red-bright-) representing the grayscale values ​​along a linear capillary length (y-axis) over time (x-axis). FIG. 32C shows a spatiotemporal map obtained using volume resampling, shown in capillary length coordinates (y-axis) as a function of the cumulative capillary volume at each cross-section of FIG. 32A. In this case, the trajectory of a white blood cell at constant velocity (red-bright-line) becomes a straight line and therefore easier and more accurate to estimate using correlation or similar techniques.

[0165] Subselection of Microvascular Events Further improvement in WBC detection and estimation (e.g., as generally described with respect to Figures 23-30) can be achieved through subselection of capillaries whose events are most easily detectable and most likely to correlate with the presence of WBCs. Subselection of capillary events can be performed according to several criteria.

[0166] In one example, sub-selection may follow the observed size, i.e., mean diameter and / or diameter profile. Size usually correlates to the presence of clear, high-contrast total capillary diameter events for a given range, such as the range between about 10 μm and about 20 μm. Figure 33 shows the range of capillary diameters where clear gaps exist. Sub-selecting capillaries based on their diameter can improve correlation with WBC.

[0167] In another example, sub-selection may follow the time distribution of arrival of clear, high-contrast, full-capillary diameter events, which have been shown to differ in the presence / absence of white blood cells. Figure 34A shows the distribution of time of arrival (TOA) between gaps in a sample containing 5,500 WBCs / μL. For this high concentration of WBCs, the TOA follows a substantially Poisson distribution. In contrast, Figure 26B shows the distribution of TOA between gaps in a sample containing 100 WBCs / μL, a much lower concentration than that shown in Figure 34A.

[0168] In yet another example, sub-selection can be performed according to the dynamic behavior of capillaries. In this approach, the method can directly sub-select capillaries based on event features including length, contrast-to-noise ratio, and velocity. For example, occlusion effects can be identified as due to red blood cells in some small diameter capillaries.

[0169] Acquiring fast, high-contrast, stable movies of multiple nailfold capillaries 35A-35C illustrate an apparatus for detecting severe neutropenia based on images of nailfold capillaries. In some embodiments, some or all aspects of the apparatus of FIGs. 35A-35C may be structurally and / or functionally similar to one or more of the systems, apparatus, and devices as described herein, such as system 2300 and / or device 2340.

[0170] Figure 35A shows a rendered 3D model of the apparatus employed to record microscopic videos of the microcirculation in a patient's nailfold capillaries along with its various components. Figure 35B illustrates the patient placing their ring finger into a 3D printed holder that serves the dual purpose of achieving stability throughout the one minute recording period and holding the oil utilized for optical coupling. Figure 35C shows that the finger is positioned such that illumination and imaging are directed toward the nailfold region (area marked in the figure and circled in purple).

[0171] The device shown in Figure 35A is used to record high quality microscopic videos of microcirculation in human nailfold capillaries, which can be utilized in ASCT patients (see Figure 36). The patient's finger is inserted into a hole in a 3D printed, hemispherical, easily disinfected hand rest (Figure 35B), which is designed to ergonomically hold the patient's hand with sufficient stability to record a one-minute video. The capillary video is acquired from the nailfold region of the subject's finger (Figure 35C).

[0172] Figures 37A and 37B show examples of raw images acquired by the device shown in Figure 35A. Wide-field movie pairs were acquired by the device from one ASCT patient at two time points where the same capillaries could be observed (three numbered pairs are shown). Figure 37A shows an image taken at baseline neutrophil concentrations (i.e., >1,500 / μL) and Figure 37B shows an image taken when the patient had severe neutropenia (neutrophil concentration <500 / μL). The images shown in Fig. 1 are the scale bars of the images, 100 μm.

[0173] It can be seen from Figures 37A and 37B that the device can optically capture capillary-nailfold movies with suitable contrast, resolution, stability, and depth of focus. The raw movies acquired by the device contain multiple capillaries in the field of view. The acquisition of multiple capillaries was particularly convenient with this simple optical approach that allows imaging multiple capillaries at once in the same field of view. In contrast, many existing techniques, such as coded confocal microscopy (SECM), can only image a single capillary at a time.

[0174] Figures 38A-E illustrate an example of an optical gap flowing through a capillary. The image sequence shows several raw frames of a movie centered on one capillary acquired by the device for one patient at baseline. The dark loop corresponds to a capillary filled with RBCs that absorb light at the illumination wavelength. Optical absorbing gaps in the microcirculation are approximately the same size as the capillary width (~15 μm) and can be seen flowing through the arterial limb of the capillary (labeled as black arrows). Frame numbers are labeled in the upper right corner. The frame rate was 60 frames per second and the exposure time was 16.7 ms. Contrast was adjusted for the indicated regions of interest.

[0175] These capillaries in the acquired images are very narrow, with a typical width of about 10 μm to about 20 μm. In this case, WBCs have no choice but to push their way through the capillaries, typically one by one. Focusing on a single capillary, the frame reveals the passage of an event in the microcirculation, which can be perceived as a moving "bright" object of approximately the same size as the capillary diameter (about 15 μm). The bright object (or gap in a dark background) has a significantly higher brightness than the surrounding RBCs and can therefore be easily observed and monitored.

[0176] The movement of the events is clearly tracked across successive frames and is therefore visually identifiable by human observers. Therefore, all events in the capillary movies were marked and labeled by three blinded human raters according to specific visual criteria (see further detailed description below). The space-time (ST) map provides a convenient alternative representation to show the trajectories of all events bearing marks obtained from these raters in the 1-min capillary movie.

[0177] Figures 39A-E show the results of blinded event assessment using images acquired by the device shown in Figure 35A. Three human raters labeled one observed event in one of the 98 capillaries employed in the study. Figures 39A-C show capillary movie frames (indexed in the upper right) with cross-shaped event marks obtained from raters 1 (blue), 2 (green) and 3 (yellow). Figure 31D shows the aggregated locations of all event marks obtained from all three raters. Figure 39E shows an ST map displaying the recorded intensity levels along the segmented capillary length (vertical axis) as a function of time (horizontal axis) during a 1.7 s interval around the event of interest. The bright trajectory generated by the passing of the event can be clearly identified in the center of the ST map. The blue, green and yellow cross marks correspond to the spatiotemporal coordinates where each of the three raters labeled the event. Event trajectories visible on the ST maps correspond well with events identified by raters from the videos.

[0178] Microcirculatory events with specific characteristics can be used as surrogates for WBCs FIG. 36 illustrates two different timing points for acquiring images from patients in a clinical trial. Patients enrolled in the trial received ASCT, which resulted in chemotherapy. This is a very predictable process of the evolution of their neutrophil count resulting from the controlled administration of chemotherapy. This provides the opportunity to record capillary videos at two different time points for each patient: (1) baseline (neutrophils >1,500 / μL) and (2) severe neutropenia (neutrophils <500 / μL).

[0179] Images from the acquired movies were analyzed at two different time points shown in Figure 36 for 10 ASCT patients who received chemotherapy: before chemotherapy (>1,500 neutrophils per μL, baseline) and during severe neutropenia (<500 neutrophils per μL). To minimize potential confounding factors as well as selection bias, the same set of capillaries was acquired for each patient at baseline and during severe neutropenia (see details below). Acquiring a 1-minute movie allows overcoming the shot noise associated with the individual nature of the events.

[0180] In these movies, the consistency with which events in the capillary microcirculation were identified by the three raters depended on whether the capillary movies were acquired at baseline or during severe neutropenia. When raters identified and counted events in capillary movies acquired during baseline, 67% of the events were corroborated (i.e., two or more raters identified the same event). In the case of capillary movies acquired during severe neutropenia, only 22% of the events were corroborated. The visual characteristics of the assessed events tended to be more consistent for baseline events (see Figures 48A and 48B).

[0181] These results suggested that events with consistently detectable visual features correlated with the presence of WBCs and neutrophils. Therefore, the count of confirmed events was treated as a surrogate for WBC count. The fact that most confirmed events occurred in capillary compartments comparable in size to WBCs (see FIG. 47) also supports previous literature findings that relate the observed gaps to a combination of WBCs and plasma flowing through the capillaries.

[0182] Noninvasive detection of severe neutropenia White blood cell (WBC) counting can be used as one of the indicators of immunological status for the diagnosis and treatment of multiple medical conditions, including cancer, infections, sepsis, and autoimmune disorders. WBC counting is also used with immunosuppressants. However, current methods of WBC counting typically involve a personal visit to a medical institution, even for finger-stick techniques, for venous blood collection performed by trained clinical personnel. This constraint limits both the time and frequency of monitoring. Furthermore, traditional blood tests typically use specific reagents and sterile conditions, which may preclude applicability in resource-limited settings. In contrast, a noninvasive approach to WBC measurement would circumvent many of these requirements and would be comparable to existing noninvasive methods for monitoring blood oxygen saturation.

[0183] One step towards non-invasive WBC analysis includes non-invasive screening for severe neutropenia, which can be defined as a common WBC type with low levels of neutrophils (e.g., less than 500 per μL). This condition is one of the main toxicities in patients undergoing common chemotherapy regimens. It is responsible for significant morbidity and significant mortality risk due to the associated increased risk of infection. However, monitoring of severe neutropenia is currently inadequate for the reasons mentioned above. This barrier to prompt clinical care hampers timely life-saving intervention of prophylactic antibiotics or growth colony-stimulating factors in afebrile patients with prolonged severe neutropenia. In that respect, non-invasive methods could substantially impact outpatient care and management of patients at high risk of immunosuppression associated with severe neutropenia.

[0184] The systems, devices, and methods described herein can provide a screening tool for severe neutropenia based on non-invasive and portable optical visualization of capillaries. When capillary diameters are close to WBC diameters (e.g., about 10 μm to about 20 μm), white blood cells can completely fill the capillary lumen. This typically results in red blood cell (RBC) depletion downstream of the WBC in the microcirculation, where the WBCs flow at a lower directional velocity than the RBCs. In this situation, appropriate illumination (e.g., of a specific wavelength) can make the WBCs transparent and the RBCs dark, and the passage of the WBCs can appear as a light-absorbing gap in the continuous flow of RBCs moving through the capillaries.

[0185] Using white light transmission microscopy, this "gap" phenomenon can be observed in a rabbit ear window model. Results explicitly showed that when capillaries and WBCs were of comparable diameter, RBCs accumulated upstream of the WBCs, with depleted areas of RBCs downstream. The same phenomenon was observed in rat cremaster muscle and bat wing microcirculation, but in this observation, blue light transmission was used to maximize the contrast between RBCs - i.e., oxyhemoglobin and deoxyhemoglobin have a peak absorption of blue at 420 nm - and low-absorbing areas without RBCs. Observing the flow of RBCs over time revealed the morphology of the capillaries, where the brighter areas were associated with light-absorbing gaps inside the capillary lumen. Fluorescent labeling could also be used to confirm that the gaps were associated with WBCs.

[0186] The idea that such absorption gaps are related to WBCs has been explored in humans by utilizing blue endophthalmology, where WBCs transmit blue light as they flow in front of the retina, thus creating bright spots that the subject can see. For example, subjects were able to shine a blue light on their eye and were asked how many bright spots they perceived. Between-group differences in the amount of perceived spots were reported between baseline, leukocytopenic, and leukocytic subjects, i.e., comparing WBC counts in the normal, abnormally low, and abnormally high ranges, respectively. However, one of the limitations of these methods is that they rely on subject self-assessment. As such, they are prone to personal bias and poor reproducibility, making them unsuitable for clinical screening.

[0187] Overall, these findings suggest that flowing gaps in capillaries provide the basis for new methods for non-invasively measuring WBC counts. The systems, devices, and methods described herein use nailfold capillaries, which are superficial (e.g., about 50 μm to about 100 μm deep), have diameters comparable to WBC size, run substantially parallel to the skin surface, and can therefore be visualized non-invasively using simple and affordable optical equipment.

[0188] In this technique, optical imaging is used to screen for severe neutropenia in human subjects by counting events defined as instances of moving light-absorbing gaps in the nailfold microcirculation. To do so, a portable device is configured to generate optical microscopic movies of capillaries (see, e.g., Figures 35A-35C). This technique can maximize the contrast between RBCs and non-RBCs across multiple capillaries in one field of view while ensuring accurate resolution, focal depth, stability, and frame rate. Based on this device, and considering the predictability of neutrophil nadir and recovery kinetics, a clinical study was performed including 10 patients undergoing high-dose chemotherapy and autologous stem cell transplantation (ASCT) (see, e.g., Figure 34). For each patient, 1-minute movies of the same set of capillaries were acquired by the device at two time points: pre-chemotherapy baseline (~1,500 neutrophils per μL) and severe neutropenia (~500 neutrophils per μL) (see, e.g., Figure 35B). (See, e.g., Figures 29A and 29B). Based on these data, methods can be developed and validated for tagging event counts (See, e.g., Figures 38A-39D) and for distinguishing baseline from severe neutropenia across all patients (See, e.g., Figures 40 and 41).

[0189] Baseline status can be non-invasively classified from severe neutrophilic status Figure 40 shows the number of confirmed events per minute in all capillary pairs tested. The values ​​at baseline (blue dots) showed statistically significant differences when compared to the corresponding values ​​at severe neutropenia (red squares). To maximize the objectivity of event selection and discard noise, only confirmed events are considered. All capillaries were analyzed at both baseline and severe neutropenia (total of 98 pairs, black dotted line).

[0190] The number of confirmed event counts in capillaries imaged during severe neutropenia was consistently lower than the counts of the same capillaries imaged during baseline, as seen in Figure 40. Specifically, paired capillary counts showed a highly statistically significant difference (P<10) between baseline and severe neutropenia. -8 , Wilcoxon signed-rank test). Separate capillary-derived counts tended to vary across the same patient and could reach low values ​​even at baseline. Such variations could be associated with multiple factors, which motivated us to aggregate multiple capillary-derived counts for every patient (see details below).

[0191] Figure 41 shows the distinction between baseline and severe neutropenia. The median number of confirmatory events observed per minute allows the distinction between baseline (blue dots) and severe neutropenia (red dots) for 20 acquired videos and 10 tested patients when aggregating all available capillaries per patient. The variability between the corresponding capillaries is also shown for each patient (blue and red bars). The optimal threshold for separating baseline from severe neutropenia is 7 events per capillary minute (black dotted line). The x-axis is labeled by patient ID together with the amount of capillaries analyzed (brackets). The median amount of capillaries used per patient was 4.

[0192] When aggregating data across all capillaries from a given patient at a given time, the distinction between neutropenic and baseline states was clear (Figure 41), with results showing a statistically significant difference between baseline and severe neutropenia (P=0.002, Wilcoxon signed-rank test). Furthermore, in this capillary aggregation case, the distribution of confirmed event counts at baseline does not overlap with the distribution during severe neutropenia. In fact, at a threshold of seven counts, the median counts for capillaries of a given patient were able to correctly classify 9 out of 10 neutropenic cases. Two or three capillaries added initially accounted for most of the increase in classification performance (see Figures 44A and 44B).

[0193] The above study demonstrated that severe neutropenia can be detected noninvasively in humans based on optical imaging, and it also validates the overall classification strategy as proof of principle, including the optical device, experimental protocol, and data analysis methods.

[0194] By design, the clinical trial included baseline absolute neutrophil counts (ANC) (>1,500 neutrophils / μL) and severe neutropenia (<500 neutrophils / μL) in the same patients. Although the classification approach was not evaluated in cases of mild (grade II, <1,500 neutrophils / μL) and moderate (grade III, <1,000 neutrophils / μL) neutropenia, the current results can be extrapolated to these additional ranges, assuming that the mean event counts change accordingly. This suggests that the severity of neutropenia across different grades of neutropenia may vary significantly. Further representative data may be included in additional clinical studies. Meanwhile, the event counts obtained for each patient in the study (shown in FIG. 41) correspond to the corresponding reference cell concentrations shown in Table 1 below.

[0195] [Table 1] Table 1. Reference values ​​obtained from the hospital clinical laboratory.

[0196] Specifically, assuming a mean blood flow velocity of 800 μm / sec and a mean capillary diameter of 15 μm, the median counts of patients tabulated for baseline and severe neutropenia, i.e., 31.73 and 1.96, yield estimates of WBC concentration of approximately 3,700 and 200 cells / μL, respectively, both of which are within the range of the corresponding references obtained from standard reference laboratory assays (Table 1).

[0197] Event assessment can be performed automatically on the input capillary video. Such algorithms can follow approaches used to detect objects moving through capillaries, or more advanced strategies, such as machine learning techniques. Several event features can be employed, such as contrast, size, or persistence. Beyond mere counting, algorithmic estimation of capillary blood flow can improve the precision and accuracy of the results by giving estimates that are physically consistent with the WBC concentration.

[0198] Further improvements to the device could increase the amount of capillaries per patient that meet the quality and consistency criteria required for further analysis, which may be useful for future translation of this technology into clinical practice: the constraint of following the same capillaries on the same patient would be relaxed, facilitating clinical application of the method.

[0199] One extension to this work could be to explore whether specific WBC ranges could be identified beyond screening for severe neutropenia. This could broaden the applicability of the techniques described herein beyond chemotherapy settings, and into new settings such as infectious diseases, while still following a similar conceptual approach. Also, knowing that distinct WBC types lead to distinct optical properties and image characteristics, e.g., non-granular and granular WBCs correspond to distinct event lengths and backscattering properties in capillaries, non-invasive differential WBC counting could be achieved based on similar techniques / data.

[0200] Overall, this study demonstrated that chemotherapy-induced severe neutropenia can be detected noninvasively through the nailfold with a provisionally developed prototype. This study represents the first proof of concept for a technology that can measure important toxicities of chemotherapy by optical means. These results represent a promising new approach to the monitoring of cancer patients at risk of severe neutropenia. Moreover, from a more general point of view, the proposed imaging method and conceptual approach could constitute one of the first steps towards non-invasive in vivo WBC counting. EXAMPLES

[0201] A pilot diagnostic validation study was performed to test the a priori hypothesis that a non-invasive technique would allow classification of patients with severe neutropenia (neutrophils < 500 / μL) from their baseline state (neutrophils > 1,500 / μL). A cohort of patients who underwent high-dose chemotherapy followed by ASCT was enrolled. The kinetics of neutrophil counts in these patients is predictable, since the intensity of chemotherapy applied before transplantation ensures a course of severe neutropenic phases followed by recovery. In the framework of this study, no power analysis was performed, and a convenience sample of 10 subjects (selected from the initial patient pool) was considered sufficient to test the study hypothesis. Non-parametric tests were used alongside receiver operating characteristic curve analysis. All human assessors who analyzed the data were blinded, as detailed below.

[0202] A total of 23 patients were recruited, 16 and 7 patients from Massachusetts General Hospital, Boston, Massachusetts, USA, and La Paz University Hospital, Madrid, Spain, respectively. Each patient signed an informed consent. All information obtained was anonymized to prevent the identification of participants.

[0203] The inclusion criteria for patients used for recruitment were as follows: (a) patients must be scheduled for ASCT of hematopoietic progenitor cells; (b) patients must be 18 years of age or older; (c) patients must be able to understand and willing to sign a written informed consent form; (d) patients must have a baseline white blood cell count of 3,000 cells / μL or greater and a neutrophil count of 1,500 cells / μL or greater. Patients were excluded if they suffered from myelodysplastic syndromes, or from a history of allergic reactions to components of the oil-like chemical composition used for optical coupling in clinical devices, or if their photodermatological type was greater than 4 on the Fitzpatrick scale.

[0204] The MGH clinical trial was approved by the Dana-Farber / Harvard Cancer Center (DFHCC) Institutional Review Board and by the MIT Committee on the Use of Humans as Experimental Subjects (COUHES) under COUHES protocol #1610717680. We also registered this study on Clinicaltrials.gov. The La Paz clinical trial was approved by the La Paz Ethics Committee under document HULP PI-2353. Analyses of the de-identified data obtained from these pilot studies were also approved by the Ethics Committee of the Polytechnic University of Madrid.

[0205] optical device An example of an apparatus configuration used for video acquisition in ASCT patients (shown in FIG. 35A) includes the following elements: This configuration is for illustrative purposes only and variations may be made by one skilled in the art.

[0206] 1. Imaging objective: Edmund Optics TECHSPEC 5X. The optical characteristics of this objective are a magnification of 5x and a maximum numerical aperture of 0.15, reduced through the use of a 2.5mm diameter 3D printed iris, which maximizes the depth of focus and images multiple capillaries simultaneously. The working distance is 16.2mm, with a maximum field of view (FOV) of 1.8x1.32mm. Its dimensions are a fixed tube length of 50mm and an overall length of 93.81mm. Its height, azimuth, and focal position are manually adjustable.

[0207] 2. CMOS camera. Thorlabs DCC3240N. This CMOS camera is attached to the objective lens and computer power via a USB connection. It has a global / rolling shutter. Its field of view (FOV) is 1280 x 1024 pixels, or 1360 x 1088um at 5x magnification, corresponding to a pixel size of 1.0625 x 1.0625um. The frame rate is approximately 60 frames per second (FPS) across the frame, thus ensuring sufficient time resolution to detect and track events, given that the blood flow velocity of the nailfold capillaries ranges from 100 to 1,000μm / s. The frame rate can reach 229FPS when limited to a FOV of 320 x 240 pixels. Its bit depth is monochrome with 10 bits per pixel.

[0208] 3. Illumination device. A hastily constructed prototype LED holder was used, attached by a cage to a heat sink at an angle of about 70 degrees from the detection axis. These contain high-power Luxeon LEDs emitting light in the deep blue, i.e. 420 nm. This illumination wavelength allows maximizing the contrast between the RBCs - which appear dark in the movie - and the light absorbing gaps. Each LED emits 161 lumens at 700 mA, using an aspheric focusing lens with F=20.1 mm, NA=0.6, and a Thorlabs VA100C with adjustable collimating slit.

[0209] 4. Power driver, used to drive both LED illuminators continuously at a constant DC power level.

[0210] 5. Disposable hand rest. A 3D printed rigidly mounted platform used to hold a finger in a stable position for imaging for at least 1 minute. The platform includes a one-size finger hole. Optical coupling oil (Johnson & Johnson, refractive index = 1.51) is retained in the finger hole.

[0211] 6. Laptop and software. A laptop connected to a CMOS camera was used for power and image acquisition. Specifically, custom LabView software was used for movie acquisition and storage. The output data collected by this software for each patient and acquisition session consisted of an uncompressed set of movies and a timestamp that gave information about the exact acquisition time associated with each frame for each movie.

[0212] As part of the clinical trial, two units of the device were installed and utilized at Massachusetts General Hospital in Boston, Massachusetts, USA, and La Paz in Madrid, Spain. Disinfectant wipes were applied to the system components after each patient use. Use of the device was approved for the clinical trial as DF / HCC protocol #15-070.

[0213] Data collection In this study, videos were acquired from the same set of capillaries for each given patient at baseline and during severe neutropenia. By tracking the same capillaries at both time points, potential bias in capillary selection can be avoided and confounding factors can be minimized. For example, this was expected to minimize changes in capillary geometry between baseline and severe neutropenia time points, thus ensuring that changes in counts within a given capillary pair most accurately reflect the underlying changes in WBC concentration.

[0214] To obtain a movie that includes common capillaries in a given patient, the user of the optical prototype During severe neutropenia, at least one capillary area similar to the baseline was located. This process was performed manually during raw data acquisition, but proved difficult in certain cases for logistical reasons. On the other hand, the presence of different capillary distributions and morphologies may have allowed for the correct identification of previously acquired areas during baseline acquisition. Furthermore, this process of identifying capillary areas was simplified by the fact that the nailfold capillaries of interest were selected from the vicinity of the nailfold border, and therefore the capillary area of ​​interest was limited accordingly.

[0215] Of the 23 patients recruited, 10 met the quality criteria and were eligible for further processing (see Capillary Selection). Specifically, six entire patient datasets were excluded due to insufficient imaging quality and four were excluded due to lack of correspondence between capillaries examined at baseline and severe neutropenia. For each of the 10 eligible patients, at least one pair of videos was selected representing the acquisition of the same capillary area in both clinical conditions. This amounted to 20 raw video datasets with two distinct capillary areas acquired for patients 01 and 02. A total of 49 distinct capillaries were selected and followed and analyzed at baseline and during severe neutropenia.

[0216] Videos were acquired within 8 hours of the corresponding blood test providing the reference information. Specifically, in addition to selected videos and capillaries, WBC and ANC concentrations from state-of-the-art blood cell analysis were acquired for each patient along with their clinical status (see Table 1). The reference blood analysis values ​​from this table reveal a decrease in WBC and ANC concentrations between baseline and severe neutropenia.

[0217] Pre-processing workflow Figure 42 illustrates the pre-processing workflow in a clinical trial. Once a patient's raw video is acquired (top left), a set of capillaries of suitable quality is selected by two human experts (top right, green rectangle). For each selected capillary (top right, examples identified by arrows), a motion-corrected video of the corresponding region of interest is then created (bottom right). The following step (bottom left) involves event labeling by three blinded human raters, which enables subsequent analysis and event counting, aggregation, and visualization.

[0218] Each raw input video of interest was processed according to the pre-processing workflow shown in Figure 42. After capillary selection, each capillary video included in the study was cropped to its corresponding region of interest and registered with respect to a reference frame in the sequence to correct for motion during acquisition. The motion corrected video was anonymized / shuffled and then used by blinded experts to identify events and derive event counts accordingly. The data with labeled events could then be visualized post-hoc. More details regarding these steps are provided below.

[0219] Capillary Selection Once raw videos were obtained from a given patient, two human experts independently defined the appropriate capillary set based on the qualitative, empirical, and a priori criteria defined below. To avoid potential bias, only capillaries selected by both blinded raters were included in the study. Each expert independently selected the best capillary in the raw video according to the following criteria:

[0220] A. Illumination. The capillaries are visible with sufficient contrast to the observer.

[0221] B. Focus. Detailed capillary structure / dynamics are visible and not blurred.

[0222] C. Flow. Blood flow is present so that potential events can be identified and counted.

[0223] D. Stability. The capillary remains completely within the field of view of the movie in every frame.

[0224] E. Visibility. No objects (e.g. air bubbles) that may block the capillaries.

[0225] F. Morphology. The capillaries exhibit distinct arterial and venous limbs.

[0226] G. Criteria A-F must be met for both baseline and severe neutropenia acquisition.

[0227] For all patients, each expert first followed this procedure for the videos in the baseline state. The goal was to obtain pairs of capillary videos for all patients, with the set of candidate capillaries in severe neutropenia already limited by the selection made during baseline. The resulting capillary video pairs - at least one per patient - were those that met the above criteria in both baseline and severe neutropenia, and by both experts (see Figures 49-60).

[0228] Creating a video of capillaries Based on the raw movie data and the capillary selection procedure described above, individual capillary movies were created based on: (a) the definition on the first movie frame of a rectangular region of interest surrounding each of the capillaries of interest, followed by (b) a video motion compensation software that locally corrects for camera movement and ensures that the position of each capillary remains stable within the corresponding region of interest throughout the duration of the movie. Note that all raw movies were first smoothed, i.e., their local intensity was normalized through Gaussian filtering to remove potential effects of non-uniform illumination.

[0229] Step (a) was performed based on a simple graphical user interface, while step (b) was performed based on a custom-made motion compensation algorithm. Both implementations were done in MATLAB. Based on the raw video and a given rectangular region of interest, the algorithm applies a rigid body registration technique to output a motion-compensated capillary video, specifically, this technique assumes that the potential camera movements in the region of interest are simply a combination of X and Y translations, excluding rotations, to align every video frame with the first one.

[0230] Although frame translation in the raw video may introduce distortions, visual inspection of the alignment results was successful when applied to each capillary field separately. As a similarity metric and optimization criterion, the algorithm uses mutual information, an information-theoretic measure that handles small contrast changes and ensures accurate sub-pixel alignment. Prior to this alignment process, a preliminary coarse alignment step is performed to ensure proper initialization. This initialization step performs a pre-alignment of the frames based on cross-correlation analysis of pixel values ​​and spatial gradients.

[0231] Evaluation of the event Based on a graphical user interface, three human raters followed specific visual criteria to identify all events in the capillary movies. Under these visual criteria, the generation of RBC-depleted regions as a consequence of the passage of WBCs through capillaries of approximately the same diameter was included. Specifically, a moving light-absorbing gap refers to an event with the following characteristics:

[0232] In one example, the event is significantly brighter than the surrounding capillary flow. An event can be identified as a distinct object that moves along the capillary flow, or, in yet another example, the event occupies the entire capillary diameter and extends along the direction of flow.

[0233] Raters were blinded to each other and to the corresponding blood analysis, physiological status, patient, and time information. Each rater labeled the corresponding frame and spatial location within the capillary where these events occurred.

[0234] The indexing of the videos made available to the assessors for the identification and counting of events was obtained by randomly shuffling the names of the original videos, so that the assessors could not access the original indexing, even if the corresponding content was similar. Furthermore, the amount of frames was always the same for all videos obtained from the same patient. The corresponding files did not contain any side information that would allow such an identification. All videos - both in the non-shuffled and shuffled indexed version - were anonymized in the sense that there was no information and naming in the video content that could be used to identify the patient or the neutropenic state. After blinded evaluation, all marked events could be visualized based on the specifically developed method.

[0235] statistical analysis The counts obtained from all three independent raters allow the determination of the quality of the joint assessment, i.e., whether one single expert, or two or more experts, agreed to observe the same event. By convention, it is assumed that at least R raters jointly mark a given event, even if the average mark time from at least R raters is within a maximum of 10 frames (1 / 6 seconds) from each other, which is substantially smaller than the expected event rate in the capillary movie (see FIG. 46 below). It does not require precise spatial superposition of the labels. The particular case of interest is the agreement of the majority of raters, R being 2 or greater in this setting, which results in a confirmed event. The counting was then performed accordingly, i.e., the events from each capillary movie were summed accordingly.

[0236] Validated counts for multiple capillaries were combined for each patient (see FIG. 41 and FIG. 44A and FIG. 44B). This is because the accuracy of individual counts in a single capillary is (a) limited by shot noise proportional to the square root of the amount of counts, (b) limited by potential WBC phenomena, e.g. margination, that do not meet the event criteria and do not occur in some capillaries, (c) dependent on the specific capillary geometry and flow rate, and (d) dependent on the specific positioning of the capillary in the dynamics of the underlying capillary network. Given a fixed amount of capillary count combinations per patient (FIG. 44A and FIG. 44B), the corresponding capillary set was picked randomly and the results were calculated based on 10,000 trials. This is akin to Monte Carlo integration, which allows to effectively handle an otherwise intractable exponentially increasing amount of combinations between patients.

[0237] When comparing counts between capillaries or their combined sets, the analytical tool used was the Wilcoxon signed rank test for paired data, which avoids the statistical assumption that counts are normally distributed while testing / rejecting the hypothesis H0 that there is no difference between counts observed at baseline and severe neutropenia for the same capillary. Furthermore, this test generates a receiver operating characteristic (ROC) curve and corresponding area under the curve (AUC) value, the latter testing the performance of the binary classification between baseline and severe neutropenia as a function of varying count thresholds.

[0238] Visualization of marked events To visualize the marked events in the capillaries, throughout the duration of the movie We developed a method to simultaneously visualize a capillary movie frame of interest and the corresponding ST intensity profile of that capillary, both in the vicinity of a given frame. This allows us to visualize events both explicitly, as moving targets through the corresponding movie frames, and as fixed profiles in the ST map representation. In the context of a clinical trial, this visualization method allowed for a large proportion of raters' agreement as well as a retrospective analysis of the distribution of labeled events in the movies.

[0239] The ST-map concept of capillary flow visualization has been described above. Motivated by the fact that events can be conveniently observed in the above representation, WBC-related events are expected to appear as dense, high-contrast, sparse, unidirectional trajectories. These characteristics are also related to the visual criteria defined for the rater, e.g., the brightness of the events. The ST-map makes the events marked by the rater appear as clearly defined, prominent trajectories surrounded by a dark background (see Figure 39E above).

[0240] To create ST maps, the method extracts capillary intensities - as averages over cross sections - based on segmented capillary boundaries, as a function of time, and as a function of cumulative capillary length. To improve visualization of event trajectories, map values ​​were normalized by subtracting local temporal averages as they were acquired for 50 frames before and after each time point. An initial capillary segmentation step was performed based on images extracted from the corresponding registered videos.

[0241] Because capillary profiles may be incomplete within a single video frame due to the presence of absorption gaps in the flow, temporally integrated images were extracted for the nailfold capillaries, in which time-varying features of the capillary flow were also enhanced to maximize the contrast between the capillaries and their surroundings. This approach is also related to the concept of motion contrast enhancement. Specifically, the images used for segmentation were obtained through the integration of time-frequency components whose duration was empirically chosen to be in the [0.25, 1.5] s interval.

[0242] The boundaries of the capillaries were first manually segmented in a first step and then refined utilizing an active contour technique. The segmentations of both capillary boundaries were then automatically resampled to contain 1,000 points each and so that the centers of all point pairs at the same index of both boundaries were on the medial axis of the capillary, where the medial axis is the locus of all circles inscribed in the capillary. Finally, based on this segmentation, the separation between the arterial limb, the venous limb, and the loop of the capillary was defined on a case-by-case basis for visualization (see Figures 45A-C).

[0243] Acquisition time under shot noise The video acquisition time t was set to 1 min because it is long enough to allow sampling of a significantly higher amount of events N in the baseline case compared to the severe neutropenic case while still being suitable for a clinical setting. Even under the shot noise resulting from the quantized nature of the events, the distribution of counts associated with both cases is consistent with their respective mean N, as expected from the calculations detailed below. b and N n It is expected that the seam will separate within at least one standard deviation of .

[0244] To determine this result, we use the lower bound case (C b= 1,500 neutrophils / μL) and the upper limit for severe neutropenia (C n The worst-case scenario was considered using a capillary diameter (D = 15 μm) and flow velocity (v = 800 μm / μL), which is the most difficult to distinguish, since the difference in cell concentration C from both categories is minimized. Then, typical values ​​from the literature were used to determine the capillary diameter (D = 15 μm) and flow velocity (v = 800 μm / μL). seconds), which allows the expected event volume to be estimated from the concentration.

number

[0245] Figure 35 shows the number of events labeled by one single rater. As obtained in the 98 capillaries of this study, baseline counts with no agreement between two or more experts (blue dots) show no statistically significant difference (P=0.12; Wilcoxon signed rank) with respect to the corresponding counts of severe neutropenia (red squares). This result indicates that labeled events by a single rater are less objective and contain less information than those by consensus of multiple raters. Capillary pairs are grouped by patient ID.

[0246] Figures 44A and 44B show the discrimination between baseline and severe neutropenia using capillary collection. Figure 44A shows the number of event counts resulting from aggregating N=1, 2, 3, 4, 5 capillaries per patient. Figure 44B shows the ROC curves for classification of baseline vs severe neutropenia based on aggregating N=1, 2, 3, 4, 5 capillaries per patient. Patient-level distribution of the resulting counts shows that their discriminatory power increases with the amount of capillaries combined per patient. Specifically, the minimum area under the curve (AUC) consistently increases with the amount of capillaries combined from 0.61 to 0.84, 0.92, 0.96, and 1.00 when including 1, 2, 3, 4, and 5 capillaries, respectively.

[0247] Figures 45A-C show examples of capillary segmentation. Figure 45A shows a capillary obtained from patient 02. Figure 45B shows the same capillary with supervised segmentation (red). Scale bar is 20 μm. Figure 45C shows that the separation between the arterial limb (green), venous limb (blue), and loop (red) of the capillary can be defined on a case-by-case basis for visualization.

[0248] Figure 46 shows the expected event volume per minute of a capillary under shot noise. Assuming typical capillary diameter and velocity values ​​taken from the literature, the baseline expected event volume (blue, 1,500 neutrophils / μL) exceeds the severely neutropenic counterpart (red; 500 neutrophils / μL) in a single minute acquisition, even under shot noise. Shown are the expected count averages (median points) along with the expected variation resulting from shot noise up to the standard deviation (bars).

[0249] Figure 47 shows the distribution of capillary diameters at event locations. The distribution of capillary diameter values ​​at the locations where events were labeled by three blinded assessors is shown. The detected events tend to appear in capillary segments in approximately the same size range as WBCs, i.e., [10-20] μm, thus confirming observations made in prior literature and the usefulness of events as a surrogate for WBCs.

[0250] Figures 48A and 48B show ST maps of capillaries with high vs. low rates of confirmed events. For each event, the first click from each of the three human raters is displayed as a red, blue, and green dot, respectively. Inter-rater agreement was based on the severity of the event, with the highest score being 1.5, followed by 1.2, and the lowest score being 1.5. It was higher in the baseline capillaries (upper map) compared to the neutropenic (lower map) cases, indicating that the baseline events corresponded to more target physical phenomena.

[0251] FIG. 49 shows the selection of capillaries from both experts in a raw movie pair from patient 01, region 1. Movies acquired in baseline and severe neutropenic conditions are displayed. Green boxes outline the selected capillary pairs that met the quality criteria by each of the two experts for both the baseline and severe neutropenic acquisitions. Red areas / capillaries are discarded for not meeting the quality criteria, i.e., in this case (b) lack of focus, (c) lack of blood flow, and (d) movement outside the field of view. Red lines / corners in the baseline movie outline the useful field of view, outside which a capillary must be discarded when it disappears for some frames due to camera movement during the acquisition. Yellow boxes outline capillaries that were initially selected in baseline but later discarded due to lack of suitability during the severe neutropenic acquisition (g). Both experts independently performed the above selection process, then only capillaries that were agreed upon by both experts (black numbers) were selected and the rest (red numbers) were discarded.

[0252] FIG. 50 shows the selection of capillaries from both experts in a raw movie pair from patient 01, region 2. Movies acquired in baseline and severe neutropenic conditions are displayed. Green boxes outline the selected capillary pairs that met the quality criteria by each of the two experts for both the baseline and severe neutropenic acquisitions. Red areas / capillaries are discarded for not meeting the quality criteria, i.e., in this case (b) lack of focus, (d) movement outside the field of view, and (e) occlusion. Red lines / corners in the baseline movie outline the useful field of view, outside which a capillary must be discarded when it disappears for some frames due to camera movement during the acquisition. Yellow boxes outline capillaries that were initially selected in baseline but later discarded due to lack of suitability during the severe neutropenic acquisition (g). Both experts independently performed the above selection process, then only capillaries that were agreed upon by both experts (black numbers) were selected and the rest (red numbers) were discarded.

[0253] FIG. 52 shows the selection of capillaries from both experts in a raw movie pair from patient 02, region 1. Movies acquired in baseline and severe neutropenic conditions are displayed. Green boxes outline the selected capillary pairs that met the quality criteria by each of the two experts for both the baseline and severe neutropenic acquisitions. Red areas / capillaries are discarded for not meeting the quality criteria, i.e., in this case, due to (a) insufficient lighting, (b) lack of focus, (c) lack of blood flow, and (e) occlusion. Red lines / corners in the baseline movie outline the useful field of view, outside which a capillary must be discarded when it disappears for some frames due to camera movement during the acquisition. Yellow boxes outline capillaries that were initially selected in baseline but later discarded due to lack of suitability during the severe neutropenic acquisition (g). Both experts independently performed the above selection process, and then only capillaries that were in agreement with both experts (black numbers) were selected and the rest (red numbers) were discarded.

[0254] FIG. 52 shows the selection of capillaries from both experts in a raw movie pair from patient 02, region 2. Movies acquired in baseline and severe neutropenic conditions are displayed. Green boxes outline the selected capillary pairs that met the quality criteria by each of the two experts for both the baseline and severe neutropenic acquisitions. Red regions / capillaries were selected for not meeting the quality criteria, i.e., in this case there is (b) lack of focus, (c) lack of blood flow, (e) occlusion, and (f) lack of clear morphology. The red lines / corners in the baseline movies outline the field of view, outside which capillaries must be discarded when they disappear for some frames due to camera movement during acquisition. The yellow boxes outline capillaries that were initially selected in the baseline but later discarded due to lack of suitability during the severe neutropenic acquisition (g). Both experts independently performed the selection process, after which only capillaries that were in agreement between both experts (black numbers) were included and the rest (red numbers) were discarded.

[0255] FIG. 53 shows the selection of capillaries from both experts in a raw movie pair from patient 03. Movies acquired in baseline and severe neutropenic conditions are displayed. Green boxes outline the selected capillary pairs that met the quality criteria by each of the two experts for both the baseline and severe neutropenic acquisitions. Red areas / capillaries are discarded for not meeting the quality criteria, i.e., in this case, due to (a) insufficient lighting, (b) lack of focus, (c) lack of blood flow, (d) movement outside the field of view, and (e) occlusion. Red lines / corners in the baseline movie outline the useful field of view, outside which a capillary must be discarded when it disappears for some frames due to camera movement during the acquisition. Yellow boxes outline capillaries that were initially selected in baseline but later discarded due to lack of suitability during the severe neutropenic acquisition (g). Both experts independently performed the above selection process, then only capillaries that were agreed upon by both experts (black numbers) were selected and the rest (red numbers) were discarded.

[0256] FIG. 54 shows the selection of capillaries obtained from both experts in raw movie pairs from patient 04. Movies acquired in baseline and severe neutropenic conditions are displayed. Green boxes outline the selected capillary pairs that met the quality criteria by each of the two experts for both the baseline and severe neutropenic acquisitions. Red areas / capillaries are discarded for not meeting the quality criteria, i.e., in this case (b) due to lack of focus. Red lines / corners in the baseline movie outline the useful field of view, outside which a capillary must be discarded when it disappears for some frames due to camera movement during the acquisition. Yellow boxes outline capillaries that were initially selected in the baseline but later discarded due to lack of suitability during the severe neutropenic acquisition (g). Both experts independently performed the above selection process, after which only capillaries that were in agreement between both experts (black numbers) were selected and the rest (red numbers) were discarded.

[0257] FIG. 55 shows the selection of capillaries obtained from both experts in the raw movie pairs from patient 05. Movies acquired in baseline and severe neutropenic conditions are displayed. The green boxes outline the selected capillary pairs that met the quality criteria by each of the two experts for both the baseline and severe neutropenic acquisitions. The red areas / capillaries are discarded for not meeting the quality criteria, i.e., in this case (b) lack of focus, (c) lack of blood flow, (d) movement outside the field of view, and (e) occlusion. The red lines / corners in the baseline movie outline the useful field of view, outside which the capillary must be discarded when it disappears for some frames due to camera movement during the acquisition. The yellow boxes outline the capillaries that were initially selected in the baseline but were later discarded due to lack of suitability during the severe neutropenic acquisition (g). Both experts independently performed the above selection process, then only capillaries that were agreed upon by both experts (black numbers) were selected and the rest (red numbers) were discarded.

[0258] Figure 56 shows a selection of capillaries obtained from both experts in a raw movie pair from patient 06. Movies acquired in baseline and severe neutropenic conditions are shown. The green boxes outline the selected capillary pairs that met the quality criteria by each of the two experts for both the baseline and the severe neutropenic acquisitions. The red areas / capillaries are discarded for not meeting the quality criteria, i.e. in this case (b) lack of focus, (c) lack of blood flow, (d) movement outside the field of view, and (f) lack of clear morphology. The red lines / corners in the baseline movie outline the useful field of view, outside which the capillary must be discarded when it disappears for some frames due to camera movement during the acquisition. The yellow boxes outline the capillaries that were initially selected in the baseline but were later discarded due to their lack of suitability during the severe neutropenic acquisition (g). Both experts independently performed the above selection process, after which only the capillaries that were in agreement between both experts (black numbers) were included and the rest (red numbers) were discarded.

[0259] FIG. 57 shows the selection of capillaries obtained from both experts in raw movie pairs from patient 07. Movies acquired in baseline and severe neutropenic conditions are displayed. Green boxes outline the selected capillary pairs that met the quality criteria by each of the two experts for both the baseline and severe neutropenic acquisitions. Red areas / capillaries are discarded for not meeting the quality criteria, i.e., in this case, due to (a) insufficient lighting, (b) lack of focus, (d) movement outside the field of view, and (e) occlusion. Red lines / corners in the baseline movie outline the useful field of view, outside which a capillary must be discarded when it disappears for some frames due to camera movement during the acquisition. Yellow boxes outline capillaries that were initially selected in baseline but later discarded due to lack of suitability during the severe neutropenic acquisition (g). Both experts independently performed the above selection process, then only capillaries that were agreed upon by both experts (black numbers) were selected and the rest (red numbers) were discarded.

[0260] FIG. 58 shows the selection of capillaries obtained from both experts in raw movie pairs from patient 08. Movies acquired in baseline and severe neutropenic conditions are displayed. Green boxes outline the selected capillary pairs that met the quality criteria by each of the two experts for both the baseline and severe neutropenic acquisitions. Red areas / capillaries are discarded for not meeting the quality criteria, i.e., in this case (b) lack of focus, (d) movement outside the field of view, (e) occlusion, and (f) lack of clear morphology. Red lines / corners in the baseline movie outline the useful field of view, outside which a capillary must be discarded when it disappears for some frames due to camera movement during the acquisition. Yellow boxes outline capillaries that were initially selected in baseline but later discarded due to lack of suitability during the severe neutropenic acquisition (g). Both experts independently performed the above selection process, then only capillaries that were agreed upon by both experts (black numbers) were selected and the rest (red numbers) were discarded.

[0261] Figure 59 shows the selection of capillaries obtained from both experts in the raw movie pairs from patient 09. Movies acquired in baseline and severe neutropenic conditions are displayed. The green boxes outline the selected capillary pairs that met the quality criteria by each of the two experts for both the baseline and severe neutropenic acquisitions. The red areas / capillaries are discarded for not meeting the quality criteria, i.e., in this case, due to (a) insufficient lighting, (b) lack of focus, (c) lack of blood flow, and (e) occlusion. The red lines / corners in the baseline movie outline the useful field of view, outside which the capillary must be discarded when it disappears for some frames due to camera movement during the acquisition. The yellow boxes outline the capillaries that were initially selected in the baseline but were later discarded due to lack of suitability during the severe neutropenic acquisition (g). Both experts independently performed the above selection process, after which both experts Only capillaries that were agreed upon by the panel members (black numbers) were included, and the rest (red numbers) were discarded.

[0262] FIG. 60 shows the selection of capillaries obtained from both experts in raw movie pairs from patient 10. Movies acquired in baseline and severe neutropenic conditions are displayed. Green boxes outline the selected capillary pairs that met the quality criteria by each of the two experts for both the baseline and severe neutropenic acquisitions. Red areas / capillaries are discarded for not meeting the quality criteria, i.e., in this case (b) lack of focus, (d) movement outside the field of view, and (f) lack of clear morphology. Red lines / corners in the baseline movie outline the useful field of view, outside which a capillary must be discarded when it disappears for some frames due to camera movement during the acquisition. Yellow boxes outline capillaries that were initially selected in baseline but later discarded due to lack of suitability during the severe neutropenic acquisition (g). Both experts independently performed the above selection process, then only capillaries that were agreed upon by both experts (black numbers) were selected and the rest (red numbers) were discarded.

[0263] Each of the following references is incorporated herein by reference in its entirety.

[0264] Tsukada, Kosuke, et al. Image correlation method for measuring blood flow velocity in microcirculation:correlationwindow'simulation and in vivo image analysis." Physiological measurement 21.4(2000):459.

[0265] Langeder, Florian, and Bernhard G. Zagar." Image processing strategies to accurately measure red blood cell motion in superficial capillaries." Systems,Signals and Devices,2009.SSD’09.6th International Multi-Conference on.IEEE,2009.

[0266] Wu,Chih-Chieh,et al."Red blood cell vel ocity measurements of complete capillary in finger nail-fold using optical flow estimation." Microvascular research 78.3 (2009):319-324.

[0267] Huang,Tzung-Chi,et al."Experimental est imation of blood flow velocity through simulation of intravital microscopic imaging in micro-vessels by different image processing methods." Microvascular resea rch 80.3(2010):477-483.

[0268] Wu,Chih-Chieh,et al."Accuracy evaluatio n of RBC velocity measurement in nail-fold capillaries." Microvascular research 81.3(2011):252-260.

[0269] Lo,Lun-Chien,et al."Pseudo three-dimens ional vision-based nail-fold morphological and hemodynamic analysis." Computers in biology and medicine 42.9(2012):873-884.

[0270] Wang,Mingyi,et al."Full-field velocity imaging of red blood cells in capillaries with spatiotemporal demodulation autocorrelation." Journal of biomedical optic s 21.3(2016):036007-036007.

[0271] J.Crawford,D.C.Dale,and G.H.Lyman(2004).Chemotherapy‐induced neutropenia:risks,consequences,and new directions for its management.Cancer,100(2),228-237.

[0272] M.E.van Wolfswinkel,K.Vliegenthart-Jongbloed,M.de Mendonca Melo,P.C.Wever,M.B.McCall,R.Koelewijn,J.J.van Hellemond,and P.J.van Genderen(2013).Predictive value of lymphocytopenia and the neutrophil-lymphocyte count ratio for severe imported malaria.Malar.J.,12(1),101.

[0273] T.Honda,T.Uehara,G.Matsumoto,S.Arai,and M.Sugano(2016).Neutrophil left shift and white blood cell count as markers of bacterial infection.Clin.Chim.Acta,457,46-53.

[0274] T.B.Newman,D.Draper,K.M.Puopolo,S.Wi,and G.J.Escobar(2014).Combining immature and total neutrophil counts to predict early onset sepsis in term and late preterm newborns:use of the I / T2.Pediat.Infect.Dis.J.,33(8),798.

[0275] A.Velo-Garcia,S.G.Castro,and D.A.Isenberg(2016).The diagnosis and management of the haematologic manifestations of lupus.J.Autoimmun.,74,139-160.

[0276] D.C.Dale(2014).Understanding neutropenia.Curr.Opin.Hematol.,21(1),1-2.

[0277] V.S.Hollis,J.A.Holloway,S.Harris,D.Spencer,C.van Berkel,and H.Morgan(2012).Comparison of venous and capillary differential leukocyte counts using a standard hematology analyzer and a novel microfluid ic impedance cytometer.PloS one,7(9),e43702.

[0278] C.L.Ghai(2012).A textbook of practical physiology.JP Medical Ltd.

[0279] S.Sharma,J.Zapatero-Rodriguez,P.Estrela,and R.O’Kennedy(2015).Point-of-care diagnostics in low resource settings:present status and future role of microfluidics.Biosensors,5(3),577-601.

[0280] Y.Mendelson(1992).Pulse oximetry:theory and applications for noninvasive monitoring.Clin.Chem.,38(9),1601-1607.

[0281] A.Trotti,A.D.Colevas,A.Setser,V.Rusch,D.Jaques,V.Budach,C.Langer,B.Murphy,R.Cumberlin,C.N.Coleman,and P.Rubin(2003,July).CTCAE v3.0:development of a comprehensive grading system for the adverse effects of cancer treatment.In Semin.Radiat.Oncol.(Vol.13,No.3,pp.176-181).WB Saunders.

[0282] G.H.Lyman,M.S.Poniewierski,J.Crawford,D.C.Dale,and E.Culakova(2015).Cost of Hospitalization in Patients with Cancer and Febrile Neutropenia and Impact of Comorbid Conditions.Blood,126(23),2089-2089.

[0283] G.H.Lyman、C.H.Lyman、O.Agboola、and Anc Study Group.(2005).Risk models for predicting chemotherapy-induced neutropenia.Oncologist,10(6),427-437.

[0284] J.De Naurois,I.Novitzky-Basso,M.J.Gill,F.M.Marti,M.H.Cullen,F.Roila,and ESMO Guidelines Working Group.(2010).Management of febrile neutropenia:ESMO clinical practice guidelines.Ann.Oncol.,21(suppl 5),v252-v256.

[0285] G.W.Schmid-Schonbein,S.Usami,R.Skalak,and S.Chien(1980).The interaction of leukocytes and erythrocytes in capillary and postcapillary vessels.Microvasc.Res.,19(1),45-70.

[0286] S.H.Sinclair,M.Azar-Cavanagh,K.A.Soper, R.F.Tuma,and H.N.Mayrovitz(1989).Investigation of the source of the blue field entoptic phenomenon.Invest.Ophthalmol.Vis.Sci.,30(4),668-673.

[0287] A.Roggan,M.Friebel,K.Dorschel,A.Hahn,and G.Muller(1999).Optical properties of circulating human blood in the wavelength range 400-2500 nm.J.Biomed.Opt.,4(1),36-46.

[0288] A.Uji,M.Hangai,S.Ooto,K.Takayama,N.Arakawa,H.Imamura,K.Nozato,and N.Yoshimura(2012).The source of moving particles in parafoveal capillaries detected by adaptive optics scanning laser ophthalmoscopy.Invest.Ophthalmol.Vis.Sci.,53(1),171-178.

[0289] U.Schmidt-Gross(1954).Entoptische Beurteilung der Leukocytenzahl.Klin.Wochenschr.,32(33-34),817-819.

[0290] T.Rimmer,E.M.,Kohner,and J.M.Goldman(1988).Retinal blood velocity in patients with leukocyte disorders.Arch.Ophthalmol.,106(11),1548-1552.

[0291] G.Fuchsjager-Mayrl,M.Malec,E.Polska,B.Jilma,M.Wolzt,and L.Schmetterer(2002).Effects of granulocyte colony stimulating factor on retinal leukocyte and erythrocyte flux in the human retina.Invest.Ophthalmol.Vis.Sci.,43(5),1520-1524.

[0292] C.E.Curtis,W.G.Iacono,and M.Beiser(1999).Relationship between nailfold plexus visibility and clinical,neuropsychological,and brain structural measures in schizophrenia.Biol.Psychiatry,46(1),102-109.

[0293] F.Lefford and J.C.Edwards(1986).Nailfold capillary microscopy in connective tissue disease:a quantitative morphological analysis.Ann.Rheum.Dis.,45(9),741-749.

[0294] Z.Nagy and L.Czirjak(2004).Nailfold digital capillaroscopy in 447 patients with connective tissue disease and Raynaud’s disease.J.Eur.Acad.Dermatol.Venereol.,1 8(1),62-68.

[0295] L.K.Mercer,T.L.Moore,H.Chinoy,A.K.Murray,A.Vail,R.G.Cooper,and A.L.Herrick(2010).Quantitative nailfold video capillaroscopy in patients with idiopathic inflammatory myopathy.Rheumatology,49(9),1699-1705.

[0296] A.K.Murray,A.Vail,T.L.Moore,J.B.Manning,C.J.Taylor,and A.L.Herrick(2012).The influence of measurement location on reliability of quantitative nailfold videocapillaroscopy in patients with SSc.Rheumatology,kes007.

[0297] H.M.Hofstee,E.H.Serne,C.Roberts,R.Hesselstrand,A.Scheja,T.L.Moore,M.Wildt,J.B.Manning,A.V.Noordegraaf,A.E.Voskuyl,and A.L.Herrick(2011).A multicentre study on the reliability of qualitative and quantitative nail-fold videocapillaroscopy assessment.Rheumatology,ker403.

[0298] S.J.Simnett,L.A.Stewart,J.Sweetenham,G.Morgan,and P.W.Johnson(2000).Autologous stem cell transplantation for malignancy:a systematic review of the literature.Clin.Lab.Haematol.,22(2),61-72.

[0299] L.Golan,D.Yeheskely-Hayon,L.Minai,E.J.Dann,and D.Yelin(2012).Noninvasive imaging of flowing blood cells using label-free spectrally encoded flow cytometry.Biomed.Opt.Express,3(6),1455-1464.

[0300] N.Mugii,M.Hasegawa,Y.Hamaguchi,C.Tanaka,K.Kaji,K.Komura,I.Ueda-Hayakawa,S.Horie,M.Ikuta,K.Tachino,F.Ogawa,S.Sato,M.Fujimoto,and K.Takehara(2009).Reduced red blood cell velocity in nail-fold capillaries as a sensitive and specific indicator of microcirculation injury in systemic sclerosis.Rheumatology,48(6),696-703.

[0301] A.Bourquard,I.Butterworth,A.Sanchez-Ferro,L.Giancardo,L.Soenksen,C.Cerrato,R.Flores,and C.Castro-Gonzalez(2015,August).Analysis of white blood cell dynamics in nailfold capillaries.In Conf.Proc.IEEE Eng.Med.Biol.Soc.,2015 37th Annual International Conference of the IEEE(pp.7470-7473).

[0302] C.Castro-Gonzalez,I.Butterworth,A.Bourquard,and A.Sanchez-Ferro(2015).Systems,apparatus,and methods for analyzing blood cell dynamics.U.S.Patent Application No.14 / 951,260.

[0303] C.C.Wu,G.Zhang,T.C.Huang,and K.P.Lin(2009).Red blood cell velocity measurements of complete capillary in finger nail-fold using optical flow estimation.Microvasc.Res.,78(3),319-324.

[0304] T.C.Huang,W.C.Lin,C.C.Wu,G.Zhang,and K.P.Lin(2010).Experimental estimation of blood flow velocity through simulation of intravital microscopic imaging in micro-vessels by different image processing methods.Microvasc.Res.,80(3),477-483.

[0305] C.C.Wu,W.C.Lin,G.Zhang,C.W.Chang,R.S.Liu,K.P.Lin,and T.C.Huang(2011).Accuracy evaluation of RBC velocity measurement in nail-fold capillaries.Microvasc.Res.,81(3),252-260.

[0306] C.H.Wu,T.D.Wang,C.H.Hsieh,S.H.Huang,J.W.Lin,S.C.Hsu,H.T.Wu,Y.M.Wu,and T.M.Liu(2016).Imaging Cytometry of Human Leukocytes with Third Harmonic Generation Microscopy.Sci.Rep.,6.

[0307] T.C.Shih,G.Zhang,C.C.Wu,H.D.Hsiao,T.H.Wu,K.P.Lin,and T.C.Huang(2011,January).Hemodynamic analysis of capillary in finger nail-fold using computational fluid dynamics and image estimation.Microvasc.Res.81(1),68-72.

[0308] D.Mattes,D.R.Haynor,H.Vesselle,T.K.Lewellyn,and W.Eubank(2001,July).Nonrigid multimodality image registration.In Med.Imaging(pp.1609-1620).International Society for Optics and Photonics.

[0309] G.W.Schmid-Schonbein,R.Skalak,S.Usami,and S.Chien(1980).Cell distribution in ca pillary networks.Microvasc.Res.,19(1),18-44.

[0310] J.H.Lee,J.Jimenez,I.R.Butterworth,C.Castro-Gonzalez,S.K.Shukla,B.Marti-Fuster,L.Elvira,D.S.Boning,and B.W.Anthony(2015,October).Measurement of very low concentration of microparticles in fluid by single particle detection using acoustic radiation force induced particle motion.In 2015 IEEE Int.Ultrason.Symp.(pp.1-4).

[0311] J.Tam,P.Tiruveedhula,and A.Roorda(2011).Characterization of single-file flow through human retinal parafoveal capillaries using an adaptive optics scanning laser ophthalmoscope.Biomed.Opt.Express,2(4),781-793.

[0312] P.D.Allen,C.J.Taylor,A.L.Herrick,and T.Moore(1998,September).Enhancement of Temporally Variable Features in Nailfold Capillary Patterns.In BMVC(pp.1-10).

[0313] M.Kass,A.Witkin,and D.Terzopoulos(1988).Snakes:Active contour models.Int.J.Comput.Vision,1(4),321-331.

[0314] Conclusion While various inventive embodiments have been described and illustrated herein, those skilled in the art will readily envision a variety of other means and / or structures for performing the functions and / or obtaining the results and / or one or more advantages described herein, and each such variation and / or modification is intended to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary, and that the actual parameters, dimensions, materials, and / or configurations will depend on the particular application or applications for which the teachings of the present invention are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, numerous equivalents to the specific inventive embodiments described herein. Thus, the foregoing embodiments are presented by way of example only, and it will be understood that within the scope of the appended claims and their equivalents, the inventive embodiments may be practiced otherwise than as specifically described and claimed. The inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the inventive scope of the present disclosure, if such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent.

[0315] The above embodiments can be implemented in any of numerous ways. For example, the embodiments disclosed herein may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code may be implemented in a single command. The present invention may be implemented on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers.

[0316] Further, it should be understood that a computer may be embodied in any of a number of forms, such as a rack-mounted computer, a desktop computer, a laptop computer, or a tablet computer, etc. In addition, a computer may be embedded within a device having suitable processing capabilities, including a personal digital assistant (PDA), a smart phone, or any other suitable portable or fixed electronic device, rather than a device generally considered a computer.

[0317] A computer may also have one or more input and output devices. These devices can be used, among other things, to present a user interface. Examples of output devices that can be used to provide a user interface include a printer or display screen for a visual representation of the output, and a speaker or other sound generating device for an audible representation of the output. Examples of input devices that can be used for a user interface include keyboards and pointing devices such as mice, touch pads, and digitizer tablets. As another example, a computer may receive input information by voice recognition or in other audible formats.

[0318] Such computers may be interconnected by one or more networks of any suitable form, including local area networks, or wide area networks such as enterprise networks, and intelligent networks (INs) or the Internet. Such networks may be based on any suitable technology and operate according to any suitable protocol, and may include wireless networks, wired networks, or fiber optic networks.

[0319] The various methods or processes outlined herein may be coded as software executable on one or more processors using any one of a variety of operating systems or platforms. In addition, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and compiled as executable machine code or intermediate code that runs on a framework or virtual machine.

[0320] Also, various inventive concepts may be embodied as one or more methods, examples of which have been provided. The acts performed as part of a method may be ordered in any suitable manner. Thus, embodiments may be constructed in which acts are performed in an order different from that illustrated, including performing some acts simultaneously even though shown as sequential acts in the exemplary embodiments.

[0321] All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety.

[0322] All definitions and those used herein should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0323] As used in the specification and claims, the indefinite articles "a" and "an" should be understood to mean "at least one," unless clearly indicated otherwise.

[0324] As used herein and in the claims, the phrase "and / or" should be understood to mean "either or both" of the elements so conjoined, i.e., elements that are conjunctive in some cases and disjunctive in other cases. Multiple elements listed with "and / or" should be interpreted in the same manner, i.e., "one or more" of the elements so conjoined. Other elements, whether related or unrelated to the elements specifically identified, may optionally be present other than the elements specifically identified by the "and / or" clause. Thus, as a non-limiting example, a reference to "A and / or B," when used in conjunction with open-ended language such as "comprising," can refer in one embodiment to only A (optionally including elements other than B), in another embodiment to only B (optionally including elements other than A), in yet another embodiment to both A and B (optionally including other elements), etc.

[0325] As used herein and in the claims, "or" should be understood to have the same meaning as "and / or" defined above. For example, when separating items in a list, "or" or "and / or" shall be interpreted as being inclusive, i.e., including at least one of, but also including two or more of, the number or recited elements, and optionally additional items not in the list. Only terms clearly indicated to the contrary, such as "only one of" or "exactly one of," or "consisting of," when used in the claims, shall refer to the inclusion of exactly one element of the number or recited elements. In general, as used herein, the term "or" shall only be interpreted as indicating exclusive alternatives (i.e., "one or the other but not both") when preceded by an exclusive term, such as "either," "one of," "only one of," or "exactly one of." "Consisting essentially of," when used in the claims, shall have its ordinary meaning as used in the field of patent law.

[0326] As used in this specification and claims, the phrase "at least one" in connection with a list of one or more elements should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of each and every element specifically listed within the list of elements, and not excluding any combination of elements in the list of elements. This definition also allows that elements other than those specifically identified in the list of elements to which the phrase "at least one" refers may optionally be present, whether related or unrelated to the elements specifically identified. Thus, as a non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B" or, equivalently, "at least one of A and / or B") can refer, in one embodiment, to at least one A, optionally including more than one, A (optionally including elements other than B), in the absence of B; in another embodiment, to at least one B, optionally including more than one, B (optionally including elements other than A), in the absence of A; in yet another embodiment, to at least one A, optionally including more than one, A, and at least one B, optionally including more than one, B (optionally including other elements);

[0327] In the claims, as well as in the above specification, all transitional phrases, such as "comprising," "including," "carrying," "having," "containing," "involving," "holding," "composed of," and the like, are open-ended, i.e., including but not limited to. Only the transitional phrases "consisting of" and "consisting essentially of" shall be closed or semi-closed transitional phrases, respectively, as set forth in the United States Patent Office Manual of Patent Examining Procedures, Section 2111.03.

Claims

1. A system for non-invasive in vivo hematological measurements, comprising: a platform for receiving, during use, a body portion of a user having a capillary structure; an imaging device coupled to the platform for acquiring a series of time lapse images of the capillary structure, each image resolving a plurality of capillaries; and an imaging objective coupled to the imaging device, the imaging device configured to provide the series of time lapse images of the capillary structure to a controller; and The controller, processing the series of time lapse images to perform the hematological measurements for one or more of the plurality of capillaries, the hematological measurements including detecting passage of cells within the capillaries that exhibit contrasting light absorption relative to red blood cells in the series of time lapse images; and generating an event count based on the detected cellular events in the one or more capillaries, each cellular event being associated with the passage of a particular type of cell through a capillary of the one or more capillaries; The controller is adapted to A system comprising:

2. The system described in claim 1, wherein the cells are circulating tumor cells or white blood cells (WBCs).

3. The cell is a WBC, The controller: generating an event count based on the detected cellular events in the one or more capillaries, each detected cellular event being associated with the passage of a particular type of WBC in a capillary of the one or more capillaries; comparing the event count to an event count threshold associated with a minimum number of cellular events for classification of the user into a first user type, wherein the one or more capillaries include capillaries having a diameter selected to be approximately equal to or less than a diameter of any of the one or more types of WBCs; and classifying the user into a first user type of a set of user types based on a comparison of the event count to the event count threshold, wherein at least one user type of the set of user types is associated with a neutropenic condition; The system of claim 1 , further adapted to:

4. The system of claim 3, wherein the controller is further adapted to detect each of the cellular events by detecting a light absorption gap during passage of the one or more types of WBCs in a capillary of the one or more capillaries.

5. The system of claim 3, wherein the controller is further adapted to detect each cellular event of the detected cellular events based on a comparison of absorption to red blood cells.

6. The system described in claim 3, further comprising an illumination source (2320) for illuminating the capillary structure with blue light.

7. The system of claim 3, wherein the diameter is between 10 microns and 20 microns.

8. The system of claim 3, wherein the controller is further adapted to apply the series of time lapse images to a neural network to identify the one or more capillaries.

9. The controller: receiving a set of training images related to capillary beds of a portion of a body of a set of training users; and Generate an event counting threshold through supervised learning based on the set of training images. The system of claim 3 , further adapted to:

10. The controller: generating a bounding box for each detected capillary in each image of the series of time lapse images; generating a confidence value related to the likelihood that the detected capillary detection corresponds to a capillary and not to another structure / artifact; and If the reliability value meets or exceeds a predetermined reliability threshold, the detected capillary is included in the one or more capillaries. The system of claim 3 , further adapted to:

11. The system of claim 3, wherein the controller is further adapted to apply a bandpass filter in a separate Fourier domain, the bandpass filter having low frequency and high frequency bandpass parameters that match a range of expected cellular event sizes.

12. The system of claim 3, wherein the controller is further adapted to compile profiles extracted from the series of time lapse images into at least one spatiotemporal profile to analyze WBC events for capillaries of the one or more capillaries.

13. The system described in claim 12, wherein the controller is further adapted to determine a velocity of the WBC from the at least one spatiotemporal profile.

14. The system of claim 12, wherein the controller is further adapted to determine a total number of WBC events per microliter from the at least one spatiotemporal profile.

15. A method for non-invasive in vivo hematological measurements, comprising: receiving a body part of a user having a capillary structure on a platform; acquiring a series of time lapse images of the capillary structure with an imaging device comprising an imaging objective; providing a series of time lapse images of the capillary structure to a controller, the controller being communicatively connected to the imaging device; processing the series of time lapse images by the controller to perform an analysis of blood cell dynamics for one or more of the plurality of capillaries, the hematological measurement including detecting passage of cells through the capillaries that exhibit contrasting light absorption relative to red blood cells in the series of time lapse images; generating, by the controller, an event count based on the detected cellular events in the one or more capillaries, each cellular event of the cellular events being associated with the passage of a particular type of cell through a capillary of the one or more capillaries; A method comprising:

16. The method of claim 15, wherein the cells are circulating tumor cells or white blood cells (WBCs). generating, by said controller, an event count based on the detected cellular events in the one or more capillaries, each detected cellular event being associated with the passage of a particular type of WBC in a capillary of the one or more capillaries; comparing, by the controller, the event count to an event count threshold associated with a minimum number of cellular events for classification of the user into a first user type, the one or more capillaries including capillaries having a diameter selected to be approximately equal to or less than a diameter of any WBC of the one or more types of WBC; classifying, by the controller, the user into a first user type of a set of user types based on a comparison of the event count to the event count threshold, wherein at least one user type of the set of user types is associated with a neutropenic condition; The method of claim 15 further comprising:

18. The method of claim 17, further comprising detecting, by the controller, each cellular event of the detected cellular events based on a comparison of absorption to red blood cells. receiving, by the controller, a set of training images related to capillary beds of body portions of a set of training users; generating an event counting threshold through supervised learning based on the set of training images; 20. The method of claim 17, further comprising: generating, by the controller, a bounding box for each detected capillary in each image of the series of time lapse images; generating, by the controller, a confidence value associated with the likelihood that the detected capillary detection corresponds to a capillary and not to another structure / artifact; including, by the controller, the detected capillary in the one or more capillaries if the reliability value meets or exceeds a predetermined reliability threshold.

20. The method of claim 17, further comprising:

Images