Systems, devices, and methods for non-invasive hematological measurements

Abstract

To solve the problem in which tests to derive the WBC count are invasive and there can be a certain bias of some parameters caused by inherent differences between measurements and true physiological properties.SOLUTION: A system for non-invasive hematological measurement includes a platform to receive a body portion of a user, and an imaging device to acquire a set of images of a capillary bed in the body portion. For each image, a controller detects one or more capillaries in the body portion of a finger so as to identify a first set of capillaries by estimating one or more attributes of each capillary (e.g., structural attributes, flow attributes, imaging attributes, or combinations thereof), where at least one attribute of each capillary meets a predetermined criterion. The controller also identifies a second set of capillaries from the first set of capillaries, so that each capillary of the second set of capillaries is visible in a predetermined number of images of the set of images.SELECTED DRAWING: Figure 23

Description

[Technical Field]

[0001] Cross-references to related applications This application claims priority to U.S. Provisional Patent Application No. 62 / 572,738, filed October 16, 2017, entitled "DEVICE AND METHODS FOR NON-INVASIVE HEMATOLOGICAL MEASUREMENTS," the entire disclosure of which is incorporated herein by reference.

[0002] Application for support This invention was made with government support under grant number U54EB015403, awarded by the National Institutes of Health. The government has certain rights in this invention.

[0003] This disclosure generally relates to systems, apparatus, and methods for analyzing the dynamics of blood cells and cell populations. More specifically, this disclosure relates to systems, apparatus, and methods for extracting information about leukocytes from non-invasive, in vivo, and / or slow-motion imaging. [Background technology]

[0004] White blood cells (WBCs, also called leukocytes) are cells of the immune system that are involved in protecting the body from both infectious diseases and foreign invaders. WBCs can be present not only in the blood but also in the lymphatic system and tissues. Some conditions can trigger a response within the immune system, leading to an increase in the number of WBCs (also called the WBC count). Other conditions can affect WBC production by the bone marrow or the survival of existing WBCs in the circulating system. In either case, these conditions can cause a change (either an increase or a decrease) in the number of circulating WBCs. Therefore, the WBC count can be a relevant physiological parameter for the diagnosis, monitoring, and treatment of various conditions, including, but not limited to, bacterial and viral infections (e.g., pneumonia or meningitis), bone marrow function associated with chemotherapy toxicity, and hematological growth processes such as leukemia.

[0005] In current clinical practice, most tests that derive WBC counts are performed in central clinical laboratories using large-scale equipment. Generally, these ex vivo tests remain invasive as they require the collection of blood samples from patients (usually a vial full of blood for each test). These blood samples are then transported, sorted, and analyzed in clinical laboratories, which can take several days to receive any results. This procedure can be burdensome for patients who require regular WBC counts, or for patients experiencing a condition and their caregivers. Furthermore, due to the ex vivo nature of conventional blood tests, there may be certain biases in some parameters due to the inherent difference between measured values ​​and true physiological characteristics. [Overview of the project]

[0006] The system includes a platform that receives a portion of the user's body during use, and an imaging device coupled to the platform for acquiring a set of images of at least the capillary bed of the portion of the body. The system also includes a controller communicatively coupled to the imaging device for detecting one or more capillaries of the portion of the body in each image of the set of images to identify a first set of capillaries across the set of images. The detection includes estimating one or more attributes of each capillary in the first set of capillaries, which may be one or more structural attributes, one or more flow attributes, one or more image attributes, or the like. This includes combinations thereof. The first attribute of one or more attributes of each capillary in the first capillary set satisfies a predetermined criterion for the first attribute. The controller also identifies the second capillary set from the first capillary set, and as a result, each capillary in the second capillary set is visible in a predetermined number of images of the above image set.

[0007] A method comprising: obtaining a set of images of at least the capillary bed of a portion of a user's body; and identifying a first set of capillaries across the set of images by detecting one or more capillaries of the portion of the body in each image of the set of images. The detection includes estimating one or more attributes of each capillary in the first set of capillaries, which include one or more structural attributes, one or more flow attributes, one or more image attributes, or a combination thereof. The first attribute of one or more attributes of each capillary in the first set of capillaries satisfies a predetermined criterion for the first attribute. The method also includes identifying a second set of capillaries from the first set of capillaries, wherein each capillary in the second set of capillaries is visible in a predetermined number of images of the set of images.

[0008] An apparatus comprising a controller that receives a set of images of the capillary bed of a part of a user's body and detects one or more capillaries in the capillary bed of the finger in each image of the set of images to identify a first set of capillaries across the set of images. The detection includes estimating one or more attributes of each capillary in the first set of capillaries, which include one or more structural attributes, one or more flow attributes, one or more image attributes, or a combination thereof. The first attribute of one or more attributes of each capillary in the first set of capillaries satisfies a predetermined criterion for the first attribute. The controller also identifies a second set of capillaries from the first set of capillaries, so that each capillary in the second set of capillaries is visible in a predetermined number of images of the set of images. The controller also detects a set of cellular events for the set of images and for the second set of capillaries, where each cellular event in this set of cellular events is related to the passage of leukocytes in a capillary in the second set of capillaries. The controller also estimates the event count for a second set of capillaries based on the set of cellular events.

[0009] It should be understood that all combinations of the aforementioned concepts and the additional concepts discussed further below (on the premise that such concepts are not contradictory) are considered to be part of the subject matter of the invention disclosed herein. In particular, all combinations of subject matter described in the claims appearing at the end of this disclosure are considered to be part of the subject matter of the invention disclosed herein. It should also be understood that any terms explicitly used in any disclosure incorporated by reference should be given meanings that most closely correspond to the specific concepts disclosed herein. [Brief explanation of the drawing]

[0010] A patent or application file shall include at least one colored drawing. A copy of this patent or patent application publication accompanied by a colored drawing shall be provided by the Office upon request and payment of the necessary fees.

[0011] Those skilled in the art will understand that the drawings are for illustrative purposes only and are not intended to limit the scope of the subject matter of the invention described herein. The drawings are not necessarily to a fixed proportion, and in some examples, various aspects of the subject matter of the invention disclosed herein may be exaggerated or enlarged in the drawings to facilitate the understanding of different features. In the drawings, similar reference letters generally mean similar features (e.g., functionally similar and / or structurally similar elements).

[0012] [Figure 1] Figure 1 is a flowchart illustrating a method for analyzing the dynamics of blood cells according to some embodiments. [Figure 2] Figure 2 is a flowchart illustrating a method for analyzing blood cell dynamics using spatial-temporal profiles and radon conversion, according to some embodiments. [Figure 3] Figure 3 is an example of an image of the nail fold that can be used in the manner shown in Figures 1 and 2, according to some embodiments. [Figure 4]Figures 4A to 4B are plots of cubic spline interpolation that can be used in the method illustrated in Figures 1 and 2 for resampling user-specified contour points of capillaries, according to some embodiments. [Figure 5] Figure 5 is an image of the nail fold, including two capillaries, a user-specified contour point, and a resampled contour curve, according to one embodiment. [Figure 6] Figures 6A and 6B show representations of the spatial and temporal profiles extracted from the two capillaries shown in Figure 5, according to some embodiments. [Figure 7] Figure 7 shows a representation of the Radon transformation performed on a spatial-temporal profile in some embodiments. [Figure 8] Figures 8A and 8B show representations of the spatial and temporal profiles in the radon domain, with local maximums highlighted to indicate WBC events, according to some embodiments. [Figure 9] Figures 9A and 9B are plots showing experimental results of WBC events occurring in the two capillaries shown in Figure 5, according to some embodiments. [Figure 10] Figure 10 is a plot comparing WBC events obtained by the method illustrated in Figure 2 according to some embodiments with WBC events identified by a trained human assessor. [Figure 11-1] Figures 11A to 11C are schematic diagrams of a system for analyzing the dynamics of blood cells using a finger holder and an imager, according to some embodiments. [Figure 11-2] Same as above. [Figure 11-3] Same as above. [Figure 12-1] Figures 12A and 12B are schematic diagrams of a system for analyzing the dynamics of blood cells using a vertically configured imager, according to some embodiments. [Figure 12-2] Same as above. [Figure 13] Figure 13 is a schematic diagram of a system for analyzing the dynamics of blood cells using a smartphone, according to one embodiment. [Figure 14-1] Figures 14A to 14D are images of systems and apparatus for analyzing the dynamics of blood cells according to some embodiments. [Figure 14-2] Same as above. [Figure 15-1] Figures 15A to 15J illustrate an adapter, according to some embodiments, for capturing images of the nail fold using a smartphone camera for the dynamic analysis of blood cells. [Figure 15-2] Same as above. [Figure 16] Figure 16 is a schematic diagram of a system, according to one embodiment, that includes a smartphone and adapter for capturing images of the nail fold for analysis of blood cell dynamics. [Figure 17] Figures 17A and 17B show images of a system, including a smartphone and adapter, for capturing images of the nail fold for dynamic analysis of blood cells, according to one embodiment. [Figure 18] Figure 18 is an example of an image captured by the system shown in Figures 17A to 17B, according to one embodiment. [Figure 19] Figure 19 is a schematic diagram of a clamp device for performing dynamic analysis of blood cells from images of the nail fold, according to one embodiment. [Figure 20] Figure 20 shows an image of a clamp device used to capture images of the nail fold and analyze the dynamics of blood cells. [Figure 21-1] Figures 21A to 21F and 21G to 21O are images and wireframes of a smartphone adapter, according to some embodiments, for capturing images of the nail fold using a smartphone camera for capillary microscopy and hematological analysis. [Figure 21-2] Same as above. [Figure 21-3] Same as above. [Figure 22] Figures 22A to 22D illustrate a method, according to some embodiments, of using a smartphone adapter to capture images of the nail fold using a smartphone camera for capillary microscopy and hematological analysis. [Figure 23] Figure 23 illustrates a system for non-invasive hematological measurement according to an embodiment. [Figure 24] Figures 24A and 24B illustrate the criteria for tracking capillaries to select appropriate capillaries. Each capillary was tracked using a given identifier (id) over 3600 frames of video. In Figure 24A, the black bars represent capillary segments used to calculate cell count / leukocyte index. An id is selected if its occurrence exceeds a confidence value of ¢=600 (green), or discarded if there are more than three selected capillaries (red). A value of ¢=1 is used if there are fewer than three capillaries. In Figure 24B, capillary segments used to calculate the leukocyte index are shown in green, and others in red. [Figure 25] Figures 25A and 25B illustrate a comparison between the detection of capillaries in raw video using the neural network-based methods / approaches described herein and the results obtained using human expert-based methods. The results are obtained by combining the results of two evaluators (green) versus the results of the neural network analysis (red, yellow where overlapping). [Figure 26]Figures 26A–26C illustrate mean time signals. Figure 26A illustrates an example of an actual mean time signal generated from one of the analyzed capillary videos, showing the first 20 seconds (frames 1–1200; see left) containing a positive peak associated with the detected event, along with a 2-second zoom around the single event (frames 650–769; see right). The zoomed time signal around this exemplary event displays a brightness peak and also a slight “drop” for some of the subsequent duration. Figure 26B illustrates the expected profile of the mean time signal around the passage of a single event, which is associated with the passage of a leukocyte in a capillary (see right). According to the zoomed example in (a), the “drop” in intensity is thought to always occur to the right after the maximum value of the intensity peak, due to the higher accumulation of erythrocytes upstream of the leukocyte. Figure 26C illustrates the expected mean-time signal profile around the passage of plasma gaps in capillaries (see right), and it is thought that no drop occurs when there are no leukocytes and therefore no accumulation of erythrocytes upstream. [Figure 27] Figures 27A and 27B illustrate the detection of cellular events by an automated approach versus manual evaluators. Figure 27A shows an example of events detected on capillary videos by one human evaluator (top row) versus a neural network (bottom row), each with blue and white event markers. Figure 27B illustrates how true positives (TP), false positives (FP), and false negatives (FN) were evaluated for the detection of reference events. On the right, the average F1 scores obtained for three evaluators (red) and algorithms (blue) across 26 analyzed capillary videos are shown, demonstrating that the performance of the neural network is equivalent according to the above metric (F1 isocurves shown by black dashed lines). The expected behavior of the neural network when adjusting the event detection threshold is shown by the red dashed line. [Figure 28]Figures 28A to 28D illustrate an example of an event detection method / approach. Figure 28A - The initial capillary video, including selected frames showing plasma gaps flowing through capillaries, with the direction of flow highlighted by arrows. Figure 28B - A pre-processed video, including an event that appeared as a prominent feature against a dark background using a different method. The reference pixel Pref is shown in red, and additional pixels P1 and P2 belonging to the capillary are shown in blue. Figure 28C - Brightness time signals at the reference pixel positions, P1, and P2, with the effect of the event on the measured brightness (highlighted in green) clearly indicated as a peak. Figure 28D - The final time signal, obtained as the average value between time signals at individual pixel positions after alignment with the reference time signal of Pref, with the threshold level used for counting shown as a vertical blue line. [Figure 29] Figures 29A and 29B illustrate the classification results generated by the fully automated approach disclosed herein, plotted on 116 data points. Figure 29A illustrates a box plot showing the classification of raw videos associated with baseline conditions, i.e., ANC > 500 (blue) vs. raw videos associated with severe neutropenia (ANC < 500). Figure 29B shows the ROC curve associated with the classification (AUC = 0.96). [Figure 30] Figure 30 is a flowchart of a method for non-invasive hematological measurement according to one embodiment. [Figure 31] Figures 31A to 31F illustrate a method for estimating blood volume based on the analysis of pixel intensity values. [Figure 32] Figures 32A to 32C illustrate the method of volume resampling of capillary profiles. [Figure 33] Figure 33 illustrates the subselection of capillaries based on observed size. [Figure 34] Figures 34A and 34B illustrate the subselection of capillaries based on the distribution of arrival times between observed gaps. [Figure 35]Figures 35A to 35C illustrate a device for detecting severe neutropenia based on images of capillaries in the nail fold. [Figure 36] Figure 36 illustrates two different timing points for acquiring images from clinical trial patients using the apparatus shown in Figure 35A. [Figure 37] Figures 37A and 37B show examples of raw images acquired by the apparatus shown in Figure 35A. [Figure 38] Figures 38A to 38E illustrate an example of an optical gap flowing through a capillary. [Figure 39] Figures 39A to 39E show the results of a blinded event assessment using images acquired by the device shown in Figure 35A. [Figure 40] Figure 40 shows the number of confirmed events per minute in all tested capillary pairs. [Figure 41] Figure 41 shows the distinction between baseline and severe neutropenia observed in clinical trials. [Figure 42] Figure 42 illustrates the pre-processing workflow in a clinical trial for processing images acquired by the apparatus shown in Figure 35A. [Figure 43] Figure 43 shows the number of events labeled by a single evaluator in the clinical trial. [Figure 44] Figures 44A and 44B show the distinction between baseline and severe neutropenia using capillary clusters. [Figure 45] Figures 45A to 45C show examples of capillary segmentation. [Figure 46] Figure 46 shows the expected number of events per minute in capillaries under shot noise. [Figure 47] Figure 47 shows the distribution of capillary diameters at the event site. [Figure 48] Figures 48A and 48B show ST maps of capillaries with confirmed high vs. low ratio events. [Figure 49]Figure 49 shows the selection of capillaries obtained from both experts in a pair of raw videos for patient 01, region 1. [Figure 50] Figure 50 shows the selection of capillaries obtained from both experts in a pair of raw videos for patient 01 and region 2. [Figure 51] Figure 51 shows the selection of capillaries obtained from both experts in a pair of raw videos for patient 02, region 1. [Figure 52] Figure 52 shows the selection of capillaries obtained from both experts in a pair of raw videos for patient 02, region 2. [Figure 53] Figure 53 shows the selection of capillaries obtained from both experts in a pair of raw videos for patient 03. [Figure 54] Figure 54 shows the selection of capillaries obtained from both experts in a pair of raw videos for patient 04. [Figure 55] Figure 55 shows the selection of capillaries obtained from both experts in a pair of raw videos for patient 05. [Figure 56] Figure 56 shows the selection of capillaries obtained from both experts in a pair of raw videos for patient 06. [Figure 57] Figure 57 shows the selection of capillaries from both experts in a pair of raw videos for patient 07. [Figure 58] Figure 58 shows the selection of capillaries from both experts in a pair of raw videos for patient 08. [Figure 59] Figure 59 shows the selection of capillaries obtained from both experts in a pair of raw videos for patient 09. [Figure 60] Figure 60 shows the selection of capillaries obtained from both experts in a pair of raw videos for patient 10. [Figure 61]Figures 61A and 61B illustrate the classification results for video segments of specific durations. Figure 61A shows the area under the curve (AUC) of the classification as a function of video segment duration, where the video segment starts at the very beginning of the entire video. It can be seen that the classification performance remains optimal for at least 29 seconds of duration, i.e., nearly half the original 1-minute duration, and remains equal to that based on the entire 1-minute video. Figure 61B illustrates the classification AUC when using different 29-second video segments while varying the start times of each video segment in the entire 1-minute video, demonstrating the stability of the classification results as the resulting AUC values ​​remain nearly constant. [Modes for carrying out the invention]

[0013] The following provides a more detailed description of various concepts and implementations related to systems, devices, and methods for non-invasive hematological measurements. It should be understood that the various concepts introduced above and discussed in more detail below can be implemented in numerous ways. Examples of specific implementations and applications are provided primarily for illustrative purposes, so as to enable those skilled in the art to practice implementations and alternatives that would be obvious to them.

[0014] The figures and implementation examples described below are not intended to limit the scope of implementation of the Application to a single embodiment. Other implementations are possible by replacing some or all of the elements described or illustrated. Furthermore, where certain elements of the disclosed exemplary implementations can be partially or completely implemented using known components, in some cases only some of such known components necessary for understanding the Implementation of the Application are described, and detailed descriptions of other parts of such known components are omitted so as not to obscure the Implementation of the Application.

[0015] Considering the challenges associated with conventional blood tests as discussed above, the inventors have recognized and understood the various advantages of non-invasive in vivo methods for deriving WBC counts. These non-invasive in vivo methods enable more frequent monitoring, fewer clinic visits, and immediate access to testing in areas lacking adequate research facilities or reagents.

[0016] A non-invasive WBC counting method that utilizes the optical properties of WBCs involves measuring WBCs in the nail fold capillaries. They can be used to observe gaps in retinal capillaries or moving particles in oral mucosal capillaries. However, specialized and / or non-portable devices (e.g., adaptive optics and confocal microscopes) may be required to derive WBC counts. Optical devices called capillary microscopes can be used to acquire optical images of the morphology of nail fold capillaries and diagnose rheumatic diseases, but the acquired images require time-consuming analysis by a trained human assessor.

[0017] Overview of non-invasive in vivo analysis of blood cell dynamics Figure 1 is a flowchart illustrating a non-invasive in vivo method for analyzing blood cell dynamics according to some embodiments, which includes, but is not limited to, detecting the number of WBCs passing through a given capillary over a period of time, the velocity of one or more WBCs flowing through the capillary, and the total number of WBC events per μL. In some embodiments, some or all aspects of Method 100 can be implemented by one or more of the systems, apparatuses, and devices described herein, such as system 2300 and / or device 2340, which are described in more detail with respect to Figure 23.

[0018] More specifically, method 100 illustrated in Figure 1 includes a step 110 of normalizing and / or registering a source image of a subject containing capillaries. By normalizing the source image (also known as contrast expansion, histogram expansion, or dynamic arrangement expansion), the range of pixel intensity values ​​may be changed, for example, so that the contrast of the image increases. By registering the source image, the source image may be aligned so that the same pixel location on different images corresponds to the same physical location on the subject. In some embodiments, the source image is captured within a finite time range in which the camera, the subject, or both may be moving. Image registration can address such movement. In image registration, one of the images (e.g., the first image in a sequence, the last image in a sequence, or an image that may contain the desired field of view) may be used as a reference image, and other images in the sequence may be modified by comparing them to the reference image. Modifications may be made to maximize a particular image similarity criterion that quantifies the similarity between the reference image and the other images. Examples of image similarity criteria include, for example, cross-correlation, cross-information, sum of squared differences in intensity, and image uniformity ratio.

[0019] In step 120 of Method 100, capillary segmentation and / or profile extraction are performed. Capillary segmentation may be used to identify capillary segments in either the original or registered image (e.g., by identifying the boundaries of the capillary segments). Profile extraction may be used to extract pixel information within capillary segments for further analysis. Since WBC information is typically contained only in capillaries, it may be useful to extract pixel information within capillary segments and exclude information from other areas of the image. The extracted pixel information may include the location and value (also called intensity) of pixels within the capillary segment. The location of pixels in each image may be represented in 2D Cartesian coordinates, and capillaries may be curved. This location may be useful when transforming images from a 2D Cartesian coordinate system to a different coordinate system, in which case the same point in the same capillary but in different images in the image sequence will have the same coordinates. One example of such a coordinate system is a curvilinear coordinate system that uses a point within a curved capillary as the origin, where any other point has a one-dimensional (1D) coordinate, which is the distance between that point and the origin.

[0020] In step 130 of Method 100, the profiles extracted from the sequence of images are compiled into a single image (also called a spatial-temporal profile) for analyzing the WBC event. The profiles extracted from each image are used for analyzing the WBC event or other purposes. This may include information about the spatial distribution (i.e., location) of elephants. Profiles extracted from different images in an image sequence, taken at different time points, may contain information about WBC events at those different time points. Therefore, a spatial-temporal profile compiled from all of these extracted profiles derived from the entire image sequence can provide rich information related to both the spatial distribution and temporal evolution of WBC events. For example, the tracing of a particular WBC can be visualized from the spatial-temporal profile described above. Also, the velocity of a particular WBC flowing through a capillary can be derived, for example, by considering the time difference between two or more images in an image sequence.

[0021] In step 140 of Method 100, the spatial-temporal profile is processed to detect and / or analyze WBC events. In some embodiments, the processing is manual. The user may identify WBC events by scrutinizing the spatial-temporal profile and, for example, detecting visual gaps within the spatial-temporal profile (e.g., pixels with higher or lower pixel values ​​compared to surrounding pixels). The user may also derive motion traces and flow velocities from the spatial-temporal profile. In some embodiments, the processing is automatic or semi-automatic. For example, the spatial-temporal profile may be converted to, for example, radon domains. WBC events may then be detected based on maximal values ​​in the radon domains. Based on the detected WBC events, additional analyses such as WBC counts and WBC flow velocities may be derived. Furthermore, WBC counts, flow velocities, etc., may be used for subsequent diagnosis, monitoring, and / or treatment of a disease or condition.

[0022] According to some embodiments, systems, apparatus, and methods for analyzing blood cell dynamics are based on in vivo images of capillaries without blood collection or other invasive procedures on the subject's body, and without removing WBCs from their natural environment within the subject's body. Furthermore, these systems, apparatus, and methods can be used for real-time or substantially real-time results. For example, image processing (from source image detection to WBC event detection and WBC count calculation) can be performed while new images are being taken. WBC events may be monitored to examine the subject's body's response to medical treatment, thereby providing feedback on the effectiveness of the treatment. Moreover, the systems, apparatus, and methods described herein can identify WBCs from relatively low-resolution, low-frame-rate, and noisy source images. For example, the source image can be a frame from a video clip captured by a commercially available camera, mobile phone, or other image-capturing device.

[0023] Non-invasive in vivo analysis methods for blood cell dynamics Figure 2 is a flowchart illustrating a method for analyzing blood cell dynamics using radon conversion of the spatial and temporal profiles of capillary images, according to some embodiments. Method 200 is used to detect WBC events from in vivo capillary data related to a subject. In some embodiments, some or all aspects of Method 200 can be implemented by one or more of the systems, apparatuses, and devices described herein, such as system 2300 and / or device 2340, which are described in more detail with respect to Figure 23.

[0024] In step 210 of method 200, capillary data is obtained non-invasively. The capillary data may include multiple images of one or more capillary segments captured over a first period.

[0025] In step 220 of method 200, the contours of one or more capillary segments in the image are specified. In particular, for each image of a group of images, a first two-dimensional (2D) coordinate set may be specified to correspond to the internal contour points of capillary segments visible in the image. Step 220 may also include specifying a second set of 2D coordinates corresponding to the outer contour points of the capillary segments. These sets of 2D coordinates for each image may define the boundaries of the capillary segments in the image.

[0026] In step 230 of method 200, the first 2D coordinate set and the second 2D coordinate set are interpolated to generate the first resampled coordinate set and the second resampled coordinate set, respectively. This interpolation fills in potential gaps between adjacent contour points specified in step 220, and thus can define smoother boundaries for one or more capillary segments.

[0027] In step 240 of method 200, multiple intermediate curves are generated based on a first set of resampled coordinates and a second set of resampled coordinates. These intermediate curves may be located within capillary segments defined by internal and external contour points. The multiple intermediate curves may include a central curve, which can be used to define multiple curve distances, as in step 250 of method 200.

[0028] In step 260 of method 200, multiple intensity values ​​are extracted from multiple images. Each extracted intensity value corresponds to one of the multiple images, one of the multiple intermediate curves, and one of the multiple curve distances. That is, each extracted intensity value can be indexed by a vector containing three values ​​representing (1) a specific image, (2) a specific intermediate curve, and (3) a specific curve distance.

[0029] In step 270 of Method 200, the extracted intensity values ​​are converted to radon domains. In step 280 of Method 200, multiple maximum positions within the radon domains correspond to the flow trajectories within the capillaries, and consequently, visual gaps in the flow trajectories within the capillaries indicate WBC events.

[0030] Collection of capillary data According to some embodiments, capillary data (e.g., source images) for the analysis of blood cell dynamics can be obtained from a variety of subjects, including but not limited to humans and other mammals. In some embodiments, capillary data is collected or taken up from one or more locations on or within the subject's body. For example, capillary data may be collected and / or taken up from nail fold capillaries, retinal capillaries, and / or oral mucosal capillaries.

[0031] Source images may be acquired using various methods. For example, source images may include a sequence of images extracted from a video clip. Each image in the sequence may be acquired at a different point in time, such as when analyzing blood cell dynamics, which may include, for example, the flow rate of WBCs.

[0032] In some embodiments, a location on or within the body of a subject is illuminated by a pulsed light source, and the source image is captured by a camera synchronized with the pulsed light source. For example, the location may be illuminated by a pulsed laser of a specific wavelength (e.g., blue light at about 430 nm) for which the WBC (or other object of interest) has good reflectivity, in order to improve the contrast of the resulting image.

[0033] In some embodiments, the source image includes one or more color images, and as a result, each pixel of the source image includes three values ​​corresponding to, for example, the values ​​of red, green, and blue components. In some embodiments, each pixel of the source image includes 2, 4, 6, 7, 8, etc., corresponding to, for example, 4, 16, 64, 128, 256, and 1024 gray levels, respectively. The source image includes one or more grayscale images, or has a bit depth of 10.

[0034] In some embodiments, the capillary data (e.g., obtained in step 210 of Figure 2) includes video acquired from a given object in 24-bit RGB format. Generally, the video data is N h ×N v ×N f It can be viewed as a 3D stack I of a sequence of image frames, where N h and N v These are the vertical and horizontal dimensions (also called the number of pixels) of each image frame, and N f is the total number of frames in the video data. Each pixel in this video data can be represented as I[k,l], which corresponds to an RGB vector (R[k;l], G[k;l], B[k;l]). The index k=(k1,k2) refers to the position of the pixel within a particular frame, and l refers to the index of a frame within multiple frames contained in the video data. R[k;l], G[k;l], and B[k;l] correspond to the values ​​of the red, green, and blue components of pixel I[k,l], respectively. For example, I[(15,20),5] refers to the pixel located in the 15th row and 20th column of the 5th frame of the video data.

[0035] Figure 3 is an example of a human nail fold image that can be used in the method described above for the analysis of blood cell dynamics in one or more nail fold capillaries. The image is a frame taken from a video of nail fold capillaries, with a frame rate of r frames per second and S p The camera has a pixel size of μm. In Figure 3, the profiles of relatively dark U-shaped capillaries 300 are easily identifiable in the image despite the presence of white saturated regions 310 associated with less-than-ideal acquisition conditions.

[0036] Preprocessing of capillary data Some image preprocessing steps may be performed within the capillary data to facilitate subsequent processing and analysis. These preprocessing steps may include image normalization and / or image registration. In some embodiments, some or all aspects of these preprocessing steps may be performed by one or more of the systems, apparatus, and devices described herein, such as system 2300 and / or device 2340, which are described in more detail with respect to Figure 23.

[0037] In some embodiments, the capillary data includes one or more grayscale images. Normalization may be used to compress and / or expand the gray levels to a desired range. In some embodiments, the capillary data includes one or more color images. Normalization may be used to convert the color images to grayscale images before compressing and / or expanding the gray levels. The conversion from color images to grayscale images can be achieved through various methods. In one example, the gray level of the resulting color pixels is the sum of the red, green, and blue values ​​in the pixel. In another example, the resulting gray level is the difference (RG) between the values ​​of the red component and the green component to highlight the structure of the capillaries. More advanced methods may be employed to calculate the gray level to highlight a specific object of interest (e.g., WBC or red blood cell). For example, a specific weighted average of the red, green, and blue components may be calculated, or other non-linear types of channel conversions (e.g., RGB to HSV (Hue, Saturation, Value) conversion) may be used.

[0038] In some embodiments, image registration includes one or more intensity-based methods that compare the intensity patterns of images via a correlation metric. The intensity-based methods can register the entire image or sub-images. If sub-images are registered, the centers of the corresponding sub-images may be treated as corresponding feature points.

[0039] In some embodiments, registration of an image includes one or more feature-based methods that find correspondences between features of the image such as points, lines, contours, etc. Feature-based methods can establish correspondences between one or more distinct points of an image. Knowing the correspondences between these distinct points within an image, a geometric transformation can be determined to map a target image to another image, thereby establishing point-by-point correspondences between a reference image and the other image.

[0040] In some embodiments, the capillary data includes a 24-bit RGB format video including a frame such as the image of FIG. 3. In these embodiments, Stack I may be converted to a single-channel version I n which can be suitably utilized for further profile segmentation and analysis. The scalar value stack I n [k;l] can be obtained by first averaging the RGB channels of I for each pixel point ("per point") and then normalizing the resulting intensity. This normalization may be performed such that the mean and standard deviation of the intensity values of I over any given frame l are 0 and 1. n The frames constituting I can be registered, for example, to correct for camera movement. Registration of the frames can be achieved by applying respective correction shifts to each frame l>1 of I, i.e., using the first image (l = 1) as a reference. n n

[0041] More specifically and first, a per-point operation is performed to obtain a temporary stack.

Number

[0042] Secondly, the threshold τ specified for each separate frame of the sequence c Using, binary threshold stack I t ' can be obtained, but this value can be determined experimentally, using a priori information present in the image (e.g., total luminance, contrast, grayscale variance, distortion, etc.), or through feature learning obtained from a training dataset. In this step, the pixel value at position [k;l] of the temporary stack I'[k;l] is obtained when I'[k;l] is τ c If it is less than τ, it is set to zero, and I'[k;l] is τ c If the value is greater than or equal to the above, it will be set to 1.

number

number

[0043] Thirdly, the applied correction shift is I for l>1. t '[k;1] and I t 'To maximize the correlation (i.e., the normalized scalar product) between [k;1] and the 2D registration stack I r This is calculated.

[0044] Capillary profile segmentation Step 220 of Method 200 in Figure 2 is performed in some embodiments to achieve capillary profile segmentation, i.e., to identify segments of capillaries in an image. In some embodiments, segmentation is performed with respect to the source image (with or without preprocessing). In some embodiments, segmentation The process is performed on the normalized and / or registered image. In some embodiments, segmentation is performed on the source image after several other processes, such as noise filtering and / or contrast enhancement intended to highlight capillary structures.

[0045] In some embodiments, capillary segmentation is manual. The user may draw lines along the boundaries of capillaries visible in the image. For example, step 220 may be performed by the user. The user may mark several internal and external contour points on the image, and the computer may then retrieve the 2D coordinates of these marked internal and external contour points for further processing (e.g., resampling in step 230).

[0046] In some embodiments, capillary segmentation is automated or semi-automated using methods such as thresholding and / or edge detection. A computer-based pattern recognition model may be trained using artificial intelligence methods. In one example, the pattern recognition model may be based on historical images of capillaries so that it can recognize similar capillary structures when new images are provided. In another example, the pattern recognition model may be based on one or more capillary structures within a portion of an image so that it can recognize similar capillary structures in other parts of the image.

[0047] In some embodiments, capillary segmentation may involve a hybrid approach in which the user creates some initial and approximate markings of the capillary boundaries, and a computer-based program refines the initial markings using interpolation (e.g., linear interpolation, polynomial interpolation, or spline interpolation), curve fitting (e.g., using linear or nonlinear regression), and / or extrapolation (e.g., linear extrapolation, polynomial extrapolation, conical extrapolation, or French extrapolation).

[0048] In some embodiments, the segmentation of each capillary is semi-automated using a minimum of user-defined 2D coordinates. For example, the user defines two sets of 2D coordinates P that follow internal and external capillary boundaries defined with respect to the first frame of stack I. int [j]P ext [j], j=1, 2, ..., N p The contour can be defined. Here, j is the index of the j-th point on the contour, and N p This is the total number of points specified by the user in each contour (internal or external). The specified number of points N p The total number may depend on several factors, including but not limited to the complexity of the capillary boundary and the length of the capillary. For straight capillaries or portions of straight capillaries, fewer points can be specified compared to capillaries with sharp curves. In some embodiments, the number of specified points N p The number of points can range from approximately 3 to approximately 20. In some embodiments, the number of specified points N p This can range from approximately 5 points to approximately 10 points.

[0049] point P int [j] or P ext The value of [j] is 3D Stack I or I r This includes a vector indicating the position of this point within P. int [5] refers to the fifth point on the internal contour of the capillary. P int The value of [5] may be, for example, (20, 30), which indicates the 2D position of the point on the image, i.e., the 20th row and 30th column. After obtaining the 2D position of this point on the curve, the pixel value (or intensity) of the pixel is taken from the image stack I or I r The data may also be read from the source. In this way, the index on the curve (j), the pixel position (k), and the pixel value are associated with each other in a one-to-one correspondence.

[0050] Resampling of capillary contour points The specified contour points are, for example, in step 220 of method 200, typically sparse and are specific characteristic points on the internal and external boundaries of capillaries (e.g., start point, end point). It may only mark the boundary (or the bend point). Therefore, in step 230 of method 200, the designated internal and external contour points are resampled to further refine the boundary for further processing according to some embodiments.

[0051] At least one of the three methods and / or a combination thereof may be used to resample the specified contour points. In some embodiments, interpolation (e.g., linear interpolation, polynomial interpolation, or spline interpolation) is used to generate new data points between two adjacent specified points to fill the gap. In some embodiments, curve fitting (e.g., using linear or nonlinear regression) is used to fit the specified contour points to a curve, which generally has an analytical formula and may contain any number of data points for further processing. In some embodiments, extrapolation (e.g., linear extrapolation, polynomial extrapolation, conical extrapolation, or French extrapolation) is employed to generate projection points beyond the curved section defined by the specified contour points. In further embodiments, a combination of two or more of these methods is used.

[0052] In some embodiments, denser points P' int and P' ext The set is generated using cubic spline interpolation of the original points with a resampling coefficient α. Therefore, the total number of contour points after resampling is α(N). p -1)+1, and in the formula, N pα is the number of specified contour points. The resampling count α may depend at least in part on the desired resolution of the resampled contours, i.e., the distance between two adjacent resampled contour points (also called the point spacing). For example, if a WBC event is detected, it is preferable to set the point spacing smaller than the size of the WBC in order to identify each WBC in subsequent processing. In some embodiments, the point spacing may be substantially similar to the pixel size in the source image of the capillary data.

[0053] In some embodiments, resampling is performed directly on the 2D coordinates specified in step 220. For example, the specified contour points may be resampled using a 2D spline (also called bicubic interpolation). In some embodiments, the point resampling operation in step 230 involves separate 1D resampling of the corresponding vertical and horizontal coordinate sequences (also called row and column sequences, respectively). Figure 4A is a plot of cubic spline interpolation of the x-coordinate, and Figure 4B is a plot of cubic spline interpolation of the y-coordinate. In Figures 4A and 4B, the contour points specified in step 220 are marked as X marks (e.g., X marks 400), and the resampled contour points connect these X marks. Note that the resampled data points, due to their large number, may appear as a continuous curve (e.g., apparent curve 410).

[0054] Definition of intermediate curves and curve distances In step 240 of method 200, a series of intermediate curves are defined between the internal and external contours of each identified capillary, according to some embodiments. In some embodiments, this series of intermediate curves is evenly distributed across the space defined by the internal and external contours. In some embodiments, the distribution of intermediate curves is non-uniform. For example, the central portion of a capillary may have a higher density of intermediate curves compared to the edge portion of the capillary.

[0055] In some embodiments, a series of N cIndividual intermediate curves P' m [j] is generated by a linear combination of the following form:

number

[0056] Total number of intermediate curves N c This may depend on the desired resolution for collecting intermediate curves, i.e., the distance between two adjacent intermediate curves (also called the curve spacing). For example, if a WBC event is detected, the curve spacing may be smaller than the size of the WBC in order to elucidate each WBC in subsequent processing. In some embodiments, the curve spacing may be substantially similar to the pixel size in the source image of the capillary data.

[0057] The resampling of contour points in step 230 can be seen as improving the longitudinal resolution (i.e., resolution along the direction of the capillaries) from the contour points specified in step 220. On the other hand, the creation of intermediate curves can be seen as improving the lateral resolution (i.e., resolution along the cross-section of the capillaries) based on the two boundaries of the capillaries (i.e., the internal contour and the external contour).

[0058] In some embodiments, step 230 is performed before step 240, so that intermediate curves can be created based on the resampled contour points obtained in step 230. In some embodiments, step 240 is performed before step 230. For example, a series of intermediate points may be created between a user-specified point on the internal contour and a user-specified point on the external contour to fill gaps along the cross-section of a capillary. The same step may be performed for each pair of user-specified points to generate a set of intermediate contour points between the internal and external contour points. Then, a resampling step, as in step 230, may be performed for each set of intermediate contour points to connect the intermediate contour points and thereby generate a set of intermediate curves.

[0059] In step 240, the central curve P' mid It can be defined as follows:

number

[0060] Figure 5 shows normalized and registered image frames I obtained from a video clip containing capillaries 500 and 510. r Both capillaries are highlighted in the processed image along corresponding user-defined contour points (shown as crosses) and interpolation points (shown as curves) that follow the internal and external curves of each profile. The central curve P' from which the curve distance is calculated. midIt is also shown that the user specified only six points, and furthermore, the resampled curve already appears to match the actual boundaries of the two capillaries quite well.

[0061] Extraction of intensity profile In step 260 of Method 200, the intensity (i.e., pixel value) of a pixel within a capillary segment is extracted as a function of curve distance to form a spatial-temporal profile f[j;m;l], where j is the curve index (the j-th point on a curve), m is the curve index (the m-th curve among several curves, including the internal boundary, the external boundary, and the intermediate curve generated between them), and l is the frame index (the l-th image in a sequence of images). More specifically, I r Each P' above m [j] is associated with the intensity value. In some embodiments, bilinear interpolation is used to subdivide a given curve index j, curve m, and sub-particles in frame l. The resulting intensity f[j;m;l] is calculated using Xel precision.

[0062] In some embodiments, if we consider f[j;m;l], then the spatial-temporal profile average f'[j;l] (i.e., from 1 to N) c The average capillary strength of all curves can be calculated for any frame and curve distance to produce an averaged value over all m up to a certain point. For example, this averaging step can be thought of as folding all capillary curves (internal contour, external contour, and mid-curves) into a single curve.

[0063] In some embodiments, the spatial-temporal profile average f'[j;l] is resampled to generate a resampled spatial-temporal profile g[j;l], which contains the same number of samples as f'[j;l] but has equidistant curve spacing. This resampling can be performed using linear interpolation.

[0064] Spatial and temporal profile analysis The profile g[j;l] may contain an intensity profile that evolves as a function of time at each point j along the curved distance of each capillary. This profile can reveal spatial and temporal correlations, which may be extracted to overcome potential challenges when analyzing noisy source images acquired at relatively low frame rates and / or low resolution.

[0065] In some embodiments, a median-normalized version of g is calculated to highlight the intensity variations created by the WBC event. More specifically, the median of the time-varying intensity line g[j;l] may first be set to zero at any curvilinear coordinate j. The same operation may then be performed column by column in any frame l. To reduce noise, the contrast-normalized profile g' may be filtered along the time axis using a symmetric unit sum rectangular function of size σ, which gives g''. The filtering operation may be performed through convolution using the rectangular function.

[0066] Figure 6A shows an example of the spatial-temporal profile g'' of 500 capillaries obtained from Figure 5. The spatial-temporal profile g'' is based on the original spatial-temporal profile f[j;m;l] and ranges from 1 to N cThe data was generated after averaging over all m up to (achieving f'[j;l]), resampling to have equidistant curve intervals (achieving g[j;l]), normalizing by the median (to achieve g'[j;l]), and filtering (achieving g''[j;l]). On the spatial-temporal profile shown in Figure 6A, the spatial trajectories corresponding to WBC events can be identified. Figure 6B illustrates these trajectories by dashed lines extending across a 2D intensity plot. The slope of each line may be related to the average velocity of the corresponding WBC event in the capillary. Thus, spatial-temporal profiles like those shown in Figures 6A and 6B already provide important information about capillary WBC events, such as the number of events and flow velocity. This information can be extracted manually, automatically, or semi-automatically using the methods described above.

[0067] Radon transformation In step 270 of Method 200, in some embodiments, a Radon transform is performed on the spatial-temporal profile generated in step 260. The Radon transform transforms 2D lines into peaks located in a specific configuration to identify events and their associated parameters. Event detection using the Radon transform is not only convenient for noise but also robust.

[0068] It is not tied to a specific theory or operating mode, but a continuous 2D image Considering f(x), in the equation, x is a 2D Cartesian coordinate defined in relation to the center of the image f(x).

number

number

[0069] In some embodiments, f(x) is a separate image, in this case a Radon transform R of a separate domain, including a finite amount of radial coordinate values ​​and angles in [0;π]. 0 You may use this. The maximum radial coordinate value can correspond to half the length of the image diagonal. Therefore, profiling in the radon domain

number

number

[0070] Figure 7 illustrates the application of the Radon transform on the spatial-temporal profile shown in Figures 6A and 6B, according to some embodiments. The projected angle θ and radial coordinate R are shown in Figure 7. Solid arrows indicate the direction of orbital line detection. Considering a predetermined set of angle θ and radial coordinate R, the Radon transform calculates the projection of the spatial-temporal profile intensity (i.e., the intensity of the untransformed domain as shown in Figure 7). More specifically, the Radon transform domain value for a given angle θ and radial coordinate R (see Figure 8A) corresponds to the integral of all values ​​(i.e., the intensity values ​​of the untransformed domain as shown in Figure 7) found on the line where the angle inclination is counterclockwise with respect to the horizontal axis is θ and the radial coordinate is R.

[0071] Figure 8A shows the radon conversion of the map in Figure 7 according to some embodiments.

number

[0072] Figure 8B shows the spatial and temporal relationship in the radon domain with the highlighted discrimination peak within the circle. The following profile is shown. The original physical time and velocity parameters of a WBC event within a capillary can be analytically estimated from the maximum position based on a known frame rate and pixel size, via elementary trigonometric relational equations. More specifically, the velocity is related to the tangent of the projection angle θ in the radon domain and is therefore proportional to the longitudinal gradient of the corresponding trajectory line in the aforementioned spatial-temporal profile g''. The original physical time is related to the intersection of the trajectory line and the time axis of the spatial-temporal profile. In some embodiments, the time of each WBC event corresponds to the first frame in which a visual gap appears in the capillary.

[0073] Figure 9A shows experimental results of WBC events occurring in capillary 500 shown in Figure 5, according to some embodiments, and Figure 9B shows experimental results of events occurring in capillary 510. The experimental results are compared with manual counts performed by four trained human assessors. In Figures 9A and 9B, WBC events detected by the method described above are marked with an "X," while event times estimated by trained human assessors are shown with dashed vertical lines. Horizontal lines represent limits related to maximum and minimum blood flow.

[0074] The source image is taken from a video clip of a human fingernail. The acquisition parameter is N. h =1280, N v =960, N f =450, r=15, and S p The threshold τ is 0.65 μm and the duration of the video clip is 30 seconds. c The intensity is heuristically set to 3 / 4 of the maximum intensity value for each frame l in I'. The number of curve points specified by the user is N p It is set to =6, and this amount can produce accurate segmentation. The number of interpolated curves is N, using α=100 for resampling. c The value is set to =10. The size of the filter used for noise reduction is set to σ=3. Finally, the window size and threshold used to detect the maximum value of the radon domain are set to S w =11 and τ m Selected as =7, the number of radon angles is N θ Set it to =400.

[0075] In Figures 9A and 9B, over 80% of the WBC events detected by the method described above match those of trained human assessors. Missed events may correspond to temporarily adjacent visual gaps, which may be difficult to analyze using automated methods due to the low frame rate.

[0076] Figure 10 shows a further comparison between the total WBC event counts obtained from the methods described above in some embodiments and those obtained from four trained human assessors. Figure 10 also shows significant inter-observer manual variability in the counts. Each capillary is associated with a box whose central mark is the median. The edges of the boxes are the 25th and 75th percentiles, and the whiskers extend to the data endpoints. The crosses and circles represent the counts completed by each of the four trained human assessors and by the approaches described herein, respectively. The results obtained from the described methods are shown in these box plots to be within the range of inter-human assessor variability.

[0077] The trajectory slopes of the visual gaps in Figures 6A to 7B can be used to calculate the velocities associated with the WBC event for both capillary 500 and capillary 510. The results shown in Figures 9A to 9B are within the range of human nail fold capillary blood flow values ​​previously reported in the literature, i.e., from approximately 450 μm / sec to approximately 1200 μm / sec.

[0078] Interpolated inner and outer capillary curves P int [j], P ext [j] may be used to derive the vascular radius r, which corresponds to approximately 7.5 μm for both capillary 500 and capillary 510, a value consistent with previously published data.

[0079] While not bound by any particular theory or operating mode, it can be assumed that capillaries have a circular cross-section. Assuming that the average velocity v derived above is approximately 600 μm / second, the total blood volume V sampled per second can be determined as follows:

number

[0080] Assuming that the range for healthy WBCs is approximately 3500 to 9000 per μL, and given that the duration of the video clip is 30 seconds, the number of WBCs c can be determined as follows.

number

[0081] Instruments and systems for non-invasive in vivo analysis of blood cell dynamics Figures 11A and 11B are schematic diagrams of a system for detecting leukocyte (WBC) events in a non-invasive manner in vivo, according to some embodiments. In some embodiments, some or all aspects of the system in Figures 11A and 11B may be structurally and / or functionally similar to one or more systems, apparatuses, and devices described herein, such as system 2300 and / or device 2340, which are described in more detail with respect to Figure 23.

[0082] Figure 11A is a side view of the system, and Figure 11B is a top view of the system. For illustrative purposes, the embodiments described herein analyze WBC events in the nail fold of a human, but these embodiments may be modified for analysis of other subjects and / or other locations within or on the body of a subject. In Figures 11A and 11B, the system 1100 includes a finger holder 1110 having a finger hole 1112 for receiving at least a portion of a human finger (e.g., a portion of the nail fold of a finger). The finger holder 1110 is also configured to receive an imager 1120, which optically communicates with the finger received by the finger hole 1112 through a transparent window 1114 to capture an image or video of the finger. The imager further includes an optical device 1122 for receiving reflected or scattered light by collecting light reflected or scattered from the finger and a detector 1124 to form an image of the finger. System 1100 further includes a processor 1130 operably coupled to processor 1120, and a memory 1140 operably coupled to processor 1130. The memory 1140 is coded using processor-executable instructions, which, when executed by processor 1130, can analyze images received from imager 1120 in the manner described above. System 1100 also includes a display 1150 which can display images or videos captured by imager 1120 and / or data related to WBC events detected by processor 1130.

[0083] In some embodiments, the finger holder 1110 has an igloo shape (as shown in Figures 11A-11B), so that the hand can be placed on the dome of the finger holder while the fingers of the hand are received into the finger holes 1112 for imaging. The finger holder 1110 may have other configurations, such as a flat top, a handle configuration (the hand can grasp the handle while the fingers are being imaged), or any other configuration known in the art.

[0084] In some embodiments, the system 1100 also includes an illumination source 1160 (shown in Figure 11B) that illuminates the finger to facilitate image acquisition by the imager 120. In some embodiments, the illumination source 1160 includes a pair of light-emitting diodes (LEDs), each located on one side of the finger hole 1112. In some embodiments, the illumination source 1160 is configured to emit monochromatic light. The capillary structure of the finger can have high reflectivity at the wavelength of monochromatic light.

[0085] Figure 11C shows a schematic diagram of system 1100, which is substantially similar to the systems shown in Figures 11A and 11B, according to some embodiments. System 1100 includes an adjustable transparent window 1115 provided between the finger hole 1112 and the imager 1120. By adjusting the height of the window 1115, images of the finger may be captured using imagers 1120 of different sizes.

[0086] Figures 12A to 12B are schematic diagrams of a system for acquiring in vivo images used for non-invasive analysis of WBC events, according to some embodiments. In some embodiments, some or all aspects of the system in Figures 12A to 12B may be structurally and / or functionally similar to one or more systems, apparatuses, and devices described herein, such as system 2300 and / or device 2340, which are described in more detail with respect to Figure 23.

[0087] Figure 12A is a side view of system 1200, and Figure 12B is a top view of system 1200. System 1200 includes a finger holder 1210 having a finger hole 1212 for receiving at least a portion of a human finger (e.g., a portion of the fingernail fold). The finger holder 1210 is also configured to receive an imager 1220, which optically communicates with the finger received through the finger hole 1212 through a transparent window 1214 and a mirror 1226 to capture an image or video of the finger. The imager further includes an optical device 1222 for receiving reflected or scattered light by collecting light reflected or scattered from the finger and a detector 1224 to form an image of the finger. In system 1200, the imager 1220 is provided in a vertical configuration. The mirror 1226 is configured to reflect the image of the finger toward the imager 1220. In some embodiments, the mirror 1226 is connected to an actuator (not shown) that can tilt the mirror 1226 in two directions. This tilt may be useful for directing the imager 1220 to a desired portion of a finger for imaging. Similar to system 1100, system 1200 may also have an illumination source 1260 to facilitate image acquisition by the imager 1220.

[0088] Figure 13 is a schematic diagram of a system for performing in vivo, non-invasive analysis of blood cell dynamics using a camera-equipped device (e.g., a smartphone), according to some embodiments. In some embodiments, some or all aspects of the system in Figure 13 may be structurally and / or functionally similar to one or more systems, apparatuses, and devices described herein, such as system 2300 and / or device 2340, which are described in more detail with respect to Figure 23.

[0089] System 1300 includes a finger holder 1310 having a finger hole 1312 for receiving at least a portion of a finger for imaging. The finger holder 1310 is also configured to receive a smartphone 1320, so that the camera in the smartphone 1320 communicates optically with the finger in the finger hole 1312 via a transparent window 1315 and a mirror 1326. System 1300 may also include a modification optical system 1322, which modifies the camera of the smartphone 1320 to adapt the smartphone 1320 for better image acquisition. It is provided in front. For example, the corrective optical system 1322 may include a lens for adjusting the focal length (optical zoom) of the camera in the smartphone 1322. Since many smartphone cameras do not have optical zoom, the lens 1322 can increase the optical flexibility of the camera. The focal length of the camera is also related to the resolution of the image captured by the camera. Therefore, the use of the lens 1322 may also allow for adjustment of the image resolution for further processing.

[0090] The smartphone 1320 generally includes its own memory and processor. The method described in the first paragraph of this application may be coded in the memory of the smartphone 1320 as processor executable instructions. During operation, images captured by the camera may be transmitted to the processor for analysis of blood cell dynamics. In some embodiments, images captured by the camera are transmitted wirelessly (e.g., Bluetooth, WiFi, 3G network, 4G network, or any other wireless communication protocol known in the art) to another processor for processing. In some embodiments, images captured by the camera are saved locally in the memory of the smartphone 1320.

[0091] Figures 14A–14D are images of systems and apparatus for the non-invasive in vivo analysis of blood cell dynamics according to some embodiments. In some embodiments, some or all aspects of the systems and apparatus in Figures 14A–14D may be structurally and / or functionally similar to one or more systems, apparatus, and devices described herein, such as system 2300 and / or device 2340, which are described in more detail with respect to Figure 23.

[0092] Figure 14A shows a finger holder connected to an imager. The imager may be a commercially available capillary microscope (e.g., a Dino-Lite digital microscope, available from BigC.com, Torrance, California). The imager may include its own illumination source so that the finger is adequately illuminated for imaging once the finger is placed in the finger hole. Figure 14B shows the imager and finger holder disconnected. Figure 14C shows the finger holder with a finger placed in the finger hole. Figure 14D shows a system including the finger holder, the imager, and a computer connected to the imager, for example, via a USB connection. The computer may display images or videos captured by the imager in real time. The method described in the first section of this application may be saved in the memory of the computer as processor-executable instructions, which may then perform image analysis to detect WBC events.

[0093] Figures 15A to 15G illustrate adapters that can be attached to a smartphone for capturing images of the nail fold using a camera-mounted device for analyzing the dynamics of blood cells, according to some embodiments. Figure 15A is a perspective view of adapter 1500, including a window compartment 1510 and several suction cups 1520. The window compartment 1510 may be aligned with a camera typically available in a smartphone. The suction cups 1520 may be used to secure adapter 1500 to, for example, a smartphone. Four suction cups are shown in Figure 15A, but the number of suction cups can be any applicable number. Alternatively or additionally, the adapter may be attached or secured to a smartphone or other device using clips, adhesives, etc.

[0094] Figure 15B shows the adapter 1500 from an oblique angle to illustrate the structure of window compartment 1510, which further includes a window section 1512 and a finger receptor 1514. The window section 1512 is substantially transparent, or at least transparent at a specific wavelength, so that a camera can capture an image of the finger. The finger receptor 1514 has an internal contour that conforms to the general shape of the finger so that the finger is securely held by the adapter 1500, thereby enabling subsequent processing To reduce the load of image registration (e.g., due to movement).

[0095] Figure 15C is a top view of adapter 1500, Figure 15D is a first side view of adapter 1500, and Figure 15F is a second side view of adapter 1500. Figure 15E is a rear view of adapter 1500 illustrating the illumination channel 1516 and window 1512. Figure 15G is a front view of adapter 1500 (finger receiving side). As shown in Figures 15E and 15G, the illumination channel 1516 does not generally extend through the entire depth of adapter 1500. Instead, the illumination channel 1516 can optically communicate with a flashlight, which is commonly available in smartphones, and other devices attached to the camera other than the camera lens. The illumination channel 1516 can receive light emitted by the flashlight and reflect it toward window 1512 to illuminate at least a portion of the finger being photographed.

[0096] Figures 15H to 15J illustrate the illumination channel 1516. Figure 15H is a perspective view of the adapter 1500. Figure 15I is a partial perspective cross-sectional view of the adapter 1500 to illustrate the window 1512 and the illumination channel 1516. Figure 15J is a cross-sectional view of the adapter 1500 to illustrate the structure of the illumination channel 1516. In particular, the illumination channel 1516 may include a curved surface 1517 that reflects the received light toward the window 1512 to illuminate a finger typically placed in front of the window 1512 for imaging.

[0097] Figure 16 is a schematic diagram of a system, according to some embodiments, that includes a smartphone and adapter for capturing images of the nail fold for analysis of blood cell dynamics. In some embodiments, some or all aspects of the system in Figure 16 may be structurally and / or functionally similar to one or more systems, apparatus, and devices described herein, such as system 2300 and / or device 2340, which are described in more detail with respect to Figure 23.

[0098] The adapter 1600 includes a window 1612 that communicates optically with the camera 1632 of the smartphone 1630. The adapter 1600 also includes an illumination channel 1616 that communicates optically with the LED light 1636 of the smartphone 1630. The illumination channel 1616 receives light emitted by the LED light 1636, and a curved surface 1617 within the illumination channel 1616 reflects the received light toward the window 1612 so as to illuminate a finger 1640 (more specifically, the nail fold 1642) that is in close contact with the window 1612.

[0099] In some embodiments, the window 1612 may include two lenses to change the focal length of the camera 1632 in the smartphone 1630. By changing the focal length, the resolution of the image captured by the camera 1632 can also be changed. In some examples, a refractive index matching fluid 1618 may be placed between the window 1612 and the claw frame 1642. The refractive index matching fluid 1618 may have a refractive index similar to that of the claw frame or the window 1612 material in order to reduce specular reflection on the surface of the window 1612 or the claw frame 1642. In some embodiments, the refractive index matching fluid 1618 has high viscosity. In some embodiments, the refractive index matching fluid 1618 may be replaced by a suitable reusable transparent plastic.

[0100] Figure 17A shows a smartphone with an adapter attached to it to enable imaging of the nail fold, according to one embodiment. Figure 17B shows the smartphone and adapter with a finger positioned in front of the adapter for imaging. Figure 18 is an image captured by the smartphone using the adapter shown in Figures 17A-17B. Multiple capillary structures are visible in the image for further analysis of blood cell dynamics, according to one embodiment.

[0101] Figure 19 is a schematic diagram of a clamp device for capturing images and performing blood cell dynamics analysis according to some embodiments. The clamp device 1900 includes a spring-loaded casing 1910 for securely receiving a finger 1950. Once the finger 1950 is positioned within the clamp device 1900, the finger 1950 is illuminated by a pair of LED lights 1940a and 1940b, allowing the camera 1930 to capture an image of the finger 1950. In some embodiments, the LED lights 1940a and 1940b emit broad-spectrum white light. In some embodiments, the LED lights 1940a and 1940b emit green light that can be effectively reflected by the WBC. The clamp device may also include a battery and / or processing unit 1920 for processing the images captured by the camera 1930 in the manner described above. Figure 20 shows a clamp device attached to a finger for blood cell dynamics analysis. As can be seen in Figure 20, the spring-loaded casing helps to secure the finger to the clamping device in order to reduce or eliminate relative movement between the finger and the camera.

[0102] Figures 21A–21O illustrate an adapter for capturing images of the nail fold with a smartphone camera for capillary microscopy and hematological analysis, according to some embodiments. Specifically, Figures 21A–21F are perspective views of the smartphone adapter. In Figures 21A–21E, the adapter is attached to a smartphone so that the smartphone camera can be used. In Figure 21F, the adapter is shown without being attached. Figures 21G–21O are wireframes showing side and perspective views of the adapter. Figures 22A–22D illustrate how to use the smartphone adapter, according to some embodiments, with the smartphone camera attached to a smartphone for use in capillary microscopy and hematological analysis. In Figure 22A, a refractive index matching fluid is placed on the surface of the user's nail fold. In Figures 22B–22D, the smartphone is placed on the finger, and therefore the adapter is placed on the finger so that the smartphone camera is aligned with the nail fold.

[0103] Figure 23 is a schematic diagram of an environment / system 2300 on which non-invasive hematological measurements can be implemented and / or performed. In some embodiments, the aspects of system 2300 can be structurally and / or functionally similar to the systems, apparatus, and / or devices described herein with respect to Figures 1 to 22 and Figures 31 to 60, and / or the methods described in Figures 1 to 2 can be implemented.

[0104] System 2300 includes a platform 2310, an illumination source 2330, an imaging device 2320, and a computing device 2340. In some embodiments, all components of System 2300 may be contained in a common casing, such as a single housing that gives System 2300 an integrated, one-piece device for the user. In other embodiments, at least some components of System 2300 may be located in separate locations, housings, and / or devices. For example, in some embodiments, the computing device 2340 may be a smartphone that communicates with the illumination source 2320 and / or the imaging device 2330 over one or more networks, each of which may be any type of network implemented as a wired and / or wireless network, such as a local area network (LAN), a wide area network (WAN), a virtual network, a telecommunications network, and / or the Internet. As is known in the Art, any or all communications may be highly protected (e.g., encrypted) or unprotected. The system 2300 and / or computing device 2340 may be or may encompass personal computers, servers, workstations, tablets, mobile devices, cloud computing environments, applications, or modules running on these platforms.

[0105] For simplicity of explanation, while described herein as a system for hematological measurements in the nail fold, it is understood that embodiments of the systems, devices, and methods disclosed herein may be useful for hematological measurements in any tissue having capillary structures that can be imaged as described herein (e.g., capillary beds, superficial capillaries, peripheral capillaries, capillaries in other parts of the user's fingers, etc.). Non-limiting examples include capillaries in the retina, earlobe, lip, and gingiva. For example, platform 2310 can be applied so as to be pressed against the user's gingival ridge while being used for hematological measurements in the capillary bed of the gingival ridge. As another example, platform 2310 may encompass a tonometer-like instrument for being pressed against the retina while being used for hematological measurements in the capillaries of the retina.

[0106] In some embodiments, the platform 2310 receives the user's fingers during use. In some embodiments, the platform 2310 may be structurally and / or functionally similar to the finger holders described in Figures 11-14 and 19. In some embodiments, the platform 2310 has a shape for guiding the placement of the user's fingers to position a portion of the fingernail folds in a predetermined position within the platform.

[0107] The illumination source can be any suitable source of substantially monochromatic light, such as, but not limited to, light-emitting diodes (LEDs), laser light, filtered light, etc. The illumination source can be positioned relative to the platform 2310 to illuminate a portion of the user's fingernail during use.

[0108] The imaging device 2330 may include any suitable imaging device disclosed herein, including a smartphone camera, a capillary microscope, and the like. In some embodiments, the imaging device 2330 captures an image of a portion of the user's fingernail fold during use in response to illumination of a portion of the nail fold by the illumination source 2330, for example, based on a synchronization / timing signal from device 2340. In some embodiments, the imaging device 2330 captures an image set, such as a series of slow-motion images, with each acquisition. In some embodiments, the imaging device 2330 captures the image set as a 60-second video / video file at an acquisition rate of, for example, 60 frames / second.

[0109] In some embodiments, the platform 2310 is optically connected to the illumination source 2330 and the imaging device 2330 via any suitable independent means, such as beam shaping and / or beam steering optical systems, one or more optical tubes such as optical fibers, or direct / optical-free connections.

[0110] The computing device 2340 includes at least a controller 2350 and memory 2360. Figure 23 also illustrates a database 2370, but it will be understood that in some embodiments, the database 2370 and memory 2360 may serve as a common data store. In some embodiments, the database 2370 constitutes one or more databases. Furthermore, in other embodiments (not shown), at least one database may be located outside the device 2340 and / or system 2300. The computing device 2340 may also include one or more input / output (I / O) interfaces (not shown) implemented in software and / or hardware for other components of system 2300 and / or outside system 2300 to interact with the device 2340.

[0111] Memory 2360 and / or database 2370 are independently, for example, random access memory (RAM), memory buffers, hard drives, databases, and erasable. Furthermore, it can be a programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), a read-only memory (ROM), flash memory, etc. Memory 2360 and / or database 2370 can store instructions and cause the controller 2350 to execute processes and / or functions associated with the system 2300.

[0112] The controller 2350 can be any suitable processing device configured to run and / or execute the instruction set or code associated with the device 2340. The controller 2350 can be, for example, a general-purpose processor, a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a digital signal processor (DSP).

[0113] The controller 2350 receives a set of image / video data from the imaging device 2330. The controller 2350 can process the image set in any suitable manner by executing computer executable instructions (e.g., those stored in memory 2360 and / or database 2370). In some embodiments, given a raw input video / image set having, for example, a resolution of 1280 × 1024 pixels and a frame rate of 60 frames / image per second, and given an initial frame index of the input, the controller 2350 can extract two uncompressed videos, limited to a duration of one minute (3600 frames) starting from that initial frame index. In some embodiments, the first video at its original pixel resolution is downsampled, for example, to half the pixel resolution (640 × 512) to accelerate processing between registration and capillary detection, as described herein. The uncompressed image is also extracted as the first frame / image of the second video.

[0114] In some embodiments, the controller 2350 performs a global registration process on the image set to eliminate motion occurring between frames / images. For example, in some embodiments, the controller 2350 aligns all images with respect to the first image in the image set. In some embodiments, alignment includes correcting the horizontal and / or vertical translation of the view captures in the image set. In some embodiments, the controller 2350 corrects the horizontal and / or vertical translation of each image in a manner that maximizes its relationship with the first image.

[0115] In some embodiments, prior to such alignment processes, the controller 2350 normalizes the intensity of each image in the image set, for example, by a spatial high-pass filter, based on predetermined characteristics (e.g., a 75x75 pixel square filter) and / or any other suitable image flattening technique. In this way, the resulting processed image set (sometimes simply called the “image set”) is cropped to a display area that remains in the field of view across the entire image set.

[0116] The controller 2350 can execute computer-executable instructions (for example, those stored in memory 2360 and / or database 2370) to detect one or more capillaries in a portion of the nail fold in each image of the processing image set and to identify a first set of capillaries across the image set. In some embodiments, detecting a first set of capillaries includes detecting each capillary of the first set of capillaries on at least one image of the image set.

[0117] In some embodiments, the controller 2350 detects each capillary in the first set of capillaries as follows: The controller 2350 detects one or more attributes of the identified capillary (for example, at least one of the one or more attributes). The system receives training data, which includes images of capillaries that meet one or more predetermined criteria. In some embodiments, one or more attributes include structural attributes of the capillaries themselves, such as, but are not limited to, capillary length (e.g., long enough so that, given the frame rate of the imager / imaging device, a cellular event moving at a typical or maximum blood flow velocity appears in more than one frame), capillary width (e.g., capillary width from about 10 μm to about 20 μm, including all values ​​and subranges in between), capillary depth (e.g., capillary depth from about 10 μm to about 350 μm, including all values ​​and subranges in between), average capillary diameter, capillary transverse diameter, capillary longitudinal diameter, and capillary shape (e.g., capillaries must exhibit distinct arterial and venous limbs).

[0118] The average capillary diameter can generally be characterized as the average of any multiple diameter measurements of a capillary, either in a single cross-section or in multiple cross-sections along the length of the capillary. In some embodiments, the structural attribute is the average capillary diameter, and a given criterion is that the average capillary diameter must be between approximately 10 μm and approximately 20 μm.

[0119] In some embodiments, one or more attributes include flow attributes, such as, but are not limited to, capillary blood flow velocity, cell migration time within the visible portion of the capillary, volumetric flow rate, mass flow rate, flow directionality, and blood flow velocity stability.

[0120] In some embodiments, one or more attributes include imaging attributes of the capillary image, such attributes include, but are not limited to, contrast (e.g., luminance contrast, root mean square (RMS) contrast, Weber contrast, Michelson contrast, histogram-based methods, etc.), focus / detail (e.g., measured by gradient-based operators, Laplace operators, wavelet operators, discrete cosine transform (DCT), frequency domain analysis, phase coherence, luminance maps, singular value decomposition, learning algorithms, etc.), signal-to-noise ratio, and image stability (e.g., measured by image registration methods, optical flow, etc.). In some embodiments, a combination of structural attributes and imaging characteristics can be employed. In some embodiments, the training data is human expert-generated data, which also contributes to the requirement that the capillaries are generally clear (e.g., without bubbles blocking the capillaries) and have clear morphology (e.g., with clear arterial and venous limbs). The controller 2350 trains a deep learning neural network (e.g., a fully convolutional neural network such as the deep learning YOLO method / technique, as broadly disclosed in Joseph Redmon et al. 2016, The IEEE Conference on Computer Vision and Pattern Recognition, whose full disclosure is incorporated herein by reference) based on training data for recognizing a first set of capillaries in the image set.

[0121] In some embodiments, the controller 2350 generates a bounding box around each detected capillary in each image of the image set, and also generates a confidence value related to the likelihood that the detection corresponds to a capillary and not to another structure / artifact. In some embodiments, the first set of capillaries includes those detected capillaries having corresponding confidence thresholds that satisfy or exceed a predetermined confidence threshold.

[0122] In some embodiments, the controller 2350 identifies a second capillary set from a first capillary set. In some embodiments, the second capillary set includes capillaries detectable in a threshold number of images in the image set. For example, if the image set includes 60 images, the second capillary set may include capillaries visible in at least 40 of the 60 images. In some embodiments, the second capillary set includes capillaries visible in a threshold number of images. The system includes capillaries that are detectable in the images and have a confidence value exceeding a confidence threshold in the minimum number or all of those images. In some embodiments, the controller 2350 generates a threshold number of image markings based on training data.

[0123] In an exemplary embodiment, the controller 2350 supplies a neural network as described herein using 130 training images with 795 manually created corresponding bounding boxes (around capillaries). The training images are extracted as the first frame / image of 130 distinct capillary video / image sets derived from 43 distinct patients, thus ensuring sufficient data diversity. The bounding boxes were manually defined by one human expert around each capillary that met the set of criteria described herein. The image dataset was divided into a first set and a second set based on the chronological order of acquisition, and a confidence threshold of ¢=0.45 was set to avoid the detection of inappropriate capillaries or artifacts. This condition resulted in the detection of 66 images from 23 patients in the first set using a total of 416 annotated bounding boxes around the identified capillaries as the first capillaries, and 64 images from 19 patients in the second set using a total of 379 annotated bounding boxes around the identified capillaries. The neural network is fed the first set and the second set for a total of 900 iterations each time, and in this way the learned weights W of the corresponding values ​​are obtained. s1 and W s2 We created the following. While we have described this using an exemplary YOLO method, in some embodiments, these learned weights W s1and W s2 is determined as a free parameter of the convolutional neural network structure and is optimized so that the capillaries detected by the neural network when executed on images from the training set best match those pre-labeled by a human evaluator on these same images as a reference.

[0124] After the above single-frame detection / identification step of the first set of capillaries, each capillary was tracked with a given identifier. The tracking of capillaries is based on overlap and Kalman filtering in cases where detection is missed between frames, although, without limitation, any suitable method including, but not limited to, mathematical morphology, cross-correlation, mutual information, optical flow, machine learning, etc. can be understood to be used for the tracking of capillaries. Capillaries are present in at least t f images (a threshold number of images), i.e., when associated with a bounding box that exceeds the confidence parameter ¢ in t f images, are selected for the second set of capillaries. Parameter t f was empirically set to 600, except when less than 3 capillaries were selected. In the latter case, t f was set so that, if possible, 3 capillaries are detected (i.e., under the condition that at least 3 distinct capillaries are detected in a single image).

[0125] To test the selection of the first set of capillaries, the steps described herein (for detecting the first set of capillaries) are the first set with weight W s2 and weight W s1The process is carried out with respect to a second set having the following characteristics. To verify the detection of the first set of capillaries, a comparison is made to the reference of 24 initial images obtained from a separate set of 24 reference images / reference videos, where the capillary boxes are annotated by two human experts who selected capillaries according to the following set of criteria: (A) Illumination. Capillaries must be visible with sufficient contrast to the observer; (B) Focus. Detailed capillary structure / dynamics must be visible and not blurred; (C) Flow. Blood flow must be present in such a way that it is possible to identify and count potential events; (D) Stability. Capillaries must remain completely within the video FOV of all frames; (E) Visibility. There must be no objects (e.g., air bubbles) that could obstruct the capillaries; (F) Morphology. Capillaries must exhibit clear arterial and venous limbs. In the test video / image set, for example, 95% of the videos have three or more detected capillaries, which is associated with an improvement in classification results. An example of capillary detection carried out as detailed herein is shown below. This is illustrated in Figure 25.

[0126] Referring again to Figure 23, in some embodiments, the controller 23 detects a set of cellular events (i.e., one or more) in the second set of capillaries. In some embodiments, each cellular event in the set of cellular events is related to the passage of leukocytes in the capillaries of the second set of capillaries. As described further herein, erythrocytes exhibit greater light absorption than leukocytes at the wavelengths described herein, so the passage of leukocytes in the capillaries results in an "absorption gap" due to the presence of leukocytes.

[0127] Therefore, as used herein, and sometimes also referred to as “gap” or simply “event,” the term “cellular event” may mean one or more of the following: a) detection of a relatively absorbent area (indicating the possibility of the presence of erythrocytes) adjacent to a relatively absorbent area (indicating the possibility of the presence of one or more leukocytes) within a capillary; b) detection of a relatively absorbent first area (indicating the possibility of the presence of erythrocytes) adjacent to a relatively absorbent second area (indicating the possibility of the presence of one or more leukocytes downstream of the first area), in which case the second area is adjacent to a third area within the capillary with different absorption than the second area (indicating either the possibility of the presence of erythrocytes downstream of the second area or an area that contains substantially no cells).

[0128] While the detection of leukocytes in capillaries is disclosed herein, aspects of this disclosure will be understood to be useful for detecting any other suitable cells insofar as they exhibit a contrast in absorption to erythrocytes. For example, circulating tumor cells (CTCs) may be detectable by selecting capillaries having a diameter similar to that of the CTCs (i.e., by selecting a first set of capillaries or a second set of capillaries). As another example, one or more types of leukocytes (e.g., neutrophils, lymphocytes, monocytes, eosinophils, and / or basophils) may be detectable by selecting suitable capillaries as described herein.

[0129] In some embodiments, for each capillary in the second set of capillaries, the controller 2350 detects one or more cellular events flowing through that capillary as follows: In some embodiments, the controller 2350 loads all images showing the capillaries as a 3D matrix into memory 2360 and / or database 2370, along with the x-pixel position, y-pixel position, and the dimension of the matrix associated with the image. Each matrix value corresponds to the brightness level of that pixel. In some embodiments, the controller normalizes the brightness level of the 3D matrix by a) spatially normalizing the brightness level with respect to the first image showing the capillaries and correcting for potential brightness variations associated with the illumination source 2320. In some embodiments, this is achieved by a) assigning pixel brightness (for each pixel) by local brightness mean estimated by a Gaussian filter with a standard deviation of 25 pixels, and b) adjusting the average brightness of all images showing the capillaries so that the brightness value remains closest to that of the first image (in terms of mean square error).

[0130] Each pixel in this preprocessed normalized 3D matrix has a corresponding time signal that is similar across all pixels through which the same cellular event passes, except that the main difference between these pixels is the pixel-specific time shift. Another difference is that not all time signals have uniformly strong / exact amplitudes. As a first approximation, the controller 2350 selects a pixel away from the edge of the image that has the "strongest" time signal to serve as the reference pixel. Generally, a strong / desirable signal is a large contrast between a bright gap (indicating the potential presence of leukocytes) and a dark erythrocyte, for example, by a given threshold. This refers to a signal that has a contrast exceeding the noise level in the signal. This signal can be a candidate for a time-aligned reference signal. In some embodiments, the controller estimates the intensity of this candidate reference signal based on the ratio of the sum of its squares to the sum of its absolutes. This ratio can then be interpreted as an estimate of the peak intensity relative to its nominal variation. In some embodiments, the estimation of the reference signal may employ the following pre-normalization and post-normalization steps: (a) subtracting the local time average from the time signal based on an averaging window (e.g., window size 200 ms); (b) for robustness, estimating the aforementioned ratio as the median of 10 estimates derived from 10 consecutive time blocks of the same size; (c) forming a 2D map of the image set showing capillaries; and further spatially filtering the resulting ratio estimate with a Gaussian filter having a standard deviation of 1 pixel. In some embodiments, because there may be intensity variations due to incomplete registration by introducing weighting factors to disadvantage candidate reference pixels near the frame boundary, the controller 2350 ignores and / or otherwise disregards other capillaries near the edge of the region of interest. The controller 2350 then selects a pixel that maximizes the peak intensity of the preprocessed time signal and its distance from the edge of the region of interest.

[0131] In some embodiments, the controller 2350 further filters out other structures that are unlikely to be cellular events through spatial filtering of each separate frame. In some embodiments, this is achieved by applying a bandpass filter in a separate Fourier domain to each frame to remove all features of inappropriate size, and the low-frequency and high-frequency bandpass parameters are adjusted to fit the expected cellular event size / expected leukocyte size range.

[0132] In some embodiments, the controller 2350 further sets the intensity of capillaries filled with red blood cells to zero. The purpose of this step is to ensure that flow without events displays adequate contrast with respect to the passage of events, the latter then ideally associated with a (significant) non-zero luminance value after bias removal. In some embodiments, the controller achieves this by subtracting a transient (frame-wise) median from all corresponding pixels in all images showing its capillaries. The pixel-wise operation is performed separately and indiscriminately on all pixels, i.e., independently of whether the pixel is inside or outside the capillary. The assumption of this step is that the capillaries are filled with red blood cells for most of their length and for most of the time.

[0133] Next, the controller 2350 compares the time signal associated with every pixel location in the image showing the capillaries of the second set of capillaries with the reference time signal at the reference pixel estimated as disclosed herein. First, the controller 2350 suppresses long-term intensity fluctuations by applying a high-pass filter to every separate time signal along the time dimension. Next, the controller 2350 estimates the correlation between every time signal and the reference time signal, resulting in amplitude and phase values ​​for each pixel. A large positive amplitude indicates a strong correlation, and the phase indicates a time shift with respect to the signal at the reference pixel. This per-pixel amplitude and phase information is used, as described below, to estimate a more noise-robust average signal. The controller 2350 performs all operations by duplicating the first and last frames to avoid false detections at the beginning or end of the image sequence showing the capillaries.

[0134] Based on the reference time signal and other time signals having related phase and amplitude correlation information, the controller 2350 generates a relatively more robust average time signal against noise. This average time signal combines all time signals that can be perceived as part of the capillaries. In some embodiments, the controller 2350 holds the arbitrary time signal if a) the correlation between an arbitrary time signal and a reference signal exceeds a threshold for the autocorrelation time of the reference signal, and b) the (positive or negative) time delay (i.e., phase) of the time signal with respect to the reference signal is on the magnitude of a relatively fixed parameter, which is the maximum time it takes for a gap / cellular event to flow through a capillary.

[0135] Next, the controller 2350 calculates the upper reference time signal as follows: (a) all time signals are aligned in time with respect to the reference time signal based on phase information; (b) the aligned time signals are averaged; and (c) the resulting average signal is high-pass filtered in time based on a fixed minimum frequency parameter. Note that the time at which a cellular event is detected is associated with the reference pixel position, so the resulting filtered one-dimensional signal is useful for detecting cellular events. Figures 26A to 26C illustrate an example of the real-time signal generated by analyzing the image set for a single capillary, and also show the types of time signal profiles that can be expected around a single cellular event vs. a plasma gap. A gap containing leukocytes is assumed to have a high concentration of erythrocytes upstream, corresponding to a dark area, as best illustrated in Figure 26B.

[0136] Each gap / cellular event is associated with a discontinuous flow of values ​​where the mean-time signal, reflecting the spatially averaged normalized brightness in the capillaries, exceeds a threshold. While all non-zero values ​​should reflect transit events, the threshold can be set as a non-zero positive value due to noise. In some embodiments, the threshold is optimized to achieve the best possible separation between signal and noise. For example, in some embodiments, the threshold is set to capture most peak signal information (related to values ​​above the threshold) while rejecting most noise artifacts (related to values ​​below the threshold). In some embodiments, the controller 2350 sets this threshold as a multiple of the noise standard deviation, in which case the noise is estimated according to a positive exponential model (i.e., the decaying exponential function fits the positive portion of the signal distribution). In some embodiments, the standard deviation of the signal itself may serve as an estimate of the noise. In some embodiments, before thresholding, the controller checks the mean time signal, specifically, if the ratio of the median filtered version of the signal (e.g., with a filter size of 240 frames) to its first quartile is greater than 1, the controller locally divides the signal by this ratio. For each detected gap / cell event, the relevant time information is defined with respect to a reference pixel. Then, if the signal exceeds a threshold for two or more images representing that capillary, the controller selects the central image from the series of images representing that capillary.

[0137] In some embodiments, the controller maps cellular events back to a set of x, y, and t in a 3D matrix that is the maximum per image of its capillary. Each cellular event may occur in multiple images, with a number of occurrences dependent on the flow velocity, capillary length, and acquisition frame rate. The ability to detect cellular events is limited by the maximum crossing time of gaps around a reference frame, in which frame the gap passes through the reference pixel. For each image, the controller 2350 marks the brightest pixel, which is assumed to be inside the capillary.

[0138] For each capillary in the second capillary set, the controller 2350 then generates an associated event count. As described in more detail herein, in some embodiments, the event counts are averaged across all capillaries in the second capillary set, and then each image set can be used for user classification.

[0139] Figures 27A and 27B illustrate the results obtained from the detection of cellular events in 26 raw video / image sets compared to human evaluators. Compared to one of the individual evaluators, the F1 score is used as an index to measure the consistency of detected events (defined as the harmonic mean of precision P and recall R, where P = TP / (TP + FP) and R = TP / (TP + FN)). For each of the three experts, the F1 score was evaluated against the cellular events consistently detected by the remaining experts (i.e., the group of two). In the embodiments disclosed herein, the F1 score was evaluated as the average of the results across the three groups of these two experts to ensure an unbiased comparison. The F1 score takes into account true positives, false positives, and false negatives.

[0140] Figures 28A to 28D illustrate an example of the overall approach to detecting cellular events as described herein with respect to controller 2350.

[0141] Referring again to Figure 23, in some embodiments, the controller 2350 classifies the user into a user type in a user type set based on the number of cellular events detected in a second capillary set. In some embodiments, at least one user type is associated with a diagnosis of neutropenia (absolute neutrophil count < 500 cells / uL), and at least one other user type is associated with users who are not neutropenic. In some embodiments, the controller 2350 employs a weighting approach (i.e., employing a weighted average of event counts across all capillaries) to improve the quality of classification. In some embodiments, the weighting approach may be based on the estimated quality of the image set.

[0142] In some embodiments, the controller 2350 generates a single event metric value (which may sometimes be referred to as a "white blood cell metric value") that summarizes all event counts. In some embodiments, the controller compares the event metric value to an event threshold μ, which (in some embodiments) can be a learning parameter, as described in more detail herein. Classification of the user as a user type associated with severe neutropenia (e.g., a first user type) occurs when the event index value < μ, and as another user type (e.g., a second user type). Although two user types are described herein, it should be understood that the controller 2350 can classify the user into three, four, or more user types depending on the threshold parameter.

[0143] As described herein, the controller 2350 can assign weights to each capillary of the second capillary set based on its estimated quality of the capillary. As an example, in some embodiments, the weight of the capillary can be set to 1 / 10 when the threshold used for event detection is greater than a fixed level relative to the standard deviation of the signal (indicating lower quality) and otherwise 1.

[0144] As described above, in some embodiments, the event threshold can be a learning parameter. For example, in some embodiments, the event threshold was determined based on a total of 116 raw video / image sets. Specifically, 116 imaging sessions from 42 patients (mostly each associated with one raw video / video set) were employed. From these sessions, 60 corresponded to reference blood draw values ANC < 500, and 56 corresponded to ANC > 500 (15 of which were 500 < ANC < 1,500). In cases where consecutive daily blood draws showed a transition from ANC < 500 to ANC > 500, an imaging session was performed within 8 hours of the first blood draw.

[0145] Figures 29A and 29B show the results obtained from the classification of the aforementioned data using the systems, devices, and methods disclosed herein. Under a detection rate of 96% for severe neutropenia (AUC=0.96), there was a 9% false alarm (i.e., 9% of cases were incorrectly identified as severe neutropenia). The results showed that the analysis was inconsistent. In these results, 20% of the sessions were deemed inappropriate during the analysis and were therefore discarded. The corresponding conditions were (a) no sessions with <3 detected capillaries, and (b) no sessions with a white blood cell index of zero, with individual capillaries (i.e., individual capillaries without detected events) also being discarded as part of (b). Figures 61A and 61B illustrate how the classification may be affected by the length of the video / number of images in the adopted image set.

[0146] Figure 30 illustrates a method 3000 for non-invasive hematological measurement according to some embodiments. Method 3000 can be performed by system 2300 or by structurally and / or functionally similar modifications thereof. Method 3000 includes acquiring a set of images of at least a portion of the nail fold of a user's finger in 3100. In some embodiments, step 3100 further includes acquiring the image set as a set of frames of a video. In some embodiments, method 3000 further includes illuminating a portion of the nail fold, and acquiring the image set is in response to the illumination of the portion of the nail fold.

[0147] Method 3000 further includes, in 3200, detecting one or more capillaries within a portion of the fingernail fold in each image of the image set to identify a first set of capillaries across the image set. The above detection may include estimating one or more attributes of each capillary in the first set of capillaries. One or more attributes include one or more structural attributes, one or more flow attributes, one or more imaging attributes, or a combination thereof, and therefore, the first attribute of one or more attributes of each capillary in the first set of capillaries satisfies a predetermined criterion for the first attribute. In some embodiments, one or more structural attributes are selected from the group consisting of mean capillary diameter, capillary transverse diameter, capillary longitudinal diameter, capillary length, capillary shape, etc. In some embodiments, one or more flow attributes are selected from the group consisting of blood flow velocity in the capillary, cell migration time in the visible portion of the capillary, volume flow rate, mass flow rate, etc. In some embodiments, one or more imaging attributes are selected from the group consisting of contrast, focus, signal-to-noise ratio, image stability, etc. In some embodiments, the first attribute is the average capillary diameter, and each capillary in the second set of capillaries has an estimated average capillary diameter ranging from about 10 μm to about 20 μm.

[0148] In some embodiments, method 3000 may further include generating confidence values ​​in the image set for each capillary in a first capillary set. The first capillary set may include capillaries for which the confidence value exceeds a confidence threshold for each image in which a capillary is detected.

[0149] In some embodiments, Method 3000 may further include receiving a training image set, which includes the specification of one or more capillaries visible in each image of the training image set. Method 3000 may further include training a neural network with respect to the training image set, and applying the image set to the neural network to detect a first set of capillaries.

[0150] In some embodiments, detecting a first set of capillaries further includes applying the image set to a neural network, which is trained on a training image set that includes the specification of one or more capillaries visible in each image of the training image set.

[0151] Method 3000 further identifies the second capillary set from the first capillary set in 3300, and as a result each capillary in the second capillary set is an image of a predetermined number of images in the image set This further includes becoming visible.

[0152] In some embodiments, Method 3000 further includes detecting a set of cellular events for an image set and for a second capillary set. Each cellular event in the set of cellular events is related to the passage of leukocytes in the capillaries of the second capillary set. Based on the set of cellular events, Method 3000 may further include estimating the event count for the second capillary set.

[0153] In some embodiments, Method 3000 further includes estimating a quality factor for each capillary in the second capillary set. Method 3000 may further include estimating an event count based on a set of cellular events and the quality factor associated with each capillary in the second capillary set.

[0154] In some embodiments, Method 3000 further includes receiving a set of training images relating to a portion of the nail folds of a set of training users. Method 3000 may further include generating event count thresholds based on the training image set via supervised learning. Method 3000 may further include classifying users into a first user type of a set of user types based on the event counts and the event count thresholds, where at least one user type of the set of user types is associated with a diagnosis of neutropenia. Method 3000 may further include transmitting an indication of the first user type to the user (e.g., via the interface of System 2300 of Device 2340).

[0155] Additional details relating to various embodiments of the systems, devices, and methods described herein are shown below.

[0156] Volume estimation and capillary subselection to improve the accuracy of counting. As described above (see, for example, equation (8)), WBC concentration can be estimated based on an estimate of the number of WBC cells (e.g., estimated by the number of cellular events) and the volume of blood passing through the target capillary during the total video acquisition time. The total blood volume passing through a given capillary compartment usually depends on the local diameter of the capillary compartment and the mean local velocity of the blood. Furthermore, volume conservation (or blood mass conservation) indicates that the target total blood volume can be maintained at a constant value during propagation within the capillary. In other words, the total blood volume can be kept constant, independently of the capillary compartment under consideration. Therefore, an estimate of the volume of a capillary compartment can be used as an estimate of the target blood volume.

[0157] Based on this observation, robust volume estimation methods are based on cross-correlation or optical flow techniques.

[0158] The volume of a capillary compartment can be estimated using at least two approaches. One approach assumes that the capillary compartment in question has a perfect cylindrical shape. In this case, the width of the capillary obtained from a video or image can also be used as the depth of the capillary.

[0159] Another approach involves estimating the volume of capillary compartments based on an analysis of pixel intensity values. Specifically, the depth of capillaries at a given location in the image plane is expected to be inversely correlated with the measured light intensity due to light absorption. The longitudinal and transverse diameters of capillaries may not be equal, and the diameter of the visible projection may not represent the longitudinal diameter. In other words, a low pixel value indicates a large value for capillary depth, and vice versa. Thus, this elucidated depth estimate can be used to more accurately calculate the cross-sectional area of ​​capillary compartments.

[0160] Figures 31A to 31F illustrate a method for estimating the volume of a capillary compartment based on the analysis of pixel intensity. In some embodiments, some or all aspects of the methods shown in Figures 31A to 31F can be performed by one or more of the systems, apparatuses, and devices described herein, such as system 2300 and / or device 2340.

[0161] Figures 31A and 31B show two images of capillary compartments, where the pixel values ​​in Figure 31A are greater than those in Figure 31B (i.e., Figure 31 is brighter). The two capillary compartments have the same width of approximately 15 μm at the given positions indicated in Figures 31A and 31B. In conventional methods, this width is also used as the depth of the capillary compartment, and the reconstructed cross-section of the capillaries is perfectly circular, as shown in Figures 31C and 31D. In contrast, when pixel intensity is considered, different pixel values ​​indicate different depths for the two capillary compartments. Figures 31E and 31F show reconstructed cross-sections corresponding to the capillary compartments shown in Figures 31A and 31B, respectively, using a method based on pixel intensity analysis. As can be seen from Figures 31E and 31F, a larger pixel value in Figure 31A indicates a smaller depth, while a lower pixel value in Figure 31B indicates a larger depth. The cross-sectional shape of the two capillary compartments is elliptical, rather than circular.

[0162] The same analysis can be performed along the entire capillary shown in Figures 31A and 31B, and the cross-sectional area A can be obtained as a function of curvilinear coordinates (e.g., length l). The total volume V of the capillaries can be calculated by integrating A(l) with respect to l, i.e., by doing the following:

number

[0163] The volume calculated using the above method can also be used to resample the volume of a capillary profile. This method allows for the analysis of flow velocity in a given capillary using a one-dimensional curvilinear coordinate system. In one example, the coordinates in the one-dimensional curvilinear coordinate system are typically a function of length along the capillary (see, e.g., Figures 6A and 6B and the description above). In another example, the coordinates in the one-dimensional curvilinear coordinate system can be a function of cumulative capillary volume. One advantage of resampling coordinates by cumulative capillary volume is that particle / event velocities can become constant within a given coordinate system, thereby facilitating accurate estimation using correlation or similar methods.

[0164] Figures 32A–32C illustrate the resampling of capillary profile volumes. Figure 32A shows a representation of capillaries where the blood flow velocity is constant but the diameter changes along the capillary length. Due to light absorption, red blood cells are represented in black, while single white blood cells pushing through the capillaries are represented in white. Figure 32B shows a spatial-temporal map (color map from blue-dark to red-bright) representing grayscale values ​​along linear capillary length (y-axis) over time (x-axis). Figure 32C shows a spatial-temporal map obtained using volume resampling, which is shown as a function of cumulative capillary volume in each cross-section of Figure 32A, with capillary length as the coordinate (y-axis). In this case, the trajectory of a white blood cell at a constant velocity (red-bright line) becomes a straight line, and therefore easier and more accurate to estimate using correlation or similar methods.

[0165] Subselection of capillary events Further improvements in WBC detection and estimation (for example, as generally described in Figures 23-30) can be achieved through subselection of capillaries where events are most easily detectable and most likely to correlate with the presence of WBCs. Subselection of capillary events can be performed according to several criteria.

[0166] In one example, subselection may follow the observed size, i.e., the mean diameter and / or diameter profile. Size typically correlates with the presence of events in a clear, high-contrast total capillary diameter within a given range, such as between approximately 10 μm and 20 μm. Figure 33 shows the range of capillary diameters in which clear gaps exist. Subselecting capillaries based on their diameters can improve the correlation with WBCs.

[0167] In another example, subselection may follow a time distribution of arrival events across the entire capillary diameter, where these events are shown to differ in the presence / absence of leukocytes. Figure 34A shows the distribution of time to arrival (TOA) between gaps in a sample containing 5,500 WBCs / μL. For this high concentration of WBCs, the TOA substantially follows a Poisson distribution. In contrast, Figure 26B shows the distribution of TOA between gaps in a sample containing 100 WBCs / μL, a much lower concentration than that shown in Figure 34A.

[0168] In yet another example, subselection can be carried out according to the dynamic behavior of capillaries. In this approach, the method can directly subselect capillaries based on event features including length, contrast-to-noise ratio, and velocity. For example, occlusive activity can be identified as being caused by erythrocytes in capillaries of a certain diameter.

[0169] Acquisition of high-speed, high-contrast, and stable video of multiple nail fold capillaries. Figures 35A to 35C illustrate an apparatus for detecting severe neutropenia based on images of nail fold capillaries. In some embodiments, some or all aspects of the apparatus in Figures 35A to 35C may be structurally and / or functionally similar to one or more systems, apparatuses, and devices described herein, such as system 2300 and / or device 2340.

[0170] Figure 35A shows a drawn 3D model of the device used to record microscopic video of microcirculation in the patient's nail fold capillaries, along with its various components. Figure 35B illustrates the patient placing their ring finger in a 3D-printed holder, which serves two purposes: to achieve stability throughout the one-minute recording time and to hold the oil used for optical bonding. Figure 35C shows the finger positioned so that illumination and imaging are directed towards the nail fold region (the area enclosed by the purple circle in the figure).

[0171] The apparatus shown in Figure 35A is used to record high-quality microscopic videos of microcirculation in human nail fold capillaries, which can be made available to ASCT patients (see Figure 36). The patient's finger is inserted into a hole in a 3D-printed, hemispherical, easily sterilized handrest (Figure 35B), which is ergonomically designed to hold the patient's hand with sufficient stability to record a one-minute video. The capillary video is acquired from the nail fold region of the target finger (Figure 35C).

[0172] Figures 37A and 37B show examples of raw images acquired by the apparatus shown in Figure 35A. Wide-field video pairs were acquired by the apparatus from one ASCT patient at two different time points in time when the same capillaries could be observed (three numbered pairs are shown). Figure 37A shows images taken at baseline neutrophil concentration (i.e., >1,500 / μL), and Figure 37B shows images taken when the patient had severe neutropenia (neutrophil concentration <500 / μL). The image shown was taken at [location]. The scale bar in the image represents 100 μm.

[0173] Figures 37A and 37B show that the device can optically capture capillary-nail fold videos with appropriate contrast, resolution, stability, and depth of field. The raw video acquired by the device contains multiple capillaries within the field of view. Acquisition of multiple capillaries was particularly easy with this simple optical approach, which allows imaging of multiple capillaries simultaneously within the same field of view. In contrast, many existing methods, such as coded confocal microscopy (SECM), can only image a single capillary at a time.

[0174] Figures 38A–38E illustrate an example of an optical gap flowing through a capillary. The image sequence shows several raw frames from a video centered on a single capillary, acquired by the instrument from one patient at baseline. Dark loops correspond to capillaries filled with RBCs that absorb light at the illumination wavelength. The light absorption gap in microcirculation is approximately the same size as the capillary width (about 15 μm) and can be observed flowing through the arterial limb of the capillary (indicated as a black arrow). Frame numbers are labeled in the upper right corner. The frame rate was 60 frames per second, and the exposure time was 16.7 milliseconds. Contrast was adjusted for the indicated area of ​​interest.

[0175] These capillaries in the acquired images are very narrow, with a typical width of approximately 10 μm to 20 μm. In this case, the whole blood cell count (WBC) has no choice but to push its way through the capillaries one by one. Focusing on a single capillary, the frame reveals the passage of an event in the microcirculation, which can be perceived as a moving "bright" object roughly the same size as the capillary diameter (approximately 15 μm). The bright object (or gap against a dark background) has significantly higher brightness than the surrounding RBC and is therefore easily observable and monitorable.

[0176] The movement of events is clearly tracked across consecutive frames and is therefore visually identifiable by human observers. Accordingly, all events in the capillary video were marked and labeled by three blinded human evaluators according to specific visual criteria (see further details below). A space-time (ST) map provides a convenient alternative representation for showing the trajectories of all events with marks obtained from these evaluators in a one-minute capillary video.

[0177] Figures 39A–39E show the results of blinded event evaluations using images acquired by the apparatus shown in Figure 35A. Three human evaluators labeled one event observed in one of the 98 capillaries used in the study. Figures 39A–39C show capillary video frames (indexed in the upper right) with cross-shaped event marks obtained from evaluators 1 (blue), 2 (green), and 3 (yellow). Figure 31D shows the aggregated location of all event marks obtained from all three evaluators. Figure 39E shows an ST map displaying the recorded brightness levels along the segmented capillary length (vertical axis) as a function of time (horizontal axis) during 1.7-second intervals around the event of interest. The bright trajectory generated by the passage of the event can be clearly identified in the center of the ST map. The blue, green, and yellow cross marks correspond to the spatial and temporal coordinates where each of the three evaluators labeled the event. The event trajectories visible on the ST map correspond well to the events identified by evaluators from the video.

[0178] Microcirculatory events with specific characteristics can be used as a substitute for whole-body cell division (WBC). Figure 36 illustrates two different timing points for acquiring images from patients in a clinical trial. Patients enrolled in the trial undergo ASCT, but ASCT is a result of This process makes the progression of their neutrophil counts due to the management administration of pharmacotherapy highly predictable. This provides the opportunity to record capillary videos for each patient at two different time points: (1) baseline (neutrophils > 1,500 / μL) and (2) severe neutropenia (neutrophils < 500 / μL).

[0179] Images from the acquired videos were analyzed at two different time points, as shown in Figure 36, for 10 ASCT patients who underwent chemotherapy: pre-chemotherapy (neutrophils >1,500 per μL, baseline) and during severe neutropenia (neutrophils <500 per μL). To minimize potential confounding factors and selection bias, the same set of capillaries was acquired for each patient at baseline and during severe neutropenia (see details below). Acquiring 1-minute videos allowed us to overcome shot noise associated with the individual nature of events.

[0180] In these videos, the consistency of event identification by three raters in capillary microcirculation depended on whether the capillary videos were acquired at baseline or during severe neutropenia. When raters identified and counted events in capillary videos acquired during baseline, 67% of those events were confirmed (i.e., two or more raters identified the same event). In the case of capillary videos acquired during severe neutropenia, only 22% of events were confirmed. The visual features of the assessed events tended to be even more consistent in baseline events (see Figures 48A and 48B).

[0181] These results suggested that events with consistently detectable visual features correlated with the presence of WBCs and neutrophils. Therefore, the count of confirmed events was treated as a surrogate for the WBC count. The fact that most confirmed events occurred in capillary compartments comparable in size to WBCs (see Fig. 47) corroborates findings from previous literature, but the findings of these literatures associate the observed gaps with combinations of WBCs and plasma flowing through capillaries.

[0182] Non-invasive Detection of Severe Neutropenia The count of white blood cells (WBCs) can be used as one of the indicators of the immunological status for the diagnosis and treatment of multiple medical conditions, including cancer, infections, sepsis, and autoimmune disorders. The WBC count is also used with immunosuppressive agents. However, current methods of WBC counting usually involve venipuncture performed by trained clinical staff, and even in the case of finger-prick techniques, they involve a personal visit to a medical institution. This constraint limits both the time and frequency of monitoring. Furthermore, conventional blood tests typically use specific reagents and sterile conditions, which may thereby eliminate their applicability in resource-limited environments. In contrast, non-invasive approaches to WBC measurement avoid many of these requirements and are comparable to existing non-invasive techniques for monitoring blood oxygen saturation.

[0183] One step towards non-invasive WBC analysis involves non-invasive screening for severe neutropenia, which can be defined as a common type of WBC characterized by low levels of neutrophils (e.g., less than 500 per μL). This condition is one of the major toxicities in patients undergoing common chemotherapy regimens. Since the associated risk of infection increases, it is involved in significant morbidity and significant risk of death. However, monitoring for severe neutropenia is currently inadequate for the reasons stated above. This barrier to prompt clinical care hinders timely life-saving interventions with prophylactic antibiotics or growth colony-stimulating factors in afebrile patients with chronic severe neutropenia. In that regard, non-invasive methods can substantially impact outpatient care and management of patients at high risk of immunosuppression associated with severe neutropenia.

[0184] The systems, devices, and methods described herein can provide a screening tool for severe neutropenia based on non-invasively and portably optically visualizing capillaries. When the capillary diameter is close to the WBC diameter (e.g., from about 10 μm to about 20 μm), white blood cells can completely fill the capillary lumen. This typically results in depletion of red blood cells (RBCs) downstream of the WBCs in the microcirculation, where the WBCs flow at a lower velocity than the RBCs in the direction of flow. In this situation, appropriate illumination (e.g., of a specific wavelength) can render the WBCs transparent and the RBCs dark, and the passage of the WBCs can appear as a light absorption gap in the continuous flow of RBCs moving through the capillary.

[0185] This "gap" phenomenon can be observed in a rabbit ear window model using a white light transmission microscope. The results explicitly showed that when capillaries and WBCs have similar diameters, RBCs accumulate upstream of WBCs, with a drastically reduced range of RBCs downstream. The same phenomenon was observed in the microcirculation of rat cremaster muscle and bat wings, but in this observation, blue light transmission was used to maximize the contrast between RBCs—i.e., the peak absorption of oxyhemoglobin and deoxyhemoglobin is blue at 420 nm—and low-absorption regions without RBCs. By observing the flow of RBCs over time, the morphology of the capillaries was revealed, where brighter regions were associated with light-absorbing gaps within the capillary lumen. Fluorescent labeling can also be used to confirm that the gaps are associated with WBCs.

[0186] The idea that such absorption gaps are related to WBCs was investigated in humans using the blue entoptic phenomenon, in which WBCs transmit blue light as they flow in front of the retina, thus creating bright spots that the subject can see. For example, subjects could have blue light shone into their eyes and were asked to report the number of bright spots they perceived. Differences in the amount of perceived spots were reported between groups—baseline, leukopenic, and leukocytogenic subjects—i.e., comparing WBC counts in the normal, abnormally low, and abnormally high ranges, respectively. However, one limitation of these methods is that they rely on the subjects' self-reporting. Therefore, they are prone to personal bias and poor reproducibility, and are therefore not suitable for clinical screening.

[0187] Overall, these findings suggest that the basis for novel methods for non-invasively measuring WBC counts lies in the flow gaps within capillaries. The systems, apparatus, and methods described herein utilize nail fold capillaries, which are superficial (e.g., at a depth of approximately 50 μm to 100 μm), have a diameter comparable to that of WBCs, run substantially parallel to the skin surface, and can therefore be visualized non-invasively using simple and readily available optical instruments.

[0188] This method screens for severe neutropenia in human subjects by using optical imaging to count events defined as examples of the movement of light absorption gaps in the microcirculation of the nail fold. To do this, a portable device is configured to produce optical microscope videos of capillaries (see, e.g., Figures 35A–35C). This method can maximize the contrast between RBCs and non-RBCs across multiple capillaries within a single field of view while ensuring accurate resolution, depth of field, stability, and frame rate. Based on this device, a clinical study was conducted involving 10 patients undergoing high-dose chemotherapy and autologous stem cell transplantation (ASCT), taking into account the predictability of neutrophil kinetics at their lowest point and recovery (see, e.g., Figure 34). For each patient, a 1-minute video of the same set of capillaries was acquired by the device at two time points: baseline before chemotherapy (approximately 1,500 neutrophils per μL) and severe neutropenia (approximately 500 neutrophils per μL) (e.g., (See, for example, Figures 29A and 29B). Based on this data, methods can be developed and validated for tagging event counts (see Figures 38A–39D) and for distinguishing baseline from severe neutropenia across all patients (see, for example, Figures 40 and 41).

[0189] The baseline state can be non-invasively classified from a severe neutrophil state. Figure 40 shows the number of confirmed events per minute in all tested capillary pairs. Baseline values ​​(blue dots) showed a statistically significant difference compared to the corresponding values ​​in severe neutropenia (red squares). To maximize the objectivity of event selection and discard noise, only confirmed events are considered. All capillaries were analyzed in both baseline and severe neutropenia (98 pairs in total, black dotted line).

[0190] The number of confirmed events in capillaries imaged during severe neutropenia was consistently lower than the number of confirmed events in the same capillaries imaged during baseline, as shown in Figure 40. Specifically, the number of paired capillaries showed a statistically significant difference (P<10) between baseline and severe neutropenia. -8 The Wilcoxon signed-rank test was used to analyze the results. Counts derived from separate capillaries tended to vary throughout the same patient, sometimes reaching low values ​​even at baseline. Such variations may be associated with multiple factors, which motivated us to aggregate multiple capillary-derived counts for every patient (see details below).

[0191] Figure 41 shows the distinction between baseline and severe neutropenia. The median number of confirming events observed per minute allows for the distinction between baseline (blue dots) and severe neutropenia (red dots) for 20 acquired videos and 10 study patients when aggregating all available capillaries per patient. Inter-capillary variability is also shown for each patient (blue and red bars). The optimal threshold for separating baseline from severe neutropenia is 7 events per capillary per minute (black dotted line). The X-axis is labeled by patient ID along with the amount of capillaries analyzed (in parentheses). The median amount of capillaries used per patient was 4.

[0192] When data was collected from all capillaries of a given patient at a given time, the distinction between neutropenia and baseline was clear (Figure 41), and the results showed a statistically significant difference between baseline and severe neutropenia (P=0.002, Wilcoxon signed-rank test). Furthermore, in the case of this capillary collection, the distribution of confirmed event counts at baseline did not overlap with the distribution in severe neutropenia. In fact, with seven count thresholds, the median count for a given patient's capillaries was able to correctly classify 9 out of 10 cases of neutropenia. The initial addition of two or three capillaries explained most of the increase in classification performance (see Figures 44A and 44B).

[0193] The above study demonstrated that severe neutropenia can be non-invasively detected in humans based on optical imaging. It also validates the overall classification strategy, demonstrating the principles, including the optical equipment, experimental protocol, and data analysis techniques.

[0194] Intentionally, the clinical trial included baseline absolute neutrophil count (ANC) (neutrophils > 1,500 / μL) and severe neutropenia (neutrophils < 500 / μL) in the same patients. Although the classification approach was not evaluated for cases of mild (grade II, neutrophils < 1,500 / μL) and moderate (grade III, neutrophils < 1,000 / μL) neutropenia, the current results can be extrapolated to these additional ranges, assuming that the mean event count changes accordingly. This is through neutropenia of different grades. Furthermore, this may include additional clinical studies using representative data. On the other hand, the event counts obtained for each patient in the trial (shown in Figure 41) correspond to the corresponding reference cell concentrations shown in Table 1 below.

[0195] [Table 1] Table 1. Reference values ​​obtained from hospital clinical laboratories.

[0196] Specifically, assuming an average blood flow velocity of 800 μm / sec and an average capillary diameter of 15 μm, the median counts for patients aggregated for baseline and severe neutropenia, namely 31.73 and 1.96, yield estimated WBC concentrations of approximately 3,700 and 200 cells / μL, respectively. Both estimates fall within the reference range of corresponding values ​​obtained from standard laboratory assays (Table 1).

[0197] Event evaluation can be performed automatically on the input capillary video. Such algorithms can follow approaches used to detect objects moving through capillaries, or more advanced strategies, such as machine learning techniques. Several event features, such as contrast, size, or persistence, can be employed. Beyond mere counting, algorithmic estimation of capillary blood flow can improve the accuracy and precision of results by providing estimates that are physically consistent with WBC concentration.

[0198] Further improvements to the device can increase the amount of capillaries per patient that meet the quality and consistency standards required for further analysis. This could be useful for future clinical applications of this technology. The constraint of tracking the same capillaries in the same patient is eased, making the method easier to apply clinically.

[0199] One possible extension to this study is to investigate whether specific WBC ranges can be identified beyond screening for severe neutropenia. This would broaden the applicability of the methods described herein, not only within the context of chemotherapy but also in new settings such as infectious diseases, while still following a similar conceptual approach. Furthermore, if we know that distinct WBC types are linked to distinct optical and imaging characteristics, for example, that non-granular and granular WBCs correspond to distinct event lengths and backscattering characteristics within capillaries, then non-invasive differential WBC counting could also be achieved based on similar methods / data.

[0200] Overall, this study demonstrated that chemotherapy-induced severe neutropenia could be non-invasively detected through the nailfold by a temporarily developed prototype device. This study represents the first proof of concept for a technology capable of measuring an important toxicity of chemotherapy by optical means. The automation, repetition, and refinement of these results could lead to a new paradigm in the monitoring of cancer patients at risk of severe neutropenia. Furthermore, from a more general perspective, the proposed imaging approach and conceptual approach could constitute one of the first steps towards non-invasive in vivo WBC counting.

Example

[0201] To test the empirical hypothesis that a non-invasive method enables the classification of severe neutropenia (neutrophils < 500 cells / μL) and baseline state (neutrophils > 1,500 cells / μL) in patients, a pilot diagnostic validation test was conducted. A cohort of patients who received high-dose chemotherapy followed by ASCT was enrolled. Due to the intensity of the chemotherapy applied before transplantation, the kinetics of neutrophil counts in these patients are predictable because the course of severe neutropenia and subsequent recovery are ensured. In the framework of this test, a power analysis was not performed, and a simple sample of 10 subjects (selected from the initial patient pool) was considered sufficient to test the test hypothesis. Non-parametric tests were used in parallel with ROC curve analysis. All human evaluators who analyzed the data were blinded as detailed below.

[0202] A total of 23 patients were recruited, 16 and 7 patients were from Massachusetts General Hospital in Boston, Massachusetts, USA, and Hospital Universitario La Paz in Madrid, Spain, respectively. Each recruited patient signed an informed consent. All the information obtained was anonymized so that the participants could not be identified.

[0203] The following criteria were used for patient inclusion in the recruitment: (a) Patients must have an ASCT of hematopoietic progenitor cells scheduled; (b) Patients must be 18 years of age or older; (c) Patients must be able to understand and be willing to sign the written consent form; (d) Patients must have a white blood cell count of 3,000 cells / μL or higher and a neutrophil count of 1,500 cells / μL or higher at baseline. Patients were excluded if they had myelodysplasia, a history of allergic reactions to components of chemical compositions similar to oils used in optical bonding in clinical devices, or if their photodermatological type was greater than 4 on the Fitzpatrick scale.

[0204] The MGH clinical trial was approved by the Dana-Farber / Harvard Cancer Center (DFHCC) Institutional Review Board and by the MIT Committee on the Use of Human Subjects in Experiments (COUHES) as COUHES Protocol #1610717680. This study was also registered on Clinicaltrials.gov. The La Paz clinical trial was approved by the La Paz Ethics Committee in document HULP PI-2353. The analysis of anonymized data obtained from these pilot trials was also approved by the Ethics Committee of the Polytechnic University of Madrid.

[0205] optical device An example of the configuration of a device used to acquire video in ASCT patients (shown in Figure 35A) includes the following elements. This configuration is for illustrative purposes only and can be modified by those skilled in the art.

[0206] 1. Imaging objective lens. Edmund Optics TECHSPEC 5X. The optical features of this objective lens are a magnification of 5× and a maximum numerical aperture of 0.15 reduced through the use of a 2.5mm diameter 3D printed iris. The above iris maximizes the depth of focus and allows simultaneous imaging of multiple capillaries. The working distance is 16.2mm and the maximum field of view (FOV) is 1.8×1.32 mm. Its dimensions are a fixed tube length of 50mm and an overall length of 93.81mm. It is compatible with [the specified feature]. Its height, azimuth, and focus position are manually adjustable.

[0207] 2. CMOS camera. SoLab DCC3240N. This CMOS camera is mounted to the objective lens and computer power supply via USB connection. It features a global / rolling shutter. Its field of view (FOV) is 1280×1024 pixels, or 1360×1088um at 5× magnification, corresponding to a pixel size of 1.0625×1.0625um. The frame rate is approximately 60 frames per second (FPS) across all frames, thus ensuring sufficient temporal resolution to detect and track events, given that the blood flow velocity range in nail fold capillaries is 100-1,000 μm / sec. The frame rate can reach 229 FPS when limited to an FOV of 320×240 pixels. Its bit depth is 10 bits per pixel in monochrome.

[0208] 3. Illumination device. A makeshift prototype LED holder was used, mounted by a cage to a heatsink at an angle of approximately 70 degrees from the detection axis. These include high-power Luxeon LEDs emitting deep blue light, i.e., at 420 nm. This illumination wavelength makes it possible to maximize the contrast between RBC—which appears dark in motion—and the light absorption gap. Each LED emits 161 lumens at 700 mA using an aspherical focusing lens with F=20.1 mm, NA=0.6, and a SoraLab VA100C with an adjustable sighting slit.

[0209] 4. Power driver. Used to continuously drive both LED lights at a constant DC power level.

[0210] 5. Disposable hand rest. A rigid, 3D-printed platform used to hold fingers in a stable position for imaging for at least one minute. The platform features a one-size-fits-all finger hole. Optical bonding oil (Johnson & Johnson, refractive index = 1.51) remains in the finger hole.

[0211] 6. Laptop and Software. A laptop connected to the CMOS camera was used for power and image acquisition. Specifically, custom-built LabVIEW software was used for video acquisition and storage. The output data collected by this software for each patient and acquisition session consisted of an uncompressed video set and a timestamp providing information about the precise acquisition time associated with each frame for each video.

[0212] As part of a clinical trial, two units of this device were installed and used at Massachusetts General Hospital in Boston, Massachusetts, USA, and at La Paz, Madrid, Spain. After each use of the device on a patient, disinfectant wipes were used on the system components. The use of this device was approved for clinical trials under DF / HCC Protocol #15-070.

[0213] Data collection In this study, videos were acquired from the same set of capillaries for each given patient at baseline and during severe neutropenia. Tracking the same capillaries at both time points helps to avoid potential bias in capillary selection and minimize confounding factors. For example, this was expected to minimize changes in the geometric arrangement of capillaries between baseline and the time of severe neutropenia, and therefore ensure that changes in counts within a given pair of capillaries most accurately reflect the underlying changes in WBC concentration.

[0214] To obtain a video containing common capillaries in a given patient, the user of the optical prototype machine can During severe neutropenia, at least one capillary range similar to the baseline was identified. This process was performed manually during raw data acquisition, but proved difficult in certain cases due to logistical reasons. On the other hand, the diverse capillary distributions and morphologies may have enabled accurate identification of previously acquired regions during baseline acquisition. Furthermore, the process of identifying this capillary range was simplified by the fact that the target nail fold capillaries were selected from the vicinity of the nail fold boundary, and therefore the target capillary region was considerably limited.

[0215] Of the 23 recruited patients, 10 met the qualitative criteria and were deemed eligible for further processing (see Capillary Selection). Specifically, six entire patient datasets were excluded due to insufficient imaging quality, and four were excluded because there was no correspondence between the capillaries tested at baseline and at severe neutropenia. For each of the 10 eligible patients, at least one pair of videos corresponding to acquisitions of the same capillary region at both clinical states were selected. This amounted to 20 raw video datasets, each containing two distinct capillary regions acquired for patients 01 and 02. A total of 49 distinct capillaries were selected and tracked and analyzed at baseline and during severe neutropenia.

[0216] Videos were acquired within 8 hours of the corresponding blood test that provided reference information. Specifically, in addition to selected videos and capillaries, WBC and ANC concentrations were obtained for each patient along with their clinical status using state-of-the-art blood cell analysis (see Table 1). The reference blood analysis values ​​from this table clearly show a decrease in WBC and ANC concentrations between baseline and severe neutropenia.

[0217] Preprocessing workflow Figure 42 illustrates the pre-processing workflow in a clinical trial. Once raw patient video is acquired (top left), a set of capillaries of appropriate quality is selected by two human experts (top right, green rectangle). For each selected capillary (top right, examples identified by arrows), motion-corrected video of the corresponding area of ​​interest is then created (bottom right). The following steps (bottom left) involve event labeling by three blinded human evaluators, which enables subsequent analysis and event counting, aggregation, and visualization.

[0218] Each of the target raw input videos was processed according to the preprocessing workflow shown in Figure 42. After selecting the capillaries, each capillary video included in the study was trimmed to the corresponding target region and registered with respect to a reference frame in the sequence to correct for motion during acquisition. The motion-corrected videos were anonymized / shuffled, and then a blinded expert used them to identify events and derive event counts accordingly. The data with labeled events could then be visualized retrospectively. Details of these steps are provided below.

[0219] Selection of capillaries Once raw video footage was obtained from a given patient, two human experts separately defined appropriate capillary sets based on the qualitative, empirical, and deductive criteria defined below. To avoid potential bias, only capillaries selected by both blinded evaluators were included in the study. Each expert individually selected the best capillaries in the raw video footage according to the following criteria.

[0220] A. Illumination. Capillaries are visible with sufficient contrast for the observer.

[0221] B. Focus. Detailed capillary structure / dynamics are visible and not blurred.

[0222] C. Flow. Blood flow exists to enable the identification and counting of potential events.

[0223] D. Stability. The capillaries remain completely within the field of view of the video in every frame.

[0224] E. Visibility. There are no objects (e.g., air bubbles) that could block the capillaries.

[0225] F. Morphology. Capillaries exhibit distinct arterial and venous limbs.

[0226] G. Both baseline acquisition and acquisition of severe neutropenia must satisfy conditions A to F.

[0227] For all patients, each specialist first followed this procedure for baseline state videos. The goal was to obtain pairs of capillary videos for all patients, and the set of candidate capillaries for severe neutropenia was already limited by selection during baseline. The resulting pairs of capillary videos—at least one per patient—met the above criteria in both baseline and severe neutropenia, and by both specialists (see Figures 49–60).

[0228] Creating a video of capillaries Based on the raw video data and the capillary selection procedure described above, individual capillary videos were created based on: (a) the demarcation of the target rectangular region surrounding each target capillary on the first video frame, and (b) video motion compensation software that locally corrects camera movement to ensure that the position of each capillary remains stable within the corresponding target region throughout the duration of the video. Note that all raw videos were first flattened, that is, their local brightness was normalized through Gaussian filtering to remove the potential effects of uneven lighting.

[0229] Step (a) was performed based on a simple graphical user interface, and step (b) was performed based on a specially designed motion compensation algorithm. Both implementations were done in MATLAB. Based on the raw video and a predetermined target rectangular region, the algorithm outputs a motion-corrected capillary video by applying a rigid body alignment technique, which specifically aligns all video frames with the first one, assuming that the potential camera movement in the target region is merely a combination of X and Y transformations, excluding rotation.

[0230] While frame movement in raw video may result in deformation, visual inspection of the alignment results was successful when applied separately to the field of view of each capillary. As a criterion for similarity and optimization, the algorithm uses cross-information, an information theory-based measure that ensures accurate subpixel alignment by addressing slight contrast changes. Prior to this alignment process, a preliminary rough alignment step is performed to ensure proper initialization. This initialization step pre-aligns frames based on cross-correlation analysis of pixel values ​​and spatial gradients.

[0231] Evaluation of events Based on a graphical user interface, three human evaluators followed specific visual criteria to identify all events in the capillary video. Under these visual criteria, the generation of RBC depletion regions was included as a consequence of WBC passage through capillaries of approximately the same diameter. Specifically, a moving light absorption gap refers to an event with the following characteristics:

[0232] In one example, the phenomenon is significantly brighter than the surrounding capillary flow. In another example, The event can be identified as a distinct object moving along the flow of the capillary. In yet another example, the event occupies the entire diameter of the capillary and extends along the direction of the flow.

[0233] The evaluators were blinded to other individuals, as well as to corresponding blood analysis, physiological status, patient, and temporal information. Each evaluator labeled the corresponding frame and spatial location within the capillary where these events occurred.

[0234] The indexing of videos available to evaluators for event identification and counting was obtained by randomly shuffling the names of the original videos, so that evaluators could not access the original indexing even if the corresponding content was similar. Furthermore, the number of frames was always kept the same for all videos obtained from the same patient. The corresponding files did not contain any secondary information that would enable such identification. All videos—both unshuffled and shuffled indexed versions—were anonymized in the sense that there was no information or naming in the video content that could be used to identify the patient or neutropenia. After blinded evaluation, all marked events could be visualized based on a method specifically developed.

[0235] statistical analysis The counts obtained from all three independent evaluators allow for the determination of the characteristics of the collaborative evaluation, i.e., whether a single expert or two or more experts agreed to observe the same event. By convention, even if the average mark times from at least R evaluators are within a maximum of 10 frames (1 / 6 second) of each other, it is assumed that at least R evaluators collaboratively marked a given event, although this time is substantially smaller than the expected event rate in capillary videos (see Figure 46 below). Precise spatial superposition of labels is not required. The specific case of interest is majority evaluator agreement, and in this setting, R is greater than or equal to 2, which produces a confirmed event. The counts were then performed accordingly, i.e., the events from each capillary video were summed accordingly.

[0236] Confirmed counts for multiple capillaries were combined for each patient (see Figures 41, 44A, and 44B). This is because the accuracy of individual counts in a single capillary is limited by (a) shot noise proportional to the square root of the count quantity, (b) potential WBC phenomena such as marginal trend that do not fit the event criteria and do not occur in some capillaries, (c) the geometric arrangement and flow velocity of a particular capillary, and (d) the specific positioning of the capillary in the dynamics of the underlying capillary network. Considering a fixed amount of capillary count combinations per patient (Figures 44A and 44B), corresponding capillary sets were randomly selected and results were calculated based on 10,000 trials. This is similar to Monte Carlo integration and allows for the effective handling of exponentially increasing inter-patient combinations that would otherwise be difficult to manage.

[0237] When comparing counts between capillaries or combinations thereof, the analytical tool used was the Wilcoxon signed-rank test for paired data, which avoids the statistical assumption that counts are distributed normally while testing / negating hypothesis H0 that there is no difference between the counts observed in baseline and severe neutropenia for the same capillaries. Furthermore, this test generates a receiver operational characteristic (ROC) curve and a corresponding area under the curve (AUC) value, the latter testing the performance of the binary classification between baseline and severe neutropenia as a function of the varying count threshold.

[0238] Visualization of marked events To visualize the marked events in the capillaries, throughout the entire duration of the video... We developed a method to simultaneously visualize the frame of a target capillary video and its corresponding ST intensity profile, both within and around a given frame. This allows for the visualization of events both explicitly as moving targets through the corresponding video frame and as fixed profiles in the ST map representation. In a clinical trial setting, this visualization technique enabled not only retrospective analysis of the distribution of labeled events in the video but also the agreement of a majority of evaluators.

[0239] The concept of ST mapping for visualizing capillary flow has been explained above. The motivation for using this representation is that events associated with WBC are expected to appear as dense, high-contrast, sparse, unidirectional trajectories. These characteristics are also related to visual criteria defined for the evaluator, such as the brightness of the events. ST mapping makes events marked by the evaluator appear as clearly defined, prominent trajectories surrounded by a dark background (see Figure 39E above).

[0240] To create the ST map, this method extracts capillary brightness—as an average across cross-sections—as a function of time and as a function of cumulative capillary length, based on the segmented capillary boundaries. To improve the visualization of event trajectories, the map values ​​were normalized by subtracting the local temporal average obtained between 50 frames before and after each time point. The initial capillary segmentation procedure was performed based on images extracted from the corresponding registered video.

[0241] Since the capillary profile may be incomplete within a single video frame due to the presence of absorption gaps in the flow, time-integrated images were extracted for the nailfold capillaries, and in those images, the time-varying features associated with capillary flow were also enhanced to maximize the contrast between the capillaries and their surroundings. This approach is also related to the concept of enhancing the contrast of motion. Specifically, the images used for segmentation were obtained through the integration of time-frequency components whose periods were empirically selected to be at intervals of [0.25, 1.5] seconds.

[0242] The boundaries of the capillaries were first manually segmented in the first step and then refined using the active contour method. Then, the segmentation of both capillary boundaries was automatically resampled to include 1,000 points each, and such that the center of all point pairs at the same index of both boundaries lies on the medial axis of the capillary, where the medial axis is the locus of all circles inscribed in the capillary. Finally, based on this segmentation, the separation between the arterial, venous, and loop segments of the capillaries was defined on a case-by-case basis for visualization (see FIGS. 45A - 45C).

[0243] Acquisition time under shot noise The video acquisition time t was set to 1 minute because it is still suitable for clinical settings and is long enough to enable sampling of a significantly higher amount of baseline case events N compared to cases of severe neutropenia. Even under shot noise due to the quantized nature of the events, the distribution of the counts associated with both cases is expected to deviate within at least one standard deviation of their respective means N b and N n as expected from the calculations detailed below.

[0244] To determine this result, the lower bound case (C b(= neutrophils 1,500 / μL) and the upper limit case for severe neutropenia (C n We considered the worst-case scenario using neutrophils (500 cells / μL), but this scenario is the most difficult to distinguish because the difference in cell concentration C from both categories is minimized. Then, we used typical values ​​from the literature, capillary diameter (D=15μm) and flow velocity (v=800μm / By assuming a time interval of seconds, it became possible to estimate the expected event quantity from the concentration. Specifically,

number

[0245] Figure 35 shows the number of events labeled by a single assessor. As obtained in 98 capillaries in this study, baseline counts for which agreement among two or more experts (blue dots) was not reached did not show a statistically significant difference (P=0.12; Wilcoxon signed rank) with respect to the corresponding count for severe neutropenia (red squares). This result indicates that single-assessor labeled events are less objective and contain less information than those with agreement among multiple assessors. Capillary pairs are grouped by patient ID.

[0246] Figures 44A and 44B illustrate the distinction between baseline and severe neutropenia using capillary aggregates. Figure 44A shows the number of event counts resulting from aggregating N=1, 2, 3, 4, and 5 capillaries per patient. Figure 44B shows the ROC curves for baseline vs. severe neutropenia classification based on aggregating N=1, 2, 3, 4, and 5 capillaries per patient. The patient-level distribution of the resulting counts shows that their discriminative power increases with the amount of combined capillaries per patient. Specifically, the minimum area under the curve (AUC) for 1, 2, 3, 4, and 5 capillaries, respectively, consistently increases from 0.61 to 0.84, 0.92, 0.96, and 1.00 with increasing amounts of combined capillaries.

[0247] Figures 45A to 45C show examples of capillary segmentation. Figure 45A shows capillaries obtained from patient 02. Figure 45B shows the same capillaries with supervised segmentation (red). The scale bar is 20 μm. Figure 45C shows that the separation between arterial limbs (green), venous limbs (blue), and loops (red) of capillaries can be defined using case-by-case criteria for visualization.

[0248] Figure 46 shows the expected event rate per minute in capillaries under shot noise. Assuming typical capillary diameter and velocity values ​​obtained from the literature, the baseline expected event rate (blue, 1,500 neutrophils / μL) exceeds the rate corresponding to severe neutropenia (red; 500 neutrophils / μL) in a single minute acquisition, even under shot noise. Shown are the expected count mean (median point) combined with expected variability resulting from shot noise up to a maximum of one standard deviation (bar).

[0249] Figure 47 shows the distribution of capillary diameters at event locations. It shows the distribution of capillary diameter values ​​at locations where three blinded evaluators labeled events. The detected events tended to appear in capillary segments of approximately the same size range as WBCs, i.e., [10-20] μm, thus confirming the usefulness of events as a substitute for WBCs, as observed in previous literature.

[0250] Figures 48A and 48B show ST maps of capillaries with confirmed high vs. low percentage events. For each event, the first clicks from each of the three human evaluators are indicated by red, blue, and green dots, respectively. Agreement among evaluators indicates a high degree of preference. The baseline capillary levels (upper map) were higher compared to the stropenia (lower map) cases, indicating that the baseline events correspond to a wider range of target physical phenomena.

[0251] Figure 49 shows the selection of capillaries from both experts in a raw video pair derived from patient 01, region 1. Videos acquired in baseline and severe neutropenia are shown. Green boxes outline selected capillary pairs that met the qualitative criteria by each of the two experts for both the baseline and severe neutropenia acquisitions. Red regions / capillaries are discarded because they do not meet the qualitative criteria, i.e., in this case (b) lack of focus, (c) lack of blood flow, and (d) out-of-field movement. In the baseline video, the red lines / corners outline the effective field, and outside of that, capillaries must be discarded if they disappear between frames due to camera movement during acquisition. Yellow boxes outline capillaries that were initially selected at baseline but later discarded due to unsuitability during the severe neutropenia acquisition (g). Both experts independently carried out the selection process described above, and then, only the capillaries agreed upon by both experts (numbered in black) were selected, while the rest (numbered in red) were discarded.

[0252] Figure 50 shows the selection of capillaries from both experts in raw video pairs from patient 01, region 2. Videos acquired in baseline and severe neutropenia are shown. Green boxes outline selected capillary pairs that met the qualitative criteria by each of the two experts for both baseline and severe neutropenia acquisitions. Red regions / capillaries are discarded because they do not meet the qualitative criteria, i.e., in this case (b) lack of focus, (d) out-of-field movement, and (e) occlusion. In the baseline video, red lines / corners outline the effective field, and outside of that, capillaries must be discarded if they disappear between frames due to camera movement during acquisition. Yellow boxes outline capillaries that were initially selected at baseline but later discarded due to unsuitability during the severe neutropenia acquisition (g). Both experts independently carried out the selection process described above, and then, only the capillaries agreed upon by both experts (numbered in black) were selected, while the rest (numbered in red) were discarded.

[0253] Figure 52 shows the selection of capillaries from both experts in a raw video pair derived from patient 02, region 1. Videos acquired in baseline and severe neutropenia are shown. Green boxes outline selected capillary pairs that met the qualitative criteria by each of the two experts for both the baseline and severe neutropenia acquisitions. Red regions / capillaries are discarded because they do not meet the qualitative criteria, i.e., in this case (a) insufficient illumination, (b) lack of focus, (c) lack of blood flow, and (e) occlusion. In the baseline video, red lines / corners outline the effective field of view, and outside of this, capillaries must be discarded if they disappear between frames due to camera movement during acquisition. Yellow boxes outline capillaries that were initially selected at baseline but later discarded due to unsuitability during the severe neutropenia acquisition (g). Both experts independently carried out the selection process described above, and then, only the capillaries agreed upon by both experts (numbered in black) were selected, while the rest (numbered in red) were discarded.

[0254] Figure 52 shows the selection of capillaries from both experts in a raw video pair derived from patient 02, region 2. Videos taken in baseline and severe neutropenia are shown. Green boxes outline the selected capillary pairs that met the qualitative criteria by each of the two experts, for both baseline and severe neutropenia acquisitions. Red regions / capillaries do not meet the qualitative criteria, i.e., in this case (b) lack of focus, (c) lack of blood flow, (e) occlusion, and (f) lack of clear morphology. Therefore, they are discarded. In the baseline video, the red lines / corners outline the effective field of view, and outside of this, capillaries must be discarded if they disappear between frames due to camera movement during acquisition. The yellow boxes outline capillaries that were initially selected at baseline but were later discarded due to unsuitability during acquisition of severe neutropenia (g). Both experts independently performed the above selection process, and then kept only the capillaries agreed upon by both experts (black numbers), discarding the rest (red numbers).

[0255] Figure 53 shows the selection of capillaries from both specialists in a raw video pair from patient 03. Videos acquired in baseline and severe neutropenia are shown. Green boxes outline selected capillary pairs that met the qualitative criteria by each of the two specialists for both the baseline and severe neutropenia acquisitions. Red areas / capillaries are discarded because they do not meet the qualitative criteria, i.e., in this case (a) insufficient illumination, (b) lack of focus, (c) lack of blood flow, (d) out-of-field movement, and (e) occlusion. In the baseline video, the red lines / corners outline the effective field, and outside of that, capillaries must be discarded if they disappear between frames due to camera movement during acquisition. Yellow boxes outline capillaries that were initially selected at baseline but later discarded due to unsuitability during the severe neutropenia acquisition (g). Both experts independently carried out the selection process described above, and then, only the capillaries agreed upon by both experts (numbered in black) were selected, while the rest (numbered in red) were discarded.

[0256] Figure 54 shows the selection of capillaries obtained by both experts in a raw video pair from patient 04. Videos acquired in baseline and severe neutropenia are shown. Green boxes outline selected capillary pairs that met the qualitative criteria of each expert for both baseline and severe neutropenia acquisitions. Red areas / capillaries are discarded because they do not meet the qualitative criteria, i.e., in this case (b) lack of focus. In the baseline video, the red lines / corners outline the effective field of view, and outside of this, capillaries must be discarded if they disappear between frames due to camera movement during acquisition. Yellow boxes outline capillaries that were initially selected at baseline but later discarded due to unsuitability during the severe neutropenia acquisition (g). Both experts independently performed the above selection process, and then kept only the capillaries agreed upon by both experts (black numbers), discarding the rest (red numbers).

[0257] Figure 55 shows the selection of capillaries from both experts in a raw video pair from patient 05. Videos acquired in baseline and severe neutropenia are shown. Green boxes outline selected capillary pairs that met the qualitative criteria of each expert for both baseline and severe neutropenia acquisitions. Red areas / capillaries are discarded because they do not meet the qualitative criteria, i.e., in this case (b) lack of focus, (c) lack of blood flow, (d) out-of-field movement, and (e) occlusion. In the baseline video, the red lines / corners outline the effective field, and outside of that, capillaries must be discarded if they disappear between frames due to camera movement during acquisition. Yellow boxes outline capillaries that were initially selected at baseline but later discarded due to unsuitability during the severe neutropenia acquisition (g). Both experts independently carried out the selection process described above, and then, only the capillaries agreed upon by both experts (numbered in black) were selected, while the rest (numbered in red) were discarded.

[0258] Figure 56 shows the selection of capillaries from both experts in a raw video pair derived from patient 06. Videos taken in baseline and severe neutropenia are displayed. The green boxes outline selected capillary pairs that met the qualitative criteria as determined by each of the two experts, for both baseline and severe neutropenia acquisitions. Red areas / capillaries are discarded because they do not meet the qualitative criteria, i.e., in this case (b) lack of focus, (c) lack of blood flow, (d) movement outside the field of view, and (f) lack of clear morphology. The red lines / corners in the baseline video outline the effective field of view, and outside of this, capillaries must be discarded if they disappear between frames due to camera movement during acquisition. The yellow boxes outline capillaries that were initially selected at baseline but later discarded due to unsuitability during the severe neutropenia acquisition (g). Both experts independently performed the above selection process, and then only capillaries agreed upon by both experts (black numbers) were kept, with the rest (red numbers) being discarded.

[0259] Figure 57 shows the selection of capillaries from both experts in a raw video pair from patient 07. Videos acquired in baseline and severe neutropenia are shown. Green boxes outline selected capillary pairs that met the qualitative criteria of each expert for both baseline and severe neutropenia acquisitions. Red areas / capillaries are discarded because they do not meet the qualitative criteria, i.e., in this case (a) insufficient illumination, (b) lack of focus, (d) out-of-field movement, and (e) occlusion. In the baseline video, the red lines / corners outline the effective field, and outside of this, capillaries must be discarded if they disappear between frames due to camera movement during acquisition. Yellow boxes outline capillaries that were initially selected at baseline but later discarded due to unsuitability during the severe neutropenia acquisition (g). Both experts independently carried out the selection process described above, and then, only the capillaries agreed upon by both experts (numbered in black) were selected, while the rest (numbered in red) were discarded.

[0260] Figure 58 shows the selection of capillaries from both experts in a raw video pair from patient 08. Videos acquired in baseline and severe neutropenia are shown. Green boxes outline selected capillary pairs that met the qualitative criteria of each expert for both baseline and severe neutropenia acquisitions. Red areas / capillaries are discarded because they do not meet the qualitative criteria, i.e., in this case (b) lack of focus, (d) movement outside the field of view, (e) occlusion, and (f) lack of clear morphology. In the baseline video, the red lines / corners outline the effective field of view, and outside of this, capillaries must be discarded if they disappear between frames due to camera movement during acquisition. Yellow boxes outline capillaries that were initially selected at baseline but later discarded due to unsuitability during the severe neutropenia acquisition (g). Both experts independently carried out the selection process described above, and then, only the capillaries agreed upon by both experts (numbered in black) were selected, while the rest (numbered in red) were discarded.

[0261] Figure 59 shows the selection of capillaries obtained by both experts in a raw video pair from patient 09. Videos acquired in baseline and severe neutropenia are shown. Green boxes outline selected capillary pairs that met the qualitative criteria by each of the two experts for both baseline and severe neutropenia acquisitions. Red areas / capillaries are discarded because they do not meet the qualitative criteria, i.e., in this case (a) insufficient illumination, (b) lack of focus, (c) lack of blood flow, and (e) occlusion. In the baseline video, the red lines / corners outline the effective field of view, and outside of this, capillaries must be discarded if they disappear between frames due to camera movement during acquisition. Yellow boxes outline capillaries that were initially selected at baseline but later discarded due to unsuitability during the severe neutropenia acquisition (g). Both experts independently performed the above selection process, and then both experts... Only the capillaries agreed upon by the family (numbered in black) were kept, and the rest (numbered in red) were discarded.

[0262] Figure 60 shows the selection of capillaries from both experts in raw video pairs from patient 10. Videos acquired in baseline and severe neutropenia are shown. Green boxes outline selected capillary pairs that met the qualitative criteria of each expert for both baseline and severe neutropenia acquisitions. Red areas / capillaries are discarded because they do not meet the qualitative criteria, i.e., in this case (b) lack of focus, (d) out-of-field movement, and (f) lack of clear morphology. In the baseline video, the red lines / corners outline the effective field of view, and outside of this, capillaries must be discarded if they disappear between frames due to camera movement during acquisition. Yellow boxes outline capillaries that were initially selected at baseline but later discarded due to unsuitability during the severe neutropenia acquisition (g). Both experts independently carried out the selection process described above, and then, only the capillaries agreed upon by both experts (numbered in black) were selected, while the rest (numbered in red) were discarded.

[0263] Each of the following references is incorporated herein by reference in its entirety.

[0264] Tsukada, Kosuke, et al. Image correlation method for measuring blood flow velocity in microcirculation:correlationwindow'simulation and in vivo image analysis." Physiological measurement 21.4(2000):459.

[0265] Langeder, Florian, and Bernhard G. Zagar.” Image processing strategies to accurately measure red blood cell motion in superficial capillaries." Systems,Signals and Devices,2009.SSD’09.6th International Multi-Conference on.IEEE,2009.

[0266] Wu,Chih-Chieh,et al."Red blood cell vel ocity measurements of complete capillary in finger nail-fold using optical flow estimation." Microvascular research 78.3 (2009):319-324.

[0267] Huang,Tzung-Chi,et al."Experimental est imation of blood flow velocity through simulation of intravital microscopic imaging in micro-vessels by different image processing methods." Microvascular resea rch 80.3(2010):477-483.

[0268] Wu,Chih-Chieh,et al."Accuracy evaluatio n of RBC velocity measurement in nail-fold capillaries." Microvascular research 81.3(2011):252-260.

[0269] Lo,Lun-Chien,et al."Pseudo three-dimens ional vision-based nail-fold morphological and hemodynamic analysis." Computers in biology and medicine 42.9(2012):873-884.

[0270] Wang,Mingyi,et al."Full-field velocity imaging of red blood cells in capillaries with spatiotemporal demodulation autocorrelation." Journal of biomedical optic s 21.3(2016):036007-036007.

[0271] J.Crawford,D.C.Dale,and G.H.Lyman(2004).Chemotherapy‐induced neutropenia:risks,consequences,and new directions for its management.Cancer,100(2),228-237.

[0272] M.E.van Wolfswinkel,K.Vliegenthart-Jongbloed,M.de Mendonca Melo,P.C.Wever,M.B.McCall,R.Koelewijn,J.J.van Hellemond,and P.J.van Genderen(2013).Predictive value of lymphocytopenia and the neutrophil-lymphocyte count ratio for severe imported malaria.Malar.J.,12(1),101.

[0273] T.Honda,T.Uehara,G.Matsumoto,S.Arai,and M.Sugano(2016).Neutrophil left shift and white blood cell count as markers of bacterial infection.Clin.Chim.Acta,457,46-53.

[0274] T.B.Newman,D.Draper,K.M.Puopolo,S.Wi,and G.J.Escobar(2014).Combining immature and total neutrophil counts to predict early onset sepsis in term and late preterm newborns:use of the I / T2.Pediat.Infect.Dis.J.,33(8),798.

[0275] A.Velo-Garcia,S.G.Castro,and D.A.Isenberg(2016).The diagnosis and management of the haematologic manifestations of lupus.J.Autoimmun.,74,139-160.

[0276] D.C.Dale(2014).Understanding neutropenia.Curr.Opin.Hematol.,21(1),1-2.

[0277] V.S.Hollis,J.A.Holloway,S.Harris,D.Spencer,C.van Berkel,and H.Morgan(2012).Comparison of venous and capillary differential leukocyte counts using a standard hematology analyzer and a novel microfluid ic impedance cytometer.PloS one,7(9),e43702.

[0278] C.L.Ghai(2012).A textbook of practical physiology.JP Medical Ltd.

[0279] S.Sharma,J.Zapatero-Rodriguez,P.Estrela,and R.O’Kennedy(2015).Point-of-care diagnostics in low resource settings:present status and future role of microfluidics.Biosensors,5(3),577-601.

[0280] Y.Mendelson(1992).Pulse oximetry:theory and applications for noninvasive monitoring.Clin.Chem.,38(9),1601-1607.

[0281] A.Trotti,A.D.Colevas,A.Setser,V.Rusch,D.Jaques,V.Budach,C.Langer,B.Murphy,R.Cumberlin,C.N.Coleman,and P.Rubin(2003,July).CTCAE v3.0:development of a comprehensive grading system for the adverse effects of cancer treatment.In Semin.Radiat.Oncol.(Vol.13,No.3,pp.176-181).WB Saunders.

[0282] G.H.Lyman,M.S.Poniewierski,J.Crawford,D.C.Dale,and E.Culakova(2015).Cost of Hospitalization in Patients with Cancer and Febrile Neutropenia and Impact of Comorbid Conditions.Blood,126(23),2089-2089.

[0283] G.H.Lyman、C.H.Lyman、O.Agboola、and Anc Study Group.(2005).Risk models for predicting chemotherapy-induced neutropenia.Oncologist,10(6),427-437.

[0284] J.De Naurois,I.Novitzky-Basso,M.J.Gill,F.M.Marti,M.H.Cullen,F.Roila,and ESMO Guidelines Working Group.(2010).Management of febrile neutropenia:ESMO clinical practice guidelines.Ann.Oncol.,21(suppl 5),v252-v256.

[0285] G.W.Schmid-Schonbein,S.Usami,R.Skalak,and S.Chien(1980).The interaction of leukocytes and erythrocytes in capillary and postcapillary vessels.Microvasc.Res.,19(1),45-70.

[0286] S.H.Sinclair,M.Azar-Cavanagh,K.A.Soper, R.F.Tuma,and H.N.Mayrovitz(1989).Investigation of the source of the blue field entoptic phenomenon.Invest.Ophthalmol.Vis.Sci.,30(4),668-673.

[0287] A.Roggan,M.Friebel,K.Dorschel,A.Hahn,and G.Muller(1999).Optical properties of circulating human blood in the wavelength range 400-2500 nm.J.Biomed.Opt.,4(1),36-46.

[0288] A.Uji,M.Hangai,S.Ooto,K.Takayama,N.Arakawa,H.Imamura,K.Nozato,and N.Yoshimura(2012).The source of moving particles in parafoveal capillaries detected by adaptive optics scanning laser ophthalmoscopy.Invest.Ophthalmol.Vis.Sci.,53(1),171-178.

[0289] U.Schmidt-Gross(1954).Entoptische Beurteilung der Leukocytenzahl.Klin.Wochenschr.,32(33-34),817-819.

[0290] T.Rimmer,E.M.,Kohner,and J.M.Goldman(1988).Retinal blood velocity in patients with leukocyte disorders.Arch.Ophthalmol.,106(11),1548-1552.

[0291] G.Fuchsjager-Mayrl,M.Malec,E.Polska,B.Jilma,M.Wolzt,and L.Schmetterer(2002).Effects of granulocyte colony stimulating factor on retinal leukocyte and erythrocyte flux in the human retina.Invest.Ophthalmol.Vis.Sci.,43(5),1520-1524.

[0292] C.E.Curtis,W.G.Iacono,and M.Beiser(1999).Relationship between nailfold plexus visibility and clinical,neuropsychological,and brain structural measures in schizophrenia.Biol.Psychiatry,46(1),102-109.

[0293] F.Lefford and J.C.Edwards(1986).Nailfold capillary microscopy in connective tissue disease:a quantitative morphological analysis.Ann.Rheum.Dis.,45(9),741-749.

[0294] Z.Nagy and L.Czirjak(2004).Nailfold digital capillaroscopy in 447 patients with connective tissue disease and Raynaud’s disease.J.Eur.Acad.Dermatol.Venereol.,1 8(1),62-68.

[0295] L.K.Mercer,T.L.Moore,H.Chinoy,A.K.Murray,A.Vail,R.G.Cooper,and A.L.Herrick(2010).Quantitative nailfold video capillaroscopy in patients with idiopathic inflammatory myopathy.Rheumatology,49(9),1699-1705.

[0296] A.K.Murray,A.Vail,T.L.Moore,J.B.Manning,C.J.Taylor,and A.L.Herrick(2012).The influence of measurement location on reliability of quantitative nailfold videocapillaroscopy in patients with SSc.Rheumatology,kes007.

[0297] H.M.Hofstee,E.H.Serne,C.Roberts,R.Hesselstrand,A.Scheja,T.L.Moore,M.Wildt,J.B.Manning,A.V.Noordegraaf,A.E.Voskuyl,and A.L.Herrick(2011).A multicentre study on the reliability of qualitative and quantitative nail-fold videocapillaroscopy assessment.Rheumatology,ker403.

[0298] S.J.Simnett,L.A.Stewart,J.Sweetenham,G.Morgan,and P.W.Johnson(2000).Autologous stem cell transplantation for malignancy:a systematic review of the literature.Clin.Lab.Haematol.,22(2),61-72.

[0299] L.Golan,D.Yeheskely-Hayon,L.Minai,E.J.Dann,and D.Yelin(2012).Noninvasive imaging of flowing blood cells using label-free spectrally encoded flow cytometry.Biomed.Opt.Express,3(6),1455-1464.

[0300] N.Mugii,M.Hasegawa,Y.Hamaguchi,C.Tanaka,K.Kaji,K.Komura,I.Ueda-Hayakawa,S.Horie,M.Ikuta,K.Tachino,F.Ogawa,S.Sato,M.Fujimoto,and K.Takehara(2009).Reduced red blood cell velocity in nail-fold capillaries as a sensitive and specific indicator of microcirculation injury in systemic sclerosis.Rheumatology,48(6),696-703.

[0301] A.Bourquard,I.Butterworth,A.Sanchez-Ferro,L.Giancardo,L.Soenksen,C.Cerrato,R.Flores,and C.Castro-Gonzalez(2015,August).Analysis of white blood cell dynamics in nailfold capillaries.In Conf.Proc.IEEE Eng.Med.Biol.Soc.,2015 37th Annual International Conference of the IEEE(pp.7470-7473).

[0302] C.Castro-Gonzalez,I.Butterworth,A.Bourquard,and A.Sanchez-Ferro(2015).Systems,apparatus,and methods for analyzing blood cell dynamics.U.S.Patent Application No.14 / 951,260.

[0303] C.C.Wu,G.Zhang,T.C.Huang,and K.P.Lin(2009).Red blood cell velocity measurements of complete capillary in finger nail-fold using optical flow estimation.Microvasc.Res.,78(3),319-324.

[0304] T.C.Huang,W.C.Lin,C.C.Wu,G.Zhang,and K.P.Lin(2010).Experimental estimation of blood flow velocity through simulation of intravital microscopic imaging in micro-vessels by different image processing methods.Microvasc.Res.,80(3),477-483.

[0305] C.C.Wu,W.C.Lin,G.Zhang,C.W.Chang,R.S.Liu,K.P.Lin,and T.C.Huang(2011).Accuracy evaluation of RBC velocity measurement in nail-fold capillaries.Microvasc.Res.,81(3),252-260.

[0306] C.H.Wu,T.D.Wang,C.H.Hsieh,S.H.Huang,J.W.Lin,S.C.Hsu,H.T.Wu,Y.M.Wu,and T.M.Liu(2016).Imaging Cytometry of Human Leukocytes with Third Harmonic Generation Microscopy.Sci.Rep.,6.

[0307] T.C.Shih,G.Zhang,C.C.Wu,H.D.Hsiao,T.H.Wu,K.P.Lin,and T.C.Huang(2011,January).Hemodynamic analysis of capillary in finger nail-fold using computational fluid dynamics and image estimation.Microvasc.Res.81(1),68-72.

[0308] D.Mattes,D.R.Haynor,H.Vesselle,T.K.Lewellyn,and W.Eubank(2001,July).Nonrigid multimodality image registration.In Med.Imaging(pp.1609-1620).International Society for Optics and Photonics.

[0309] G.W.Schmid-Schonbein,R.Skalak,S.Usami,and S.Chien(1980).Cell distribution in ca pillary networks.Microvasc.Res.,19(1),18-44.

[0310] J.H.Lee,J.Jimenez,I.R.Butterworth,C.Castro-Gonzalez,S.K.Shukla,B.Marti-Fuster,L.Elvira,D.S.Boning,and B.W.Anthony(2015,October).Measurement of very low concentration of microparticles in fluid by single particle detection using acoustic radiation force induced particle motion.In 2015 IEEE Int.Ultrason.Symp.(pp.1-4).

[0311] J.Tam,P.Tiruveedhula,and A.Roorda(2011).Characterization of single-file flow through human retinal parafoveal capillaries using an adaptive optics scanning laser ophthalmoscope.Biomed.Opt.Express,2(4),781-793.

[0312] P.D.Allen,C.J.Taylor,A.L.Herrick,and T.Moore(1998,September).Enhancement of Temporally Variable Features in Nailfold Capillary Patterns.In BMVC(pp.1-10).

[0313] M.Kass,A.Witkin,and D.Terzopoulos(1988).Snakes:Active contour models.Int.J.Comput.Vision,1(4),321-331.

[0314] Conclusion While various embodiments of the invention have been described and illustrated herein, those skilled in the art will readily conceive of various other means and / or structures for carrying out the functions described herein and / or for obtaining the results and / or one or more advantages, and each of such variations and / or modifications will be considered to fall within the scope of the embodiments of the invention described herein. More generally, those skilled in the art will readily understand that all parameters, dimensions, materials and configurations described herein are illustrative, and that actual parameters, dimensions, materials and / or configurations will depend on the specific one or more uses in which the teachings of the present invention are used. Those skilled in the art can recognize, or simply verify by routine experimentation, numerous equivalents of the specific embodiments of the invention described herein. Therefore, it should be understood that the embodiments described herein are presented only as examples, and within the scope of the appended claims and their equivalents, embodiments of the invention may be practiced in ways other than those specifically described and claimed. Embodiments of the invention of this disclosure cover the individual features, systems, articles, materials, kits and / or methods described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the scope of the present invention as long as such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent.

[0315] The embodiments described above can be implemented by any of a number of means. For example, the embodiments disclosed herein may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code is a single code. It can run on any suitable processor or set of processors, whether provided to a single computer or distributed across multiple computers.

[0316] Furthermore, it should be understood that computers can be embodied in one of many forms, such as rack-mounted computers, desktop computers, laptop computers, or tablet computers. In addition, computers can be embedded in devices with sufficient processing power, including personal digital assistants (PDAs), smartphones, or any other suitable portable or fixed electronic device, rather than being devices generally considered computers.

[0317] Furthermore, a computer may have one or more input and output devices. These devices can, among other things, be used to present a user interface. Examples of output devices that can be used to provide a user interface include a printer or a display screen for a visual representation of output, and a speaker or other sound-generating device for an audible representation of output. Examples of input devices that can be used for a user interface include a keyboard, as well as pointing devices such as a mouse, touchpad, and digitizer tablet. As another example, a computer may receive input information by speech recognition or in other audible formats.

[0318] Such computers may be interconnected by one or more networks of any suitable form, including local area networks, wide area networks such as enterprise networks, and intelligent networks (INs) or the Internet. Such networks may be based on any suitable technology, may operate according to any suitable protocol, and may include wireless networks, wired networks, or fiber optic networks.

[0319] The various methods or processes outlined herein may be coded as software executable on one or more processors using any one of a variety of operating systems or platforms. In addition, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine code or intermediate code that runs on a framework or virtual machine.

[0320] Furthermore, various concepts relating to the invention may be embodied in one or more methods, and examples of such methods have been provided. Actions performed as part of a method can be ordered by any suitable means. Thus, embodiments may be constructed in which actions are performed in a different order than those exemplified, and this may include performing some actions simultaneously, even if they are shown as a sequence of actions in the exemplary embodiments.

[0321] All publications, patent applications, patents, and other references mentioned herein are incorporated in their entirety by reference.

[0322] All definitions defined and used herein should be understood to govern dictionary definitions, definitions in documents incorporated by reference, and / or the ordinary meanings of the defined terms.

[0323] As used herein and in the claims, the indefinite articles "a" and "an" should be understood to mean "at least one" unless explicitly stated otherwise.

[0324] As used herein and in the claims, the phrase “and / or” should be understood to mean “either or both” of the elements thus coordinated, that is, elements that are sometimes connective and sometimes disjunctive. The multiple elements listed by “and / or” should be interpreted in the same form, that is, “one or more” of the elements thus coordinated. Other elements may be present, as they see fit, whether related to or unrelated to the elements specifically identified by the “and / or” clause. Thus, as a non-restrictive example, a reference to “A and / or B,” when used in conjunction with open-ended usage such as “equipped with,” may refer in one embodiment to A only (optionally including elements other than B), in another embodiment to B only (optionally including elements other than A), in yet another embodiment to both A and B (optionally including other elements), and so on.

[0325] Where used herein and in the claims, “or” should be understood to have the same meaning as “and / or” as defined above. For example, when separating items in a list, “or” or “and / or” shall be interpreted as inclusive, that is, including at least one of the many or enumerated elements and any additional items not enumerated at will, but also including two or more. Only terms that are explicitly indicated to be the opposite, such as “only one of” or “exactly one of” or, when used in the claims, “consisting of,” refer to including exactly one of the many or enumerated elements. In general, where used herein, the term “or” shall be interpreted only as indicating an exclusive choice (i.e., “one or the other, but not both”) when preceded by an exclusive term such as “either,” “one of,” “only one of,” or “exactly one of.” Where used in the claims, “consisting of” shall have the usual meaning as used in the field of patent law.

[0326] As used herein and in the claims, the phrase “at least one” in relation to an enumeration of one or more elements should be understood to mean at least one element selected from any one or more elements in the enumeration, but not necessarily including at least one of all elements specifically listed in the enumeration, nor excluding any combination of elements in the enumeration. This definition also allows for the optional presence of elements other than those specifically identified in the enumeration, to which the phrase “at least one” refers, whether related to or unrelated to the specifically identified elements. Therefore, as a non-restrictive example, "at least one of A and B" (or equivalently, "at least one of A or B," or equivalently, "at least one of A and / or B") may refer, in one embodiment, to at least one A in which B is absent and A can optionally include more than one (including elements other than B); in another embodiment, to at least one B in which A is absent and B can optionally include more than one (including elements other than A); and in yet another embodiment, to at least one A in which A can optionally include more than one, and at least one B in which B can optionally include more than one (including other elements); and so on.

[0327] In the claims and the above specification, all transitional phrases, such as "comprising," "including," "carrying," "having," "containing," "involving," "holding," and "composed of," are open-ended, meaning they include but are not limited to. It is understood to mean "not." Only the transitional phrases "consisting of" and "consisting essentially of" are considered closed or semi-closed transitional phrases, respectively, as described in Section 2111.03 of the U.S. Patent and Trademark Office's Patent Examination Procedure Manual.

Claims

1. A system for non-invasive in vivo hematological measurement, A platform for receiving a part of the user's body that has a capillary structure during use, A light source for illuminating the aforementioned capillary structure, An imaging device coupled to the platform and for acquiring a series of time-lapse images of the capillary structure, wherein each image resolves multiple capillaries; An imaging objective lens optically coupled to the imaging device, wherein the imaging device is configured to provide the controller with a series of time-lapse images of the capillary structure, The controller, Receiving a series of time-lapse images of the capillary structure from the imaging device; Identifying a first set of capillaries in the image set from the aforementioned series of time-lapse images; Identifying a second capillary set from the first capillary set, wherein the second capillary set includes capillaries from the first capillary set that are detectable within a threshold number of images in the series of time-lapse images and have a confidence value exceeding a confidence threshold within the minimum number of images in the series of time-lapse images; The detection of one or more cellular events from the second set of capillaries, wherein the cellular events include the passage of cells that show contrast in light absorption to red blood cells in the series of time-lapse images within the capillaries; and Based on the one or more cellular events detected, an event count is generated for each capillary in the second set of capillaries. The controller that conforms to the above The system comprising the above.

2. The system according to claim 1, wherein the cells are circulating tumor cells or leukocytes (WBCs).

3. The aforementioned cells are WBCs, The aforementioned controller: For the purpose of classifying the user to a first user type, the comparison is made by comparing the event count to an event count threshold associated with the minimum number of cellular events, wherein the one or more capillaries include capillaries having a diameter selected to be approximately equal to or less than the diameter of any WBC among the one or more types of WBCs; and Classifying the user to a first user type of a set of user types based on a comparison of the event count with the event count threshold, wherein at least one user type in the set of user types is associated with a state of neutropenia; The system according to claim 1, further adapted to the following.

4. The system according to claim 3, wherein the controller is further adapted to detect each cellular event of the cellular event by detecting the light absorption gap between the passage of the one or more types of WBCs in the capillaries of the one or more capillaries.

5. The system according to claim 3, wherein the controller is further adapted to detect each of the detected cellular events based on the comparison of absorption to red blood cells.

6. The system according to claim 3, wherein the illumination source (2320) is configured to illuminate the capillary structure with blue light.

7. The system according to claim 3, wherein the diameter is 10 microns to 20 microns.

8. The system according to claim 3, wherein the controller is further adapted to apply the series of time-lapse images to a neural network in order to identify the one or more capillaries.

9. The aforementioned controller The training user receives a set of training images related to the capillary bed of a part of the body of the training user; and Based on the aforementioned set of training images, an event count threshold is generated via supervised learning. The system according to claim 3, further adapted as follows.

10. The aforementioned controller In each of the aforementioned series of time-lapse images, a bounding box is generated for each detected capillary; The detection of the aforementioned capillaries generates a confidence value related to the possibility that it corresponds to capillaries and not to other structures / artifacts; and If the reliability value meets or exceeds a predetermined reliability threshold, the detected capillary is included in the one or more capillaries. The system according to claim 3, further adapted as follows.

11. The system according to claim 3, wherein the controller is further adapted to apply a bandpass filter in a separate Fourier domain, and the bandpass filter has low-frequency and high-frequency bandpass parameters that fit a range of expected cellular event sizes.

12. The system according to claim 3, wherein the controller is further adapted to compile the profiles extracted from the series of time-lapse images into at least one spatial-temporal profile to analyze WBC events for one or more capillaries.

13. The system according to claim 12, wherein the controller is further adapted to determine the velocity of the WBC from the at least one spatial-temporal profile.

14. The system according to claim 12, wherein the controller is further adapted to determine the total number of WBC events per microliter from the at least one spatial-temporal profile.

15. A method for non-invasive in vivo hematological measurement performed by a system for non-invasive in vivo hematological measurement, comprising a platform, an illumination source, an imaging device having an imaging objective lens, and a controller, The user's body, which has a capillary structure, is received on the platform, Illuminating the aforementioned capillary structure with the aforementioned illumination source, The imaging device acquires a series of time-lapse images of the capillary structure, The provision of a series of time-lapse images of the capillary structure to the controller, wherein the controller is communicatively connected to the imaging device. The controller processes the series of time-lapse images, thereby, In the image set from the aforementioned series of time-lapse images, the first set of capillaries is identified; Identifying a second set of capillaries from the first set of capillaries, wherein the second set of capillaries includes capillaries from the first set of capillaries that are detectable in a threshold number of images in the series of time-lapse images and have a confidence value exceeding a confidence threshold in the minimum number of images in the series of time-lapse images; One or more cellular events are detected from the second set of capillaries, the cellular events including the passage of cells within the capillaries, which show contrast in light absorption to red blood cells in the series of time-lapse images; The controller generates an event count for each capillary in the second set of capillaries based on the one or more cellular events detected; The above process The method, including the method described above.

16. The method according to claim 15, wherein the cells are circulating tumor cells or leukocytes (WBCs).

17. For the purpose of classifying the user to a first user type, the controller compares the event count to an event count threshold associated with a minimum number of cellular events, wherein the one or more capillaries include capillaries having a diameter selected to be approximately equal to or less than the diameter of any WBC among the one or more types of WBCs; The controller classifies the user into a first user type of a set of user types based on a comparison of the event count with the event count threshold, wherein at least one user type in the set of user types is associated with a state of neutropenia; The method according to claim 15, further comprising:

18. The method according to claim 17, further comprising detecting each of the detected cellular events based on the comparison of absorption to red blood cells using the controller.

19. The controller receives a set of training images related to the capillary bed of a part of the body of a set of training users; Based on the aforementioned set of training images, an event count threshold is generated through supervised learning. The method according to claim 17, further comprising:

20. The controller generates a bounding box for each detected capillary in each of the series of time-lapse images; The controller generates a confidence value related to the possibility that the detected capillaries correspond to capillaries and not to other structures / artifacts; The controller, if the reliability value satisfies or exceeds a predetermined reliability threshold, includes the detected capillary in the one or more capillaries. The method according to claim 17, further comprising:

Images