Systems and methods for the detection of cellular entities

Abstract

A device for investigating a target includes an imaging module and an interfacing module. The interfacing module includes a processor for analyzing a first image of a first plurality of images, the first image being a fluorescence-based image comprising fluorescence from the target, using an analytical model. The processor analyzes the three-dimensional image of the target using the analytical model to determine variations in emitted light intensity across the spatial region of the target by compensating for variations in distance across the spatial region of the target from the three-dimensional image capture sensor and variations in curvature across the spatial region of the target. The processor uses the analytical model to detect the presence of problematic cellular entities based on the analysis. The analytical model is trained to detect the presence of problematic cellular entities in the target.

Description

[Technical Field]

[0001] The present subject matter relates generally to the detection of problematic cellular entities, such as pathogens, in targets, and specifically to systems and methods for the detection of problematic cellular entities. [Background technology]

[0002] A cellular entity can be an entity consisting of one or more biological cells, such as a unicellular organism, a multicellular organism, or a tissue. A problematic cellular entity can be a cellular entity that can cause harm to the health of a plant, an animal, or a human. For example, a cellular entity can be a pathogen that causes disease in humans and a pathogen that delays wound healing. A problematic cellular entity can be a cellular entity that indicates a disease in a plant, an animal, or a human. For example, cancerous tissue can be a problematic cellular entity, indicating the presence of a tumor. For example, the presence of a problematic cellular entity in a target such as a human body, an animal, or a plant should be detected to prevent the occurrence of disease, provide timely treatment to avoid death, etc. Similarly, the presence of a problematic cellular entity in a target such as food, sanitary ware, or laboratory ware should be detected to determine contamination in food, contamination of the surface of sanitary ware or laboratory ware, determine the effectiveness of a disinfectant for laboratory ware, etc. Summary of the Invention [Means for solving the problem]

[0003] The detailed description is provided with reference to the accompanying drawings, in which the most significant digit(s) of a reference number identifies the drawing in which the reference number first appears, and the same numbers are used throughout the drawings to reference like features and components. [Brief explanation of the drawings]

[0004] [Figure 1] FIG. 1 is a block diagram of a device for investigating a target according to an implementation of the present subject matter. [Figure 2a] FIG. 1 is a front perspective view of a device for investigating a target according to an implementation of the present subject matter. [Figure 2b] FIG. 1 is a rear perspective view of a device for investigating a target according to an implementation of the present subject matter. [Figure 2c] 1 is an exploded view of a device for investigating a target according to an implementation of the present subject matter. [Figure 3] FIG. 1 is a block diagram of a device for investigating a target according to an implementation of the present subject matter. [Figure 4a] 1 is a perspective view of a device for investigating a target according to an implementation of the present subject matter. [Figure 4b] 1 is a perspective view of a device for investigating a target according to an implementation of the present subject matter. [Figure 4c] 1 is an exploded view of a device for investigating a target according to an implementation of the present subject matter. [Figure 4d] FIG. 1 is an exploded view of a portable power module of a device for investigating a target in accordance with an implementation of the present subject matter. [Figure 4e] FIG. 1 is an exploded view of an interfacing module of a device for investigating a target, according to an implementation of the present subject matter. [Figure 5] FIG. 1 illustrates a method for training an analytical model to detect problematic cellular entities in a target, according to an implementation of the present subject matter. [Figure 6] FIG. 1 illustrates an example for training an analytical model to detect problematic cellular entities in a target, according to an implementation of the present subject matter. [Figure 7] FIG. 1 illustrates a method for detecting problematic cellular entities according to an implementation of the present subject matter. [Figure 8] FIG. 1 illustrates a method for detecting problematic cellular entities according to an implementation of the present subject matter. [Figure 9] FIG. 1 illustrates a method for detecting problematic cellular entities according to an implementation of the present subject matter. [Figure 10] FIG. 1 illustrates a method for an auto-exposure process according to an implementation of the present subject matter. [Figure 11] FIG. 1 illustrates a method for detecting problematic cellular entities according to an implementation of the present subject matter. [Figure 12a] 1 is a perspective view of a device for investigating a target according to an implementation of the present subject matter. [Figure 12b] 1 is a perspective view of a device for investigating a target according to an implementation of the present subject matter. [Figure 12c] 1 is a perspective view of a device for investigating a target according to an implementation of the present subject matter. [Figure 12d] FIG. 1 is a top view of a device for investigating a target according to an implementation of the present subject matter. [Figure 12e] FIG. 1 is a top view of a device for investigating a target according to an implementation of the present subject matter. [Figure 12f] 1 is an exploded view of a device for investigating a target according to an implementation of the present subject matter. [Figure 12g] FIG. 1 is a front view of a device for investigating a target according to an implementation of the present subject matter. [Figure 12h] FIG. 1 is a top view of a device for investigating a target according to an implementation of the present subject matter. [Figure 12i] FIG. 1 is a side view of a device for investigating a target according to an implementation of the present subject matter. [Figure 13] FIG. 1 illustrates a device for investigating a target according to an implementation of the present subject matter. [Figure 14] FIG. 1 illustrates the detection of problematic cellular entities according to an implementation of the present subject matter. [Figure 15] FIG. 1 illustrates a system for surveying a target according to an implementation of the present subject matter. [Figure 16a] FIG. 1 illustrates a method for surveying a target according to an implementation of the present subject matter. [Figure 16b] FIG. 1 illustrates a method for surveying a target according to an implementation of the present subject matter. [Figure 17] 1A-1C illustrate results corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. [Figure 18]1A-1C illustrate results corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. [Figure 19] 1A-1C illustrate results corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. [Figure 20] 10A-10C illustrate results corresponding to tissue oxygen saturation according to an implementation of the present subject matter. [Figure 21] 1A-1D show results corresponding to the detection of biofilm in a wound according to an implementation of the present subject matter. [Figure 22] 1A-1C illustrate results corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. [Figure 23a] 1A-1C illustrate results corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. [Figure 23b] 1A-1C illustrate results corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. [Figure 24] 1A-1C illustrate results corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. [Figure 25] 1A-1C illustrate results corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. [Figure 26] 1A-1C illustrate results corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. [Figure 27] 1A-1C illustrate results corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. DETAILED DESCRIPTION OF THE INVENTION

[0005] The presence of problematic cellular entities in a target should be accurately detected. The target may be, for example, a wound area in the human body, food, a tissue sample extracted from the human body, or a surface that should be sterile, such as the surface of laboratory equipment, medical equipment, or sanitary equipment. Traditionally, culture methods are used to detect problematic cellular entities such as pathogens. In other words, to detect problematic cellular entities, a sample is obtained from an area suspected to be infected with a pathogen using a swab or deep tissue biopsy. The obtained sample is then stored in an appropriate culture medium, where the pathogens suspected to be present there grow over time. If a pathogen is present in the sample, the pathogen is isolated and identified using biochemical methods.

[0006] Similarly, for problematic cellular entities, such as cancerous tissue, tissue biopsies are performed. To identify whether the tissue is cancerous, the tissue biopsy is examined under a microscope with stains such as hematoxylin and eosin, mucicarmine, or Papanicolaou stain. In some cases, the examination can be performed without stains. As can be appreciated, these methods are laborious, require specialized microbiology facilities, and can take one to two days to accurately identify the infection and classify the pathogen or cancerous tissue.

[0007] In some cases, the detection and classification of problematic cellular entities is performed based on the autofluorescence resulting from natural biomarkers in the problematic cellular entities. The natural biomarkers can be, for example, nicotinamide adenine dinucleotide hydrogen phosphate (NAD(P)H), flavin, porphyrin, pyoverdine, tyrosine, and tryptophan. The autofluorescence resulting from the biomarkers can be unique to each biomarker and can be useful for the detection and classification of problematic cellular entities.

[0008] Although autofluorescence can be used for detection and classification, the autofluorescence typically produced by natural biomarkers can be weak and not easily detected. Furthermore, in addition to autofluorescence, the light emitted from the target can include background and excitation light, which can interfere with the emitted autofluorescence. Therefore, the detection and classification of problematic cellular entities using autofluorescence can be time-consuming, complicated, and relatively inaccurate.

[0009] Additionally, in some scenarios, the intensity of autofluorescence, reflection, and / or scattering emitted or reflected from different regions of a target may be the same. For example, assume that a target, such as a wound containing pathogens, is spread across a spatial region of the wound. Furthermore, assume that a first spatial region of the wound and a second spatial region of the wound are at different depths within the wound. In this regard, the intensities of autofluorescence emitted by the first spatial region and the second spatial region may be the same. Therefore, when autofluorescence is captured using a camera, such as a CMOS camera or a CCD camera, the spatial region of the wound farther from the camera appears weaker than the spatial region of the wound closer to the camera. For example, assume that the first spatial region of the wound is closer to the camera and the second spatial region of the wound is farther from the camera. In this regard, the autofluorescence emitted by the second spatial region appears weaker than the autofluorescence emitted by the first spatial region, regardless of the presence or number of pathogens.

[0010] Additionally, intensity variations from spatial regions of a target at the same distance from the camera can also be caused by target curvature, which results in different reflections, scattering, or autofluorescence. For example, assume a target, such as a wound containing pathogens, is spread across a spatial region of the wound. Further assume that a first spatial region of the wound and a second spatial region of the wound contain the same pathogens and the same pathogen densities. Furthermore, assume that the first spatial region is flat and the second spatial region is curved. Because the pathogens and pathogen densities are the same, the camera should capture the same densities of fluorescence, reflection, and / or scattering. However, due to the curvature, the fluorescence, reflection, and / or scattering intensities corresponding to the second spatial region may differ from those of the first spatial region.

[0011] Thus, detection of problematic cellular entities may be inaccurate and / or incorrect. Inaccuracy in detection of problematic cellular entities hinders accurate diagnosis of disease, prevention of disease onset, provision of timely treatment to avoid death, etc. Similarly, inaccurate and / or incorrect detection of problematic cellular entities in targets such as food, sanitary or laboratory ware, bodily fluids such as blood, medical devices such as catheters, etc., impacts determination of contamination in food, contamination on surfaces of sanitary or laboratory ware, etc.

[0012] The present subject matter relates to systems and methods for the detection of problematic cellular entities, such as pathogens, cancerous tissue, necrotic tissue, and the like, and implementations of the present subject matter can make the detection of problematic cellular entities, such as pathogens, cancerous tissue, necrotic tissue, and the like, rapid, accurate, simple, and cost-effective.

[0013] According to some implementations, a device for investigating a target may include an imaging module, an interfacing module, and a display. The target may be suspected of containing a problematic cellular entity, such as a pathogen or cancerous tissue. In some examples, the target may consist of one or more cells, such as a wound or tissue specimen in a body part. In other examples, the target may be an item that should be free of pathogens, such as food, laboratory equipment, or sanitary equipment. In some other examples, the target may be a bodily fluid, such as pus, blood, urine, saliva, sweat, semen, mucus, plasma, water, an injectable fluid, or the like, that may be suspected of containing a pathogen.

[0014] The imaging module may include a first plurality of light sources, an imaging sensor, and a three-dimensional image capture sensor. Each of the first plurality of light sources should emit excitation radiation in a predetermined range of wavelengths. Specifically, the emitted excitation radiation may be a single wavelength or a band of wavelengths that, when illuminated, causes one or more markers in the target to fluoresce. The first plurality of light sources may be, for example, homogeneous or heterogeneous light sources. In some examples, the use of heterogeneous light sources may reduce or eliminate background light in the light emitted by the target.

[0015] One or more markers may be part of the problematic cellular entity. Fluorescence emitted by a marker that is part of the problematic cellular entity may be referred to as autofluorescence. In some instances, an exogenous marker, such as indocyanine green (ICG) or a synthetic marker like methylene blue, may be sprayed onto a target to trigger detection of the problematic cellular entity in the target. The exogenous marker may bind to a cellular entity, such as deoxyribonucleic acid (DNA), ribonucleic acid (RNA), protein, blood, or biochemical marker, causing the target to fluoresce. Fluorescence emitted by the added synthetic marker may also be referred to as exogenous fluorescence.

[0016] In one example, the imaging sensor may be configured to directly receive light emitted by the target in response to illumination of the target by at least one light source of the first plurality of light sources, without an optical bandpass filter disposed between the imaging sensor and the target, and to capture a first plurality of images formed based on the emitted light. If the target includes a marker that emits fluorescence, the captured image may include fluorescence and be referred to as a fluorescence-based image. Thus, the fluorescence-based image may include fluorescence emitted from the target. Here, the light is said to be received directly from the imaging sensor because the emitted light is not filtered by an optical bandpass filter before capturing the image.

[0017] The imaging sensor may be a multispectral camera configured to capture light emitted by a target at multiple wavelengths. Specifically, the multispectral camera may capture light emitted at wavelengths in the visible range, the ultraviolet (UV) range, the near-infrared (NIR) range, or a combination thereof. In another example, the imaging sensor may be a charge-coupled device (CCD) sensor, a CCD digital camera, a complementary metal-oxide semiconductor (CMOS) sensor, a CMOS digital camera, a single-photon avalanche diode (SPAD), a single-photon avalanche diode (SPAD) array, an avalanche photodetector (APD) array, a photomultiplier tube (PMT) array, a near-infrared (NIR) sensor, a red-green-blue (RGB) sensor, or a combination thereof. In one example, the device may include one or more lenses, which may be integral with the imaging sensor, to focus light onto the imaging sensor and capture an image.

[0018] The three-dimensional image capture sensor may illuminate the target, receive light reflected by the target in response to the illumination of the target by the three-dimensional image capture sensor, and generate a three-dimensional image of the target based on the reflected light. Additionally, use of the three-dimensional image capture sensor may enable determination of variations in the intensity of light reflected by the target across a spatial region of the target. This intensity variation may need to be taken into account due to differences in the distances of multiple regions across the spatial region of the target from the three-dimensional image capture sensor and differences in curvature across the spatial region of the target. For example, a first spatial region of the target may be at a different distance from the three-dimensional image capture sensor than a second spatial region of the target. Thus, the first spatial region and the second spatial region may emit fluorescent light at the same intensity. Because the fluorescent light of the first spatial region and the second spatial region is the same intensity, the spatial region farther from the device may appear weaker relative to the spatial region closer to the device. For example, assume that the second spatial region is farther from the device than the first spatial region. Correspondingly, the fluorescent light emitted by the second spatial region may appear weaker.

[0019] Additionally, intensity variations from spatial regions of a target at the same distance from the camera can also occur due to curvature of the target, resulting in different reflections, scattering, or autofluorescence. For example, assume that a target, such as a wound containing pathogens, is spread across a spatial region of the wound. Further assume that a first spatial region of the wound and a second spatial region of the wound contain the same pathogens and the same pathogen densities. Furthermore, assume that the first spatial region is flat and the second spatial region is curved. Because the pathogens and pathogen densities are the same, the camera should capture the same intensity of fluorescence, reflection, and / or scattering. However, due to the curvature, the fluorescence, reflection, and / or scattering intensities corresponding to the second spatial region may differ from those of the first spatial region.

[0020] Therefore, variations in distance and curvature over the spatial region of the target relative to the device may need to be compensated for in the light reflected by the target. In one example, the three-dimensional image capture sensor may be a structured light-based sensor, a time-of-flight sensor, a stereo sensor, or a combination thereof.

[0021] The interfacing module may be coupled to the imaging module. The interfacing module may include a processor configured to analyze an image corresponding to the target. Specifically, the processor may analyze a first image of the first plurality of images using an analytical model. The first plurality of images may be a fluorescence-based image comprising fluorescence emitted from the target. Furthermore, the processor may analyze the three-dimensional image of the target by compensating for variations in distance across a spatial region of the target relative to the three-dimensional image capture sensor in reflected light and by compensating for variations in curvature across the spatial region of the target. In this regard, the processor may determine variations in the intensity of emitted light across a spatial region of the target by compensating for variations in distance across the spatial region of the target relative to the three-dimensional image capture sensor and by compensating for variations in curvature across the spatial region of the target. The analytical model may be, for example, an artificial neural network model (ANN), a machine learning (ML) model, or a combination thereof. In one example, the ANN model may include a deep learning model, such as a transformer model, a convolutional neural network (CNN), a generative adversarial network (GAN), an autoencoder-decoder network, a transformer model, or a combination thereof. The ML model may be, for example, a support vector machine (SVM) model or a random forest model, or a combination thereof.

[0022] The processor may use an analytical model to detect the presence of problematic cellular entities in the target based on analysis of the first image and the three-dimensional image. The analytical model is trained to detect the presence of problematic cellular entities in the target. Specifically, the analytical model may be trained using multiple reference fluorescence-based images to detect the presence of problematic cellular entities in the target. The analytical model may be trained to distinguish between fluorescence in the fluorescence-based image arising from problematic cellular entities and fluorescence in the fluorescence-based image arising from areas other than the problematic cellular entities.

[0023] In one example, in addition to being trained with multiple reference fluorescence-based images, the analytical model may be trained using multiple reference three-dimensional images of the target to detect the presence of problematic cellular entities in the target. In this regard, the analytical model may be trained to distinguish between fluorescence in the fluorescence-based images arising from problematic cellular entities and fluorescence in the fluorescence-based images arising from regions other than the problematic cellular entities. Additionally, the analytical model may be trained by compensating for differences in distance across a spatial region of the target relative to the three-dimensional image capture sensor and by compensating for variations in curvature across the spatial region of the target by determining variations in the intensity of emitted light across the spatial region of the target. The variations in the intensity of emitted light across the spatial region of the target may be determined based on variations in distance across the spatial region of the target relative to the three-dimensional image capture sensor, variations in curvature across the spatial region of the target, and the intensity measured across the spatial region of the target.

[0024] Additionally, the processor may use the analytical model to create a composite image of the first image and the three-dimensional image of the target. The interface may display results corresponding to the detection of the problematic cellular entity and the composite image of the first image and the three-dimensional image of the target.

[0025] In one example, the device may include a system-on-module (SOM). The SOM may include an imaging module, an interfacing module, and a plurality of light source drivers. The plurality of light source drivers may include metal-oxide-semiconductor field-effect transistors (MOSFETs), bipolar junction transistors (BJTs), phase-locked loops (PLLs), or any combination thereof configured to control each light source of the first plurality of light sources.

[0026] In one example, one or more light sources of the first plurality of light sources are pulsed light-emitting diodes (LEDs). The processor can be configured to operate one or more of the light source drivers of the plurality of light source drivers to control the pulsed LEDs to emit pulses of excitation radiation. The one or more light source drivers can be operated by the processor to control the pulsed LEDs at a pulse width and frequency to enable high-speed imaging and reduce ambient light interference in the light emitted by the target. In one example, the pulse width can range from several hundred nanoseconds to 0.005 ms, and the frequency of the pulsed LED can be from 100 Hz to several tens of megahertz. Thus, the present subject matter enables faster capture of the first plurality of images and three-dimensional images and reduces ambient light interference (background interference).

[0027] In one example, the processor may be configured to operate the imaging sensor and the three-dimensional image capture sensor to capture and process the first plurality of images and the three-dimensional image at greater than 30 frames per second. In this regard, the processor may include a central processing unit (CPU) and a graphics processing unit (GPU). Specifically, the CPU and GPU may be part of the SOM. In other words, the CPU and GPU may be provided on-board. The CPU may operate the imaging sensor and the three-dimensional image capture sensor to capture the first plurality of images and the three-dimensional image. Furthermore, the GPU may process images captured by the first plurality of images and the three-dimensional image. Providing a GPU and a CPU, specifically, providing an on-board GPU and CPU, may enable faster processing and capture of the first plurality of images and the three-dimensional image at greater than 30 frames per second.

[0028] In some examples, in addition to using fluorescence-based and three-dimensional images to detect the presence of problematic cellular entities, the device may detect the presence of problematic cellular entities based on oxygen saturation. In this regard, the device may include a second plurality of light sources for illuminating the target without causing markers in the target to fluoresce. Each of the second plurality of light sources may be configured to emit light at wavelengths in the near-infrared (NIR) or visible ranges.

[0029] The imaging sensor may be configured to capture a second plurality of images formed based on light reflected by the target in response to illumination of the target by at least one light source of the second plurality of light sources. The processor may use the analytical model to analyze a second image obtained from the second plurality of images to identify oxygen saturation levels in multiple regions within the target. The processor may use the analytical model to analyze the three-dimensional image of the target to determine variations in the intensity of reflected light across the spatial region of the target by compensating for variations in distance across the spatial region of the target from the three-dimensional image capture sensor and variations in curvature across the spatial region of the target. The processor may use the analytical model to detect the presence of a problematic cellular entity in the target based on analysis of a first image of the first plurality of images, a second image obtained from the second plurality of images, and the three-dimensional image. In such a case, the processor may create a composite image of the first image, the second image, and the three-dimensional image of the target. The interface may display results corresponding to the detection of the problematic cellular entity and the composite image of the first image, the second image, and the three-dimensional image of the target.

[0030] In some examples, the analytical model may utilize a white-light image in addition to the first image and the three-dimensional image of the target to detect problematic cellular entities. In this regard, in some examples, at least one or more of the second plurality of light sources may be configured to emit light with wavelengths in the visible range. The imaging sensor may be configured to capture a third plurality of images formed based on light reflected by the target in response to illumination of the target by at least one or more of the second plurality of light sources. The third plurality of images are white-light images. The processor may be configured to analyze the third image obtained from the third plurality of images using the analytical model. The processor may analyze the three-dimensional image of the target using the analytical model to determine variations in the intensity of reflected light across the spatial region of the target by compensating for variations in distance across the spatial region of the target from the three-dimensional image capture sensor and variations in curvature across the spatial region of the target. The processor may be configured to detect the presence of problematic cellular entities in the target based on analysis of the first image, the third image, and the three-dimensional image using the analytical model. The processor may be configured to create a composite image of the target using the first image, the third image, and the three-dimensional image. The interface may be configured to display results corresponding to the detection of the problematic cellular entity and a composite image of the first image, the third image, and the three-dimensional image of the target. As will be appreciated, in such cases, the analytical model may be trained using a plurality of reference fluorescence-based images, a plurality of reference white-light images, and a plurality of reference three-dimensional images to detect the presence of the problematic cellular entity in the target.

[0031] The processor may be configured to operate the first plurality of light sources to emit light at the target and to operate the second plurality of light sources to emit light at the target. Additionally, the processor may be configured to operate the imaging sensor to capture light emitted by the target in response to illumination of the target by at least one or more light sources of the first plurality of light sources and to capture light emitted by the target in response to illumination of the target by at least one or more light sources of the second plurality of light sources.

[0032] In an example, to reduce and / or eliminate the effect of background light in the captured image, the processor may be configured to control the first plurality of light sources and the second plurality of light sources to illuminate at a frequency other than the frequency of the ambient light source.

[0033] In some examples, in addition to detecting problematic cellular entities, the device may classify the detected problematic cellular entities. Thus, in some examples, when the target is a wound area, the processor may be configured to extract spatial and spectral features of the wound area from the first image and the three-dimensional image using the analytical model. Furthermore, the processor may identify the location of the wound area based on the extraction of the spatial and spectral features using the analytical model. The processor may determine a contour of the wound area based on the extraction of the spatial and spectral features using the analytical model. In some examples, based on the determination of the contour of the wound area, the processor may be configured to determine a length of the wound area, a width of the wound, a perimeter of the wound, an area of ​​the wound, a depth of the wound, or a combination thereof. Furthermore, the processor may detect pathogens in the wound area based on the extraction of the spatial and spectral features using the analytical model. The processor may classify the pathogens by at least one of a pathogen family, a family, a species, or a strain using the analytical model.

[0034] In some examples, in addition to detecting problematic cellular entities, the device may determine other parameters corresponding to the detected problematic cellular entities. For example, when the target is a wound area, the processor may be configured to determine the extent of infection of the wound area, the scab area, the spatial distribution of pathogens in the wound area, the rate of healing of the wound area, or a combination thereof, in response to detecting the presence of the problematic cellular entity. When the target is tissue, the processor may be configured to detect the presence of the problematic cellular entity as cancerous tissue, necrotic tissue, or a combination thereof in a tissue specimen. When the target is sanitary equipment, medical equipment, sanitary equipment, laboratory equipment, biochemical assay chips, microfluidic chips, and / or bodily fluids, the processor may be configured to determine the presence of the problematic cellular entity as a pathogen and classify the pathogen in the target.

[0035] Furthermore, in addition to detecting problematic cellular entities, the processor may be configured to detect time-dependent changes in fluorescence emanating from the target. In other words, the processor may be configured to detect changes in fluorescence between an initial imaging of the target and a later imaging of the target. For example, the processor may be configured to detect changes in fluorescence between pre-debridement of the wound and post-debridement of the wound. This detection may enable accurate removal of dead / unhealthy tissue from the wound. In another example, the processor may be configured to detect changes in fluorescence between images of the wound taken on the first day and images of the wound taken on a later day. This detection may help ensure wound healing and allow medical professionals to prescribe medication accordingly.

[0036] In some examples, the device may be portable and may comprise a smartphone. The smartphone may include a processor and an imaging sensor. In some examples, the device may include other components. In some examples, the device may include a first set of excitation filters. Each of the first set of excitation filters may be configured to filter and pass excitation radiation emitted by a light source of the first plurality of light sources in a predetermined range of wavelengths to illuminate the target. In addition, one or more excitation filters may also be configured to filter and pass excitation radiation emitted by a light source of the second plurality of light sources in a predetermined range of wavelengths.

[0037] The device may include a thermal sensor for thermal imaging of the target. In this regard, the processor may be configured to use the analytical model to detect the problematic cellular entity based on a first image of the first plurality of images, a second image obtained from the second plurality of images, a three-dimensional image, and a thermal image of the target. In such a scenario, the processor may use the analytical model to create a composite image of the first image, the second image, the three-dimensional image, and the thermal image. Further, the interface may display results corresponding to the detection of the problematic cellular entity based on the composite image of the first image, the second image, the three-dimensional image, and the thermal image of the target.

[0038] The device may include a ranging sensor operable to determine the distance of a target from the device to position the device at a predetermined distance from the target. In one example, a three-dimensional image capture sensor may be used as the ranging sensor. For example, the three-dimensional image capture sensor may be operable to determine the distance of a target from the device to position the device at a predetermined distance from the target.

[0039] The device may include multiple polarizers. For example, the device may include a first polarizer positioned between the first plurality of light sources and the target to pass excitation radiation of the first plurality of light sources of a first polarization. The device may include a second polarizer positioned between the target and the imaging sensor to pass light emitted by the target of a second polarization. In some examples, the first polarization and the second polarization may be the same. In other examples, the first polarization and the second polarization may be different. In some examples, the first polarization and the second polarization may be the same. For example, in some examples, the first polarization and the second polarization may be left-handed circular polarization (LHCP). In other examples, the first polarization and the second polarization may be right-handed circular polarization (RHCP). In other examples, the first polarization and the second polarization may be different. For example, the first polarization may be one of LHCP or RHCP, and the second polarization may be the other of LHCP or RHCP. The multiple polarizers may be combined with the first set of excitation filters.

[0040] The device may include a housing for housing the components. Specifically, the device may include a first housing, a second housing, and a bridge. The first housing may house the imaging module, and the second housing may house the interfacing module. The bridge may connect the imaging module and the interfacing module. The bridge may include an electrical interface for enabling electrical communication between the processor of the interfacing module and the imaging module. The electrical interface may include a camera serial interface (CSI), a serial management bus such as an I2C interface, a system packet interface (SPI), a universal asynchronous receiver-transmitter (UART), a general-purpose input / output (GPIO) interface, a universal serial bus (USB) interface, a pulse-width modulation (PWM) interface, a display serial interface (DSI), a high-definition multimedia interface (HDMI), or a combination thereof.

[0041] The device may include a portable power module operable to power components of the device, such as the imaging module and the interfacing module. The third housing may house the portable power module.

[0042] In some examples, the device may transmit the results to a remote system, such as a cloud server. For example, the processor may be configured to transmit the results and the first image, a composite image of the three-dimensional image, to a remote system, such as a cloud server. The remote system may be in electronic communication with the device. The device enables transmission of the results and the composite image to a cloud server so that a non-medical professional or medical professional can transmit an image or series of images to a remote medical professional for further advice before treatment using the device of the present disclosure.

[0043] The interface may be configured to receive input from a user corresponding to operation of the device by using an application programming interface (API). For example, using the API, a user may be able to select one or more of the first plurality of light sources and one or more of the second plurality of light sources to illuminate a target. In addition, a user may be able to select the frequency of light emission of the first plurality of light sources and the second plurality of light sources.

[0044] The interface may be configured to respond to input and, using the API, transmit results corresponding to the detection and classification of pathogens in the target once the pathogens are detected and classified. In this regard, the interface may allow a user to store and analyze results corresponding to the detection and classification of pathogens in the target. Additionally, the interface may allow a user to select a composite image to be obtained, may enable transmission of the results to a remote system or server, and may allow a user to select various views of the composite image.

[0045] In one example, the processor may be configured to detect time-dependent changes in fluorescence emanating from the target. In other words, the processor may be configured to detect changes in fluorescence between an initial imaging of the target and a later imaging of the target. For example, the processor may be configured to detect changes in fluorescence between pre-debridement of the wound and post-debridement of the wound. This detection may enable accurate removal of dead / unhealthy tissue from the wound. In another example, the processor may be configured to detect changes in fluorescence between images of the wound taken on a first day and images of the wound taken on a later day. This detection may help ensure wound healing and allow medical personnel to prescribe medication according to the detection.

[0046] In the above examples, devices have been described that do not include optical bandpass filters for filtering light emitted by the target. However, in some examples, one or more optical bandpass filters, such as absorption filters, may be used.

[0047] Thus, in one example, a device for investigating a target may include an imaging module, an interfacing module, and an interface. The imaging module may include a first plurality of light sources, a first plurality of optical bandpass filters, an image sensor, and a three-dimensional image capture sensor. Each of the first plurality of light sources may be configured to emit excitation radiation in a predetermined range of wavelengths that causes one or more markers in the target to fluoresce. In one example, each of the first plurality of light sources may be an LED. In another example, one or more of the light sources of the first plurality of light sources may be a pulsed light-emitting diode (LED) for emitting pulses of excitation radiation to enable faster imaging and reduce ambient light interference in the light emitted by the target. The first plurality of light sources may, for example, be homogeneous or heterogeneous light sources.

[0048] Each of the first plurality of optical bandpass filters may be configured to filter and pass light emitted by the target in response to illumination of the target by at least one or more light sources of the first plurality of light sources of a predetermined wavelength. The imaging sensor may capture filtered light filtered by an optical bandpass filter of the first plurality of optical bandpass filters and capture a first plurality of images formed based on the filtered light. In an example, the device may include one or more lenses integral with the imaging sensor to focus light onto the imaging sensor to capture the images.

[0049] The three-dimensional image capture sensor may illuminate the target, receive light reflected by the target in response to the illumination of the target by the three-dimensional image capture sensor, and generate a three-dimensional image of the target based on the reflected light. In one example, the three-dimensional image capture sensor may be a structured light sensor, a time-of-flight sensor, a stereo sensor, or a combination thereof.

[0050] The interfacing module may be coupled to the imaging module. The interfacing module may include a processor. The processor may be configured to analyze a first image of the first plurality of images using an analytical model. The first image may be a fluorescence-based image comprising fluorescence emitted from the target. The processor may analyze a three-dimensional image of the target and determine variations in the intensity of emitted light across a spatial region of the target by compensating for variations in distance across the spatial region of the target from the three-dimensional image capture sensor and by compensating for variations in curvature across the spatial region of the target using the analytical model. The processor may detect the presence of a problematic cellular entity in the target based on analysis of the first image and the three-dimensional image using the analytical model. The analytical model may be trained to detect the presence of a problematic cellular entity in the target.

[0051] The analytical model is trained to detect the presence of problematic cellular entities in the target. Specifically, the analytical model can be trained using a plurality of reference fluorescence-based images to detect the presence of problematic cellular entities in the target. The analytical model can be trained to distinguish between fluorescence in the fluorescence-based images arising from problematic cellular entities and fluorescence in the fluorescence-based images arising from regions other than the problematic cellular entities.

[0052] In one example, in addition to being trained with multiple reference fluorescence-based images, the analytical model may be trained using multiple reference three-dimensional images of the target to detect the presence of problematic cellular entities in the target. In this regard, the analytical model may be trained to distinguish between fluorescence in the fluorescence-based images arising from problematic cellular entities and fluorescence in the fluorescence-based images arising from regions other than the problematic cellular entities. Additionally, the analytical model may be trained by compensating for differences in distance across a spatial region of the target relative to the three-dimensional image capture sensor and by compensating for differences in curvature across the spatial region of the target by determining variations in the intensity of emitted light across the spatial region of the target. The variations in the intensity of emitted light across the spatial region of the target may be determined based on variations in distance across the spatial region of the target relative to the three-dimensional image capture sensor, variations in curvature across the spatial region of the target, and the intensity measured across the spatial region of the target.

[0053] The processor may create a composite image of the first image and the three-dimensional image of the target. The interface may display results corresponding to the detection of the problematic cellular entity and the composite image of the first image and the three-dimensional image of the target.

[0054] In one example, the device can include a first set of excitation filters, each of which can be configured to filter and pass excitation radiation emitted by a light source of the first plurality of light sources in a predetermined range of wavelengths to illuminate the target. In addition, one or more excitation filters can also be configured to filter and pass excitation radiation emitted by a light source of the second plurality of light sources in a predetermined range of wavelengths.

[0055] In an example, the device may include a system-on-module (SOM). The SOM may include an imaging module, an interfacing module, and a plurality of light source drivers. The plurality of light source drivers may be configured to control each light source of the first plurality of light sources.

[0056] The processor may also be configured to operate the first plurality of light sources to emit light at the target and to operate the imaging sensor to capture light emitted by the target in response to illumination of the target by at least one or more light sources of the first plurality of light sources.

[0057] In one example, the device may include an absorption filter wheel rotatably disposed within the imaging module. The absorption filter wheel may be operably coupled to a servo motor. The absorption filter wheel may include a first plurality of optical bandpass filters. As will be appreciated, based on a required optical bandpass filter from the first plurality of optical bandpass filters, the servo motor may be operated to position the required optical bandpass filter between the target and the imaging sensor. In this regard, the processor may be configured to operate the servo motor to rotate the absorption filter wheel to position an optical bandpass filter from the first plurality of optical bandpass filters positioned between the target and the imaging sensor.

[0058] In the above examples, image capture and processing of the device have been described with reference to a single device. In some examples, image capture and processing may be performed by different components. Thus, in one example, a system for investigating a target may include a processor. The processor may analyze a first image of a first plurality of images using an analytical model. The first plurality of images may be a fluorescence-based image comprising fluorescence emitted from the target. The processor may be configured to analyze a three-dimensional image of the target to determine variations in emitted light intensity across a spatial region of the target by compensating for variations in distance across the spatial region of the target from the three-dimensional image capture sensor and by compensating for variations in curvature across the spatial region of the target using the analytical model.

[0059] The processor may detect the presence of problematic cellular entities in the target based on analysis of the first image and the three-dimensional image using the analytical model. The analytical model may be trained to detect the presence of problematic cellular entities in the target. The processor may create a composite image of the first image and the three-dimensional image of the target. The processor may transmit to the device a result corresponding to the detection of the problematic cellular entities and the composite image of the first image and the three-dimensional image of the target.

[0060] The system may include a device. The device may include an imaging module including a first plurality of light sources, an imaging sensor, and a three-dimensional image capture sensor. Each of the first plurality of light sources may emit excitation radiation in a predetermined range of wavelengths that causes one or more markers in the target to fluoresce. The imaging sensor may be configured to directly receive light emitted by the target in response to illumination of the target by one or more light sources of the first plurality of light sources, without an optical bandpass filter disposed between the imaging sensor and the target, and to capture a first plurality of images formed based on the emitted light. Here, the light is said to be directly received by the imaging sensor because the emitted light is not filtered by an optical bandpass filter before image capture.

[0061] The three-dimensional image capture sensor may illuminate a target and receive light reflected by the target in response to the illumination of the target by the three-dimensional image capture sensor, and may generate a three-dimensional image of the target based on the reflected light. In some examples, the three-dimensional image capture sensor may be a structured light sensor, a time-of-flight sensor, a stereo sensor, or a combination thereof. The target may be a wound area, food, laboratory equipment, medical equipment, bodily fluids, sanitary equipment, sanitary equipment, biochemical assay chips, microfluidic chips, or a combination thereof. The analytical model may be trained using multiple reference fluorescence-based images and multiple reference three-dimensional images to detect the presence of problematic cellular entities in the target. The analytical model may be trained to distinguish between fluorescence in the fluorescence-based image arising from problematic cellular entities and fluorescence in the fluorescence-based image arising from areas other than the problematic cellular entities.

[0062] In some examples, when the target is a wound, the present subject matter enables detection of biofilm in the wound. In this regard, a device for investigating a wound may include an imaging module, an interfacing module, and an interface. The imaging module may include a first plurality of light sources, a second plurality of light sources, an imaging sensor, and a three-dimensional image capture sensor. Each of the first plurality of light sources may emit excitation radiation in a predetermined range of wavelengths that causes one or more markers in the wound to fluoresce. The first plurality of light sources may be, for example, homogeneous or heterogeneous light sources.

[0063] Each of the second plurality of light sources may emit excitation radiation in a predetermined range of wavelengths without causing markers in the wound to fluoresce. The imaging sensor may directly receive light emitted by the wound in response to illumination of the wound by at least one or more of the first plurality of light sources and directly receive light reflected by at least one or more of the second plurality of light sources without an optical bandpass filter disposed between the imaging sensor and the wound. The imaging sensor may capture a first plurality of images formed based on light emitted by the wound and may capture a second plurality of images formed based on light reflected by the wound. Here, the light is said to be directly received by the imaging sensor because the emitted light and reflected light are not filtered by an optical bandpass filter prior to image capture.

[0064] The three-dimensional image capture sensor may illuminate the wound, may receive light reflected by the wound in response to the illumination of the wound by the three-dimensional image capture sensor, and may generate a three-dimensional image of the wound based on the reflected light. In one example, the three-dimensional image capture sensor may be a structured light sensor, a time-of-flight sensor, a stereo sensor, or a combination thereof.

[0065] The interfacing module may be coupled to the imaging module. The interfacing module may include a processor. The processor may be configured to analyze a first image of the first plurality of images using the analytical model, the first image being a fluorescence-based image comprising fluorescence emitted from the wound. The processor may analyze a second image obtained from the second plurality of images using the analytical model. Further, the processor may analyze the three-dimensional image of the wound using the analytical model to determine variations in emitted and reflected light intensity across the spatial region of the wound by compensating for variations in distance across the spatial region of the wound from the three-dimensional image capture sensor and by compensating for variations in curvature across the spatial region of the wound.

[0066] In this regard, the processor may detect the presence of a biofilm in the wound based on analysis of the first image, the second image, and the three-dimensional image using an analytical model. The analytical model may be trained to detect the presence of a biofilm in the wound. The analytical model may create a composite image of the first image, the second image, and the three-dimensional image of the wound. The interface may display results corresponding to the detection of a biofilm in the wound and the composite image of the first image, the second image, and the three-dimensional image of the wound.

[0067] In an example, the device may include a first set of excitation filters, each of which may be configured to filter and pass excitation radiation emitted by a light source of the first plurality of light sources in a predetermined range of wavelengths to illuminate the target.

[0068] The present subject matter enables faster image capture and processing for detecting problematic cellular entities. Because the present subject matter provides an on-board processor and imaging module, the present subject matter enables faster image capture and processing. Specifically, by using a combination of a CPU and a GPU, the present subject matter enables image capture and processing at a rate of more than 30 images per second. To detect the presence of problematic cellular entities in a target, thereby increasing detection accuracy, an analytical model is trained on several reference fluorescence-based images and several reference three-dimensional images. The present subject matter ensures that the light emitted by the light source has a different frequency from the ambient light source. Thus, the present subject matter enables the interference of ambient light with the light emitted by the target to be eliminated. Furthermore, the present subject matter allows the pulsed LED to operate at a shorter pulse width, such as from several hundred nanoseconds to 0.005 ms, and at a faster frequency, such as from 100 Hz to several tens of megahertz. Thus, the present subject matter enables faster capture of the first plurality of images and the three-dimensional image and reduces ambient light interference (background interference). Thus, the present subject matter removes background information and increases detection accuracy.

[0069] Furthermore, in certain examples, the analytical model can ignore background light and excitation light in the fluorescence-based image and can pick up weak fluorescence information in the fluorescence-based image. Thus, in certain examples, the present subject matter also eliminates the use of absorption filters to filter background light and excitation light, as well as the use of filter wheels. Thus, the subject matter devices are simple and cost-effective.

[0070] In the present subject matter, the variation in distance between the imaging sensor and multiple regions across the spatial region of the target, and the variation in curvature of multiple regions across the spatial region of the target, are determined by a three-dimensional image capture sensor. Thus, the present subject matter enhances the accuracy of detecting problematic cellular entities, particularly for targets such as wounds. Because the device enables transmission of results and composite images to a cloud server, a non-medical professional or medical professional can transmit an image or series of images to a remote medical professional for further advice before treatment using the disclosed device.

[0071] Thus, the present subject matter provides for rapid, optionally filter-free, non-invasive, automatic, in-situ pathogen detection and classification using "optical computational biopsy" techniques, in which multispectral imaging is used in conjunction with computational models, such as machine learning models, artificial neural network (ANN) models, and deep learning models, for non-invasive biopsy to detect and classify problematic cellular entities.

[0072] The present subject matter can be used to detect the presence of problematic cellular entities in diabetic foot ulcers, surgical site infections, burns, skin, and inside the body, such as the esophagus, stomach, and colon. The subject devices can be used in the fields of dermatology, cosmetic surgery, plastic surgery, infection control, photodynamic therapy monitoring, and antimicrobial susceptibility testing.

[0073] Furthermore, the device can be used to detect time-dependent changes in fluorescence to understand pathogen colonization and necrotic tissue. In other words, the processor can be configured to detect changes in fluorescence between an initial imaging of a target and a subsequent imaging of the target. For example, the processor can be configured to detect changes in fluorescence between before debridement of a wound and after debridement of the wound. This detection can enable accurate removal of dead / unhealthy tissue from the wound. In another example, the processor can be configured to detect changes in fluorescence between images of the wound taken on the first day and images of the wound taken on a later day. This detection can help ensure wound healing and allow medical professionals to prescribe medication according to the detection.

[0074] The device may be incorporated into routine clinical diagnostics and may be used in telemedicine and telenursing. Furthermore, the majority of clinically important pathogens can be detected and classified within minutes. Furthermore, data acquisition and analysis can be performed automatically. Therefore, the device can be easily operated without the need for skilled technicians. This feature aids in quickly determining treatment protocols. The device may also be used for pathogen detection and classification in resource-poor settings. The subject device may also be used in endoscopic investigations. For example, the subject imaging module may be incorporated into the imaging unit of an endoscopic device.

[0075] The subject devices can be used to quantify various pathogens present in a specimen. The devices can also be used to monitor wound healing and wound closure. The devices can also be used to study antimicrobial susceptibility by exposing targets to various antibiotics and observing and analyzing them. For example, the devices can be used to study bacteria growing in the presence of nutrients and antibiotics, and corresponding biomarker signatures can be recorded. This information can be used to inform antibiotic prescriptions based on the antimicrobial susceptibility of specific bacteria. It should be understood that the antimicrobial susceptibility of other pathogens, such as fungi, can also be studied. Furthermore, antibiotic doses and concentrations can also be determined based on dilution factors to determine the dose of antibiotic and antifungal agents to be administered.

[0076] The device can be configured to study the biomolecular composition and dynamic behavior of various pathogens based on their fluorescent signatures. The device can also be used in cosmetics. For example, the device can be used to detect the presence of acne-causing Propionibacterium. The device can also be used during tissue transplantation to ensure that the tissue is pathogen-free. The device can be used for forensic detection, for example, to detect pathogens in bodily fluids such as saliva, blood, and mucus. The device can be configured to study the effectiveness of disinfection on various hospital surfaces, such as beds, walls, hands, gloves, bandages, clothing, catheters, endoscopes, hospital instruments, and sanitary equipment.

[0077] The device can also be used to detect the presence of pathogens on hands and surfaces, for example, in hospitals and other places where pathogens should not be present. The device can be used to detect pathogen contamination in food products, such as food, fruits, and vegetables.

[0078] FIG. 1 shows a block diagram of a device 100 for investigating a target 101 according to an implementation of the present subject matter. The device 100 for investigating a target may include an imaging module 102, an interfacing module 104, and an interface 108. The target 101 may be suspected of having a problematic cellular entity, such as a pathogen or cancerous tissue. In some examples, the target 101 may consist of one or more cells, such as a wound or tissue specimen of a body part. In other examples, the target 101 may be an item that should be free of pathogens, such as food, labware, or sanitary equipment. In some other examples, the target 101 may be pus, blood, urine, saliva, sweat, semen, mucus, plasma, water, etc., that may be suspected of having a pathogen.

[0079] The imaging module 102 may include a first plurality of light sources 130, an imaging sensor 122, and a three-dimensional image capture sensor 120. Each of the first plurality of light sources 130 emits excitation radiation in a predetermined range of wavelengths. Specifically, the wavelength of the emitted excitation radiation may be a single wavelength or a band of wavelengths that, when illuminated, causes one or more markers in the target to fluoresce. In certain examples, the wavelength band of light used to elicit fluorescence from the target 101 may include 200 nm to 300 nm, 300 nm to 400 nm, 400 nm to 500 nm, or 500 nm to 600 nm. In certain examples, the wavelengths of light used to elicit fluorescence from the target 101 may include 280 nm, 310 nm, 330 nm, 365 nm, 395 nm, 405 nm, 415 nm, 430 nm, 480 nm, and 520 nm. In some examples, the wavelength band of light may include 600 nm to 700 nm, 700 nm to 800 nm, or 800 nm to 1000 nm. In certain examples, the wavelengths of light used to elicit fluorescence from the target 101 may also include 430 nm, 630 nm, 660 nm, 680 nm, 735 nm, 830 nm, 880 nm, 940 nm, and 970 nm.

[0080] The first plurality of light sources 130 can be, for example, homogeneous or heterogeneous light sources. In some examples, the use of heterogeneous light sources can reduce or eliminate background light in the light emitted by the target.

[0081] The one or more markers may be part of the problematic cellular entity. Fluorescence emitted by a marker that is part of the problematic cellular entity may be referred to as autofluorescence. In one example, an exogenous marker, such as a synthetic marker, may be sprayed onto the target 101 to cause detection of the problematic cellular entity in the target 101. The exogenous marker may bind to a cellular entity, such as deoxyribonucleic acid (DNA), ribonucleic acid (RNA), protein, or biochemical marker, which may cause the target 101 to fluoresce. Fluorescence emitted by an additional synthetic marker may be referred to as exogenous luminescence.

[0082] In one example, the imaging sensor 122 may be configured to directly receive light emitted by the target 101 in response to illumination of the target 101 by at least one light source of the first plurality of light sources 130, without an optical bandpass filter disposed between the imaging sensor 122 and the target 101, and to capture a first plurality of images formed based on the emitted light. If the target 101 includes a marker that fluoresces, the captured images may include fluorescence and be referred to as fluorescence-based images. Thus, the fluorescence-based images may include fluorescence emitted from the target 101. Here, the light is said to be directly received by the imaging sensor 122 because the emitted light is not filtered by an optical bandpass filter before capturing the image.

[0083] The imaging sensor 122 may be a multispectral camera configured to capture light emitted by the target 101 at multiple wavelengths. Specifically, the multispectral camera may capture light emitted at wavelengths in the visible range, the ultraviolet (UV) range, the near-infrared (NIR) range, or a combination thereof. In another example, the imaging sensor 122 may be a charge-coupled device (CCD) sensor, a CCD digital camera, a complementary metal-oxide semiconductor (CMOS) sensor, a CMOS digital camera, a single-photon avalanche diode (SPAD), a single-photon avalanche diode (SPAD) array, an avalanche photodetector (APD) array, a photomultiplier tube (PMT) array, a near-infrared (NIR) sensor, a red-green-blue (RGB) sensor, a thermal camera, or a combination thereof. In some examples, one or more lenses (not shown in FIG. 1 ) may be integral with the imaging sensor to focus light onto the imaging sensor 122 and capture an image.

[0084] The three-dimensional image capture sensor 120 may illuminate the target 101, may receive light reflected by the target 101 in response to the illumination of the target 101 by the three-dimensional image capture sensor 120, and may generate a three-dimensional image of the target 101 based on the reflected light. To illuminate the target 101, the three-dimensional image capture sensor 120 may include one or more light sources (not shown in FIG. 1 ) integral with the three-dimensional image capture sensor 120. However, in some examples, a separate light source may also be coupled to the three-dimensional image capture sensor 120 to illuminate the target 101 and enable capture of light reflected by the target 101 due to the illumination.

[0085] Additionally, use of the three-dimensional image capture sensor 120 may enable determination of variations in the intensity of light emitted by the target 101 across spatial regions of the target 101. Variations in intensity may need to be considered due to differences in the distances of multiple regions across the spatial regions of the target 101 from the three-dimensional image capture sensor 120. For example, a first spatial region of the target 101 may be at a different distance from the three-dimensional image capture sensor 120 from a second spatial region of the target 101. Thus, the first spatial region and the second spatial region may emit fluorescent light at the same intensity. Because the fluorescent light of the first spatial region and the second spatial region is the same intensity, a spatial region farther from the device 100 may appear weaker relative to a spatial region closer to the device 100. For example, assume that the second spatial region is farther from the device 100 than the first spatial region. Thus, the fluorescent light emitted by the second spatial region may appear weaker. Therefore, variations in distance across a spatial region of the target 101 relative to the device 100 may need to be compensated for in the light emitted by the target 101. In some examples, different spatial regions across the target 101 have different curvatures. Thus, the fluorescence emitted from different spatial regions of the target may be different even at the same distance from the imaging sensor 122. Thus, variations in curvature across a spatial region of the target 101 relative to the device 100 may need to be compensated for in the light emitted by the target 101. In some examples, the three-dimensional image capture sensor may be a structured light sensor, a time-of-flight sensor, a stereo sensor, or a combination thereof.

[0086] The interfacing module 104 may be coupled to the imaging module. The interfacing module 104 may include a processor 140. The processor 140 may be a microprocessor, a microcomputer, a microcontroller, a digital signal processor, a central processing unit, a combination of a central processing unit and a graphics processing unit, a state machine, a logic circuit, and / or any device capable of manipulating signals based on operational instructions. Among other capabilities, the processor 140 may fetch and execute computer-readable instructions contained in a memory (not shown in FIG. 1 ) of the device 100.

[0087] The processor 140 may be configured to analyze an image corresponding to the target 101. Specifically, the processor 140 may analyze a first image of the first plurality of images using an analytical model. The first plurality of images may be a fluorescence-based image comprising fluorescence emitted from the target 101. Furthermore, the processor 140 may analyze the three-dimensional image of the target 101 by compensating for variations in the distance of the light emitted by the target 101 across a spatial region of the target 101 relative to the three-dimensional image capture sensor 120. In this regard, the processor 140 may determine variations in the intensity of the emitted light across a spatial region of the target 101 by compensating for variations in the distance of the target 101 across a spatial region of the target 101 relative to the three-dimensional image capture sensor 120 and by compensating for variations in the distance of the target 101 across a spatial region of the target 101 relative to the three-dimensional image capture sensor 120. The analytical model may be, for example, an artificial neural network model (ANN), a machine learning (ML) model, or a combination thereof. In one example, the ANN model may include a deep learning model, such as a transformer model, a convolutional neural network (CNN), a generative adversarial network (GAN), an autoencoder-decoder network, or a combination thereof. The ML model may be, for example, a support vector machine (SVM) model or a random forest model, or a combination thereof.

[0088] The processor 140 may detect the presence of problematic cellular entities in the target 101 based on analysis of the first image and the three-dimensional image using an analytical model. The analytical model is trained to detect the presence of problematic cellular entities in the target. Specifically, the analytical model may be trained using multiple reference fluorescence-based images to detect the presence of problematic cellular entities in the target. The analytical model may be trained to distinguish between fluorescence in the fluorescence-based image arising from problematic cellular entities and fluorescence in the fluorescence-based image arising from areas other than the problematic cellular entities.

[0089] In one example, in addition to being trained with multiple reference fluorescence-based images, the analytical model may be trained using multiple reference three-dimensional images of the target to detect the presence of problematic cellular entities in the target. In this regard, the analytical model may be trained to distinguish between fluorescence in the fluorescence-based images arising from problematic cellular entities and fluorescence in the fluorescence-based images arising from areas other than the problematic cellular entities. Additionally, the analytical model may be trained to compensate for differences in distance across the spatial region of the target 101 relative to the three-dimensional image capture sensor 120 by determining the variation in the intensity of emitted light across the spatial region of the target 101. The variation in the intensity of emitted light across the spatial region of the target 101 may be determined based on the variation in distance and curvature across the spatial region of the target relative to the three-dimensional image capture sensor 120 and the intensity measured across the spatial region of the target 101. Training of the analytical model is described with reference to FIGS. 5 and 6.

[0090] Additionally, processor 140 may use the analytical model to create a composite image of the first image and the three-dimensional image of target 101. Interface 108 may display results corresponding to the detection of the problematic cellular entity and the composite image of the first image and the three-dimensional image of target 101. Detecting the presence of the problematic cellular entity is described below with reference to Figures 7 through 11.

[0091] In one example, the device 100 may include a system-on-module (SOM). The SOM may include an imaging module 102, an interfacing module 104, and a plurality of light source drivers 150. The plurality of light source drivers 150 may be trained to control each light source of the first plurality of light sources 130. In other words, the device 100 also utilizes an integrated circuit board that typically includes the SOM. The SOM includes the imaging module 102, the interfacing module 104, and a plurality of light source drivers 150. The plurality of light source drivers 150 may include metal-oxide-semiconductor field-effect transistors (MOSFETs), bipolar junction transistors (BJTs), phase-locked loops (PLLs), or combinations thereof. In one example, the processor 140 may include a central processing unit (CPU) and a graphics processing unit (GPU). The SOM may also include a field-programmable gate array (FPGA) module. Additionally, the SOM may include a battery charging module. A SOM may include an integrated circuit (also known as a "chip") that integrates all or most of the components of a computer or other electronic system. These components almost always include a processor 140, a memory interface, on-chip input / output devices, an input / output interface, and a secondary storage interface, often along with other components such as a modem, including a wireless modem, all on a single substrate or microchip. The SOM 106 may include digital, analog, mixed, and often radio frequency signal processing capabilities (otherwise it would be considered only an applications processor). Alternatively, the device 100 may include a system-on-chip (SOC) instead of a SOM. A SOC may resemble a SOM.

[0092] The SOM 106 may also include a GPU or FPGA, or a combination thereof, which enables faster capture, processing, and thereby image capture by the imaging sensor 122. The on-board FPGA or GPU uniquely enables the imaging sensor 122 to capture images at up to 100 frames per second, more typically greater than 30 frames per second, and most typically greater than 40 frames per second, up to 100 frames per second. This faster image pulsing, capture, and processing reduces background noise and allows for accurate extraction of fluorescence / oxygen saturation information. The device 100, which utilizes this faster pulsing, eliminates the need for a shroud or shield to eliminate or reduce ambient light. The device 100 may be used indoors as well as outdoors and still obtain an accurate scan of a wound or other target imaging surface. Thus, the device 100 is not shielded from ambient light.

[0093] In one example, one or more light sources of the first plurality of light sources 130 are pulsed light-emitting diodes (LEDs). The processor 140 may be configured to activate one or more of the light source drivers of the plurality of light source drivers to control the pulsed LEDs to emit pulses of excitation radiation. The one or more light source drivers may be activated by the processor 140 to control the pulsed LEDs with shorter pulse widths and faster frequencies to enable faster imaging and reduce ambient light interference in the light emitted by the target 101. In one example, the pulse width may be in the range of 0.005 ms to several hundred ns. In one example, the frequency of the pulsed LEDs may be from 100 Hz to several tens of MHz. Thus, the present subject matter enables faster capture of the first plurality of images and three-dimensional images and reduces ambient light interference (background interference).

[0094] In one example, processor 140 may be configured to operate imaging sensor 122 and 3D image capture sensor 120 to capture and process the first plurality of images and the three-dimensional image at greater than 30 frames per second. In this regard, processor 140 may include a central processing unit (CPU) and a graphics processing unit (GPU). Specifically, the CPU and GPU may be part of the SOM. In other words, the CPU and GPU may be provided on-board. The CPU may operate imaging sensor 122 and 3D image capture sensor 120 to capture the first plurality of images and the three-dimensional image. Furthermore, the GPU may process images captured by the first plurality of images and the three-dimensional image. Providing a GPU and a CPU, specifically, providing an on-board GPU and CPU, may enable faster processing and capture of the first plurality of images and the three-dimensional image at greater than 30 frames per second.

[0095] In some examples, in addition to using fluorescence-based images and three-dimensional images to detect the presence of problematic cellular entities, device 100 may detect the presence of problematic cellular entities based on oxygen saturation. In this regard, device 100 may include a second plurality of light sources 156 for illuminating target 101 without causing markers therein to fluoresce. Each of second plurality of light sources 156 may be configured to emit light at wavelengths in the near-infrared (NIR) or visible ranges.

[0096] Imaging sensor 122 may be configured to capture a second plurality of images formed based on light reflected by target 101 in response to illumination of target 101 by at least one or more light sources of second plurality of light sources 156. Processor 140 may analyze a second image obtained from the second plurality of images to identify oxygen saturation in multiple regions of target 101 using an analytical model. Processor 140 may analyze the three-dimensional image of target 101 using the analytical model to determine variations in the intensity of reflected light across the spatial region of target 101 by compensating for variations in distance across the spatial region of target 101 from three-dimensional image capture sensor 120 and variations in curvature across the spatial region of target 101.

[0097] Processor 140 may detect the presence of problematic cellular entities in target 101 based on analysis of a first image of the first plurality of images, a second image obtained from the second plurality of images, and the three-dimensional image using the analytical model. In such a case, processor 140 may create a composite image of the first image, the second image, and the three-dimensional image of target 101. Interface 108 may display results corresponding to the detection of the problematic cellular entities and the composite image of the first image, the second image, and the three-dimensional image of target 101.

[0098] In one example, the analytical model may utilize a white-light image in addition to the first image and the three-dimensional image of the target 101 to detect problematic cellular entities. In this regard, in one example, at least one or more of the second plurality of light sources 156 may be configured to emit light with wavelengths in the visible range. The imaging sensor 122 may be configured to capture a third plurality of images formed based on light reflected by the target 101 in response to illumination of the target 101 by at least one or more of the second plurality of light sources 156. The third plurality of images are white-light images. The processor 140 may be configured to analyze the third image obtained from the third plurality of images using the analytical model.

[0099] The processor 140 may use the analytical model to analyze the three-dimensional image of the target 101 and determine variations in the intensity of reflected light across the spatial region of the target 101 by compensating for variations in distance across the spatial region of the target 101 from the three-dimensional image capture sensor and variations in curvature across the spatial region of the target 101. The processor 140 may be configured to use the analytical model to detect the presence of problematic cellular entities in the target based on analysis of the first image, the third image, and the three-dimensional image of the target 101. The processor 140 may be configured to create a composite image of the target 101 using the first image, the third image, and the three-dimensional image. The interface 108 may be configured to display results corresponding to the detection of problematic cellular entities and the composite image of the first image, the third image, and the three-dimensional image of the target 101. As will be appreciated, in such cases, the analytical model may be trained using multiple reference fluorescence-based images, multiple reference white-light images, and multiple reference three-dimensional images to detect the presence of problematic cellular entities in the target 101.

[0100] The processor 140 may be configured to operate the first plurality of light sources 130 to emit light at the target 101 and to operate the second plurality of light sources 156 to emit light at the target 101. Additionally, the processor 140 may be configured to operate the imaging sensor 122 to capture light emitted by the target 101 in response to illumination of the target 101 by at least one or more light sources of the first plurality of light sources 130 and to capture light emitted by the target 101 in response to illumination of the target 101 by at least one or more light sources of the second plurality of light sources 156.

[0101] In one example, to reduce and / or eliminate the effect of background light in the captured images, the processor 140 can be configured to control the first and second light sources 130, 156 to illuminate at a frequency other than that of the ambient light source. Typically, ambient lighting in a room pulsates at a frequency, for example, approximately 50 Hz. The analytical model can compare different images captured by the imaging sensor 122 to remove background noise because the frequency of the ambient light source and the frequency of the first and second light sources 130 are different. Minimizing background lighting improves image quality, which allows the imaging sensor 122 to more accurately detect fluorescence and oxygen saturation levels, leading to better analysis by the analytical model.

[0102] The processor 140 may activate the first plurality of light sources 130 when light emitted by the target 101 should be captured. To this end, the processor 140 may activate the imaging sensor 122 when activating the first plurality of light sources 130 to emit light. Typically, when the first plurality of light sources 130 are pulsed LEDs, the first plurality of light sources 130 pulse at a known rate, and the imaging sensor 122 captures the first plurality of images at a rate that is a multiple of the pulse rate of the first plurality of light sources 130, such that the first plurality of light sources 130 are always "on" when the visible light camera is taking images. Preferably, the first plurality of images are captured by the imaging sensor 122 simultaneously with the emission of light by the first plurality of light sources 130. The frame rate is typically a multiple of the rate at which light is pulsed in the first plurality of light sources 130. The ability to pulse light at a faster rate reduces background noise and allows for time-dependent fluorescence. A faster pulse rate also reduces blurring and fluctuations in the captured image. Additionally, because the frequency of the pulsed light is known, device 100 can examine only the fluctuations of the first plurality of light sources 130 because that frequency is known. Because the background is constant, it can be filtered out. Device 100 has a hardware high-speed switch that uses elements such as high-speed MOSFETs, high-speed BJTs, phase-locked loops (PLLs), or a combination thereof to rapidly turn on and off the first plurality of light sources 130.

[0103] In one example, in addition to detecting problematic cellular entities, device 100 may classify the detected problematic cellular entities. Thus, in one example, when target 101 is a wound area, processor 140 may be configured to extract spatial and spectral features of the wound area from the first image and the three-dimensional image using the analytical model. Further, processor 140 may identify the location of the wound area based on the extraction of spatial and spectral features using the analytical model. Processor 140 may determine the contour of the wound area based on the extraction of spatial and spectral features using the analytical model. In one example, based on the determination of the contour of the wound area, processor 140 may be configured to determine the length of the wound area, the width of the wound, the perimeter of the wound, the area of ​​the wound, the depth of the wound, or a combination thereof. Further, processor 140 may detect pathogens in the spatial domain based on the extraction of spatial and spectral features using the analytical model. The processor 140 may classify the pathogen by at least one of a family, genus, species, or strain of the pathogen by using the analytical model.

[0104] In some examples, in addition to detecting problematic cellular entities, device 100 may determine other parameters corresponding to the detected problematic cellular entities. For example, when target 101 is a wound area, processor 140 may be configured to determine the degree of infection in the wound area, the spatial distribution of pathogens in the wound area, the rate of healing of the wound area, or a combination thereof, in response to detecting the presence of the problematic cellular entity. When target 101 is tissue, processor 140 may be configured to detect the presence of the problematic cellular entity in the tissue specimen as cancerous tissue, necrotic tissue, or a combination thereof. When target 101 is a sanitary device, sanitary implement, medical implement, biochemical assay chip, microfluidic chip, or bodily fluid, processor 140 may be configured to determine the problematic cellular entity as a pathogen and classify the pathogen in target 101.

[0105] Furthermore, in addition to detecting problematic cellular entities, the processor 140 may be configured to detect time-dependent changes in fluorescence emanating from the target 101. In other words, the processor 140 may be configured to detect changes in fluorescence between an initial imaging of the target 101 and a subsequent imaging of the target 101. For example, the processor 140 may be configured to detect changes in fluorescence before and after wound debridement. This detection may enable accurate removal of dead / unhealthy tissue from the wound. In another example, the processor 140 may be configured to detect changes in fluorescence between images of the wound taken on a first day and images of the wound taken on a later day. This detection may help ensure wound healing and allow medical personnel to prescribe medication accordingly.

[0106] In one example, device 100 may be portable and may comprise a smartphone. The smartphone may include a processor 140 and an imaging sensor 122. In addition, the smartphone may include a three-dimensional image capture sensor 120. In one example, the smartphone may be integrated with various light sources 130, 156, polarizers, filters 142, etc.

[0107] In an example, device 100 may include other components. Device 100 may include a first set of excitation filters 142. Each of the first set of excitation filters 142 may be configured to filter and pass excitation radiation emitted by a light source of the first plurality of light sources 130 in a predetermined range of wavelengths to illuminate target 101. In addition, one or more excitation filters may also be configured to filter and pass excitation radiation emitted by a light source of the second plurality of light sources 156 in a predetermined range of wavelengths.

[0108] Device 100 may include a thermal sensor (not shown in FIG. 1 ) for thermal imaging of target 101. The thermal sensor may be part of imaging module 102, for example. In this regard, processor 140 may be configured to use the analytical model to detect problematic cellular entities based on a first image of the first plurality of images of target 101, a second image obtained from the second plurality of images, a three-dimensional image, and a thermal image. In such a scenario, processor 140 may use the analytical model to create a composite image of the first image, the second image, the three-dimensional image, and the thermal image. Furthermore, interface 108 may display results corresponding to the detection of the problematic cellular entities and a composite image of the first image, the second image, the three-dimensional image of target 101, and the thermal image of target 101.

[0109] The device 100 may include a ranging sensor 132 operable to determine a distance of the target 101 from the device 100 in order to position the device 100 at a predetermined distance from the target 101. The ranging sensor 132 may be, for example, part of the imaging module 102. In one example, the three-dimensional image capture sensor 120 may be used as the ranging sensor 132. In this regard, the three-dimensional image capture sensor 132 may be operable to determine a distance of the target 101 from the device 100 in order to position the device 100 at a predetermined distance from the target 101.

[0110] In some examples, device 100 may not have a polarizer. In other examples, device 100 may include multiple polarizers (not shown in FIG. 1 ). For example, device 100 may include a first polarizer positioned between first plurality of light sources 130 and target 101 to pass excitation radiation of first plurality of light sources 130 of a first polarization. Device 100 may include a second polarizer positioned between target 101 and imaging sensor 122 to pass light emitted by target 101 of a second polarization. In some examples, the first polarizer may be oriented 90 degrees from the second polarizer. Providing a polarizer in front of imaging sensor 122 can prevent excitation light from entering imaging sensor 122.

[0111] In some examples, the first polarization and the second polarization may be the same. For example, in some examples, the first polarization and the second polarization may be left-handed circular polarization (LHCP). In another example, the first polarization and the second polarization may be right-handed circular polarization (RHCP). In another example, the first polarization and the second polarization may be different. For example, the first polarization may be one of LHCP or RHCP, and the second polarization may be the other of LHCP or RHCP. Multiple polarizers may be combined with the first set of excitation filters 142.

[0112] Additionally, an optional light diffuser may also be placed in front of the first plurality of light sources 130, and / or the second plurality of light sources 156, and / or the excitation filter 142 to spread the light more widely onto the target 101.

[0113] As described with reference to FIGS. 2a to 2c, device 100 may include a housing for housing components. Specifically, device 100 may include a first housing, a second housing, and a bridge. The first housing may house an imaging module, and the second housing may house an interfacing module 104. The bridge may connect the imaging module and the interfacing module 104. The bridge may include an electrical interface for enabling electrical communication between the processor 140 of the interfacing module 104 and the imaging module. The electrical interface may include a camera serial interface 108 (CSI), a serial management bus such as an I2C interface, a system packet interface (SPI), a universal asynchronous receiver-transmitter (UART), a general-purpose input / output (GPIO) interface, a universal serial bus (USB) interface, a pulse-width modulation (PWM) interface, a display serial interface (DSI), a high-definition multimedia interface (HDMI), or a combination thereof.

[0114] To enable powering of components of device 100, such as imaging module and interfacing module 104, device 100 may include a portable power module 136. Portable power module 136 may include a third housing 37 for accommodating portable power module 136.

[0115] In an example, device 100 may transmit the results to a remote system, such as cloud server 160. For example, processor 140 may be configured to transmit the results and the first image, the composite image of the three-dimensional image, to a remote system, such as a cloud server. The remote system may be in electronic communication with device 100. Device 100 enables transmission of the results and the composite image to a cloud server so that a non-medical professional or medical professional can transmit an image or series of images to a remote medical professional for further advice prior to treatment using device 100 of the present disclosure.

[0116] In some examples, the interface 108 may be an interactive display, such as an LED display, a liquid crystal display, a thin film transistor display, an organic light emitting diode (OLED) display, a capacitive touch screen, a resistive touch screen, a toggle switch, buttons, etc. The digital display and buttons allow a user to easily use and operate the device 100. The interface 108 may also be a standalone device 100, such as a laptop, a desktop, a tablet, a smartphone, a smart accessory such as a smartwatch, or a combination thereof.

[0117] Interface 108 may be configured to receive input from a user corresponding to operation of device 100 by using an application programming interface 108 (API). For example, using the API, a user may be able to select one or more of first plurality of light sources 130, one or more of second plurality of light sources 156 for illuminating target 101. In addition, a user may be able to select the frequency of light emission of first plurality of light sources 130 and second plurality of light sources 156.

[0118] In response to input, interface 108 may be configured to transmit results corresponding to the detection and classification of pathogens in target 101 upon pathogen detection and classification using the API. In this regard, interface 108 may allow a user to store and analyze results corresponding to the detection and classification of pathogens in target 101. Additionally, interface 108 may allow a user to select a composite image to be acquired, may enable transmission of the results to a remote system or server, and may also allow a user to select various views of the composite image.

[0119] Figure 2a shows a front perspective view of a device 100 for investigating a target 101 according to an implementation of the present subject matter. Figure 2b shows a rear perspective view of a device 100 for investigating a target 101 according to an implementation of the present subject matter. Figure 2c shows an exploded view of an apparatus 100 for investigating a target 101 according to an implementation of the present subject matter. For simplicity, Figures 2a-2c will be described in relation to each other.

[0120] Illustrated herein is a SOM 210. In one example, the imaging module 102 assembly may be held together by a back frame 134 and a connecting bracket 236. The back frame 134 and the connecting bracket 236 together form a first housing for enclosing the imaging module 102.

[0121] In one example, the imaging module 102 and the interfacing module 104 may be joined by a bridge (not shown in FIGS. 2a-2c). The bridge may hold the imaging module 102 and the interfacing module 104 stable together and enable electrical communication between elements of the imaging module 102 and the interfacing module 104 through a camera serial interface (CSI), a serial management bus such as an I2C interface, a system packet interface (SPI), a universal asynchronous receiver-transmitter (UART), a general-purpose input / output (GPIO) interface, a universal serial bus (USB) interface, a pulse-width modulation (PWM) interface, a display serial interface (DSI), a high-definition multimedia interface (HDMI), or any other electrical connection known in the art.

[0122] The processor 140 and the interface 108 may each be fixedly mechanically attached to a bracket (not shown in FIGS. 2a-2c) while being directly electrically coupled to each other through a bus bar from the processor to the user interface 108, a serial cord, or any other cord known in the art. A bracket may then be sandwiched between the back frame 112 and the front frame (not shown in FIGS. 2a-2c) to hold the interfacing module 104 together. The back frame 112 and the front frame may together form a second housing for housing the interfacing module 104. The processor 140 may include, for example, random access memory (RAM), flash memory, a WiFi and / or cellular data antenna, a BLUETOOTH® antenna, and other interfaces that allow various peripheral devices to be electrically connected. BLUETOOTH® is a short-range wireless technology standard used to exchange data between fixed and mobile devices over short distances and build personal area networks (PANs). BLUETOOTH® utilizes UHF radio waves in the ISM band from 2.402 GHz to 2.48 GHz.

[0123] The processor 140 may connect to a cloud server 160 (shown in FIG. 1 ) via a WiFi or cellular data antenna for uploading and downloading data in the direction of the imaging sensor 122, as well as for further analysis of captured images and three-dimensional point clouds. In one example, all hardware drivers for the device 100 may be on-board, along with one or more components of the imaging module 102, such as the first plurality of light sources 130, the first set of excitation filters 142, the light source driver 150, the second plurality of light sources 156, etc. The processor 140 allows for extremely fast switching / commands for activating the light sources 130, 156, which provides many advantages to the device 100.

[0124] The portable power module 136 may include a rechargeable battery electrically coupled to a power printed circuit board (PCB) 44. The power PCB 44 and the rechargeable battery 46 may be sandwiched between a front cover and a back cover (not shown in FIGS. 2a-2c). The front cover and the back cover may form a third housing. The third housing may thus accommodate the portable power module 136. A power cord (not shown in FIGS. 2a-2c) may be electrically attached to the power PCB 44 and exit through the cover. Additionally, in some examples, the device 100 may operate with a cloth (not shown in FIGS. 2a-2c) to reduce ambient light.

[0125] Additionally, although not shown herein, device 100 may be coupled to a portable stand such as stand 410 shown in Figures 4a-4b.

[0126] In the above examples, devices have been described that do not include optical bandpass filters to filter the light emitted by the target. However, in some examples, one or more optical bandpass filters, such as absorption filters, may be used.

[0127] 3 shows a block diagram of a device 300 for investigating a target 101 according to an implementation of the present subject matter. Device 300 may correspond to device 100 and may include the same components as device 100. Accordingly, components of device 100 included in device 300 are described using the same reference numerals. Additionally, device 300 may include an absorption filter, as described below. As will be appreciated, in addition to the functions described herein, device 300 may perform some or all of the functions performed by device 100 using appropriate components mentioned in connection with device 100.

[0128] The device 300 for investigating the target 101 may include an imaging module 102, an interfacing module 104, and an interface 108. The imaging module 102 may include a first plurality of light sources 130, a first plurality of optical bandpass filters 126, an imaging sensor 122, and a three-dimensional image capture sensor 120.

[0129] The first plurality of light sources 130 may emit light to illuminate the target 101. The target 101 may be suspected of harboring a pathogen or a problematic cellular entity, such as cancerous tissue. In one example, the target 101 may consist of one or more cells, such as a wound or tissue specimen from a body part. In another example, the target 101 may be an item that should be pathogen-free, such as food, labware, or sanitary equipment. The emitted light may be within a wavelength range that causes a marker in the target 101 to fluoresce when illuminated. Specifically, the emitted light may be light of a single wavelength that causes a marker in the target 101 to fluoresce when illuminated. The light from the first plurality of light sources 130 may also be emitted at a particular frequency. This frequency may be tuned to an integer multiple of the frequency of the imaging sensor 122 so that the imaging sensor 122 captures an image when the first plurality of light sources 130 are illuminated. This frequency may also be tuned to be different from the frequency of ambient light sources in the room. This ensures that the first plurality of light sources 130 is illuminating the target when the ambient light source is off, allowing the background image to be more easily filtered and removed from analysis.

[0130] The marker is usually part of the cellular entity of interest. Fluorescence emitted by a marker that is part of the cellular entity of interest may be referred to as autofluorescence. In some instances, an exogenous marker, such as a synthetic marker, may be sprayed onto a target to trigger detection of the cellular entity of interest in the target. The exogenous marker may bind to a cellular entity, such as deoxyribonucleic acid (DNA), ribonucleic acid (RNA), protein, or biochemical marker, causing the target to fluoresce. Fluorescence emitted by the added synthetic marker may also be referred to as exogenous fluorescence.

[0131] Each of the first plurality of light sources 130 may be configured to emit excitation radiation in a predetermined range of wavelengths that causes one or more markers in the target 101 to fluoresce. In one example, each of the first plurality of light sources 130 may be a light emitting diode (LED). In another example, one or more light sources of the first plurality of light sources 130 may be a pulsed LED for emitting pulses of excitation radiation to enable faster imaging and to reduce ambient light interference in the light emitted by the target 101. The first plurality of light sources 130 may be, for example, homogeneous or heterogeneous light sources.

[0132] In certain examples, the wavelength range of light used to induce fluorescence from the target 101 may include 300 nm to 300 nm, 300 nm to 400 nm, 400 nm to 500 nm, or 500 nm to 600 nm. In certain examples, the wavelengths of light used to induce fluorescence from the target 101 may include 280 nm, 310 nm, 330 nm, 365 nm, 395 nm, 405 nm, 415 nm, 430 nm, 480 nm, and 520 nm. In certain examples, the wavelength range of light may include 600 nm to 700 nm, 700 nm to 800 nm, or 800 nm to 3000 nm. In certain examples, the wavelengths of light used may also include 430 nm, 630 nm, 660 nm, 680 nm, 735 nm, 830 nm, 880 nm, 940 nm, and 970 nm to capture reflection and / or scattering.

[0133] Each of the first plurality of optical bandpass filters 126 may be configured to filter and pass light emitted by the target 101 in response to illumination of the target 101 by at least one or more light sources of the first plurality of light sources 130 at a predetermined wavelength. In some examples, the optical bandpass filters 126 may have center wavelengths corresponding to peak emitted fluorescence from various autofluorescent biomarkers or exogenous fluorophores. The optical bandpass filters 126 may be low-pass filters, high-pass filters, single-bandpass filters, or multiple-bandpass filters. The imaging sensor 122 may capture filtered light filtered by an optical bandpass filter of the first plurality of optical bandpass filters 126 and capture a first plurality of images formed based on the filtered light. The three-dimensional image capture sensor 120 may illuminate the target 101 and receive light reflected by the target 101 in response to illumination of the target 101 by the three-dimensional image capture sensor 120 of the first plurality of light sources 130 and generate a three-dimensional image of the target 101 based on the reflected light. To illuminate the target 101, the three-dimensional image capture sensor 120 may include one or more light sources (not shown in FIG. 3 ) integral with the three-dimensional image capture sensor 120. However, in some examples, a separate light source may also be coupled with the three-dimensional image capture sensor 120 to illuminate the target 101 and to enable capture of light reflected by the target 101 due to the illumination. In certain examples, the three-dimensional image capture sensor 120 may be a structured lighting sensor, a time-of-flight sensor, a stereo sensor, or a combination thereof.

[0134] The interfacing module 104 may include a processor 140. The processor 140 may be implemented as a microprocessor, a microcomputer, a microcontroller, a digital signal processor, a central processing unit, a combination of a central processing unit and a graphics processing unit, a state machine, a logic circuit, and / or any device capable of manipulating signals based on operational instructions. Among other capabilities, the processor 140 may fetch and execute computer-readable instructions contained in a memory (not shown in FIG. 3) of the device 300.

[0135] The processor 140 may be configured to analyze a first image of the first plurality of images using the analytical model. The first image may be a fluorescence-based image comprising fluorescence emanating from the target 101. The processor 140 may analyze the three-dimensional image of the target 101 to determine variations in the intensity of emitted light across the spatial region of the target 101 by compensating for variations in distance across the spatial region of the target 101 from the three-dimensional image capture sensor 120 and by compensating for variations in curvature across the spatial region of the target 101 relative to the three-dimensional image capture sensor 120 using the analytical model. The processor 140 may detect the presence of problematic cellular entities in the target 101 based on analysis of the first image and the three-dimensional image using the analytical model. The analytical model may be trained to detect the presence of problematic cellular entities in the target.

[0136] The analytical model is trained to detect the presence of problematic cellular entities in the target. Specifically, the analytical model can be trained using multiple reference fluorescence-based images to detect the presence of problematic cellular entities in the target. The analytical model can be trained to distinguish between fluorescence in the fluorescence-based images arising from problematic cellular entities and fluorescence in the fluorescence-based images arising from areas other than the problematic cellular entities.

[0137] In one example, in addition to being trained with multiple reference fluorescence-based images, the analytical model may be trained using multiple reference three-dimensional images of the target to detect the presence of problematic cellular entities in the target. In this regard, the analytical model may be trained to distinguish between fluorescence in the fluorescence-based images arising from problematic cellular entities and fluorescence in the fluorescence-based images arising from areas other than the problematic cellular entities. Additionally, the analytical model may be trained to compensate for differences in distance and curvature across the spatial region of the target 101 relative to the three-dimensional image capture sensor 120 by determining the variation in the intensity of emitted light across the spatial region of the target 101. The variation in the intensity of emitted light across the spatial region of the target 101 may be determined based on the variation in distance across the spatial region of the target relative to the three-dimensional image capture sensor 120, the variation in curvature across the spatial region of the target 101, and the intensity measured across the spatial region of the target 101. Training of the analytical model is described in connection with FIGS. 5-6.

[0138] The processor 140 may create a composite image of the first image and the three-dimensional image of the target 101. The interface may display results corresponding to the detection of the problematic cellular entity and the composite image of the first image and the three-dimensional image of the target 101. Detecting the presence of the problematic cellular entity is described below in connection with Figures 7 through 11.

[0139] In one example, the device 300 may include a system-on-module (SOM). The SOM may include the imaging module 102, the interfacing module 104, and the plurality of light source drivers 150. The SOM may also include an FPGA module. The plurality of light source drivers 150 may be configured to control certain light sources of the first plurality of light sources 130. In other words, the device 300 also utilizes an integrated circuit board that typically includes the SOM 106. The SOM 106 includes the imaging module 102, the interfacing module 104, and the plurality of light source drivers 150. In one example, the processor 140 may include a central processing unit (CPU) and a graphics processing unit (GPU). In addition, the SOM 106 may include a battery charging module. The SOM 106 may include an integrated circuit (also known as a "chip") that integrates all or most of the components of a computer or other electronic system. These components almost always include a processor 140, a memory interface, on-chip input / output devices 300, an input / output interface, and a secondary storage interface, often along with other components such as a modem, including a wireless modem, all on a single substrate or microchip. The SOM 106 may include digital, analog, mixed-signal, and often radio frequency signal processing capabilities (otherwise it would be considered only an applications processor). Alternatively, the device 300 may include a system-on-chip (SOC) instead of a SOM. A SOC may resemble a SOM.

[0140] The processor 140 may also be configured to operate the first plurality of light sources 130 to emit light onto the target 101 and to operate the imaging sensor 122 to capture light emitted by the target 101 in response to illumination of the target 101 by at least one or more light sources of the first plurality of light sources 130.

[0141] In one example, the device 300 may include an absorption filter wheel 124 rotatably disposed within the imaging module. The absorption filter wheel 124 is operably coupled to a servo motor 128. The absorption filter wheel 124 may include a first plurality of optical bandpass filters 126. As will be appreciated, the servo motor 128 may be actuated to position a required optical bandpass filter between the target 101 and the imaging sensor 122 based on a required optical bandpass filter from the first plurality of optical bandpass filters 126. In this regard, the processor 140 may be configured to actuate the servo motor 128 to rotate the absorption filter wheel to position an optical bandpass filter from the first plurality of optical bandpass filters 126 positioned between the target 101 and the imaging sensor 122. In one example, the device 300 may include a ranging sensor 132 operable to determine a distance of the target 101 from the device 300 in order to position the device 300 at a predetermined position from the target 101.

[0142] The device 300 may include a first set of excitation filters 142. Each of the first set of excitation filters 142 may be configured to filter and pass excitation radiation emitted by a light source of the first plurality of light sources 130 in a predetermined range of wavelengths to illuminate the target 101. In addition, one or more excitation filters may also be configured to filter and pass excitation radiation emitted by a light source of the second plurality of light sources 156 in a predetermined range of wavelengths.

[0143] In one example, the three-dimensional image capture sensor 120 may be used as a distance measurement sensor 132. In this regard, the three-dimensional image capture sensor 132 may be operable to determine the distance of the target 101 from the device 100 in order to position the device 100 at a predetermined distance from the target 101. Additionally, in one example, the device 300 may operate in conjunction with a fabric to reduce ambient light.

[0144] Although only a few examples of problematic cellular entity detection are described in connection with device 300, it will be understood that device 300 may further include additional components, such as a thermal sensor, similar to device 100. Additionally, device 300 may also perform similar functions as device 300 and may perform the problematic cellular entity detection, classification, etc., described in connection with device 100.

[0145] In one example, the analytical model may utilize a white-light image in addition to the first image and the three-dimensional image of the target 101 to detect problematic cellular entities. In this regard, in one example, at least one or more of the second plurality of light sources 156 may be configured to emit light at wavelengths in the visible range. The imaging sensor 122 may be configured to capture a plurality of white-light images formed based on light reflected by the target 101 in response to illumination of the target 101 by at least one or more of the second plurality of light sources 156. The processor 140 may be configured to analyze a white-light image obtained from the plurality of white-light images using the analytical model. The processor 140 may analyze the three-dimensional image of the target 101 using the analytical model to determine variations in the intensity of reflected light across the spatial region of the target 101 by compensating for variations in distance across the spatial region of the target 101 from the three-dimensional image capture sensor 120 and variations in curvature across the spatial region of the target 101.

[0146] The processor may be configured to use the analytical model to detect the presence of problematic cellular entities in the target based on analysis of the first image, the white-light image, and the three-dimensional image of the target 101. The processor 140 may be configured to create a composite image of the target 101 using the first image, the white-light image, and the three-dimensional image. The interface 108 may be configured to display results corresponding to the detection of the problematic cellular entities and the composite image of the first image, the white-light image, and the three-dimensional image of the target 101. As will be appreciated, in such cases, the analytical model may be trained using multiple reference fluorescence-based images, multiple reference white-light images, and multiple reference three-dimensional images to detect the presence of problematic cellular entities in the target 101.

[0147] The device 300 may include multiple polarizers. For example, the device 300 may include a first polarizer positioned between the first plurality of light sources 130 and the target 101 to pass excitation radiation of the first plurality of light sources 130 in a first polarization. The device 300 may include a second polarizer positioned between the target 101 and the imaging sensor 122 to pass light emitted by the target in a second polarization. In some examples, the first polarization and the second polarization may be the same. In other examples, the first polarization and the second polarization may be different. In some examples, the first polarization and the second polarization may be the same. For example, in some examples, the first polarization and the second polarization may be left-handed circular polarization (LHCP). In other examples, the first polarization and the second polarization may be right-handed circular polarization (RHCP). In other examples, the first polarization and the second polarization may be different. For example, the first polarization may be one of LHCP or RHCP, and the second polarization may be the other of LHCP or RHCP. Multiple polarizers may be combined with the first set of excitation filters 142, or the first plurality of absorption filters 126, or both.

[0148] FIG. 4a shows a perspective view of a device 300 for investigating a target 101 according to an exemplary implementation of the present subject matter. Illustrated herein is a device 300 for investigating a target (not shown in FIG. 4a) and a portable stand 410. The portable stand 410 may easily position or move the device 300 to a desired location, particularly a desired local location within a given room or medical area, such as a hospital floor or triage space. The portable stand 410 may include a base 411, an extendable arm 413 that engages with the base 411 at a first end 421, and an articulated arm 416 that engages with a second end 423. The base 411 may include one or more legs 418 having wheels 420, at least one of which may include a brake (not shown). The wheels 420 may be, for example, casters or other similar wheels having a mount, a stem, and at least one wheel. There may be additional parts depending on the type of caster and its intended surface of use. The casters may be plate casters attached using a mounting plate and may have a single wheel or dual wheels. Instead of wheels 420, floor guides may be attached to the floor-contacting side of one or more legs 418. Base 411 may allow a user to roughly position device 300 to be closer to an area required for lighting and imaging, or for easy movement from room to room.

[0149] The telescoping arm 413 may include a lower arm 422 and an upper arm 426 joined by a collar 424 that allows one of the arms 422, 426 to slide within the other and be held in place by a locking screw 424a that has a handle so that it can be easily tightened and loosened by hand. The telescoping arm 413 may allow the device 300 to be positioned vertically (up / down) at an appropriate height for use. As will be appreciated, the collar 424 may also be integral with one of the arms 422, 426.

[0150] The articulated arm 416 may be configured to enable precise placement of the device 300 necessary to illuminate and image a specific location on a target when using the device 300 to take an image, without the user having to take any images or hold the device 300 in their hand. Typically, the articulated arm 416 may be installed / positioned after the base 411 and the telescoping arm 413 are roughly installed near the target. The articulated arm 416 may then be used to precisely place the device 300 in place to target the location of an area of ​​a patient or subject to be scanned or otherwise evaluated using the device 300. The articulated arm 416 may include a lower arm 430, an upper arm 432, and a connection bracket 434 that engages and connects the device 300 to the portable stand 410. The connecting bracket 434 may include one or more handles 434a in the shape of a generally D-shaped opening that allows a user to move the device 300 into position without applying force to, or more typically, even touching, the device 300 itself during the positioning process. Although a generally D-shaped opening forms the handle 434a, the handle 434a may be created by any shaped opening in the connecting bracket 434, such as a rectangular opening or a circular opening. It is also conceivable that the handle 434a or handles may be created by a separate knob or knobs that engage the connecting bracket 434 using one or more fasteners, such as a screw or bolt system. The lower arm 430 and the upper arm 432 may include hinges that allow rotational and vertical movement relative to one another so that the device 300 can be placed precisely where the physician using the device desires.

[0151] A bracket 428 may be inserted between the telescoping arm 413 and the articulating arm 416. The bracket 428 may be used to place or hang any medical and / or dental instruments required by the physician. The bracket 428 may also engage or include the portable power module 136, which is either secured to the bracket 428 or placed within a housing 437 formed on one or more sides of the bracket and sized to receive the portable power module 136. The housing 437 is usually the open-topped portion of the bracket 428, or it is a separate component that is open-topped and sized to receive the portable power module 136 during use, typically having a front, back, two sides, and a bottom. The locations where these enclosure sides meet other surfaces may be partially or completely enclosed. The housing 437 may be made of any material, such as cloth or leather, but is more typically a sterilizable, medical-grade metal. In fact, the entire portable stand 410 is typically made of materials that can be sanitized periodically as needed using UV or other means.

[0152] The portable power module 136 may include a rechargeable battery 46 as illustrated in FIG. 4d that may be plugged into a standard electrical outlet while not in use, or possibly during use, if the portable power module 136 has already been depleted before the next use or may be depleted during the next use. Using a rechargeable battery system allows the entire assembly to be easily moved from one location to another without constantly plugging a power cord into the assembly, which can be cumbersome for a physician to move around. Multiple portable power modules 136 may be utilized throughout the systems of the present disclosure, including the device 300. For example, one or more “kits” of portable power modules 136 or simply rechargeable batteries may be provided in a single travel case or enclosure before being deployed for use.

[0153] 4b shows a perspective view of the device 300 for investigating a target according to an implementation of the present subject matter. The interface between the articulated arm 416 and the device 300 is shown in more detail. The distal end of the upper arm 432 may include a two-axis hinge 38, which allows the device 300 to be rotated both vertically and laterally (i.e., left / right and up / down). The bracket 434 may be attached via an x-bracket 440, which allows the device 300 to be stably coupled to the articulated arm 416 by eliminating as much "slop" as possible between the bracket 434 and the articulated arm 416. As will be appreciated, coupling the bracket 434 to the articulated arm 416 may be achieved by any coupling known in the art.

[0154] The portable power module 136 may have a power cord 442 that extends from batteries 446 (shown in FIG. 4d) on the arms 430, 432 through hinge 437 to the device 300. As will be appreciated, the power cord 442 may be contained within a cord holder attached to any or all of the arms 430, 432 and hinge 437, or may hang freely between the portable power module 136 and the device 300, or any combination thereof. Typically, the power cord 442 is protected within the lower and upper arms 430, 432 so that it cannot be damaged by engagement with or become dislodged from the lower and upper arms 430, 432.

[0155] FIG. 4c shows an exploded view of a device 300 for investigating a target according to an implementation of the present subject matter. The device 300 may include an imaging module 102 and an interfacing module 104 joined by a bridge 438. The processor 140 and the interface 108 may each be fixedly mechanically attached to a bracket 409 and directly electrically coupled to each other through a bus bar, serial cord, or any other cord known in the art from the processor to the interface 108. The bracket 409 may then be sandwiched between a back frame 412 and a front frame 414, holding the interfacing module 104 together. The back frame 412 and the front frame 414 may together form a second housing for accommodating the interfacing module 104. The processor 140 may include, for example, random access memory (RAM), flash memory, a Wi-Fi and / or cellular data antenna, a Bluetooth antenna, and other interfaces that allow various peripherals to be electrically attached. BLUETOOTH® is a short-range wireless technology standard used to exchange data between fixed and mobile devices over short distances and to create personal area networks (PANs). BLUETOOTH® utilizes UHF radio waves in the ISM band from 2.402 GHz to 2.48 GHz.

[0156] The processor 140 may connect to a cloud server via WiFi or a cellular data antenna for uploading and downloading data in the direction of the imaging sensor 122, as well as for further analysis of the captured images and 3D point clouds. Typically, all hardware drivers for the device 300 are on-board, along with one or more components of the imaging module 102, such as the light sources 130, 156, the imaging sensor 122, the filters 142, 126, the filter wheel 124, the servo motor 128, the light source driver 150, the 3D image capture sensor 120, the imaging sensor 122, and the ranging sensor 132. The processor 140 enables extremely fast switching / commands for operating the light sources 130, 156, which provides many advantages to the device 300.

[0157] The ability of device 300 to obtain oxygen saturation and fluorescence data at different distances is a significant advantage of the present system, making it more likely that less trained or non-medical personnel will use the device while obtaining accurate data. The imaging module 102 assembly may be held together by a back frame 435 and a connecting bracket 436. Together, the back frame 435 and the connecting bracket 436 form a first housing for enclosing the imaging module 102. The imaging module 102 and interfacing module 104 may be constructed to be substantially or completely waterproof. While the device 300 shown utilizes multiple visible and other light filters, the device 300 may lack visible or other light filters.

[0158] 4d shows an exploded view of the portable power module 136 of the device 300 for investigating a target according to an implementation of the present subject matter. The portable power module 136 may include a rechargeable battery 446 electrically coupled to a power printed circuit board (PCB) 444. The power PCB 444 and the rechargeable battery 446 may be sandwiched between a front cover 448 and a back cover 450. The front cover 448 and the back cover 450 may form a third housing for enclosing the portable power module 136. A power cord 442 is electrically attached to the power PCB 444 and may exit through the covers 448, 450. The power cord 442 typically extends along the arm of the portable stand 410 to the device 300, although it is contemplated that the power cord 442 may instead be untethered from the portable stand 410.

[0159] 4e shows an exploded view of the interfacing module 104 of the device 300 for investigating a target in accordance with an implementation of the present subject matter. Here, a power cord 442 is shown passing through the bottom of the interfacing module 104 and entering the interfacing module 104 upward. The imaging module 102 and the interfacing module 104 may be mechanically and electrically connected through a bridge 438, which holds the imaging module 102 and the interfacing module 104 stable together and enables electrical communication between elements of the imaging module 102 and the interfacing module 104 through a camera serial interface (CSI), a serial management bus such as an I2C interface, a system packet interface (SPI), a universal asynchronous receiver-transmitter (UART), a general-purpose input / output (GPIO) interface, a universal serial bus (USB) interface, a pulse-width modulation (PWM) interface, a display serial interface (DSI), a high-definition multimedia interface (HDMI), or any other electrical connection known in the art.

[0160] FIG. 5 illustrates a method 500 for training an analytical model to detect problematic cellular entities in a target according to an implementation of the present subject matter. The order in which the method blocks are described is not intended to be limiting, and some of the described method blocks may be combined in any order to implement method 500 or alternative methods. Additionally, some of the individual blocks may be omitted from method 500 without departing from the scope of the subject matter described herein. In this specification, the target is described in relation to a wound, and the problematic cellular entities are described in relation to a pathogen. However, the target may also be a tissue specimen, food, laboratory equipment, sanitary equipment, sanitary implements, biochemical assay chips, microfluidic chips, medical equipment, bodily fluids, or a combination thereof, and the problematic cellular entities may also be cancerous tissue, necrotic tissue, etc.

[0161] In block 502, the reference fluorescence-based image, the reference white-light image, and the reference three-dimensional image are tagged with various reference labels, such as the type of target (i.e., skin or wound), the type of wound area (i.e., scab, bone, etc.), the species of infected pathogen, gram type, etc. In some examples, various spatial features, such as texture, porosity, various spectral features, such as fluorescence hue, of the wound and adjacent area, or a combination thereof, are extracted. In some examples, tagging may be performed only on the white-light image.

[0162] At block 504, the tagged image is preprocessed. For example, the image is converted to grayscale, resized, and enhanced. Enhancing the image may include rotating the image, flipping the image, etc.

[0163] In block 506, various features, such as spatial features, spectral features, or a combination thereof, are extracted from the image. In some examples, spatial features, such as histogram of oriented gradient (HOG) features, entropy features, Local Binary Patterns (LBP), and Scale Invariant Feature Transforms (SIFT), may be extracted from the image. Similarly, in some examples, spectral features may be extracted from white-light images in RGB wavelengths and fluorescence images at various excitation wavelengths. For white-light and fluorescence images, spectral features are extracted using red-green-blue (RGB) values, hue-saturation-value (HSV) values, or any other color map values ​​at each pixel / region. In some examples, machine learning or deep learning models may be used to extract spatial and spectral features.

[0164] In block 508, the extracted spatial and spectral features and tags may be stored in a database (not shown in FIG. 5) in the memory of processor 140. The extracted features are then passed to an analytical model for pathogen detection and spatial mapping, as described below. For example, for some pathogens, such as Pseudomonas Aeruginosa, the pathogen can be detected by using spatial features and excitation wavelengths. For some pathogens, such as Escherichia coli (E-coli), Klebsiella, and Staphylococcus, detection may be performed by extracting a combination of both spatial and spectral features.

[0165] Steps 502-508 are repeated for several reference fluorescence-based images, several white light images, and several three-dimensional images until a desired predetermined target training accuracy is achieved. At block 510, the information in the database can be used to train an analytical model.

[0166] This training enables the analytical model to identify wounds in a given image based on the extracted spatial features, spectral features, or a combination thereof of the image. That is, the analytical model is capable of performing wound segmentation. In one example, after block 510, method 500 may include post-processing steps, such as connected component labeling, hidden Markov models, etc., which may be used to smooth the wound segmentation results and thereby improve the accuracy of the wound segmentation.

[0167] Once the analytical model is trained, it may be tested to verify whether it can correctly identify wounds in an image. Accordingly, at block 512, a region of interest in the test image is selected. In one example, the region of interest may be selected automatically, such as by the analytical model. In another example, the region of interest may be selected manually, such as by a user. Further, at block 514, the test image is preprocessed, and at block 516, spatial features of the test image are extracted. At block 518, the extracted features are provided to the analytical model to perform wound segmentation and problematic cellular entity detection and classification. Subsequently, results of the wound segmentation, problematic cellular entity detection, and classification as performed by the analytical model may be received.

[0168] In some implementations, the analytical model used for wound segmentation may be different from the one used for pathogen detection and analysis. Thus, the wound segmentation output may be provided by the first analytical model to a second analytical model. The second analytical model may then analyze the fluorescence from the wound region as identified by the first analytical model and then detect and classify pathogens in the wound region. Alternatively, in some examples, the second analytical model may also use spatial features, information from the first analytical model about the wound, bone, tissue region, etc., in combination with spectral features for pathogen detection and classification.

[0169] In one example, the analytical model may include an ANN model and an ML model, each performing a different function. For example, an ML model may be trained to perform wound segmentation, while an ANN model may be trained to detect and classify pathogens. In another example, an ANN model may generate spectral images from fluorescence-based images, and an ML model may detect and classify pathogens based on the generated spectral images. In one example, in addition to the fluorescence-based images, an ANN model may additionally generate spectral images from white-light images, and an ML model.

[0170] In some instances, the analytical model may classify pathogens in the wound into Gram-positive (GP) and Gram-negative (GN) pathogens. Additionally, the analytical model may identify the species of pathogens in the wound.

[0171] The analytical model described herein is the same as the analytical model described in connection with Figures 1 to 4e.

[0172] 6 shows an example for training an analytical model to detect problematic cellular entities in a target according to an implementation of the present subject matter. In the example illustrated herein, image 602a shows a white-light image, and image 604a shows an image tagged with a reference label.

[0173] Similarly, autofluorescence images 602b, 602c, and 602d show autofluorescence images of the target at different excitation wavelengths, such as 365 nm, 395 nm, and 415 nm, respectively. Images 604b, 604c, and 604d show autofluorescence images tagged with reference labels. Images 604b, 604c, and 604d correspond to images 602b, 602c, and 602d with reference labels. All images 602a-602d and images 604a-604d are fed to an analytical model 606 for training. The analytical model may provide an output: a composite image 608 of the autofluorescence images superimposed with the predicted bacterial species distribution. In Figure 6, red corresponds to Staphylococcus aureus, and green corresponds to Pseudomonas aeruginosa.

[0174] The analytical model described herein is the same as the analytical model described with reference to FIGS.

[0175] FIG. 7 illustrates a method 700 for detecting problematic cellular entities according to an implementation of the present subject matter. The order in which the method blocks are described is not intended to be limiting, and some of the described method blocks may be combined in any order to implement method 700 or alternative methods. Additionally, some of the individual blocks may be deleted from method 700 without departing from the scope of the subject matter described herein. Herein, target 101 is described in relation to a wound. However, it will be understood that the target may be a tissue specimen, food, laboratory equipment, sanitary equipment, medical equipment, sanitary equipment, biochemical assay chips, microfluidic chips, bodily fluids, or a combination thereof. Method 700 may be performed by device 100 or device 300.

[0176] In block 702, a Red-Green-Blue Depth (RGBD) image is taken using imaging sensor 122. In block 704, three-dimensional image capture sensor 120 performs depth measurements of the image. In block 708, a multispectral image is obtained using imaging sensor 122 using different excitation wavelengths and different emission wavelengths.

[0177] Prior to capturing each of the images at the multispectral wavelengths, in block 706, the transfer function of the imaging sensor 122 is frozen and an auto-exposure model is implemented to maintain appropriate brightness levels, as described in connection with FIG. 10 and method 1000. The transfer function is used to convert raw Red-Blue-Green (RGB) sensor values ​​into a more realistic representation of colors as perceived by the human eye. The transfer function may be, for example, a 3*3 matrix. The transfer function is frozen before imaging so that the color mixture is known and reproducible across imaging sessions. After the multispectral images are captured, a region of interest (ROI) is selected by the physician. The images are then oriented using models such as Kaze descriptors and K-nearest neighbor (KNN) matching of features.

[0178] After the image is oriented, in block 714, the oriented image is sent for federated learning. For example, the oriented image may be securely transferred to a remote medical professional via a cloud-based server system, an email system, or otherwise transmitted electronically. The analytical model may enable continuous improvement of the analyzed image. After medical professionals around the world use a device such as device 100 or device 300 and provide input on the type of information being displayed, future users of the same or other device 100 or device 300, whether near the previous user or far from the previous user / medical professional, benefit from the “learning” provided based on previous human input from the knowledge of medical professionals who previously used the system. This is called federated learning, a machine learning technique that trains models across multiple decentralized edge devices or servers with local data samples without exchanging them.

[0179] The wound may be segmented for spatial and size parameters to be entered into the final report in block 716. Spatial parameters may be, for example, the extent of granulation, crusting, necrotic tissue, maceration, etc. Size parameters may be, for example, the length of the wound area, the width of the wound, the perimeter of the wound, the depth of the wound, the area of ​​the wound, or a combination thereof. In blocks 718-722, the wound is subsequently segmented into subregions of interest, which are then spatially segmented by connected components and passed through a sparse filter. In block 724, the analysis model may classify the output as Gram-positive or Gram-negative, which is displayed on the report page in block 726, as described with reference to FIGS. 1 and 3.

[0180] FIG. 8 illustrates a method 800 for detecting objectionable cellular entities according to an implementation of the present subject matter. The order in which the method blocks are described is not intended to be limiting, and some of the described method blocks may be combined in any order to implement method 800 or alternative methods. Additionally, some of the individual blocks may be deleted from method 800 without departing from the scope of the subject matter described herein. Herein, steps 718-724 of method 700 are described. As will be appreciated, blocks 718a-718c correspond to block 718 in FIG. 7, blocks 720a-720c correspond to block 720 in FIG. 7, blocks 722a-722c correspond to block 722 in FIG. 7, and blocks 724a-724c correspond to block 724 in FIG. 7. Method 800 may be performed by device 100 or device 300.

[0181] In block 802, a 395 nm unfiltered image is selected for ROI portion selection. In blocks 718a, 718b, and 718c, three hue-based filters are used to distinguish different colors of fluorescence emitted from the target. The hue-based filters comprise blue, green, and red filters, respectively. In blocks 720a, 720b, and 720c, the filtered hue generated by the binary mask is passed through a connected component analyzer. The connected component analysis separates disconnected components and labels them. The labels are then individually passed through a sparse filter in blocks 722a, 722b, and 722c. Any region less than approximately or just 1% of the total wound area is rejected for processing, and the wound edge is found using an inference method in blocks 724a, 724b, and 724c.

[0182] 9 illustrates a method 900 for detecting problematic cellular entities according to an implementation of the present subject matter. The order in which the method blocks are described is not intended to be limiting, and some of the described method blocks may be combined in any order to implement method 900 or alternative methods. Additionally, some of the individual blocks may be deleted from method 900 without departing from the scope of the subject matter described herein. Method 900 may be implemented by device 100 or device 300.

[0183] In blocks 902-908, the Red-Blue-Green (RGB)+Depth map image is superpixelized into superpixels, e.g., 8x8 superpixels. Then, in block 910, spatial and spectral features are extracted. In block 912, the spatial features are individually passed through an analytical model that can predict the probability that a given superpixel is part of a wound or skin. In block 914, the image is Gaussian blurred, and that image becomes the starting point in block 916. In block 916, a contour is drawn on the image, and the largest contour is selected as the wound contour. In block 920, the length, width, depth, and area of ​​the wound are derived from the drawn contour, and the output is presented using interface 108 in block 922.

[0184] 10 shows a method 1000 for a self-exposure process according to an implementation of the present subject matter. The order in which the method blocks are described is not included to be construed as limiting, and some of the described method blocks may be combined in any order to implement method 1000 or alternative methods. Additionally, some of the individual blocks may be deleted from method 1000 without departing from the scope of the subject matter described herein. Method 1000 may be implemented by device 100 or device 300.

[0185] The brightness of the image captured by the imaging sensor 122 may need to be optimal. In other words, the image brightness should not be too low or too saturated. If the image brightness is too high, the image captured by the imaging sensor 122 may be saturated and appear white. If the image brightness is too low, the image captured by the imaging sensor 122 may be too low and appear dark. Therefore, an optimal brightness of the image may need to be set. The brightness may depend on the exposure of the imaging sensor 122. In this regard, a self-exposure model is used to control the optimal exposure of the imaging sensor 122 by enabling the optimal brightness of the image to be set. In steps 1002-1010, the self-exposure model is used to set the optimal exposure of the imaging sensor 122 by setting the optimal brightness of the image. In one example, the optimal brightness may be set to 100, 200, or the like. While imaging the target 101, the first plurality of light sources 130 or the second plurality of light sources 156 of the device 100 or device 300 is illuminated. The self-exposure model is an iterative model that is run until a brightness setpoint is met. A secant method is used to find the next exposure value. Once the self-exposure model reaches a set point, it will not be executed any further until and unless it is called again. Because the brightness of each of the first plurality of light sources 130 may differ, a self-exposure model is set for each of the first plurality of excitation filters 142.

[0186] In the above example, the target is described in relation to a wound, but in other examples, the target may be food, laboratory equipment, sanitary equipment, sanitary equipment, biochemical assay chips, microfluidic chips, medical equipment, bodily fluids, or combinations thereof.

[0187] Furthermore, the analytical model referred to in the description relating to FIGS. 7 to 11 corresponds to the analytical model described in relation to FIG. 1 or the analytical model described in relation to FIG.

[0188] 11 illustrates a method 1100 for detecting problematic cellular entities according to an implementation of the present subject matter. The order in which the method blocks are described is not intended to be limiting, and some of the described method blocks may be combined in any order to implement method 1100 or alternative methods. Additionally, some of the individual blocks may be deleted from method 1100 without departing from the scope of the subject matter described herein. Method 1100 may be implemented by device 100 or device 300.

[0189] A point cloud is formed in block 604 from the 3D depth image taken in block 1102 and the white-light visible image taken in block 1110. Based on the point cloud image, a homography may be performed in block 1106, and the depth image is superimposed on the white-light image captured by the CMOS visible-light camera 122 in block 1108. A homography is an isomorphism of a projective space, induced by an isomorphism of the vector space from which the projective space is derived. It is a bijection that maps lines to lines, and is therefore a collinear transformation. In general, some collinear transformations are not homographies, but basic theory of projective geometry dictates that this is not the case, at least for two-dimensional real projective spaces.

[0190] FIG. 12a shows a perspective view of a device 1200 for investigating a target according to an implementation of the present subject matter. FIG. 12b shows a perspective view of a device 1200 for investigating a target according to an implementation of the present subject matter. FIG. 12c shows a perspective view of a device 1200 for investigating a target according to an implementation of the present subject matter. FIG. 12d shows a top view of a device 1200 for investigating a target according to an implementation of the present subject matter. FIG. 12e shows a top view of a device 1200 for investigating a target according to an implementation of the present subject matter. FIG. 12f shows an exploded view of a device 1200 for investigating a target according to an implementation of the present subject matter. FIG. 12g shows a front view of a device 1200 for investigating a target according to an implementation of the present subject matter. FIG. 12h shows a top view of a device 1200 for investigating a target according to an implementation of the present subject matter. FIG. 12i shows a side view of a device 1200 for investigating a target according to an implementation of the present subject matter. For simplicity, Figures 12a-12i will be described in conjunction with each other.

[0191] The device may interrogate a target, such as target 101. Furthermore, device 1200 may correspond to device 100 or device 300. Device 1200 may perform similar functions as device 100 or device 300.

[0192] Device 1200 may include a front cover 1236 and a back cover 1234. A portable power module, such as power module 136 (similar to the power module in device 100 or device 300), may connect to physician's mobile phone 1250 via USB cable 1238 or a similar power and / or data cable. Physician's phone 1250 or other mobile computing device, such as a desktop, tablet, laptop, smart accessory such as a smartwatch with a touch-activated user input screen, connects to a cloud server via a wired or wireless connection. The mobile computing device may even be considered a virtual reality headset, allowing the wearer to view the patient and possibly even the tissue during a medical procedure while viewing composite imaging of the wound site in real time, thus allowing the surgeon to view wound-related data in real time while performing surgery.

[0193] The mobile computing device may connect wirelessly to the main PCB board 1206, such as through a Bluetooth connection. An application programming interface (API) on the physician's phone uses the wireless connection to the main PCB board 1206 to send instructions through the API on the PCB board 1206 to the PCB board 1206, which then sends instructions to other elements of the device 1200 to start, continue, or complete the imaging process. The physician's phone and other devices 1250 then receive the images and other output from methods 700-1100, as described with reference to FIGS. 7-11 , and may then display the composite image to the user, who may delete, save, or otherwise use the images and data generated by the devices and communicated to the physician's phone 1250.

[0194] In some examples, images can be securely transferred to distant medical professionals via a cloud-based server system, email system, or otherwise transmitted electronically. The analytical models used in connection with device 1200 enable continuous improvement of the images analyzed by device 1200. After medical professionals around the world use device 1200 and provide input on the type of information being displayed, future users of the same or different device 1200, whether near the previous user or far away, benefit from the “learning” the system provides based on previous human input from the knowledge of medical professionals who previously used the system. This is called federated learning, a machine learning technique that trains models across multiple decentralized edge devices or servers with local data samples without exchanging them. Typically, detailed cumulative analyses are performed remotely from individual devices using previous wound imaging data stored in a non-patient-specific manner on a cloud-based computer system that communicates with the device via wired or wireless signals during use. Device 1200 even allows continuous improvement based on knowledge from medical professionals around the world to be used to improve device 1200 output to users who may not have the same level of advanced training as some other previous users. It is also possible that analysis based on previous imaging can be performed on device 1200 itself, instead of or in addition to remote detailed analysis. The analytical models of the present disclosure can be run faster, but perhaps at a lower level of detail, using the graphics processor of device 1200, which provides faster inference. Faster inference provides essentially instantaneous assessment of image features, such as oxygen saturation, bioburden, and wound analysis. This essentially instantaneous availability of data helps physicians provide immediate and accurate treatment to patients.

[0195] Similar to devices 100 and 300, device 1200 includes an imaging sensor 1222, a first plurality of light sources 1230, a three-dimensional image capture sensor 1220, and a ranging sensor 1232. First plurality of light source shields 1224 are typically used to house first plurality of light sources 1230 and protect them within device 1200. They may also prevent light from one light source from intersecting with other light sources. Device 1200 may include a charging board 1240 and an optional on / off switch 1210.

[0196] 13 shows a device 1300 for investigating a target according to an implementation of the present subject matter. The device 1300 may include a first plurality of optical bandpass filters or polarizers 1302, a first plurality of light sources 1304 optionally integrated with polarizers or excitation filters, or a combination thereof. Additionally, the device 1300 may include a computing device 1308, such as a smartphone, laptop, desktop, or a smart accessory such as a smartwatch. In the example illustrated herein, the computing device 1308 is illustrated as a smartphone. The computing device 1308 may be coupled using a clip 1330. Thus, in some examples, the device 1300 may utilize the first plurality of light sources 1304 for target illumination. Additionally, the device 1300 may include a power button 1306 for turning the device 1300 on or off. Additionally, computing device 1308 may include an imaging sensor or camera, such as imaging sensor or camera 122, a three-dimensional image capture sensor, such as three-dimensional image capture sensor 120, and a ranging sensor, such as ranging sensor 132. As will be appreciated, a three-dimensional image capture sensor may be used as a ranging sensor.

[0197] Device 1300 may correspond to device 100, device 300, or device 1200 and may include other similar components for detecting problematic cellular entities, such as those mentioned in connection with Figures 1 through 4e and 12a through 12i. First plurality of optical bandpass filters 1302 may correspond to first plurality of optical bandpass filters 126. First plurality of light sources 1304 may correspond to first plurality of light sources 130. Furthermore, device 1300 may detect problematic cellular entities similar to device 100 or device 300 as described in connection with Figures 7 through 11.

[0198] 1-11 may be performed by computing device 300. In some scenarios, device 1300 may include a processor, such as processor 140. The processor may process the images and transmit the results of the detection of problematic cellular entities to computing device 1308. In another example, some processing may be performed by the processor and some processing may be performed by computing device 1308. For example, analysis of the images may be performed by the processor, and detection of problematic cellular entities based on the analysis may be performed by computing device 1308. Alternatively, analysis of the images may be performed by computing device 1308, and detection of problematic cellular entities based on the analysis may be performed by the processor.

[0199] In some instances, when the target is a wound, the present subject matter allows for the detection of biofilms in the wound, as described below.

[0200] FIG. 14 illustrates detection of problematic cellular entities according to an implementation of the present subject matter. A device for investigating a wound is described herein. In other words, a target is described in relation to a wound. A device for investigating a wound may include an imaging module, an interfacing module, and an interface. The device may correspond to device 100, device 300, device 1200, and / or device 1300. Accordingly, components referred to herein may be similar to components of device 100, device 300, device 1200, and / or device 1300. The device described in relation to FIG. 14 may perform functions similar to device 100, device 300, device 1200, and / or device 1300 in addition to the functions referred to herein.

[0201] The imaging module may include a first plurality of light sources, a second plurality of light sources, an imaging sensor, and a three-dimensional image capture sensor. Each of the first plurality of light sources may emit excitation radiation in a predetermined range of wavelengths that causes one or more markers in the wound to fluoresce. The first plurality of light sources may be, for example, homogeneous or heterogeneous light sources.

[0202] Each of the second plurality of light sources may emit excitation radiation in a predetermined wavelength range without causing markers in the wound to fluoresce. The imaging sensor may directly receive light emitted by the wound in response to illumination of the wound by at least one or more of the first plurality of light sources and directly receive light reflected by at least one or more of the second plurality of light sources without an optical bandpass filter disposed between the imaging sensor and the wound. The imaging sensor may capture a first plurality of images formed based on the light emitted by the wound and may capture a second plurality of images formed based on the light reflected by the wound. Here, the light is said to be directly received by the imaging sensor because the emitted light and reflected light are not filtered by an optical bandpass filter prior to image capture.

[0203] The three-dimensional image capture sensor may illuminate the wound, receive light reflected by the wound in response to illumination of the wound by the three-dimensional image capture sensor, and generate a three-dimensional image of the wound based on the reflected light. To illuminate the target, the three-dimensional image capture sensor may include one or more light sources (not shown in FIG. 14 ) integral with the three-dimensional image capture sensor. However, in some examples, a separate light source may also be coupled to the three-dimensional image capture sensor to illuminate the target and enable capture of light reflected by the target by the illumination. In certain examples, the three-dimensional image capture sensor may be a structured lighting sensor, a time-of-flight sensor, a stereo sensor, or a combination thereof.

[0204] The interfacing module may be coupled to the imaging module. The interfacing module may include a processor. The processor may be configured to analyze a first image of the first plurality of images using an analytical model, the first image being a fluorescence-based image comprising fluorescence emitted from the wound. The processor may analyze a second image obtained from the second plurality of images using the analytical model. Further, the processor may analyze the three-dimensional image of the wound using the analytical model to determine variations in the intensity of emitted light across the spatial region of the wound by compensating for variations in distance across the spatial region of the wound from the three-dimensional image capture sensor and by compensating for variations in curvature of the wound relative to the three-dimensional image capture sensor. In addition, the processor may analyze the three-dimensional image of the wound using the analytical model to determine variations in the intensity of reflected light across the spatial region of the wound by compensating for variations in distance across the spatial region of the wound from the three-dimensional image capture sensor and by compensating for variations in curvature of the wound relative to the three-dimensional image capture sensor.

[0205] In this regard, the processor may detect the presence of a biofilm in the wound based on analysis of the first image, the second image, and the third image using an analytical model. The analytical model may be trained to detect the presence of a biofilm in the wound. The analytical model may create a composite image of the first image, the second image, and the three-dimensional image of the wound. The interface may display results corresponding to the detection of the biofilm in the wound and the composite image of the first image, the second image, and the three-dimensional image of the wound.

[0206] To detect biofilm in a wound, an analytical model is trained using a plurality of reference fluorescence-based images with biofilm, a plurality of three-dimensional images with biofilm, and a plurality of reference fluorescence-based images with biofilm, and the analytical model is trained to distinguish between fluorescence in the fluorescence-based images arising from the biofilm and fluorescence in the fluorescence-based images arising from areas other than the biofilm.

[0207] The analytical model may include, for example, multiple neural networks. Each of the multiple neural networks may extract important parameters from each modality, such as a first image, a second image, and a three-dimensional image. For example, a first neural network may extract important parameters from the first image, a second neural network may extract important parameters from the second image, and a third neural network may extract important parameters from the third image. Furthermore, a fourth neural network may perform fusion of parameters extracted by the three neural networks from the first image, the second image, and the three-dimensional image to detect biofilm in the wound. Alternatively, all images may be sent to a single neural network to identify spatial regions within the target that contain biofilm.

[0208] In addition to the first image, the second image, and the three-dimensional image, the analytical model may utilize polarized images. Thus, the device may include a first polarizer positioned between the first plurality of light sources and the target to pass excitation radiation of the first plurality of light sources of a first polarization. The device may include a second polarizer positioned between the target and the imaging sensor to pass light emitted by the target of a second polarization.

[0209] In one example, the first polarizer and the second polarizer can be in an orthogonal configuration, oriented 90 degrees from each other. In yet another example, the first polarizer and the second polarizer can be in a parallel configuration. When polarizers are used, the analytical model can include another neural network for extracting parameters from the polarized image. Furthermore, the neural network can perform fusion of parameters extracted by the neural network from the first image, the second image, the three-dimensional image, and the polarized image to detect biofilm in the wound.

[0210] In some examples, the first polarization and the second polarization may be the same. For example, in some examples, the first polarization and the second polarization may be left-handed circular polarization (LHCP). In other examples, the first polarization and the second polarization may be right-handed circular polarization (RHCP). In other examples, the first polarization and the second polarization may be different. For example, the first polarization may be one of LHCP or RHCP, and the second polarization may be the other of LHCP or RHCP.

[0211] In some examples, the plurality of polarizers may include a third polarizer positioned between the second plurality of light sources and the target to pass excitation radiation of the second plurality of light sources of the third polarization. The plurality of polarizers may be combined with the first set of excitation filters. In some examples, when the device includes a first plurality of optical bandpass filters acting as absorption filters positioned between the target and the imaging sensor, the plurality of polarizers may be combined with the first plurality of optical bandpass filters.

[0212] The device may include a second plurality of light sources for illuminating the target without causing one or more markers in the target to fluoresce. One or more of the second plurality of light sources may be configured to emit light with wavelengths in the visible range. The imaging sensor may be configured to capture a second plurality of images formed based on light reflected by the target in response to illumination of the target by at least one or more of the second plurality of light sources. The processor 140 may analyze the three-dimensional images of the wound using the analytical model to determine variations in the intensity of reflected light across a spatial region of the wound by compensating for variations in distance across the spatial region of the wound from the three-dimensional image capture sensor and by compensating for variations in curvature of the wound relative to the three-dimensional image capture sensor. The processor may be configured to analyze a second image obtained from the third plurality of images using the analytical model. The processor may be configured to detect the presence of problematic cellular entities in the target based on analysis of the first image, the second image, and the three-dimensional image using the analytical model. The processor may create a composite image of the target using the first image, the second image, and the three-dimensional image. The interface may be configured to display results corresponding to the detection of the problematic cellular entity and a composite image of the first image, the second image, and the three-dimensional image of the target.

[0213] In one example, the device can include a first set of excitation filters, each of which can be configured to filter and pass excitation radiation emitted by a light source of the first plurality of light sources in a predetermined range of wavelengths to illuminate the target. In addition, one or more excitation filters can also be configured to filter and pass excitation radiation emitted by a light source of the second plurality of light sources in a predetermined range of wavelengths.

[0214] In the example illustrated herein, reflectance image 1402 and fluorescence image 1404 corresponding to a wound are provided as input to analytical model 1406, which includes multiple neural networks, to detect biofilm in the wound, as shown in image 1408. Analytical model 1406 is the same as the analytical model previously mentioned or described with reference to Figures 1 through 11. In one example, the device may be capable of detecting autofluorescence signals emitted from the extracellular matrix (ECM) of the biofilm. The device may also detect autofluorescence from quorum sensing elements emitting into the ECM.

[0215] Furthermore, the device can distinguish biofilms in wounds from planktonic bacteria in wounds. For example, the reflectance scattering and fluorescence at multiple wavelengths can differ between biofilms and planktonic bacteria. Furthermore, biofilms can have characteristics such as higher specular reflectance compared to planktonic bacteria, which can be captured from the reflectance image. Therefore, by analyzing the first multiple images, the second multiple images, and the three-dimensional image, the device can distinguish planktonic bacteria from biofilms. In addition, multiple polarizers can be used to capture differences in reflectance coefficients to enable differentiation between planktonic bacteria and biofilms. Reflection can consist of specular and diffuse reflection. In reflectance imaging, a polarizer positioned between the light source and the target and a polarizer positioned between the imaging sensor and the target can have parallel polarizations to obtain specular and diffuse reflection. A polarizer positioned between the light source and the target and a polarizer positioned between the imaging sensor and the target can have orthogonal polarizations to obtain diffuse reflection.

[0216] Detection of biofilms can facilitate better and more rapid wound treatment management. For example, wounds with biofilms may be resistant to antibiotics and take longer to heal. Thus, when the subject device enables detection of biofilms in a wound, treatment can be provided accordingly. For example, interventional procedures such as wound debridement can be performed to effectively remove biofilms from the wound. This allows for faster wound healing.

[0217] In the above example, the device is described in connection with detecting biofilm in a wound by capturing the wound. Alternatively or in addition to capturing the wound, the device can also capture blotting paper for detecting biofilm. The blotting paper can be impregnated with a chemical, such as ruthenium red or alcian blue, and pressed against the wound. The blotting paper can then be imaged by an imaging sensor. An analytical model can analyze the image of the blotting paper and detect the presence of biofilm. For example, polysaccharides in exudate can be collected by applying a nitrocellulose membrane to the surface of the wound, and biofilm can be visualized by staining with either ruthenium red or alcian blue. In another example, charged blotting paper can be used to detect the presence of biofilm in a wound. For example, charged blotting paper can be pressed against the wound and captured by an imaging sensor. An analytical model can analyze the image and detect the presence of biofilm in the wound.

[0218] FIG. 15 illustrates a system 1500 for investigating a target according to an implementation of the present subject matter. The processing device 1501 may be a computing device, such as a server, located at a remote location, such as the cloud. The processing device 1501 may include a computer, a server, a cloud device, or any combination thereof. The device 100 may be connected to the processing device 1501 via a communication network 1501. According to the present implementation, an analytical model is stored on the processing device 1501. The analytical model may correspond to the analytical model described in connection with FIGS. 1 to 2c and 7 to 11. The analytical model may also correspond to the analytical model mentioned in connection with FIGS. 3 to 4e. The processing device 1501 may include a processor 2402 that implements the analytical model. The processor 1502 may correspond to the processor 140. Thus, the device 100 may capture fluorescence-based images and white-light images of the target and transmit them to the processing device 1501. Upon detecting and classifying a pathogen, the processing device 1501 may transmit the results of the analysis to the device 100, which may then display the results on the interface 108.

[0219] In some implementations, device 100 may perform detection and classification as described in connection with Figures 1-2c and 7-11. Additionally, training may be similar to the training described in connection with Figure 5. In the examples shown herein, the devices are described in connection with device 100, but in some examples, the devices of system 1500 may also correspond to device 300, device 1200.

[0220] 16a-16b illustrate a method for investigating a target according to an implementation of the present subject matter. The order in which method 1600 is described is not intended to be limiting, and any number of the described method blocks may be combined in any order to implement method 1600 or alternative methods. Furthermore, method 1600 may be implemented by a processor or computing device through any suitable hardware, non-transitory machine-readable instructions, or combination thereof. The method may be implemented by device 100, device 300, device 1200, device 1300, and / or system 1500. Accordingly, components described in connection with method 1600 may correspond to corresponding components of device 100, device 300, device 1200, device 1300, and / or system 1500.

[0221] In step 1602, the method 1600 may include illuminating a target using at least one or more light sources of a first plurality of light sources of the device, wherein the light emitted by each of the first plurality of light sources has a wavelength band.

[0222] In step 1604, a first plurality of images may be captured by an imaging sensor. The imaging sensor may be configured to receive light emitted by the target in response to illumination of the target by one or more light sources of the first plurality of light sources. The first plurality of images may be formed based on the light emitted by the target.

[0223] In step 1606, method 1600 may include capturing a three-dimensional image of the target by a three-dimensional image capture sensor. The three-dimensional image capture sensor may be configured to illuminate the target, receive light reflected by the target in response to illumination of the target by the three-dimensional image capture sensor, and generate a three-dimensional image of the target based on the reflected light. To illuminate the target, the three-dimensional image capture sensor may include one or more light sources integral with the three-dimensional image capture sensor. However, in some examples, a separate light source may also be coupled to the three-dimensional image capture sensor to illuminate the target and enable capture of light reflected by the target due to the illumination.

[0224] In step 1608, a first image of the first plurality of images may be analyzed by the processor using the analytical model. The first image may be a fluorescence-based image comprising fluorescence from the target in response to light emitted by at least one or more light sources of the first plurality of light sources.

[0225] In step 1610, the three-dimensional image of the target may be analyzed by a processor to determine variations in the intensity of emitted light across the spatial region of the target by compensating for variations in distance across the spatial region of the target from the three-dimensional image capture sensor and by compensating for variations in curvature across the spatial region of the target.

[0226] At step 1612, method 1600 includes detecting, by the processor, the presence of problematic cellular entities in the target based on the analysis of the first image and based on the three-dimensional image of the target using an analytical model. The analytical model can be trained to detect the presence of problematic cellular entities in the target.

[0227] In step 1614, a composite image of the first image and the three-dimensional image of the target may be created. In step 1616, results corresponding to the presence of problematic cellular entities and a composite image of the first image and the three-dimensional image may be displayed by the interface.

[0228] In one example, an analytical model can be trained using multiple reference fluorescence-based images and multiple reference three-dimensional images to detect the presence of problematic cellular entities in a target. The analytical model can be trained to distinguish between fluorescence in the fluorescence-based images arising from the problematic cellular entities and fluorescence in the fluorescence-based images arising from areas other than the problematic cellular entities.

[0229] The target may be a wound area. In this regard, the method includes extracting, by a processor, spatial and spectral features of the wound area from the first image and the three-dimensional image using the analytical model. The location of the wound area may be identified by the processor based on the extraction of the spatial and spectral features using the analytical model. The method includes determining, by the processor, a contour of the wound area based on the extraction of the spatial and spectral features using the analytical model. Pathogens in the wound area may be detected by the processor based on the extraction of the spatial and spectral features using the analytical model. The method includes classifying, by the processor, the pathogen by at least one of a pathogen family, genus, species, or strain using the analytical model.

[0230] In an example, the method 1600 may include determining, by the processor, a length of the wound area, a width of the wound, a perimeter of the wound, a depth of the wound, an area of ​​the wound, or a combination thereof based on determining the contour of the wound area using the analytical model.

[0231] Furthermore, in certain examples, the target is one of a wound area, food, laboratory equipment, medical equipment, sanitary equipment, sanitary implements, biochemical assay chips, microfluidic chips, bodily fluids, or combinations thereof. Furthermore, the method may include, in response to detecting the presence of the problematic cellular entity, when the target is a wound area, determining, by the processor, at least one of the degree of infection of the wound area, the spatial distribution of pathogens in the wound area, or the rate of healing of the wound area. Furthermore, when the target is tissue, the method may include detecting, by the processor, the presence of the problematic cellular entity as at least one of cancerous tissue, necrotic tissue, or a combination thereof in a tissue sample. Furthermore, when the target is one of sanitary equipment, sanitary implements, medical equipment, biochemical assay chips, bodily fluids, or microfluidic chips, the method may include determining, by the processor, the problematic cellular entity as a pathogen, and classifying, by the processor, the pathogen in the target.

[0232] In an example, method 1600 may include filtering and passing light emitted by a target in response to illumination of the target by at least one or more light sources of the first plurality of light sources with an optical bandpass filter of the first plurality of optical bandpass filters. The optical bandpass filter may be positioned between the target and an imaging sensor. The imaging sensor may capture the filtered light from the optical bandpass filter.

[0233] 17 shows results 1700 corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. In the example illustrated herein, a fluorescence-based image, such as an autofluorescence image of a wound, is captured by illuminating the wound with various UV-visible wavelengths, such as 365 nm, 395 nm, 415 nm, and 450 nm, from a suitable light source after passing the light through an appropriate narrow bandpass filter and linear polarizer. The autofluorescence image is captured after linearly polarizing the fluorescent response from the wound by placing a linear polarizer in front of the imaging sensor so that the polarization axis of the imaging sensor is orthogonal to the polarization axis of the polarizer in front of the light source. An exemplary autofluorescence image of a wound at an excitation wavelength of 365 nm is shown by image 1702.

[0234] Additionally, a 3D depth image and a white space image of the wound are obtained using a three-dimensional image capture sensor such as a depth camera. The 3D depth image is shown by image 1704.

[0235] The autofluorescence image, such as image 1702, and the 3D depth image, such as image 1704, along with a white light image of the wound, are fed to analysis model 1706, which predicts areas of the wound where problematic cellular entities are present. Analysis model 1706 provides a depth image superimposed with autofluorescence intensity indicative of the presence of problematic cellular entities, such as that shown by image 1708.

[0236] FIG. 18 shows results 1800 corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter.

[0237] In this example, images are captured using multiple modalities, including fluorescence-based imaging such as autofluorescence imaging, and reflectance imaging such as NIR reflectance imaging, and 3D depth imaging is fed into the analytical model.

[0238] Image 1802 shows an autofluorescence image captured when the excitation wavelength was 365 nm. Image 1804 shows an autofluorescence image when the excitation wavelength was 395 nm. Image 1806 shows a frame from NIR diffuse reflectance video captured when the excitation wavelength was 660 nm. Image 1808 shows a frame from NIR diffuse reflectance video captured when the excitation wavelength was 850 nm. Image 1810 shows a 3D depth image. Images 1802, 1804, 1810 and videos 1806, 1808 are provided as inputs to analysis model 1812. In this example, analysis model 1812 is a deep convolutional neural network. Analysis model 1812 predicts areas of the wound that indicate the presence of problematic cellular entities. For example, in image 1814, the autofluorescence image is superimposed with a mask indicating the predicted areas of problematic cellular entities. The region marked as 1816 corresponds to the pathogen Pseudomonas aeruginosa.

[0239] 19 shows results 1900 corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. In this example, images captured with multiple modalities, including fluorescence-based imaging such as autofluorescence imaging, reflectance imaging such as NIR reflectance imaging, and 3D depth imaging, are fed into an analytical model. The analytical model predicts areas of the wound that indicate the presence of problematic cellular entities, as well as regions of the wound that exhibit low, medium, and high oxygen saturation.

[0240] Image 1902 shows an autofluorescence image captured when the excitation wavelength was 365 nm. Image 1904 shows an autofluorescence image when the excitation wavelength was 395 nm. Image 1906 shows an NIR diffuse reflectance image captured when the excitation wavelength was 660 nm. Image 1908 shows a frame from an NIR diffuse reflectance video captured when the excitation wavelength was 850 nm. Image 1910 shows a 3D depth image. Images 1902, 1904, 1906, 1910, and video 1908 are provided as inputs to analysis model 1912. In this example, analysis model 1912 is a deep neural network. Analysis model 1912 predicts areas of the wound that exhibit the presence of problematic cellular entities, as shown by image 1914, as well as problematic cellular regions of the wound that exhibit low, medium, and high tissue oxygen saturation, as shown by image 1916. In image 1914, the autofluorescence image is superimposed with a mask showing the predicted area of ​​the problematic cellular entity, which (represented by region 1915) is identified as the pathogen Pseudomonas aeruginosa.

[0241] FIG. 20 shows results 2000 corresponding to tissue oxygen saturation according to an implementation of the present subject matter. In this example, time-varying NIR reflectance maps captured at different NIR excitation wavelengths, such as 660 nm, 740 nm, and 850 nm, captured as individual videos, such as those illustrated by 2002, 2004, and 2006, are first passed through an image and video processing module 2008. The image and video processing module 2008 may be part of a processor, such as processor 140, including a GPU. The image and video processing module 2008 may obtain a target heart rate from the video and may filter the video in time so that only a narrow band of frequencies around the heart rate frequency is retained. The filtered set of frames is then passed through an analysis model 2010, which predicts problematic cellular regions of low, moderate, and high tissue oxygen saturation, as shown by image 2012. In this example, the analysis model 2010 used is a deep convolutional neural network.

[0242] 21 shows results 2100 corresponding to the detection of biofilm in a wound according to an implementation of the present subject matter. In this example, white-light image 2102 and fluorescence-based images, such as autofluorescence images 2104, 2106, and 2108 of the wound, captured at different illumination wavelengths, such as 365 nm, 395 nm, and 450 nm, respectively, are used to train an analytical model 2110 to predict areas of the wound with biofilm, as shown by image 2112. In image 2112, the white-light image of the wound is superimposed with the detected biofilm (region 2113 in image 2112). In this example, the analytical model is a deep neural network. Additionally, oxygen saturation and thermal images can also be added to the analytical model to improve the accuracy of biofilm detection.

[0243] 22 illustrates results corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. In this example, a multispectral camera is used to capture fluorescence-based images, such as autofluorescence images, of a wound in different wavelength bands. Furthermore, multi-channel images, such as images 2202, 2204, 2206, 2208, and 2210, are processed by an analytical model 2212, such as a deep neural network, such as a convolutional neural network, to predict areas of the wound with specific problematic cellular entities. In image 2214, a white-light image of the wound is superimposed with predicted areas of problematic cellular entities. In image 2214, area 2215 corresponds to the pathogen Staphylococcus aureus.

[0244] FIG. 23a shows results 2300 corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter.

[0245] A multispectral camera is utilized to acquire autofluorescence images of the wound across various wavelength bands, including visible and UV wavelengths. For example, image 2302 shows an image captured with an illumination wavelength of 365 nm. Image 2304 shows an image captured with an illumination wavelength of 395 nm, and image 2306 shows an image captured with an illumination wavelength of 450 nm. Furthermore, image 2304 shows a three-dimensional image of the wound. All of the above images are provided as inputs to analysis model 2308. Analysis model 2308 may be referred to as a tissue detection network (TDN). TDN 2308 may process the multi-channel and three-dimensional images to identify specific problematic tissue regions within the wound. TDN 2308 may predict the composition of the wound tissue, including elements such as granulation, scab, necrotic tissue, etc. As shown by image 2312, this predicted image is overlaid onto a white-light image of the wound to facilitate layering of the wound healing trajectory. Image 2310 is a white light image of the wound with predicted areas of scab 2314 and granulation 2316 superimposed.

[0246] FIG. 23b shows results 2300 corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. In this example, a white-light image 2312 highlighting the problematic tissue regions predicted by TDN 2310, as described in connection with FIG. 23a, is used for further detection. Image 2312 is provided to analysis model 2318. Analysis model 2318 may be, for example, a deep learning network and may be referred to as a tissue-aware oxygen saturation prediction deep learning network. Additionally, images 2320 and 2322 corresponding to NIR diffuse reflectance wavelength images at illumination wavelengths of 660 nm and 850 nm, respectively, are provided as inputs to tissue-aware oxygen saturation prediction deep learning network 2318. Tissue-aware oxygen prediction deep learning network 2318 generates a tissue-aware oxygen saturation image highlighting the problematic tissue regions, as shown by image 2324. Image 2324 corresponds to an oxygen saturation image predicted by the tissue-aware oxygen saturation prediction deep learning network 2318, with areas of scab and granulation highlighted.

[0247] FIG. 24 shows results 2400 corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. In the example shown herein, a multi-spectral camera is utilized to capture autofluorescence images and a thermal camera captures temperature distribution images of the wound in a multi-modal detection mode. Image 2412 corresponds to a wound image captured using the multi-spectral camera. Image 2414 corresponds to a wound image captured using a thermal imaging sensor. Images 2412 and 2414 are fed to an analysis model 2416. Analysis model 2416 is a deep neural network. Deep neural network 2416 predicts areas of the wound that indicate the presence of problematic cellular elements based on both the autofluorescence and thermal signatures. Image 2418 corresponds to an autofluorescence image with the problematic cellular entities detected.

[0248] FIG. 25 shows results 2500 corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. In this example, as shown by image 2502, the wound is excited by a set of pulsed UV LEDs at wavelengths of 395 nm, 365 nm, which are rapidly repeated at a predetermined period. In general, ambient light can be involved in the imaging process, resulting in either a constant offset in the measured intensity across all Red, Green, and Blue (R,G,B) channels of the imaging sensor, or an oscillatory component of 50 / 60 Hz (region dependent) in all (R,G,B) channels. To counteract this effect, the excitation is applied at a frequency f different from these frequencies. ex As shown in image 2504, an image is captured (an autofluorescence image captured with ambient light) and the resulting image is pre-processed by image processing block 2506, which, among other things, processes f exThe image is filtered and processed by focusing on temporal frequencies around . The preprocessed data is then fed to an analytical model 2508, such as a deep neural network, which determines areas with problematic cellular entities based on autofluorescence signatures as shown by image 2510. ex ) is used, the same number of target autofluorescence frames required for subsequent detection can be obtained in a proportionally shorter time. In the above example, image preprocessing is described separately from analysis model 2508, but in some examples, image preprocessing can be performed by analysis model 2508. In this example, faster pulsing allows images to be captured in a shorter time, thereby shortening the overall imaging time. Additionally, images can be captured using a single high-power pulse of the LED, thereby reducing the exposure time of the imaging sensor and reducing the impact of ambient light on the light emitted by the high-power illumination light. Thus, the overall imaging time is significantly reduced. For example, if the pulse width is shortened from 1 ms to 0.1 ms, the overall imaging time is reduced by a factor of 10. Thus, any noise due to patient or device movement is significantly reduced.

[0249] 26 shows results 2600 corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. In the example shown herein, the wound is excited by a set of pulsed UV LEDs, indicated by 2602, which are repeated at a predetermined period. In general, ambient light may be involved in the imaging process, resulting in either a constant offset in the measured intensity across all R, G, B channels of the imaging sensor, or an oscillatory component of 50 / 60 Hz (region dependent) in all (R, G, B) channels. To counteract this effect, the excitation is applied at a frequency f different from these frequencies. exThe resulting image with environmental noise (autofluorescence image frame captured with ambient light 2604) is passed to an analysis model 2606, such as a long short-term memory (LSTM) detection model, which combines the image processing required to remove noise due to environmental disturbances from the raw data and subsequently detect and present areas with problematic cellular entities. Image 2608 shows a white-light image with an area with problematic cellular entities superimposed. In this example, faster pulsing allows images to be captured in a shorter time, thereby shortening the overall imaging time. Additionally, images can be captured using a single high-power pulse of the LED, thereby reducing the exposure time of the imaging sensor and the impact of ambient light on the light emitted by the high-power illumination light. Therefore, the overall imaging time is significantly reduced. For example, if the pulse width is reduced from 1 ms to 0.2 ms, the overall imaging time is reduced by a factor of five. Therefore, any noise due to patient or device movement is significantly reduced.

[0250] 27 shows results 2700 corresponding to the detection of problematic cellular entities according to an implementation of the present subject matter. In the example illustrated herein, an architectural variant is used in which the overall functionality of the device for investigating targets is split between a CPU and a GPU. The CPU detects different excitation wavelengths λ, as shown by images 2708, 2710, and 2712. ex The GPU is responsible for excitation and detection processes 2704, 2706, which capture autofluorescence images of the wound by exciting at 365 nm, 395 nm, and 415 nm. Images 2708, 2710, 2712 correspond to autofluorescence images at illumination wavelengths of 365 nm, 395 nm, and 415 nm. The GPU may include and execute an analytical model 2714, such as a deep neural network, that performs boundary inference functions to determine areas with problematic cellular entities. Image 2716 corresponds to a white-light image of the wound with areas of problematic cellular entities marked.

[0251] The fact that the disclosed system creates composite images provides another significant benefit to users of such systems. Specifically, the system can be used to create accurate composite renderings of images and information about a wound that can be provided to a medical professional or other user, even while taking one or more images at any angle and any distance. Thus, a user may not require significant or any significant training in using the device, but can simply use the device to take images in a manner similar to taking a regular portrait image. This allows non-medical professionals or less-trained medical professionals to obtain accurate information while using the device. As discussed herein, a non-medical professional or medical professional may send an image or series of images to a remote medical professional for further advice before treatment using the disclosed device.

[0252] The present subject matter enables faster image capture and processing for detecting problematic cellular entities. Because the present subject matter provides an on-board processor and imaging module, the present subject matter enables faster image capture and processing. Specifically, by using a combination of a CPU, a GPU, and an optional FPGA, the present subject matter enables image capture and processing at a rate of more than 30 images per second. To detect the presence of problematic cellular entities in a target, an analytical model is trained on several reference fluorescence-based images and several reference three-dimensional images, thereby increasing the accuracy of detection. The present subject matter ensures that the light emitted by the light source is at a different frequency from the ambient light source. Thus, the present subject matter enables the elimination of ambient light interference with the light emitted by the target. Furthermore, the present subject matter allows the pulsed LED to operate at a faster frequency, such as 100 Hz to tens of MHz. Thus, the present subject matter enables faster capture of the first plurality of images and the three-dimensional image, reducing ambient light interference (background interference). Thus, the present subject matter removes background information and increases the accuracy of detection.

[0253] Furthermore, in certain examples, the analytical model can ignore background light and excitation light in the fluorescence-based image and can pick up even weak fluorescence information in the fluorescence-based image. Thus, in certain examples, the present subject matter also eliminates the use of absorption filters to filter background light and excitation light, and the use of filter wheels. Thus, the subject device is simple and cost-effective.

[0254] The present subject matter compensates for variations in distance over a spatial region of a target from a three-dimensional image capture sensor and variations in curvature over a spatial region of a target relative to the three-dimensional image capture sensor. Thus, the present subject matter can increase the accuracy of detecting problematic cellular entities, particularly for targets such as wounds. Because the device enables transmission of composite image results to a cloud server, a non-medical professional or a medical professional can transmit an image or series of images to a remote medical professional for further advice before treatment using the disclosed device.

[0255] Thus, the present subject matter provides for rapid, optionally filter-free, non-invasive, automatic, in-situ pathogen detection and classification using "optical computational biopsy" techniques, in which multispectral imaging is used in conjunction with computational models, such as machine learning models, artificial neural network (ANN) models, and deep learning models, for non-invasive biopsy to detect and classify problematic cellular entities.

[0256] The present subject matter can be used to detect the presence of problematic cellular entities in diabetic foot ulcers, surgical site infections, burns, skin, and inside the body, such as the esophagus, stomach, and colon. The subject devices can be used in the fields of dermatology, cosmetic surgery, plastic surgery, infection control, photodynamic therapy monitoring, and drug susceptibility testing.

[0257] The device may be integrated into routine clinical diagnostics and may be used in telemedicine and telenursing. Furthermore, the majority of clinically important pathogens can be detected and classified within minutes. Furthermore, data acquisition and analysis can be performed automatically. Therefore, the device can be easily operated without the need for skilled technicians. This feature aids in rapidly determining treatment protocols. The device may also be used for pathogen detection and classification in resource-poor settings. The subject device may also be used in endoscopic investigations.

[0258] The subject devices can be used to quantify various pathogens present in a target. The devices can also be used to monitor wound healing and wound closure. The devices can also be used to study antimicrobial susceptibility by exposing the target to various antibiotics and observing and analyzing the target. For example, the devices can be used to study bacteria growing in the presence of antibiotics, and corresponding biomarker signatures can be recorded. This information can be used to inform antibiotic prescriptions based on the antimicrobial susceptibility of specific bacteria. It should be understood that the antimicrobial susceptibility of other pathogens, such as fungi, can also be studied. Furthermore, antibiotic doses and concentrations can also be determined based on dilution factors to determine the dose of antibiotic and antifungal agents to be administered.

[0259] The device can be configured to study the biomolecular composition and dynamic behavior of various pathogens based on their fluorescent signatures. The device can also be used in cosmetics. For example, the device can be used to detect the presence of acne-causing Propionibacterium. The device can also be used during tissue transplantation to ensure that the tissue is pathogen-free. The device can be used for forensic detection, for example, to detect pathogens in bodily fluids such as saliva, blood, and mucus. The device can be configured to study the effectiveness of disinfection on various hospital surfaces, such as beds, walls, hands, gloves, bandages, clothing, catheters, endoscopes, hospital instruments, and sanitary equipment.

[0260] The device can also be used to detect the presence of pathogens on hands and surfaces, for example, in hospitals and other places where pathogens should not be present. The device can be used to detect pathogen contamination in food products, such as food, fruits, and vegetables.

[0261] Although examples and implementations of the present subject matter have been described in language specific to structural features and / or methods, it should be understood that the present subject matter is not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed and described in the context of a few example implementations of the present subject matter. [Explanation of symbols]

[0262] 37 Third Housing 44 Power PCB 100 devices 101 Target 102 Imaging Module 104 Interfacing Module 106 SOM 108 Interface 112 Rear frame 120 3D image capture sensor 122 Imaging Sensor 124 Absorption filter wheel 126 first plurality of absorption filters 128 Servo Motor 130 first plurality of light sources 132 Distance Sensor 136 Power Supply Module 140 processors 142 First set of excitation filters 150 Light Source Driver 156 Secondary Multiple Light Sources 160 Cloud 210 SOM 236 Connection Blanket 300 devices 409 Bracket 410 Stand 411 Base 412 Rear Frame 414 Front Frame 416 Articulated Arm 418 legs 420 wheels 421 First End 422 Lower Arm 423 Second End 424 Color 426 Upper Arm 428 Bracket 432 Upper Arm 434 Connecting Bracket 435 rear frame 437 Hinge 438 Bridge 440 x Brackets 442 power cord 444 Power PCB 446 Batteries 448 Cover 450 cover 602 images 604 images 606 Analysis Model 1200 devices 1206 PCB board 1210 Switch 1220 3D image capture sensor 1222 Imaging Sensor 1224 Light Source Shield 1230 light source 1232 Distance Sensor 1234 Rear cover 1236 Front cover 1238 USB cable 1240 Charging Board 1250 phone 1300 devices 1302 Polarizing plate 1304 Light source 1306 Power button 1308 Computing Devices 1330 Grip 1402 Reflection Images 1404 Fluorescence Images 1406 Analysis Model 1408 images 1501 Network 1502 processor 1702 images 1704 images 1706 Analysis Model 1708 images 1802 images 1804 images 1806 images 1808 images 1810 images 1812 analytical model 1814 images 1902 images 1904 images 1906 images 1908 Videos 1910 images 1912 analytical model 1914 images 1915 area 1916 images 2008 Image and Video Processing Module 2010 Analysis Model 2012 Images 2102 White Light Image 2104 Autofluorescence Image 2106 Autofluorescence Images 2108 Autofluorescence Image 2112 images 2113 area 2202 images 2204 images 2206 images 2208 images 2210 images 2212 Analysis Model 2214 images 2215 area 2302 images 2304 images 2306 images 2308 Analysis Model 2310 images 2312 images 2314 scab 2316 Granulation formation 2318 Analysis Model 2320 images 2322 images 2324 images 2412 images 2414 images 2416 Analysis Model 2418 images 2502 images 2504 images 2508 Analysis Model 2510 images 2602 Pulsed UV LED 2604 Ambient light 2606 Analysis Model 2608 images 2704 Excitation Process 2706 Discovery Process 2708 images 2710 images 2712 images 2714 Analysis Model 2716 images

Claims

1. 1. A device for investigating a target, comprising: a first plurality of light sources, each configured to emit excitation radiation in a predetermined range of wavelengths that causes one or more markers in the target to fluoresce; the imaging sensor configured to directly receive light emitted by the target in response to illumination of the target by at least one light source of the first plurality of light sources without an optical bandpass filter disposed between the imaging sensor and the target, and to capture a first plurality of images formed based on the emitted light; and a three-dimensional image capture sensor for illuminating the target and receiving light reflected by the target in response to the illumination of the target by the three-dimensional image capture sensor, and generating a three-dimensional image of the target based on the reflected light; an imaging module comprising: an interfacing module coupled to the imaging module, the interfacing module comprising: analyzing a first image of the first plurality of images using an analytical model, the first image being a fluorescence-based image comprising fluorescence from the target; analyzing the three-dimensional image of the target using the analytical model to determine variations in the intensity of the emitted light across the spatial region of the target by compensating for variations in distance across the spatial region of the target from the three-dimensional image capture sensor and variations in curvature across the spatial region of the target; using the analytical model to detect the presence of problematic cellular entities in the target based on the analysis of the first image and the three-dimensional image, the analytical model being trained to detect the presence of problematic cellular entities in the target; Creating a composite image of the first image and the three-dimensional image of the target a processor configured to: displaying results corresponding to the detection of the problematic cellular entity and the composite image of the target. and an interface for

2. 2. The device of claim 1, wherein the analytical model is trained using a plurality of reference fluorescence-based images and a plurality of reference three-dimensional images to detect the presence of problematic cellular entities in a target, and the analytical model is trained to distinguish between fluorescence in the fluorescence-based images arising from the problematic cellular entities and fluorescence in the fluorescence-based images from areas of the target other than the problematic cellular entities.

3. The system further comprises a system on module (SOM), the SOM comprising: the imaging module; the processor; 10. The device of claim 1, further comprising: a plurality of light source drivers, each of the plurality of light source drivers configured to control a respective light source of the first plurality of light sources.

4. 4. The device of claim 3, wherein one or more light sources of the first plurality of light sources are pulsed light emitting diodes (LEDs), and the processor is configured to operate one or more of the light source drivers of the plurality of light source drivers to control the pulsed LEDs to emit pulses of excitation radiation to enable faster imaging and reduce ambient light interference in the light emitted by the target.

5. The device of claim 1 , wherein the processor is configured to operate the imaging sensor to capture and process the first plurality of images at greater than 30 frames per second.

6. the imaging module further comprising: a second plurality of light sources for illuminating the target without causing the one or more markers in the target to fluoresce, each of the second plurality of light sources configured to emit light at a wavelength in the near-infrared (NIR) range or the visible range; the imaging sensor is configured to capture a second plurality of images formed based on light reflected by the target in response to illumination of the target by at least one light source of the second plurality of light sources; the processor: analyzing a second image obtained from the second plurality of images using the analytical model to identify oxygen saturation in a plurality of regions within the target; analyzing the three-dimensional image of the target using the analytical model to determine variations in the intensity of the reflected light across the spatial region of the target by compensating for variations in distance across the spatial region of the target from the three-dimensional image capture sensor and variations in curvature across the spatial region of the target; using the analytical model to detect the presence of problematic cellular entities in the target based on the analysis of the first image, the second image, and the three-dimensional image; creating a composite image of the target using the first image, the second image, and the three-dimensional image; It is configured as follows: The interface: displaying results corresponding to the detection of the problematic cellular entity and the composite image of the first image, the second image, and the three-dimensional image of the target. The device of claim 1 configured to:

7. the processor: activating the first plurality of light sources to emit light at the target; activating the second plurality of light sources to emit light at the target; activating the imaging sensor to capture light emitted by the target in response to illumination of the target by the at least one or more light sources of the first plurality of light sources and to capture light emitted by the target in response to illumination of the target by the at least one or more light sources of the second plurality of light sources. The device of claim 6 , configured to:

8. a second plurality of light sources for illuminating the target without causing the one or more markers in the target to fluoresce, at least one or more of the second plurality of light sources configured to emit light of wavelengths in the visible range; the imaging sensor is configured to capture a third plurality of images formed based on light reflected by the target in response to illumination of the target by the at least one light source of the second plurality of light sources, the third plurality of images being white light images; the processor: analyzing a third image obtained from the third plurality of images using the analytical model; analyzing the three-dimensional image of the target using the analytical model to determine variations in the intensity of the reflected light across the spatial region of the target by compensating for variations in distance across the spatial region of the target from the three-dimensional image capture sensor and variations in curvature across the spatial region of the target; using the analytical model to detect the presence of problematic cellular entities in the target based on the analysis of the first image, the third image, and the three-dimensional image; creating a composite image of the target using the first image, the third image, and the three-dimensional image; It is configured as follows: The interface: displaying results corresponding to the detection of the problematic cellular entity and the composite image of the first image, the third image, and the three-dimensional image of the target. The device of claim 1 configured to:

9. The device of claim 1 , wherein the processor is configured to control the first plurality of light sources to illuminate at a frequency other than a frequency of an ambient light source.

10. The device of claim 1, further comprising a thermal sensor for thermal imaging of the problematic cellular entity.

11. a first housing for accommodating the imaging module; a second housing for receiving the interfacing module; 10. The device of claim 1, comprising: a bridge for connecting the imaging module and the interfacing module, the bridge comprising an electrical interface for enabling electrical communication between the processor and the imaging module.

12. 12. The device of claim 11, wherein the electrical interface comprises a camera serial interface (CSI), a serial management bus such as an I2C interface, a system packet interface (SPI), a universal asynchronous receiver-transmitter (UART), a general-purpose input / output (GPIO) interface, a universal serial bus (USB) interface, a pulse-width modulation (PWM) interface, a display serial interface (DSI), and a high-definition multimedia interface (HDMI).

13. a portable power module operable to power components of the imaging module and the interfacing module; and a third housing configured to house the portable power module.

14. The device of claim 1 , further comprising a ranging sensor operable to determine a distance of the target from the device to position the device at a predetermined distance from the target.

15. The device of claim 1 , wherein the three-dimensional image capture sensor is operable to determine a distance of the target from the device to position the device at a predetermined distance from the target.

16. the target is a wound area, and the processor further extracting spatial and spectral features of a wound region from the first image and the three-dimensional image using the analytical model; using the analytical model to locate the wound area based on the extraction of the spatial and spectral features; determining a contour of the wound area based on the extraction of the spatial and spectral features by using the analytical model; detecting pathogens in the wound area based on the extraction of the spatial and spectral features by using the analytical model; Classifying the pathogen by at least one of family, genus, species, or strain of the pathogen using the analytical model. The device of claim 1 configured to:

17. 17. The device of claim 16, wherein the processor is further configured to determine a length of the wound area, a width of the wound, a depth of the wound, a circumference of the wound, or an area of ​​the wound based on the determination of the contour of the wound area.

18. the target is one of a wound area, food, laboratory equipment, sanitary equipment, hygiene equipment, medical equipment, biochemical assay chips, microfluidic chips, or bodily fluids; and when the target is a wound area, the processor is configured to, in response to detecting the presence of the problematic cellular entity, determine at least one of an extent of infection of the wound area, a spatial distribution of pathogens in the wound area, or a rate of healing of the wound area; when the target is tissue, the processor is further configured to detect the presence of the problematic cellular entity as at least one of cancerous tissue or necrotic tissue in a tissue sample; 10. The device of claim 1, wherein when the target is one of a sanitary instrument, a sanitary implement, a laboratory implement, a medical implement, a biochemical assay chip, a microfluidic chip, or a bodily fluid, the processor is configured to determine the problematic cellular entity as a pathogen and classify the pathogen in the target.

19. a first polarizer positioned between the first plurality of light sources and the target for passing excitation radiation of the first plurality of light sources of a first polarization; 10. The device of claim 1, further comprising: a second polarizer positioned between the target and the image sensor for passing the light emitted by the target of a second polarization.

20. the processor: The device of claim 1 , configured to transmit the results and the composite image of the first image and the three-dimensional image to a remote system in electrical communication with the device.

21. The interface:

10. The device of claim 1, configured to, in response to an input, transmit the results corresponding to the detection and classification of the pathogen upon the detection and classification of the pathogen by using an application programming interface.

22. The device of claim 1 , wherein the device is a smartphone.

23. 10. The device of claim 1, wherein the imaging sensor is a charge-coupled device (CCD) sensor, a CCD digital camera, a complementary metal-oxide semiconductor (CMOS) sensor, a CMOS digital camera, a single-photon avalanche diode (SPAD), a SPAD array, an avalanche photodetector (APD) array, a photomultiplier tube (PMT) array, a near-infrared (NIR) sensor, a red-green-blue (RGB) sensor, or a combination thereof.

24. The device of claim 1 , comprising a lens integral with the imaging sensor for capturing the image.

25. The device of claim 1 , wherein the imaging sensor is a multispectral camera configured to capture the light emitted by the target at multiple wavelengths.

26. 10. The device of claim 1, wherein the analytical model comprises an artificial neural network (ANN) model, a machine learning model (ML), or a combination thereof.

27. The device of claim 1 , wherein the processor is configured to detect time-dependent changes in fluorescence emanating from the target.

28. The device of claim 1 , wherein the fluorescence from the target is one of autofluorescence or extrinsic fluorescence.

29. 10. The device of claim 1, comprising a first set of excitation filters, each of the first set of excitation filters configured to filter the excitation radiation emitted by a light source of the first plurality of light sources in a predetermined range of wavelengths to pass through each of the first set of excitation filters and illuminate the target.

30. 1. A device for investigating a target, comprising: a first plurality of light sources, each configured to emit excitation radiation in a predetermined range of wavelengths that causes one or more markers in the target to fluoresce; a first plurality of optical bandpass filters, each optical bandpass filter configured to filter light emitted by the target in response to illumination of the target by at least one light source of the first plurality of light sources at a predetermined wavelength to pass through the respective optical bandpass filter; an imaging sensor configured to capture the filtered light filtered by an optical bandpass filter of the first plurality of optical bandpass filters and to capture a first plurality of images formed based on the filtered light; and a three-dimensional image capture sensor for illuminating the target and receiving light reflected by the target in response to the illumination of the target by the three-dimensional image capture sensor, and generating a three-dimensional image of the target based on the reflected light; an imaging module comprising: an interfacing module coupled to the imaging module, the interfacing module comprising: analyzing a first image of the first plurality of images using an analytical model, the first image being a fluorescence-based image comprising fluorescence from the target; analyzing the three-dimensional image of the target using the analytical model to determine variations in the intensity of the emitted light across the spatial region of the target by compensating for variations in distance across the spatial region of the target from the three-dimensional image capture sensor and variations in curvature across the spatial region of the target; using the analytical model to detect the presence of problematic cellular entities in the target based on the analysis of the first image and the three-dimensional image, the analytical model being trained to detect the presence of problematic cellular entities in the target; creating a composite image of the target using the first image and the three-dimensional image; a processor configured to: displaying the composite image and results corresponding to the detection of the problematic cellular entity. and an interface for

31. 31. The device of claim 30, wherein the analytical model is trained using a plurality of reference fluorescence-based images and a plurality of reference three-dimensional images to detect the presence of problematic cellular entities in a target, and the analytical model is trained to distinguish between fluorescence in the fluorescence-based images arising from the problematic cellular entities and fluorescence in the fluorescence-based images arising from areas of the target other than the problematic cellular entities.

32. 31. The device of claim 30, further comprising an absorptive filter wheel rotatably disposed within the imaging module and operably coupled to a servo motor, the absorptive filter wheel comprising the first plurality of optical bandpass filters.

33. the processor: actuating the servo motor to rotate the absorption filter wheel to position an optical bandpass filter of the first plurality of optical bandpass filters between the target and the imaging sensor; activating the first plurality of light sources to emit light at the target; 33. The device of claim 32, configured to operate the imaging sensor to capture light emitted by the target in response to illumination of the target by the at least one or more light sources of the first plurality of light sources.

34. a system on module (SOM), the SOM comprising: the imaging module; the processor; 31. The device of claim 30, comprising: a plurality of light source drivers, each of the plurality of light source drivers configured to control a light source of the first plurality of light sources.

35. 35. The device of claim 34, wherein one or more light sources of the first plurality of light sources is a pulsed light emitting diode (LED) configured to emit pulses of excitation radiation to enable faster imaging and reduce ambient light interference in the light emitted by the target.

36. 31. The device of claim 30, comprising a first set of excitation filters, each of the first set of excitation filters configured to filter the excitation radiation emitted by a light source of the first plurality of light sources in a predetermined range of wavelengths to pass through each of the first set of excitation filters and illuminate the target.

37. 1. A system for surveying a target, comprising: analyzing a first image of the first plurality of images using the analytical model, the first image being a fluorescence-based image comprising fluorescence emitted from the target; analyzing a three-dimensional image of the target using the analytical model to determine variations in emitted light intensity across the spatial region of the target by compensating for variations in distance across the spatial region of the target from a three-dimensional image capture sensor and variations in curvature across the spatial region of the target; using the analytical model to detect the presence of problematic cellular entities in the target based on the analysis of the first image and the three-dimensional image, the analytical model being trained to detect the presence of problematic cellular entities in the target; creating a composite image of the first image and the three-dimensional image of the target; transmitting a result corresponding to the detection of the problematic cellular entity and the composite image of the first image and the three-dimensional image of the target to a device.

12. A system comprising: a processor configured to:

38. a device, the device comprising: a first plurality of light sources, each of which emits excitation radiation in a predetermined range of wavelengths that causes one or more markers in the target to fluoresce; the imaging sensor configured to directly receive light emitted by the target in response to illumination of the target by one or more light sources of the first plurality of light sources without an optical bandpass filter disposed between the imaging sensor and the target, and to capture a first plurality of images formed based on the emitted light; a three-dimensional image capture sensor configured to illuminate the target, receive light reflected by the target in response to the illumination of the target by the three-dimensional image capture sensor, and generate the three-dimensional image of the target based on the reflected light; an imaging module comprising:

38. The system of claim 37, wherein the target is one of a wound area, food, laboratory equipment, medical equipment, sanitary equipment, sanitary equipment, biochemical assay chip, microfluidic chip, or bodily fluid, and the analytical model is trained using a plurality of reference fluorescence-based images and a plurality of reference three-dimensional images to detect the presence of problematic cellular entities in the target, and the analytical model is further trained to distinguish between fluorescence in the fluorescence-based images arising from the problematic cellular entities and fluorescence in the fluorescence-based images arising from areas other than the problematic cellular entities.

39. 1. A device for investigating a wound, comprising: a first plurality of light sources, each of the first plurality of light sources configured to emit excitation radiation in a predetermined range of wavelengths that causes one or more markers in the wound to fluoresce; a second plurality of light sources, each of the second plurality of light sources configured to emit excitation radiation in a predetermined range of wavelengths without causing the one or more markers in the wound to fluoresce; an imaging sensor configured to directly receive light emitted by the wound in response to illumination of the wound by at least one or more light sources of the first plurality of light sources and directly receive light reflected by at least one or more light sources of the second plurality of light sources, wherein no optical bandpass filter is disposed between the imaging sensor and the wound, and the imaging sensor is configured to capture a first plurality of images formed based on the light emitted by the wound and a second plurality of images formed based on the light reflected by the wound; and a three-dimensional image capture sensor for illuminating the wound, receiving light reflected by the wound in response to the illumination of the wound by the three-dimensional image capture sensor, and generating a three-dimensional image of the wound based on the reflected light. an imaging module comprising: an interfacing module coupled to the imaging module, the interfacing module comprising: analyzing a first image of the first plurality of images using an analytical model, the first image being a fluorescence-based image comprising fluorescence from the wound; analyzing a second image of the second plurality of images using the analytical model; analyzing the three-dimensional image of the wound using the analytical model to determine variations in the intensity of the reflected and emitted light across the spatial region of the wound by compensating for variations in distance across the spatial region of the wound from the three-dimensional image capture sensor and variations in curvature across the spatial region of the wound; detecting the presence of a biofilm in the wound based on the analysis of the first image, the second image, and the three-dimensional image using the analytical model, wherein the analytical model is trained to detect the presence of a biofilm in the wound; creating a composite image using the first image, the second image, and the three-dimensional image of the wound; a processor configured to: Displaying results corresponding to the detection of the biofilm in the wound and the composite image of the wound. and an interface configured to:

40. 1. A method for investigating a target, comprising: illuminating the target using at least one light source of a first plurality of light sources of a device, wherein the light emitted by each of the first plurality of light sources has a wavelength range; capturing, by an imaging sensor, a first plurality of images based on the light emitted by the target, the imaging sensor being configured to receive light emitted by the target in response to illumination of the target by the at least one or more light sources of the first plurality of light sources; capturing a three-dimensional image of the target with a three-dimensional image capture sensor, the three-dimensional image capture sensor configured to illuminate the target, receive light reflected by the target in response to the illumination of the target by the three-dimensional image capture sensor, and generate the three-dimensional image of the target based on the reflected light; analyzing, by a processor, a first image of the first plurality of images using an analytical model, the first image being a fluorescence-based image comprising fluorescence from the target in response to light emitted by the at least one or more light sources of the first plurality of light sources; analyzing, by the processor, the three-dimensional image of the target to determine variations in the intensity of the emitted light across the spatial region of the target by compensating for variations in distance across the spatial region of the target from the three-dimensional image capture sensor and variations in curvature across the spatial region of the target; detecting, by the processor, the presence of problematic cellular entities in the target using the analytical model based on the analysis of the first image and based on the three-dimensional image of the target, wherein the analytical model is trained to detect the presence of problematic cellular entities in the target; creating a composite image of the first image and the three-dimensional image of the target; and displaying, by an interface, a result corresponding to the presence of the problematic cellular entity and the composite image of the first image and the three-dimensional image.

41. 41. The method of claim 40, wherein the analytical model is trained using a plurality of reference fluorescence-based images and a plurality of reference three-dimensional images to detect the presence of problematic cellular entities in a target, and the analytical model is trained to distinguish between fluorescence in the fluorescence-based images arising from the problematic cellular entities and fluorescence in the fluorescence-based images arising from areas other than the problematic cellular entities.

42. wherein the target is a wound area, and the method comprises: extracting, by the processor, spatial and spectral features of the wound region from the first image and the three-dimensional image using the analytical model; locating, by the processor, the location of the wound area based on the extraction of the spatial and spectral features using the analytical model; determining, by the processor, a contour of the wound area based on the extraction of the spatial and spectral features using the analytical model; detecting, by the processor, pathogens in the wound area based on the extraction of the spatial and spectral features using the analytical model; and classifying, by the processor, the pathogen by at least one of a family, genus, species, or strain of the pathogen using the analytical model.

43. 43. The method of claim 42, comprising determining, by the processor, a length of the wound area, a width of the wound, a depth of the wound, a circumference of the wound, and / or an area of ​​the wound based on the determination of the contour of the wound area using the analytical model.

44. the target is one of a wound area, food, laboratory equipment, medical equipment, sanitary equipment, sanitary equipment, biochemical assay chips, microfluidic chips, or bodily fluids, and the method comprises: When the target is a wound area, determining, by the processor, in response to detecting the presence of the problematic cellular entity, at least one of the degree of infection of the wound area, the spatial distribution of pathogens in the wound area, or the rate of healing of the wound area; when the target is tissue, detecting, by the processor, the presence of the problematic cellular entity in the tissue sample as at least one of cancerous tissue or necrotic tissue; determining, by said processor, the problematic cellular entity as a pathogen when said target is one of a sanitary instrument, a laboratory instrument, a sanitary instrument, a biochemical assay chip, a medical instrument, a microfluidic chip, or a bodily fluid; and classifying, by said processor, the pathogen in said target.

41. The method of claim 40, comprising:

45. 41. The method of claim 40, comprising filtering, with an optical bandpass filter of a first plurality of optical bandpass filters, light emitted by the target in response to the illumination of the target by the at least one or more light sources of the first plurality of light sources of a predetermined wavelength to pass through the optical bandpass filter, wherein the optical bandpass filter is positioned between the target and the imaging sensor, and the imaging sensor is configured to capture the filtered light from the optical bandpass filter.

Images