Systems and methods for detecting cellular entities

Abstract

An apparatus for inspecting a target includes an imaging module and an interface module. The interface module includes a processor to analyze a first image of the plurality of first images using an analysis model, the first image being a fluorescence-based image including fluorescence from the target. The processor analyzes the three-dimensional image of the target using an analysis model to determine a change in intensity of light emitted by the spatial region of the target by compensating for a change in distance of the spatial region of the target from the three-dimensional image capture sensor and compensating for a change in curvature of the spatial region of the target. The processor detects the presence of problematic cellular entities in the target based on the analysis using the analysis model. The analytical model is trained to detect the presence of problematic cell entities in the target.

Description

Technical Field

[0001] The present subject matter generally relates to the detection of problematic cellular entities, such as pathogens, in a target, and specifically to systems and methods for detecting problematic cellular entities. Background Art

[0002] A cellular entity may be an entity made of one or more biological cells (such as a unicellular organism, a multicellular organism, a tissue, etc.). A problematic cellular entity may be a cellular entity that may cause damage to the health of plants, animals or humans. Problematic cellular entities are, for example, pathogens that cause human diseases and pathogens that delay wound healing. Problematic cellular entities may be indicators of diseases in plants, animals or humans. For example, cancerous tissue may be a problematic cellular entity, indicating the presence of a tumor. Detecting the presence of problematic cellular entities on targets such as humans, animals or plants can, for example, prevent the occurrence of disease, provide timely treatment to avoid death, etc. Similarly, detecting the presence of problematic cellular entities on targets such as edible products, sanitary equipment or laboratory equipment can determine whether edible products are contaminated, whether the surface of sanitary equipment or laboratory equipment is contaminated, to study the effectiveness of disinfectants on laboratory equipment, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The detailed description is provided with reference to the accompanying drawings. In the drawings, the leftmost digit(s) of a reference number identifies the drawing in which the reference number first appears. The same numerals are used throughout the drawings to refer to similar features and components.

[0004] Figure 1 A block diagram illustrating an apparatus for inspecting a target according to an embodiment of the present subject matter is shown;

[0005] Figure 2a shows a front perspective view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0006] Figure 2b shows a rear perspective view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0007] Figure 2c shows an exploded view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0008] Figure 3 A block diagram illustrating an apparatus for inspecting a target according to an embodiment of the present subject matter is shown;

[0009] Figure 4a shows a perspective view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0010] Figure 4bshows a perspective view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0011] Figure 4c shows an exploded view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0012] Figure 4d shows an exploded view of a portable power module of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0013] Figure 4e shows an exploded view of an interface module of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0014] Figure 5 A method for training an analytical model to detect problematic cellular entities in a target according to an embodiment of the present subject matter is shown;

[0015] Figure 6 An example of training an analytical model to detect problematic cellular entities in a target according to an embodiment of the present subject matter is shown;

[0016] Figure 7 A method of detecting problematic cellular entities according to an embodiment of the present subject matter is shown;

[0017] Figure 8 A method of detecting problematic cellular entities according to an embodiment of the present subject matter is shown;

[0018] Figure 9 A method of detecting problematic cellular entities according to an embodiment of the present subject matter is shown;

[0019] Figure 10 A method for an automatic exposure process according to an embodiment of the present subject matter is shown;

[0020] Figure 11 A method of detecting problematic cellular entities according to an embodiment of the present subject matter is shown;

[0021] Figure 12a shows a perspective view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0022] Figure 12b shows a perspective view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0023] Figure 12c shows a perspective view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0024] Figure 12dshows a top view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0025] Figure 12e shows a top view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0026] Figure 12f shows an exploded view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0027] Figure 12g shows a front view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0028] Figure 12h shows a top view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0029] Figure 12i shows a side view of an apparatus for inspecting a target according to an embodiment of the present subject matter;

[0030] Figure 13 An apparatus for inspecting a target according to an embodiment of the present subject matter is shown;

[0031] Figure 14 Detection of problematic cellular entities according to embodiments of the present subject matter is shown;

[0032] Figure 15 A system for inspecting a target according to an embodiment of the present subject matter is shown;

[0033] Figures 16a-16b A method for inspecting a target according to an embodiment of the present subject matter is shown;

[0034] Figure 17 shows results corresponding to detection of problematic cellular entities according to embodiments of the present subject matter;

[0035] Figure 18 shows results corresponding to detection of problematic cellular entities according to embodiments of the present subject matter;

[0036] Figure 19 shows results corresponding to detection of problematic cellular entities according to embodiments of the present subject matter;

[0037] Figure 20 shows results corresponding to tissue oxygenation saturation according to an embodiment of the present subject matter;

[0038] Figure 21 shows results corresponding to detecting a biofilm in a wound, according to an embodiment of the present subject matter;

[0039] Figure 22 shows results corresponding to detection of problematic cellular entities according to embodiments of the present subject matter;

[0040] Figure 23a shows results corresponding to detection of problematic cellular entities according to embodiments of the present subject matter;

[0041] Figure 23b shows results corresponding to detection of problematic cellular entities according to embodiments of the present subject matter;

[0042] Figure 24 shows results corresponding to detection of problematic cellular entities according to embodiments of the present subject matter;

[0043] Figure 25 shows results corresponding to detection of problematic cellular entities according to embodiments of the present subject matter;

[0044] Figure 26 showing results corresponding to detection of problematic cellular entities according to embodiments of the present subject matter; and

[0045] Figure 27 Results corresponding to the detection of problematic cellular entities according to embodiments of the present subject matter are shown. DETAILED DESCRIPTION

[0046] To accurately detect whether there is a problematic cell entity on the target. The target can be, for example, a wound area in the human body, an edible product, a tissue sample extracted from the human body, or a surface to be disinfected, such as a laboratory equipment surface, a medical device surface, a sanitary equipment surface, etc. Conventionally, a culture method is used to detect problematic cell entities, such as pathogens. In other words, in order to detect problematic cell entities, a swab or deep tissue biopsy is used to obtain a sample from an area expected to be infected with a pathogen. Subsequently, the obtained sample is stored in an appropriate culture medium, where the pathogen in the site is expected to grow over time. If a pathogen is present in the sample, a biochemical method is used to separate and identify the pathogen.

[0047] Similarly, for problematic cell entities, such as cancerous tissue, a tissue biopsy is performed. The tissue biopsy is examined under a microscope using staining methods (e.g., hematoxylin and eosin staining, mucin carmine staining, Papanicolaou staining, etc.) to identify whether the tissue is cancerous tissue. In some examples, the examination can be performed without staining methods. It should be understood that the above methods are complex, require specialized microbiology facilities, and may take 1-2 days to accurately identify the infection and classify the pathogen or cancerous tissue.

[0048] In some cases, detection and classification of problematic cellular entities is performed based on autofluorescence generated by natural biomarkers in the problematic cellular entity. Natural biomarkers can be, for example, nicotinamide adenine dinucleotide hydrogen phosphate (NAD(P)H), flavins, porphyrins, fluorescent siderophores (pyoverdine), tyrosine, and tryptophan. The autofluorescence generated by the biomarkers may be unique to each biomarker and may help detect and classify problematic cellular entities.

[0049] While autofluorescence can be used for detection and classification, the autofluorescence emitted by natural biomarkers is often weak and may not be easily detected. Furthermore, in addition to autofluorescence, the light emitted from the target can include background light and excitation light, which can interfere with the emitted autofluorescence. Consequently, detection and classification of problematic cellular entities using autofluorescence can be time-consuming, complex, and relatively inaccurate.

[0050] In addition, in some scenarios, the intensity of autofluorescence at different areas in the target, or the reflection intensity, and / or the emitted or reflected scattered intensity can be the same. For example, assume that the target (such as a wound) has pathogens scattered over the spatial area of the wound. In addition, assume that the first spatial area of the wound and the second spatial area of the wound are at different depths in the wound. In this regard, the intensity of the autofluorescence emitted by the first spatial area and the second spatial area can be the same. Therefore, when using a camera (such as a CMOS camera, a CCD camera, etc.) to capture autofluorescence, the intensity of the spatial area of the wound farther from the camera will be weaker than the intensity of the spatial area of the wound closer to the camera. For example, assume that the first spatial area of the wound is closer to the camera, and the second spatial area of the wound is farther away from the camera. In this regard, the autofluorescence emitted by the second spatial area is weaker than the autofluorescence emitted by the first spatial area, regardless of the presence of pathogens or their number.

[0051] In addition, due to the curvature of the target resulting in different reflections or scattering or autofluorescence, intensity variations in the spatial region of the target at the same distance from the camera may also occur. For example, assume that the target (such as a wound) has pathogens scattered over the spatial region of the wound. In addition, assume that the first spatial region of the wound and the second spatial region of the wound have the same pathogen and the same concentration of pathogen. In addition, assume that the first spatial region is planar and the second spatial region is curved. Since the pathogen and the concentration of the pathogen are the same, the camera should capture the same intensity of fluorescence, reflection and / or scattering. However, due to the curvature, the intensity of the fluorescence, reflection and / or scattering corresponding to the second spatial region may be different from the intensity of the fluorescence, reflection and / or scattering of the first spatial region.

[0052] Therefore, the detection of problematic cellular entities may be inaccurate and / or incorrect. Inaccurate detection of problematic cellular entities can hinder accurate diagnosis of disease, prevention of disease occurrence, provision of timely treatment to avoid death, etc. Similarly, inaccurate and / or incorrect detection of problematic cellular entities on targets (such as edible products, sanitary or laboratory equipment, bodily fluids (such as blood), medical devices (such as catheters)) can affect the determination of contamination in edible products, contamination on sanitary or laboratory equipment surfaces, etc.

[0053] The present subject matter relates to systems and methods for detecting problematic cellular entities. By implementing the present subject matter, the detection of problematic cellular entities (such as pathogens, cancerous tissue, necrotic tissue, etc.) can be rapid, accurate, simple, and cost-effective.

[0054] According to an embodiment, the device for inspecting a target may include an imaging module, an interface module, and a display. The target may be suspected of having problematic cellular entities, such as pathogens or cancerous tissue. In an example, the target may be made of one or more cells and may be, for example, a wound or tissue sample in a body part. In other examples, the target may be an item that does not contain pathogens, such as an edible product, laboratory equipment, or sanitary equipment. In some other examples, the target may be a body fluid suspected of possibly having pathogens, such as pus, blood, urine, saliva, sweat, semen, mucus, plasma, water, injectable fluids, etc.

[0055] The imaging module may include a plurality of first light sources, an imaging sensor, and a three-dimensional image capture sensor. Each of the plurality of first light sources emits excitation radiation within a predetermined wavelength range. In particular, the emitted excitation radiation may have a single wavelength or wavelength band that causes one or more markers in the target to fluoresce when illuminated. The plurality of first light sources may be, for example, homogeneous light sources or inhomogeneous light sources. In some examples, using inhomogeneous light sources can reduce or eliminate background light in the light emitted by the target.

[0056] One or more markers can be part of the problematic cell entity. The fluorescence emitted by the marker that is part of the problematic cell entity can be referred to as autofluorescence. In an example, an exogenous marker (such as a synthetic marker, such as indocyanine green (ICG) or methylene blue) can be sprayed on the target to detect the problematic cell entity in the target. The exogenous marker can be combined with cell entities such as deoxyribonucleic acid (DNA), ribonucleic acid (RNA), protein, blood, biochemical markers, etc., so that the target can emit fluorescence. The fluorescence emitted by the added synthetic marker can also be referred to as exogenous fluorescence.

[0057] In some examples, an imaging sensor can be configured to directly receive light emitted by a target in response to illumination thereof by at least one or more of a plurality of first light sources, without providing an optical bandpass filter between the imaging sensor and the target, and to capture a plurality of first images formed based on the emitted light. If the target includes a fluorescent marker, the captured image includes fluorescence and can be referred to as a fluorescence-based image. Thus, a fluorescence-based image can include fluorescence emitted from the target. Here, the light is said to be directly received by the imaging sensor because the emitted light is not filtered by the optical bandpass filter prior to image capture.

[0058] The imaging sensor may be a multispectral camera configured to capture multiple wavelengths of light emitted by a target. In particular, the multispectral camera may capture light emitted at wavelengths in the visible region, the ultraviolet (UV) region, the near infrared (NIR) region, or a combination thereof. In another example, the imaging sensor may be a charge coupled device (CCD) sensor, a CCD digital camera, a complementary metal oxide semiconductor (CMOS) sensor, a CMOS digital camera, a single photon avalanche diode (SPAD), a single photon avalanche diode (SPAD) array, an avalanche photodetector (APD) array, a photomultiplier tube (PMT) array, a near infrared (NIR) sensor, a red, green, and blue (RGB) sensor, or a combination thereof. In an example, the device may include one or more lenses that may be integrated with the imaging sensor to focus light onto the imaging sensor and capture an image.

[0059] A three-dimensional image capture sensor can illuminate a target and, in response to the illumination of the target by the three-dimensional image capture sensor, receive light reflected from the target to generate a three-dimensional image of the target based on the reflected light. Furthermore, the use of the three-dimensional image capture sensor can determine variations in the intensity of light reflected from the target within a spatial region of the target. This variation in intensity may need to account for differences in the distances of multiple regions within the spatial region of the target from the three-dimensional image capture sensor, as well as differences in curvature within the spatial region of the target. For example, a first spatial region of the target and a second spatial region of the target may have different distances relative to the three-dimensional image capture sensor. Therefore, the first and second spatial regions may emit fluorescent light of the same intensity. Because the fluorescent light from the first and second spatial regions has the same intensity, the fluorescent light from spatial regions farther away from the device may appear weaker than that from spatial regions closer to the device. For example, assume that the second spatial region is farther away from the device than the first spatial region. Therefore, the fluorescent light emitted by the second spatial region may appear weaker.

[0060] In addition, due to the curvature of the target causing different reflections or scattering or autofluorescence, intensity variations in the spatial region of the target at the same distance from the camera may also occur. For example, assume that the target (such as a wound) has pathogens scattered over the spatial region of the wound. In addition, assume that the first spatial region of the wound and the second spatial region of the wound have the same pathogen and the same concentration of pathogen. In addition, assume that the first spatial region is planar and the second spatial region is curved. Since the pathogen and the concentration of the pathogen are the same, the camera should capture the same intensity of fluorescence, reflection and / or scattering. However, due to the curvature, the intensity of the fluorescence, reflection and / or scattering corresponding to the second spatial region may be different from the intensity of the fluorescence, reflection and / or scattering of the first spatial region.

[0061] Thus, changes in the distance and curvature of the target relative to the device may need to be compensated in the light reflected by the target.In an example, the three-dimensional image capture sensor may be a structured light based sensor, a time of flight sensor, a stereo vision sensor, or a combination thereof.

[0062] The interface module can be coupled to the imaging module. The interface module can include a processor configured to analyze an image corresponding to the target. In particular, the processor can use an analysis model to analyze a first image from a plurality of first images. The plurality of first images can be fluorescence-based images including fluorescence emitted from the target. In addition, the processor can analyze the three-dimensional image of the target by compensating for changes in the distance of the spatial region of the target relative to the three-dimensional image capture sensor in the reflected light and compensating for changes in the curvature of the spatial region of the target. In this regard, the processor can determine changes in the intensity of light emitted from the spatial region of the target by compensating for changes in the distance of the spatial region of the target relative to the three-dimensional image capture sensor and by compensating for changes in the curvature of the spatial region of the target. The analysis model can be, for example, an artificial neural network model (ANN), a machine learning (ML) model, or a combination thereof. In an example, the ANN model can include a deep learning model such as a transformer model, a convolutional neural network (CNN), a generative adversarial network (GAN), an autoencoder-decoder network, a transformer model, or a combination thereof. The ML model can be, for example, a support vector machine (SVM) model or a random forest model, or a combination thereof.

[0063] The processor may use an analysis model to detect the presence of a problematic cellular entity in the object based on an analysis of the first image and the three-dimensional image. The analysis model may be trained to detect the presence of a problematic cellular entity in the object. Specifically, the analysis model may be trained using a plurality of fluorescence-based reference images to detect the presence of a problematic cellular entity in the object. The analysis model may be trained to distinguish between fluorescence emitted from the problematic cellular entity in the fluorescence-based image and fluorescence emitted from areas other than the problematic cellular entity in the fluorescence-based image.

[0064] In an example, in addition to being trained using multiple fluorescence-based reference images, an analysis model can also be trained using multiple reference three-dimensional images of a target to detect the presence of problematic cellular entities in the target. In this regard, the analysis model can be trained to distinguish between fluorescence emitted from problematic cellular entities in the fluorescence-based images and fluorescence emitted from areas other than the problematic cellular entities in the fluorescence-based images. Additionally, the analysis model can be trained by compensating for differences in the distance of the spatial region of the target relative to the three-dimensional image capture sensor and by compensating for changes in the curvature of the spatial region of the target by determining changes in the intensity of light emitted from the spatial region of the target. Changes in the intensity of light emitted at various locations in the spatial region of the target can be determined based on changes in the distance at various locations in the spatial region of the target relative to the three-dimensional image capture sensor, changes in the curvature at various locations in the spatial region of the target, and intensities measured at various locations in the spatial region of the target.

[0065] Furthermore, the processor may create a composite image of the first image and the three-dimensional image of the target using the analysis model. The interface may display a result corresponding to the detection of the problematic cellular entity and the composite image of the first image and the three-dimensional image of the target.

[0066] In an example, a device may include a system on module (SOM). The SOM may include an imaging module, an interface module, and a plurality of light source drivers. The plurality of light source drivers may include metal oxide semiconductor field effect transistors (MOSFETs), bipolar junction transistors (BJTs), phase locked loops (PLLs), or a combination thereof or any combination thereof, and the plurality of light source drivers may be configured to regulate corresponding light sources in the plurality of first light sources.

[0067] In an example, one or more of the multiple first light sources are pulsed light emitting diodes (LEDs). The processor can be configured to actuate one or more of the multiple light source drivers to adjust the pulsed LEDs to emit excitation radiation pulses. One or more light source drivers can be actuated by the processor to adjust the width and frequency of the pulsed LEDs to achieve faster imaging and reduce ambient light interference in the light emitted by the target. In an example, the pulse width can range from 100ns to 0.005ms, and the frequency of the pulsed LEDs can range from 100Hz to tens of MHz. Therefore, the present subject matter enables faster capture of multiple first images and three-dimensional images and reduces ambient light interference (background interference).

[0068] In an example, the processor may be configured to operate the imaging sensor and the three-dimensional image capture sensor to capture and process multiple first images and three-dimensional images at more than 30 frames per second. In this regard, the processor may include a central processing unit (CPU) and a graphics processing unit (GPU). In particular, the CPU and GPU may be part of the SOM. In other words, the CPU and GPU may be provided onboard. The CPU may operate the imaging sensor and the three-dimensional image capture sensor to capture multiple first images and three-dimensional images. In addition, the GPU may process images captured by the multiple first images and three-dimensional images. The provision of the GPU and the CPU, in particular the provision of the GPU and the CPU onboard, may enable faster processing and capture of multiple first images and three-dimensional images, reaching more than 30 frames per second.

[0069] In some examples, in addition to using fluorescence-based images and three-dimensional images to detect the presence of problematic cellular entities, the device can also detect the presence of problematic cellular entities based on oxygenation. In this regard, the device can include multiple second light sources for illuminating the target without causing markers in the target to fluoresce. Each of the multiple second light sources can be configured to emit light with a wavelength in the near-infrared (NIR) region or the visible region.

[0070] The imaging sensor can be configured to capture a plurality of second images formed based on light reflected by the target in response to illumination thereof by at least one or more of the plurality of second light sources. The processor can analyze the second images obtained from the plurality of second images using an analytical model to identify oxygenation at a plurality of regions in the target. The processor can analyze the three-dimensional image of the target using an analytical model to determine changes in the intensity of light reflected from a spatial region of the target by compensating for changes in the distance of the spatial region of the target from the three-dimensional image capture sensor and compensating for changes in the curvature of the spatial region of the target. The processor can use the analytical model to detect the presence of a problematic cellular entity in the target based on an analysis of a first image from the plurality of first images, a second image obtained from the plurality of second images, and a three-dimensional image. In this case, the processor can create a composite image of the first image, the second image, and the three-dimensional image of the target. The interface can display results corresponding to the detection of the problematic cellular entity and a composite image of the first image, the second image, and the three-dimensional image of the target.

[0071] In an example, in addition to the first image and the three-dimensional image of the target, the analysis model may also utilize white light images to detect problematic cellular entities. In this regard, in an example, at least one or more of the plurality of second light sources may be configured to emit light with a wavelength in the visible region. The imaging sensor may be configured to capture a plurality of third images formed based on light reflected from the target in response to illumination thereof by at least one or more of the plurality of second light sources. The plurality of third images are white light images. The processor may be configured to analyze a third image obtained from the plurality of third images using the analysis model. The processor may analyze the three-dimensional image of the target using the analysis model to determine variations in the intensity of light reflected from the spatial region of the target by compensating for variations in the distance of the spatial region of the target from the three-dimensional image capture sensor and for variations in the curvature of the spatial region of the target. The processor may be configured to use the analysis model to detect the presence of problematic cellular entities in the target based on an analysis of the first image, the third image, and the three-dimensional image. The processor may be configured to create a composite image of the target using the first image, the third image, and the three-dimensional image. The interface may be configured to display a result corresponding to the detection of the problematic cellular entity and a composite image of the first image, the third image, and the three-dimensional image of the target. It will be appreciated that in this case, multiple fluorescence-based reference images, multiple reference white light images, and multiple reference three-dimensional images may be used to train the analysis model to detect the presence of problematic cellular entities in the target.

[0072] The processor may be configured to activate the plurality of first light sources to emit light toward the target, and to activate the plurality of second light sources to emit light toward the target. Furthermore, the processor may be configured to activate the imaging sensor to capture light emitted by the target in response to illumination of the target by at least one or more of the plurality of first light sources, and to capture light emitted by the target in response to illumination of the target by at least one or more of the plurality of second light sources.

[0073] In an example, to reduce and / or eliminate the effect of background light in the captured image, the processor may be configured to control the plurality of first light sources and the plurality of second light sources to illuminate at a frequency different from a frequency of the ambient light source.

[0074] In an example, in addition to the detection of problematic cellular entities, the device can also classify the detected problematic cellular entities. Therefore, in an example, when the target is a wound area, the processor can be configured to extract spatial features and spectral features of the wound area from the first image and the three-dimensional image using an analysis model. In addition, the processor can identify the location of the wound area based on the extraction of spatial features and spectral features using an analysis model. The processor can determine the outline of the wound area based on the extraction of spatial features and spectral features using an analysis model. In an example, based on determining the outline of the wound area, the processor can be configured to determine the length of the wound area, the width of the wound, the perimeter of the wound, the area of the wound, the depth of the wound, or a combination thereof based on determining the outline of the wound area. In addition, the processor can detect pathogens in the wound area based on the extraction of spatial and spectral features using an analysis model. The processor can classify the pathogen by at least one of the family, genus, species, or strain of the pathogen using the analysis model.

[0075] In an example, in addition to the detection of problematic cellular entities, the device may also determine other parameters corresponding to the detected problematic cellular entities. For example, when the target is a wound area, the processor may be configured to determine the degree of infection of the wound area, the area of slough, the spatial distribution of pathogens in the wound area, the healing rate of the wound area, or a combination thereof in response to detecting the presence of the problematic cellular entity. When the target is tissue, the processor is configured to detect the presence of problematic cellular entities (such as cancerous tissue, necrotic tissue, or a combination thereof) in a tissue sample. When the target is a sanitary device, medical equipment, sanitary equipment, laboratory equipment, biochemical assay chip, microfluidic chip and / or body fluid, the processor may be configured to determine the presence of the problematic cellular entity as a pathogen and classify the pathogen in the target.

[0076] Furthermore, in addition to detecting problematic cellular entities, the processor can be configured to detect changes in fluorescence emitted from the target over time. In other words, the processor can be configured to detect changes in fluorescence between a first imaging of the target relative to subsequent imaging of the target. For example, the processor can be configured to detect changes in fluorescence between before debridement of a wound and after debridement of the wound. This detection can enable accurate removal of necrotic / unhealthy tissue from the wound. In another example, the processor can be configured to detect changes in fluorescence between an image of the wound taken on the first day and an image of the wound taken on a subsequent day. This detection can help determine wound healing and allow medical practitioners to administer medication based on the detection.

[0077] In an example, the device may be portable and may include a smartphone. The smartphone may include a processor and an imaging sensor. In an example, the device may include other components. In an example, the device may include a first set of excitation filters. Each of the first set of excitation filters may be configured to filter excitation radiation within a predetermined wavelength range emitted by a light source in the plurality of first light sources, allowing the excitation radiation within the predetermined wavelength range to pass through the excitation filter to illuminate the target. In addition, one or more excitation filters may be configured to filter excitation radiation within a predetermined wavelength range emitted by a light source in the plurality of second light sources, allowing the excitation radiation to pass through the excitation filter.

[0078] The device may include a thermal sensor for thermal imaging of a target. In this regard, the processor may be configured to use an analytical model to detect problematic cellular entities based on a first image from a plurality of first images, a second image obtained from a plurality of second images, a three-dimensional image, and a thermal image of the target. In this case, the processor may use the analytical model to create a composite image of the first image, the second image, the three-dimensional image, and the thermal image. Furthermore, the interface may display a result corresponding to the detection of the problematic cellular entity and a composite image of the first image, the second image, the three-dimensional image, and the thermal image of the target.

[0079] The device may include a ranging sensor operable to determine a distance from an object to the device, for positioning the device at a predetermined distance from the object. In an example, a three-dimensional image capture sensor may be used as the ranging sensor. For example, the three-dimensional image capture sensor may be used to determine the distance from the object to the device, for positioning the device at a predetermined distance from the object.

[0080] The device may include multiple polarizers. For example, the device may include a first polarizer located between the multiple first light sources and the target so that excitation radiation from the multiple first light sources of a first polarization passes through the first polarizer. The device may include a second polarizer located between the target and the imaging sensor so that light emitted by the target of a second polarization passes through the second polarizer. In an example, the first polarization and the second polarization may be the same. In another example, the first polarization and the second polarization may be different. In an example, the first polarization and the second polarization may be the same. For example, in an example, the first polarization and the second polarization may be left-handed circular polarization (LHCP). In another example, the first polarization and the second polarization may be right-handed circular polarization (RHCP). In another example, the first polarization and the second polarization may be different. For example, the first polarization may be one of LHCP or RHCP, and the second polarization may be the other of LHCP or RHCP. Multiple polarizers may be combined with the first set of excitation filters.

[0081] The device may include a housing for accommodating components. In particular, the device may include a first housing, a second housing, and a bridge. The first housing may accommodate an imaging module, and the second housing may accommodate an interface module. The bridge may connect the imaging module and the interface module. The bridge may include an electronic interface to enable electronic communication between the processor of the interface module and the imaging module. The electronic interface may include a camera serial interface (CSI), a serial management bus (such as an I2C interface), a system packet interface (SPI), a universal asynchronous receiver-transmitter (UART), a general-purpose input / output (GPIO) interface, a universal serial bus (USB) interface, a pulse width modulation (PWM) interface, a display-serial interface (DSI), a high-definition multimedia interface (HDMI), or a combination thereof.

[0082] The device may include a portable power module operable to supply power to components of the device, such as the imaging module and the interface module.The third housing may house the portable power module.

[0083] In an example, the device can transmit the results to a remote system, such as a cloud server. For example, the processor can be configured to transmit the results and a composite image of the first image, the three-dimensional image, to a remote system, such as a cloud server. The remote system can be in electronic communication with the device. Because the device can transmit the results and the composite image to the cloud server, a non-medical professional or a medical professional can transmit the image or image sequence to a remotely located medical professional for additional consultation prior to treatment using the device(s) of the present disclosure.

[0084] The interface can be configured to receive input corresponding to the operation of the device from the user using an application programming interface (API). For example, using the API, the user can select one or more of the plurality of first light sources and one or more of the plurality of second light sources to illuminate a target. In addition, the user can select the frequency of light emission of the plurality of first light sources and the plurality of second light sources.

[0085] The interface can be configured to transmit, in response to input, results corresponding to the detection and classification of pathogens in the target when the API is used to detect and classify pathogens. In this regard, the interface can allow a user to store and analyze the results corresponding to the detection and classification of pathogens in the target. Furthermore, the interface can allow a user to select a composite image to be obtained, and can transmit the results to a remote system or remote server, and can also allow a user to select various views of the composite image.

[0086] In an example, the processor can be configured to detect changes in fluorescence emitted from a target over time. In other words, the processor can be configured to detect changes in fluorescence between a first imaging of the target relative to subsequent imaging of the target. For example, the processor can be configured to detect changes in fluorescence between before and after debridement of a wound. This detection can enable accurate removal of necrotic / unhealthy tissue from the wound. In another example, the processor can be configured to detect changes in fluorescence between an image of the wound taken on the first day and an image of the wound taken on a subsequent day. This detection can help determine wound healing and allow medical practitioners to administer medication based on the detection.

[0087] In the foregoing examples, the apparatus was explained without providing an optical bandpass filter for filtering light emitted by the target. However, in some examples, one or more optical bandpass filters, such as emission filters, may be used.

[0088] Thus, in an example, an apparatus for inspecting a target may include an imaging module, an interface module, and an interface. The imaging module may include a plurality of first light sources, a plurality of first optical bandpass filters, an imaging sensor, and a three-dimensional image capture sensor. Each of the plurality of first light sources may be configured to emit excitation radiation in a predetermined wavelength range to cause one or more markers in the target to fluoresce. In an example, each of the plurality of first light sources may be an LED. In another example, one or more of the plurality of first light sources may be a pulsed light emitting diode (LED) to emit pulses of excitation radiation to achieve faster imaging and reduce ambient light interference in the light emitted by the target. The plurality of first light sources may be, for example, a homogeneous light source or an inhomogeneous light source.

[0089] Each of the plurality of first optical bandpass filters may be configured to filter light of a predetermined wavelength emitted by an object in response to illumination by at least one or more of the plurality of first light sources, thereby allowing the light of the predetermined wavelength to pass therethrough. An imaging sensor may capture the filtered light filtered by the optical bandpass filters of the plurality of first optical bandpass filters and capture a plurality of first images formed based on the filtered light. In an example, the device may include one or more lenses integrated with the imaging sensor to focus light onto the imaging sensor and capture the image.

[0090] The 3D image capture sensor may illuminate a target, and in response to the 3D image capture sensor illuminating the target, may receive light reflected from the target, and may generate a 3D image of the target based on the reflected light. In an example, the 3D image capture sensor may be a structured light sensor, a time-of-flight sensor, a stereo sensor, or a combination thereof.

[0091] The interface module can be coupled to the imaging module. The interface module can include a processor. The processor can be configured to analyze a first image of the plurality of first images using an analysis model. The first image can be a fluorescence-based image including fluorescence emitted from a target. The processor, using the analysis model, can analyze the three-dimensional image of the target to determine a change in the intensity of light emitted from the spatial region of the target by compensating for changes in the distance of the spatial region of the target from the three-dimensional image capture sensor and compensating for changes in the curvature of the spatial region of the target. The processor can, using the analysis model, detect the presence of a problematic cellular entity in the target based on the analysis of the first image and the three-dimensional image. The analysis model can be trained to detect the presence of a problematic cellular entity in the target.

[0092] The analysis model is trained to detect the presence of problematic cellular entities in the target. In particular, the analysis model can be trained using a plurality of fluorescence-based reference images to detect the presence of problematic cellular entities in the target. The analysis model can be trained to distinguish between fluorescence emitted from the problematic cellular entity in the fluorescence-based image and fluorescence emitted from areas other than the problematic cellular entity in the fluorescence-based image.

[0093] In an example, in addition to being trained using multiple fluorescence-based reference images, an analysis model can also be trained using multiple reference three-dimensional images of a target to detect the presence of problematic cellular entities in the target. In this regard, the analysis model can be trained to distinguish between fluorescence emitted from problematic cellular entities in the fluorescence-based images and fluorescence emitted from areas other than the problematic cellular entities in the fluorescence-based images. Additionally, the analysis model can be trained by compensating for differences in the distance of the spatial region of the target relative to the three-dimensional image capture sensor and by compensating for differences in the curvature of the spatial region of the target by determining changes in the intensity of light emitted from the spatial region of the target. Changes in the intensity of light emitted at various locations in the spatial region of the target can be determined based on changes in the distance at various locations in the spatial region of the target relative to the three-dimensional image capture sensor, changes in the curvature at various locations in the spatial region of the target, and intensities measured at various locations in the spatial region of the target.

[0094] The processor may create a composite image of the first image and the three-dimensional image of the target. The interface may display a result corresponding to the detection of the problematic cellular entity and the composite image of the first image and the three-dimensional image of the target.

[0095] In an example, a device may include a first set of excitation filters. Each of the first set of excitation filters may be configured to filter excitation radiation within a predetermined wavelength range emitted by a light source in the plurality of first light sources, allowing the excitation radiation to pass through the excitation filter to illuminate the target. Furthermore, one or more excitation filters may be further configured to filter excitation radiation within a predetermined wavelength range emitted by a light source in the plurality of second light sources, allowing the excitation radiation to pass through the excitation filter.

[0096] In an example, a device may include a system on module (SOM). The SOM may include an imaging module, an interface module, and a plurality of light source drivers. The plurality of light source drivers may be configured to adjust corresponding light sources in the plurality of first light sources.

[0097] The processor may be further configured to activate the plurality of first light sources to emit light toward the target, and activate the imaging sensor in response to illumination of the target by at least one or more of the plurality of first light sources to capture light emitted by the target.

[0098] In an example, the apparatus may include an emission filter wheel rotatably disposed within the imaging module. The emission filter wheel may be operably coupled to a servo motor. The emission filter wheel may include a plurality of first optical bandpass filters. It should be understood that based on a desired optical bandpass filter from the plurality of first optical bandpass filters, the servo motor may be actuated to position the desired optical bandpass filter between the target and the imaging sensor. In this regard, the processor may be configured to activate the servo motor to rotate the emission filter wheel, thereby positioning one of the plurality of first optical bandpass filters between the target and the imaging sensor.

[0099] In the above examples, the capture of images and the processing of the device are explained with reference to a single device. In some examples, the capture and processing of images can be performed by different components. Thus, in an example, a system for inspecting a target may include a processor. The processor may analyze a first image of a plurality of first images using an analysis model. The plurality of first images may be fluorescence-based images that include fluorescence emitted from the target. The processor may be configured to analyze the three-dimensional image of the target using the analysis model to determine a change in the intensity of light emitted from the spatial region of the target by compensating for changes in the distance of the spatial region of the target from the three-dimensional image capture sensor and compensating for changes in the curvature of the spatial region of the target.

[0100] The processor may, using the analysis model, detect the presence of a problematic cellular entity in the target based on analysis of the first image and the three-dimensional image. The analysis model may be trained to detect the presence of the problematic cellular entity in the target. The processor may create a composite image of the first image and the three-dimensional image of the target. The processor may transmit a result corresponding to the detection of the problematic cellular entity and the composite image of the first image and the three-dimensional image of the target to a device.

[0101] The system may include the device. The device may include an imaging module, the imaging module including a plurality of first light sources, an imaging sensor, and a three-dimensional image capture sensor. Each of the plurality of first light sources may emit excitation radiation of a predetermined wavelength range to cause one or more markers in the target to emit fluorescence. The imaging sensor may be configured to directly receive light emitted by the target in response to illumination thereof by one or more of the plurality of first light sources, without providing an optical bandpass filter between the imaging sensor and the target, and to capture a plurality of first images formed based on the emitted light. Here, the light is said to be directly received by the imaging sensor because the emitted light is not filtered by the optical bandpass filter before capturing the image.

[0102] A three-dimensional image capture sensor can illuminate a target to receive light reflected by the target, and in response to the three-dimensional image capture sensor illuminating the target, can generate a three-dimensional image of the target based on the reflected light. In an example, the three-dimensional image capture sensor can be a structured light sensor, a time-of-flight sensor, a stereo sensor, or a combination thereof. The target can be a wound area, an edible product, laboratory equipment, a medical device, a bodily fluid, a sanitary device, a sanitary device, a biochemical assay chip, a microfluidic chip, or a combination thereof. An analysis model can be trained using multiple fluorescence-based reference images and multiple reference three-dimensional images to detect the presence of a problematic cellular entity in the target. The analysis model can be trained to distinguish between fluorescence emitted from the problematic cellular entity in the fluorescence-based image and fluorescence emitted from areas other than the problematic cellular entity in the fluorescence-based image.

[0103] In some examples, when the target is a wound, the present subject matter can detect biofilms in the wound. In this regard, a device for inspecting a wound may include an imaging module, an interface module, and an interface. The imaging module may include a plurality of first light sources, a plurality of second light sources, an imaging sensor, and a three-dimensional image capture sensor. Each of the plurality of first light sources may emit excitation radiation of a predetermined wavelength range to cause one or more markers in the wound to fluoresce. The plurality of first light sources may be, for example, homogeneous light sources or inhomogeneous light sources.

[0104] Each of the plurality of second light sources can emit excitation radiation within a predetermined wavelength range without causing markers in the wound to fluoresce. The imaging sensor can directly receive light emitted by the wound in response to illumination thereof by at least one or more of the plurality of first light sources, and directly receive light reflected by at least one or more of the plurality of second light sources, without providing an optical bandpass filter between the imaging sensor and the wound. The imaging sensor can capture a plurality of first images based on light emitted by the wound, and can capture a plurality of second images based on light reflected from the wound. Here, the light is said to be directly received by the imaging sensor because the emitted light and the reflected light are not filtered by the optical bandpass filter before the image is captured.

[0105] The three-dimensional image capture sensor can illuminate the wound, and in response to the three-dimensional image capture sensor illuminating the wound, light reflected from the wound can be received, and a three-dimensional image of the wound can be generated based on the reflected light. In an example, the three-dimensional image capture sensor can be a structured light sensor, a time-of-flight sensor, a stereo sensor, or a combination thereof.

[0106] The interface module can be coupled to the imaging module. The interface module can include a processor. The processor can be configured to analyze a first image from a plurality of first images using an analysis model, wherein the first image is a fluorescence-based image that includes fluorescence emitted from the wound. The processor can analyze a second image obtained from the plurality of second images using the analysis model. In addition, the processor can analyze a three-dimensional image of the wound using the analysis model to determine variations in the intensity of emitted light and the intensity of reflected light across the spatial region of the wound by compensating for variations in the distance from the three-dimensional image capture sensor across the spatial region of the wound and compensating for variations in the curvature across the spatial region of the wound.

[0107] In this regard, the processor may use an analysis model to detect the presence of a biofilm in a wound based on an analysis of the first image, the second image, and the three-dimensional image. The analysis model may be trained to detect the presence of a biofilm in a wound. The analysis model may create a composite image of the first image, the second image, and the three-dimensional image of the wound. The interface may display a result corresponding to the detection of the biofilm in the wound and the composite image of the first image, the second image, and the three-dimensional image of the wound.

[0108] In an example, the apparatus may include a first set of excitation filters. Each of the first set of excitation filters may be configured to filter excitation radiation of a predetermined wavelength range emitted by a light source of the plurality of first light sources, allowing the excitation radiation to pass through the excitation filter to illuminate the target.

[0109] The present invention is capable of providing faster image capture and processing to detect problematic cellular entities. Because the processor and imaging module are provided onboard in the present invention, the present invention is capable of capturing and processing images faster. In particular, by using a combination of a CPU and a GPU, the present invention is capable of capturing and processing images at a frequency of more than 30 images per second. An analysis model is trained on several fluorescence-based reference images and several reference three-dimensional images to detect the presence of problematic cellular entities in the target, thereby improving the accuracy of detection. The present invention ensures that the light emission of the light source is at a different frequency from the ambient light source. Therefore, the present invention is capable of eliminating the interference of ambient light on the light emitted by the target. In addition, in the present invention, the pulsed LED can be actuated with a shorter pulse width (such as from 100ns to 0.005ms) and a faster frequency (such as from 100Hz to tens of MHz). Therefore, the present invention enables faster capture of multiple first images and three-dimensional images and reduces ambient light interference (background interference). Therefore, the present invention eliminates background information and improves the accuracy of detection.

[0110] Furthermore, in the example, the analysis model can ignore background light and excitation light in the fluorescence-based image and can even pick up weak fluorescence information in the fluorescence-based image. Thus, in the example, the present subject matter also eliminates the use of emission filters for filtering background light and excitation light, as well as the use of filter wheels. Therefore, the present subject matter apparatus is simple and cost-effective.

[0111] In the present subject matter, changes in distance between the imaging sensor and multiple regions of the spatial region of the target, as well as changes in curvature of the multiple regions of the spatial region of the target, are determined by a three-dimensional image capture sensor. Thus, the present subject matter can improve the accuracy of detecting problematic cellular entities, particularly for targets such as wounds. Because the device can transmit the resulting, composite images to a cloud server, non-medical professionals or medical professionals can transmit images or image sequences to a remotely located medical professional for additional consultation prior to treatment using the disclosed device.

[0112] Thus, the present subject matter provides rapid, optionally filter-free, non-invasive, automated, and in situ detection and classification of pathogens using "optical computational biopsy" technology, which uses multispectral imaging along with computational models (such as machine learning models, artificial neural network (ANN) models, deep learning models, etc.) for non-invasive biopsy to detect and classify problematic cellular entities.

[0113] The subject matter can be used to detect the presence of problematic cellular entities in diabetic foot ulcers, surgical site infections, burns, skin, and the interior of the body (such as the esophagus, stomach, and colon). The subject matter device can be used in the fields of dermatology, cosmetology, plastic surgery, infection management, photodynamic therapy monitoring, and antimicrobial susceptibility testing.

[0114] In addition, the device can be used to detect changes in fluorescence over time to understand the colonization of pathogens and necrotic tissue. In other words, the processor can be configured to detect changes in fluorescence between the first imaging of the target relative to subsequent imaging of the target. For example, the processor can be configured to detect changes in fluorescence between before debridement of a wound and after debridement of the wound. This detection can enable the accurate removal of necrotic / unhealthy tissue from the wound. In another example, the processor can be configured to detect changes in fluorescence between an image of the wound taken on the first day and an image of the wound taken on the subsequent day. This detection can help determine the healing of the wound and allow medical practitioners to administer medication based on the detection.

[0115] The device can be integrated into normal clinical procedures and can be used for telemedicine and telehealthcare. In addition, most clinically relevant pathogens can be detected and classified within minutes. In addition, data acquisition and analysis can occur automatically. Therefore, the device can be easily operated without the need for skilled technicians. This feature helps to quickly decide on treatment options. The device can also be used to detect and classify pathogens in resource-scarce settings. The device of the present subject matter can also be used for endoscopy. For example, the imaging module of the present subject matter can be incorporated into the imaging unit of an endoscopic device.

[0116] The device of the present subject can be used for quantifying various pathogens present in a sample. The device can also be used to monitor wound healing and wound closure. The device can also be used to study antimicrobial sensitivity by exposing the target to various antibiotics, observing and analyzing the target. For example, the device can be used to study bacteria grown with nutrients and antibiotics, and the corresponding biomarker characteristics can be recorded. This information can be used to obtain information about antibiotics to be prescribed based on the antimicrobial sensitivity of specific bacteria. It should be understood that the antimicrobial sensitivity of other pathogens (such as fungi) can also be studied. In addition, the dosage and concentration of the antibiotic can also be determined based on the dilution factor to determine the dosage of the antibiotic or antifungal agent to be administered.

[0117] The device can be configured to study the biomolecular composition and kinetic behavior of various pathogens based on their fluorescence signatures. The device can also be used for cosmetic purposes. For example, the device can be used to detect the presence of Propionibacterium acnes, which causes acne. The device can also be used during tissue transplants to ensure that the tissue is free of pathogens. The device can be used for forensic testing, for example, to detect pathogens in body fluids (such as saliva, blood, mucus, etc.). The device can be configured to study the effectiveness of disinfectants on various hospital surfaces (such as beds, walls, hands, gloves, bandages, dressings, catheters, endoscopes, hospital settings, sanitary equipment, etc.).

[0118] The device can also be used to detect the presence of pathogens on hands and surfaces, for example, in hospitals and other places that need to be pathogen-free. The device can be used to detect pathogen contamination in edible products such as food, fruits and vegetables.

[0119] Figure 1 A block diagram of a device 100 for inspecting a target 101 according to an embodiment of the present invention is shown. The device 100 for inspecting a target may include an imaging module 102, an interface module 104, and an interface 108. The target 101 may be suspected of having a problematic cellular entity, such as a pathogen or cancerous tissue. In an example, the target 101 may be made of one or more cells and may be, for example, a wound or tissue sample in a body part. In other examples, the target 101 may be an item that does not contain pathogens, such as an edible product, laboratory equipment, or sanitary equipment. In some other examples, the target 101 may be pus, blood, urine, saliva, sweat, semen, mucus, plasma, water, etc., which may be suspected of having a pathogen.

[0120] Imaging module 102 may include a plurality of first light sources 130, an imaging sensor 122, and a three-dimensional image capture sensor 120. Each of the plurality of first light sources 130 emits excitation radiation at a predetermined wavelength range. In particular, the emitted excitation radiation may have a single wavelength or wavelength band that causes one or more markers in the target to fluoresce when illuminated. In examples, the wavelength band of light used to cause target 101 to fluoresce may include 200 nm-300 nm, 300 nm-400 nm, 400 nm-500 nm, or 500 nm-600 nm. In specific examples, the wavelength of light used to cause target 101 to fluoresce may include 280 nm, 310 nm, 330 nm, 365 nm, 395 nm, 405 nm, 415 nm, 430 nm, 480 nm, and 520 nm. In examples, the wavelength band of light may include 600 nm-700 nm, 700 nm-800 nm, or 800 nm-1000 nm. In certain examples, the wavelengths of light used to cause target 101 to fluoresce may also include 430 nm, 630 nm, 660 nm, 680 nm, 735 nm, 830 nm, 880 nm, 940 nm, and 970 nm.

[0121] The plurality of first light sources 130 may be, for example, homogeneous light sources or inhomogeneous light sources. In an example, using an inhomogeneous light source may reduce or eliminate background light in the light emitted by the target.

[0122] One or more markers can be part of the problematic cell entity. The fluorescence emitted by the marker that is part of the problematic cell entity can be referred to as autofluorescence. In an example, an exogenous marker (such as a synthetic marker) can be sprayed on the target 101 to start the detection of the problematic cell entity in the target 101. The exogenous marker can be combined with cell entities such as deoxyribonucleic acid (DNA), ribonucleic acid (RNA), protein, biochemical markers, etc., so that the target 101 can emit fluorescence. The fluorescence emitted by the added synthetic marker can also be referred to as exogenous fluorescence.

[0123] In an example, imaging sensor 122 can be configured to directly receive light emitted by target 101 in response to illumination thereof by at least one or more of plurality of first light sources 130, without interposing an optical bandpass filter between imaging sensor 122 and target 101, and to capture a plurality of first images formed based on the emitted light. If target 101 includes a fluorescent marker, the captured images include fluorescence and can be referred to as fluorescence-based images. Thus, the fluorescence-based images can include fluorescence emitted from target 101. Here, the light is said to be directly received by imaging sensor 122 because the emitted light is not filtered by an optical bandpass filter prior to image capture.

[0124] The imaging sensor 122 may be a multispectral camera configured to capture multiple wavelengths of light emitted by the target 101. In particular, the multispectral camera may capture light emitted at wavelengths in the visible region, the ultraviolet (UV) region, the near infrared (NIR) region, or a combination thereof. In another example, the imaging sensor 122 may be a charge coupled device (CCD) sensor, a CCD digital camera, a complementary metal oxide semiconductor (CMOS) sensor, a CMOS digital camera, a single photon avalanche diode (SPAD), a single photon avalanche diode (SPAD) array, an avalanche photodetector (APD) array, a photomultiplier tube (PMT) array, a near infrared (NIR) sensor, a red, green, and blue (RGB) sensor, a thermal camera, or a combination thereof. In an example, one or more lenses ( Figure 1 ) may be integrated with the imaging sensor 122 to focus light onto the imaging sensor 122 and capture an image.

[0125] The three-dimensional image capture sensor 120 may illuminate the target 101, and may receive light reflected by the target 101 in response to the illumination of the target 101 by the three-dimensional image capture sensor 120, and may generate a three-dimensional image of the target 101 based on the reflected light. To illuminate the target 101, the three-dimensional image capture sensor 120 may include one or more light sources ( Figure 1 ). However, in some examples, a separate light source may also be coupled to three-dimensional image capture sensor 120 to illuminate target 101 and enable light reflected by target 101 to be captured as a result of the illumination.

[0126] Furthermore, the use of three-dimensional image capture sensor 120 can determine how the intensity of light emitted by target 101 varies across a spatial region of target 101. This intensity variation may need to be accounted for due to differences in the distances of various regions distributed across the spatial region of target 101 from three-dimensional image capture sensor 120. For example, a first spatial region of target 101 may be located at a different distance from three-dimensional image capture sensor 120 than a second spatial region of target 101. Consequently, the first and second spatial regions may emit fluorescent light of the same intensity. Since the fluorescent light from the first and second spatial regions has the same intensity, the fluorescent light from spatial regions farther from device 100 may appear weaker than that from spatial regions closer to device 100. For example, assume that the second spatial region is farther from device 100 than the first spatial region. Consequently, the fluorescent light emitted by the second spatial region may appear weaker. Therefore, it may be necessary to compensate for the variations in distance across the spatial region of target 101 relative to device 100 in the light emitted by target 101. In this example, different spatial regions across target 101 have different curvatures. Thus, even at the same distance from imaging sensor 122, fluorescent light emitted from different spatial regions of the target may differ. Therefore, variations in curvature across spatial regions of target 101 relative to device 100 may have to be compensated for in the light emitted by target 101. In an example, the three-dimensional image capture sensor may be a structured light sensor, a time-of-flight sensor, a stereo sensor, or a combination thereof.

[0127] The interface module 104 may be coupled to the imaging module. The interface module 104 may include a processor 140. The processor 140 may be implemented as a microprocessor, a microcomputer, a microcontroller, a digital signal processor, a central processing unit, a combination of a central processing unit and a graphics processing unit, a state machine, a logic circuit, and / or any device that can manipulate signals based on operational instructions. Among other functions, the processor 140 may retrieve and execute data stored in the memory ( Figure 1 ) in computer-readable instructions.

[0128] Processor 140 is configured to analyze an image corresponding to target 101. Specifically, processor 140 may use an analysis model to analyze a first image from a plurality of first images. The plurality of first images may be fluorescence-based images including fluorescence emitted from target 101. Furthermore, processor 140 may analyze the three-dimensional image of target 101 by compensating for variations in the distance of a spatial region of target 101 relative to three-dimensional image capture sensor 120 in light emitted by target 101. In this regard, processor 140 may determine variations in the intensity of light emitted from a spatial region of target 101 by compensating for variations in the distance of the spatial region of target 101 relative to three-dimensional image capture sensor 120 and by compensating for variations in the distance of the spatial region of target 101 relative to three-dimensional image capture sensor 120. The analysis model may be, for example, an artificial neural network model (ANN), a machine learning (ML) model, or a combination thereof. In some examples, the ANN model may include a deep learning model such as a transformer model, a convolutional neural network (CNN), a generative adversarial network (GAN), an autoencoder-decoder network, or a combination thereof. The ML model may be, for example, a support vector machine (SVM) model or a random forest model, or a combination thereof.

[0129] Processor 140 can use the analysis model to detect the presence of problematic cellular entities in target 101 based on analysis of the first image and the three-dimensional image. The analysis model is trained to detect the presence of problematic cellular entities in the target. Specifically, the analysis model can be trained using a plurality of fluorescence-based reference images to detect the presence of problematic cellular entities in the target. The analysis model can be trained to distinguish between fluorescence emitted from problematic cellular entities in the fluorescence-based images and fluorescence emitted from areas other than the problematic cellular entities in the fluorescence-based images.

[0130] In an example, in addition to being trained using multiple fluorescence-based reference images, the analysis model can also be trained using multiple reference three-dimensional images of the target to detect the presence of problematic cellular entities in the target. In this regard, the analysis model can be trained to distinguish between fluorescence emitted from the problematic cellular entity in the fluorescence-based image and fluorescence emitted from areas other than the problematic cellular entity in the fluorescence-based image. Additionally, the analysis model can be trained to compensate for differences in distances at various locations in the spatial region of the target 101 relative to the three-dimensional image capture sensor 120 by determining variations in the intensity of light emitted at various locations in the spatial region of the target 101. Variations in the intensity of light emitted at various locations in the spatial region of the target 101 can be determined based on variations in the distance and curvature at various locations in the spatial region of the target relative to the three-dimensional image capture sensor 120 and based on the intensities measured at various locations in the spatial region of the target 101. Reference Figure 5 and Figure 6 Explain the training of analytical models.

[0131] In addition, the processor 140 can use the analysis model to create a composite image of the first image and the three-dimensional image of the target 101. The interface 108 can display the results corresponding to the detection of the problematic cell entity and the composite image of the first image and the three-dimensional image of the target 101. Figure 7-11 Explain the detection of the presence of problematic cellular entities.

[0132] In an example, the device 100 may include a system on module (SOM). The SOM may include an imaging module 102, an interface module 104, and multiple light source drivers 150. The multiple light source drivers 150 may be configured to adjust corresponding light sources in the multiple first light sources 130. In other words, the device 100 also uses an integrated circuit board that typically contains a SOM. The SOM includes the imaging module 102, the interface module 104, and multiple light source drivers 150. The multiple light source drivers 150 may include metal oxide semiconductor field effect transistors (MOSFETS), bipolar junction transistors (BJTs), phase-locked loops (PLLs), or a combination thereof. In an example, the processor 140 may include a central processing unit (CPU) and a graphics processing unit (GPU). The SOM may also include a field programmable gate array (FPGA) module. In addition, the SOM may include a battery charging module. The SOM may include an integrated circuit (also referred to as a "chip") that integrates all or most components of a computer or other electronic system. These components almost always include a processor 140, memory interfaces, on-chip input / output devices, input / output interfaces, and auxiliary storage interfaces, often along with other components such as a modem, including a radio modem, all on a single substrate or microchip. The SOM 106 may contain digital, analog, mixed-signal, and often radio frequency signal processing functionality (otherwise it is considered just an application processor). Alternatively, the device 100 may include a system on a chip (SOC) rather than a SOM. The SOC may be similar to the SOM.

[0133] The SOM 106 may also include a GPU or FPGA, or a combination thereof, which allows for faster capture, processing, and therefore image capture by the imaging sensor 122. Having an onboard FPGA or GPU uniquely allows the imaging sensor 122 to capture images at up to 100 frames per second, and more typically at greater than 30 frames per second, and most typically at greater than 40 frames per second and up to 100 frames per second. The ability to pulse and capture and process images at such faster rates reduces background noise and enables accurate extraction of fluorescence / oxygenation information. The device 100, employing such faster pulses, eliminates any need for protective covers or shielding to eliminate or reduce ambient light. The device 100 can even be used outdoors as well as indoors while still obtaining accurate scans of wounds or other target image surfaces. Thus, the device 100 is not shielded from ambient light.

[0134] In an example, one or more of the plurality of first light sources 130 are pulsed light emitting diodes (LEDs). The processor 140 can be configured to actuate one or more of the plurality of light source drivers to adjust the pulsed LEDs to emit excitation radiation pulses. The one or more light source drivers can be actuated by the processor 140 to adjust the pulsed LEDs at shorter pulse widths and faster frequencies to achieve faster imaging and reduce ambient light interference in the light emitted by the target 101. In an example, the pulse width can be in the range of 0.005ms to 100ns. In an example, the frequency of the pulsed LEDs can be from 100Hz to tens of MHz. Therefore, the present subject matter enables faster capture of multiple first images and three-dimensional images and reduces ambient light interference (background interference).

[0135] In an example, the processor 140 may be configured to operate the imaging sensor 122 and the three-dimensional image capture sensor 120 to capture and process multiple first images and three-dimensional images at more than 30 frames per second. In this regard, the processor 140 may include a central processing unit (CPU) and a graphics processing unit (GPU). In particular, the CPU and GPU may be part of the SOM. In other words, the CPU and GPU may be provided onboard. The CPU may operate the imaging sensor 122 and the three-dimensional image capture sensor 120 to capture multiple first images and three-dimensional images. In addition, the GPU may process the images captured by the multiple first images and the three-dimensional images. The provision of the GPU and the CPU, in particular the provision of the GPU and the CPU onboard, may enable faster processing and capture of the multiple first images and the three-dimensional images, reaching more than 30 frames per second.

[0136] In some examples, in addition to using fluorescence-based images and three-dimensional images to detect the presence of problematic cellular entities, device 100 can also detect the presence of problematic cellular entities based on oxygenation. In this regard, device 100 can include a plurality of second light sources 156 for illuminating target 101 without causing markers in target 101 to fluoresce. Each of the plurality of second light sources 156 can be configured to emit light having a wavelength in the near-infrared (NIR) region or the visible region.

[0137] Imaging sensor 122 may be configured to capture a plurality of second images formed based on light reflected from target 101 in response to illumination thereof by at least one or more of plurality of second light sources 156. Processor 140 may analyze, using an analytical model, a second image obtained from the plurality of second images to identify oxygenation at a plurality of regions in target 101. Processor 140 may analyze a three-dimensional image of target 101 using the analytical model to determine changes in the intensity of light reflected from a spatial region of target 101 by compensating for changes in the distance of the spatial region of target 101 from three-dimensional image capture sensor 120 and compensating for changes in the curvature of the spatial region of target 101.

[0138] Processor 140 can use the analysis model to detect the presence of a problematic cellular entity in target 101 based on analysis of a first image among the plurality of first images, a second image obtained from the plurality of second images, and a three-dimensional image. In this case, processor 140 can create a composite image of the first image, the second image, and the three-dimensional image of target 101. Interface 108 can display a result corresponding to the detection of the problematic cellular entity and the composite image of the first image, the second image, and the three-dimensional image of target 101.

[0139] In an example, in addition to the first image and the three-dimensional image of target 101, the analysis model can also utilize white light images to detect problematic cellular entities. In this regard, in an example, at least one or more of the plurality of second light sources 156 can be configured to emit light with a wavelength in the visible region. Imaging sensor 122 can be configured to capture a plurality of third images formed based on light reflected from target 101 in response to illumination thereof by at least one or more of the plurality of second light sources 156. The plurality of third images are white light images. Processor 140 can be configured to analyze a third image obtained from the plurality of third images using the analysis model.

[0140] Processor 140 can analyze the three-dimensional image of target 101 using an analysis model to determine changes in the intensity of light reflected from the spatial region of the target by compensating for changes in the distance of the spatial region of the target from the three-dimensional image capture sensor and compensating for changes in the curvature of the spatial region of target 101. Processor 140 can be configured to use the analysis model to detect the presence of problematic cellular entities in the target based on an analysis of the first image, the third image, and the three-dimensional image of target 101. Processor 140 can be configured to create a composite image of target 101 using the first image, the third image, and the three-dimensional image. Interface 108 can be configured to display results corresponding to the detection of problematic cellular entities and a composite image of the first image, the third image, and the three-dimensional image of target 101. It should be understood that in this case, the analysis model can be trained using multiple fluorescence-based reference images, multiple reference white light images, and multiple reference three-dimensional images to detect the presence of problematic cellular entities in target 101.

[0141] Processor 140 may be configured to activate plurality of first light sources 130 to emit light toward target 101, and to activate plurality of second light sources 156 to emit light toward target 101. Furthermore, processor 140 may be configured to activate imaging sensor 122 to capture light emitted by target 101 in response to illumination of target 101 by at least one or more of plurality of first light sources 130, and to capture light emitted by target 101 in response to illumination of target 101 by at least one or more of plurality of second light sources 156.

[0142] In an example, to reduce and / or eliminate the effects of background light in captured images, processor 140 may be configured to control the plurality of first light sources 130 and the plurality of second light sources 156 to illuminate at a frequency that is different from the frequency of the ambient light source. Typically, ambient lighting in a room may pulse at a frequency of, for example, approximately 50 Hz. The analysis model can compare different images captured by imaging sensor 122 and remove background noise because the frequency of the ambient light source differs from the frequency of the plurality of first light sources 130. Minimizing background illumination improves image quality, which allows for more accurate fluorescence and oxygenation sensing by imaging sensor 122, leading to better analysis by the analysis model.

[0143] When capturing light emitted by target 101, processor 140 may activate multiple first light sources 130. To this end, processor 140 may activate imaging sensor 122 when activating multiple first light sources 130 to emit light. Typically, when multiple first light sources 130 are pulsed LEDs, multiple first light sources 130 pulse at a known rate, and imaging sensor 122 captures multiple first images at a rate that is a multiple of the pulse rate of multiple first light sources 130, such that multiple first light sources 130 are always on while the visible light camera is capturing images. Preferably, imaging sensor 122 captures the multiple first images and multiple first light sources 130 emit light simultaneously. The frame rate is typically a multiple of the pulse rate of multiple first light sources 130. A faster pulse rate reduces background noise and enables time-dependent fluorescence measurement. A faster pulse rate also reduces blur and variation in the captured image. Furthermore, because the frequency of the pulsed light is known, device 100 can only detect fluctuations in the multiple first light sources 130, as the frequency is known. This eliminates background noise because the background is constant. The device 100 has a hardware fast switch that uses elements such as fast MOSFETS, fast BJTs, a phase-locked loop (PLL), or a combination thereof to quickly turn the plurality of first light sources 130 on and off.

[0144] In an example, in addition to detecting problematic cellular entities, the device 100 can also classify the detected problematic cellular entities. Therefore, in an example, when the target 101 is a wound area, the processor 140 can be configured to extract spatial and spectral features of the wound area from the first image and the three-dimensional image using an analysis model. In addition, the processor 140 can identify the location of the wound area based on the extraction of spatial and spectral features using the analysis model. The processor 140 can determine the contour of the wound area based on the extraction of spatial and spectral features using the analysis model. In an example, based on determining the contour of the wound area, the processor 140 can be configured to determine the length of the wound area, the width of the wound, the perimeter of the wound, the area of the wound, the depth of the wound, or a combination thereof based on the determination of the contour of the wound area. In addition, the processor 140 can detect pathogens in the wound area based on the extraction of spatial and spectral features using the analysis model. The processor 140 can classify the pathogen by at least one of the family, genus, species, or strain of the pathogen using the analysis model.

[0145] In an example, in addition to the detection of problematic cellular entities, the device 100 may also determine other parameters corresponding to the detected problematic cellular entities. For example, when the target 101 is a wound area, the processor 140 may be configured to determine the degree of infection of the wound area, the spatial distribution of pathogens in the wound area, the healing rate of the wound area, or a combination thereof in response to detecting the presence of problematic cellular entities. When the target 101 is a tissue, the processor 140 is configured to detect the presence of problematic cellular entities (such as cancerous tissue, necrotic tissue, or a combination thereof) in a tissue sample. When the target 101 is a sanitary device, sanitary equipment, medical equipment, a biochemical assay chip, a microfluidic chip, or a body fluid, the processor 140 may be configured to determine the problematic cellular entity as a pathogen and classify the pathogens in the target 101.

[0146] Furthermore, in addition to detecting problematic cellular entities, processor 140 can be configured to detect changes in fluorescence emitted from target 101 over time. In other words, processor 140 can be configured to detect changes in fluorescence between a first imaging of target 101 relative to subsequent imaging of target 101. For example, processor 140 can be configured to detect changes in fluorescence between before debridement of a wound and after debridement of the wound. This detection can enable accurate removal of necrotic / unhealthy tissue from the wound. In another example, processor 140 can be configured to detect changes in fluorescence between an image of the wound taken on the first day and an image of the wound taken on the subsequent day. This detection can help determine wound healing and allow medical practitioners to administer medication based on the detection.

[0147] In an example, device 100 may be portable and may include a smartphone. The smartphone may include a processor 140 and an imaging sensor 122. Additionally, the smartphone may include a three-dimensional image capture sensor 120. In an example, the smartphone may be integrated with various light sources 130, 156, polarizers, filters 142, and the like.

[0148] In an example, the device 100 may include other components. The device 100 may include a first set of excitation filters 142. Each of the first set of excitation filters 142 may be configured to filter excitation radiation of a predetermined wavelength range emitted by a light source in the plurality of first light sources 130 so that the excitation radiation passes through the excitation filter to illuminate the target 101. In addition, one or more excitation filters may be configured to filter excitation radiation of a predetermined wavelength range emitted by a light source in the plurality of second light sources 156 so that the excitation radiation passes through the excitation filter.

[0149] The device 100 may include a thermal sensor for thermal imaging of the target 101 ( Figure 1(not shown in the figure). The thermal sensor can, for example, be part of the imaging module 102. In this regard, the processor 140 can be configured to detect the problematic cellular entity based on the first image of the plurality of first images, the second image obtained from the plurality of second images, the three-dimensional image, and the thermal image of the target 101 using an analysis model. In this case, the processor 140 can create a composite image of the first image, the second image, the three-dimensional image, and the thermal image using the analysis model. In addition, the interface 108 can display a result corresponding to the detection of the problematic cellular entity and a composite image of the first image of the target 101, the second image, the three-dimensional image, and the thermal image of the target 101.

[0150] Device 100 may include a ranging sensor 132 operable to determine a distance of target 101 from device 100 for positioning device 100 at a predetermined distance from target 101. Range sensor 132 may be, for example, part of imaging module 102. In an example, three-dimensional image capture sensor 120 may be used as ranging sensor 132. In this regard, three-dimensional image capture sensor 132 may be used to determine a distance of target 101 from device 100 for positioning device 100 at a predetermined distance from target 101.

[0151] In an example, the device 100 may not have a polarizer. In another example, the device 100 may include multiple polarizers ( Figure 1 (not shown). For example, device 100 may include a first polarizer positioned between the plurality of first light sources 130 and target 101 to allow excitation radiation from the plurality of first light sources 130 of a first polarization to pass through the first polarizer. Device 100 may also include a second polarizer positioned between target 101 and the imaging sensor to allow light emitted by target 101 of a second polarization to pass through the second polarizer. In an example, the first polarizer may be aligned 90 degrees with the second polarizer. Providing a polarizer in front of imaging sensor 122 may prevent excitation light from entering imaging sensor 122.

[0152] In an example, the first polarization and the second polarization can be the same. For example, in an example, the first polarization and the second polarization can be left-handed circular polarization (LHCP). In another example, the first polarization and the second polarization can be right-handed circular polarization (RHCP). In another example, the first polarization and the second polarization can be different. For example, the first polarization can be one of LHCP or RHCP, and the second polarization can be the other of LHCP or RHCP. Multiple polarizers can be combined with the first set of excitation filters 142.

[0153] Additionally, optionally, a light diffuser may also be placed in front of the plurality of first light sources 130 and / or the plurality of second light sources 156 and / or the excitation filter 142 to better spread the light onto the target 101 .

[0154] The device 100 may include a housing for housing components, such as those described with reference to FIG. Figure 2a-2c As explained. In particular, the device 100 may include a first housing, a second housing, and a bridge. The first housing may accommodate the imaging module, and the second housing may accommodate the interface module 104. The bridge may connect the imaging module and the interface module 104. The bridge may include an electronic interface to enable electronic communication between the processor 140 of the interface module 104 and the imaging module. The electronic interface may include a camera serial interface 108 (CSI), a serial management bus (such as an I2C interface), a system packet interface (SPI), a universal asynchronous receiver-transmitter (UART), a general-purpose input / output (GPIO) interface, a universal serial bus (USB) interface, a pulse width modulation (PWM) interface, a display-serial interface (DSI), a high-definition multimedia interface (HDMI), or a combination thereof.

[0155] To be able to power components of the device 100, such as the imaging module and the interface module 104, the device 100 may include a portable power module 136. The portable power module 136 may include a third housing 37 to house the portable power module 136.

[0156] In an example, the device 100 can transmit the results to a remote system, such as a cloud server 160. For example, the processor 140 can be configured to transmit the results and a composite image of the first image, the three-dimensional image, to a remote system, such as a cloud server, etc. The remote system can be in electronic communication with the device 100. Because the device 100 is capable of transmitting the results and the composite image to the cloud server, a non-medical professional or a medical professional can transmit the image or image sequence to a remotely located medical professional for additional consultation before performing treatment using the device(s) 100 of the present disclosure.

[0157] In an example, the interface 108 can be an interactive display, such as an LED display, a liquid crystal display, a thin film transistor display, an organic light emitting diode (OLED) display, a capacitive touch screen, a resistive touch screen, a toggle switch, or a button. The digital display and buttons enable the user to easily use and manipulate the device 100. The interface 108 can also be a standalone device 100, such as a laptop computer, a desktop computer, a tablet computer, a smartphone, a smart accessory (such as a smart watch), or a combination thereof.

[0158] The interface 108 may be configured to receive input from a user corresponding to the operation of the device 100 using an application programming interface 108 (API). For example, using the API, the user may be able to select one or more of the plurality of first light sources 130 and one or more of the plurality of second light sources 156 to illuminate the target 101. In addition, the user may be able to select the frequency of light emission from the plurality of first light sources 130 and the plurality of second light sources 156.

[0159] Interface 108 may be configured to transmit, in response to input, results corresponding to the detection and classification of pathogens in target 101 when the pathogens are detected and classified using the API. In this regard, interface 108 may allow a user to store and analyze the results corresponding to the detection and classification of pathogens in target 101. Furthermore, interface 108 may allow a user to select a composite image to be obtained and may transmit the results to a remote system or remote server, and may further allow a user to select various views of the composite image.

[0160] Figure 2a Shown is a front perspective view of an apparatus 100 for inspecting a target 101 , according to an embodiment of the present subject matter. Figure 2b A rear perspective view of an apparatus 100 for inspecting a target 101 is shown, in accordance with an embodiment of the present subject matter. Figure 2c An exploded view of an apparatus 100 for inspecting a target 101 according to an embodiment of the present subject matter is shown. Figure 2a-2c .

[0161] Here, SOM 210 is depicted. In an example, the components of imaging module 102 may be held together by rear frame 134 and connecting bracket 236. Rear frame 134 and connecting bracket 236 together form a first housing to enclose imaging module 102.

[0162] In an example, imaging module 102 and interface module 104 may be coupled via a bridge ( Figures 2a-2c The bridge may stably hold imaging module 102 and interface module 104 together and allow electronic communication between elements of imaging module 102 and interface module 104 via a camera serial interface (CSI), a serial management bus (such as an I2C interface), a system packet interface (SPI), a universal asynchronous receiver / transmitter (UART), a general purpose input / output (GPIO) interface, a universal serial bus (USB) interface, a pulse width modulation (PWM) interface, a display-serial interface (DSI), a high-definition multimedia interface (HDMI), or any other electronic connection known in the art.

[0163] The processor 140 and the interface 108 can each be fixedly, mechanically attached to the bracket ( Figure 2a-2cThe brackets can then be clamped between the rear frame 112 and the front frame (not shown) while being directly electrically coupled to each other via busbars, serial wires, or any other wires known in the art from the processor to the user interface 108. Figure 2a-2c The rear frame 112 and the front frame may together form a second housing to accommodate the interface module 104. The processor 140 may include, for example, random access memory (RAM), flash memory, WiFi and / or cellular data antennas, Antennas, and other interfaces that allow various peripheral devices to be electronically attached. It is a short-range wireless technology standard used for short-range data exchange between fixed and mobile devices and for building personal area networks (PANs). UHF radio waves in the ISM band from 2.402 GHz to 2.48 GHz are employed.

[0164] Processor 140 can be connected to cloud server 160 (such as Figure 1 ) for uploading and downloading data for the orientation of the imaging sensor 122 and further analyzing the captured images and three-dimensional point clouds. In an example, all hardware drivers for the device 100 can be separated onboard from one or more components of the imaging module 102 (such as the plurality of first light sources 130, the first set of excitation filters 142, the light source driver 150, the plurality of second light sources 156, etc.). The processor 140 allows for extremely fast switching / instructions to activate the light sources 130, 156, which provides many advantages to the device 100.

[0165] The portable power module 136 may include a rechargeable battery 46 electrically coupled to a power printed circuit board (PCB) 44. The power PCB 44 and rechargeable battery 46 may be sandwiched between the front cover and the back cover ( Figures 2a-2c (not shown). The front cover and the rear cover may form a third housing. Thus, the third housing enables the portable power module 136 to be accommodated. Power cord ( Figures 2a-2c (not shown) can be electrically attached to the power PCB 44 and exit through the cover. In addition, in an example, the device 100 can be provided with a shade cloth ( Figure 2a-2c (not shown) operates to reduce ambient light.

[0166] Additionally, although not shown herein, the device 100 may be coupled to a portable stand, such as stand 410, as shown. Figure 4a-4b shown.

[0167] In the foregoing examples, the apparatus was explained without providing an optical bandpass filter for filtering light emitted by the target. However, in some examples, one or more optical bandpass filters, such as emission filters, may be used.

[0168] Figure 3 A block diagram of a device 300 for inspecting a target 101 according to an embodiment of the present subject matter is shown. Device 300 may correspond to device 100 and may include the same components as device 100. Therefore, components of device 100 included in device 300 are explained using the same reference numerals. In addition, device 300 may include an emission filter, as will be explained below. It should be understood that in addition to the functions explained herein, device 300 may use appropriate components mentioned with reference to device 100 to perform some or all of the functions performed by device 100.

[0169] Apparatus 300 for inspecting target 101 may include imaging module 102, interface module 104, and interface 108. Imaging module 102 may include a plurality of first light sources 130, a plurality of first optical bandpass filters 126, imaging sensor 122, and three-dimensional image capture sensor 120.

[0170] Multiple first light sources 130 can emit light for illuminating a target 101. Target 101 may be suspected of harboring problematic cellular entities, such as pathogens or cancerous tissue. In one example, target 101 may be composed of one or more cells and may be, for example, a wound or tissue sample in a body part. In other examples, target 101 may be an item that does not contain pathogens, such as an edible product, laboratory equipment, or sanitary fixtures. The emitted light may be in a wavelength band that causes markers in target 101 to fluoresce when illuminated. Specifically, the emitted light may have a single wavelength that causes the markers in target 101 to fluoresce when illuminated. The light from multiple first light sources 130 may also be emitted at a specific frequency. This frequency may be tuned to an integer multiple of the frequency of imaging sensor 122, allowing imaging sensor 122 to capture images when illuminated by multiple first light sources 130. The frequency may also be tuned to a frequency different from that of ambient light sources in the room. This ensures that multiple first light sources 130 illuminate the target when ambient light sources are off, allowing background images to be more easily filtered out and removed from analysis.

[0171] The marker is typically a part of the problematic cell entity. The fluorescence emitted by the marker as a part of the problematic cell entity can be referred to as autofluorescence. In an example, an exogenous marker (such as a synthetic marker) can be sprayed on the target to start the detection of the problematic cell entity in the target. The exogenous marker can be combined with cell entities such as deoxyribonucleic acid (DNA), ribonucleic acid (RNA), protein, biochemical markers, so that the target can be made to fluoresce. The fluorescence emitted by the added synthetic marker can also be referred to as exogenous fluorescence.

[0172] Each of the plurality of first light sources 130 can be configured to emit excitation radiation at a predetermined wavelength range to cause one or more markers in the target 101 to fluoresce. In an example, each of the plurality of first light sources 130 can be a light emitting diode (LED). In another example, one or more of the plurality of first light sources 130 can be a pulsed LED to emit pulses of excitation radiation to achieve faster imaging and reduce ambient light interference in the light emitted by the target 101. The plurality of first light sources 130 can be, for example, homogeneous light sources or inhomogeneous light sources.

[0173] In an example, the wavelength band of light used to cause target 101 to fluoresce may include 200nm-300nm, 300nm-400nm, 400nm-500nm, or 500nm-600nm. In a specific example, the wavelength of light used to cause target 101 to fluoresce may include 280nm, 310nm, 330nm, 365nm, 395nm, 405nm, 415nm, 430nm, 480nm, and 520nm. In an example, the wavelength band of light may include 600nm-700nm, 700nm-800nm, or 800nm-3000nm. In a specific example, the wavelength of light used may also include 430nm, 630nm, 660nm, 680nm, 735nm, 830nm, 880nm, 940nm, and 970nm to capture reflection and / or scattering.

[0174] Each of the plurality of first optical bandpass filters 126 can be configured to filter light of a predetermined wavelength emitted by the target 101 in response to illumination by at least one or more of the plurality of first light sources 130, thereby allowing the light of the predetermined wavelength to pass through the plurality of first optical bandpass filters 126. In an example, the optical bandpass filters 126 can have a center wavelength corresponding to the peak emission fluorescence from various autofluorescent biomarkers or exogenous fluorophores. The optical bandpass filters 126 can be low-pass, high-pass, single-pass, or multi-pass bandpass filters. The imaging sensor 122 can capture the filtered light filtered by the optical bandpass filters in the plurality of first optical bandpass filters 126 and capture a plurality of first images formed based on the filtered light. The three-dimensional image capture sensor 120 can illuminate the target 101 and, in response to illumination of the target 101 by the three-dimensional image capture sensor 120 by the plurality of first light sources 130, receive light reflected from the target 101 and generate a three-dimensional image of the target 101 based on the reflected light. To illuminate the target 101, the three-dimensional image capture sensor 120 may include one or more light sources ( Figure 3(not shown). However, in some examples, a separate light source may also be coupled to 3D image capture sensor 120 to illuminate target 101, and the illumination may enable light reflected by target 101 to be captured. In some examples, 3D image capture sensor 120 may be a structured light sensor, a time-of-flight sensor, a stereo sensor, or a combination thereof.

[0175] The interface module 104 may include a processor 140. The processor 140 may be implemented as a microprocessor, a microcomputer, a microcontroller, a digital signal processor, a central processing unit, a combination of a central processing unit and a graphics processing unit, a state machine, a logic circuit, and / or any device that can manipulate signals based on operational instructions. Among other functions, the processor 140 may retrieve and execute the memory ( Figure 3 ) in computer-readable instructions.

[0176] Processor 140 may be configured to analyze a first image from the plurality of first images using an analysis model. The first image may be a fluorescence-based image including fluorescence emitted from target 101. Processor 140, using the analysis model, may analyze the three-dimensional image of target 101 to determine variations in the intensity of light emitted from the spatial region of target 101 by compensating for variations in the distance of the spatial region of target 101 from the three-dimensional image capture sensor 120 and for variations in the curvature of the spatial region of target 101 relative to the three-dimensional image capture sensor 120. Processor 140 may use the analysis model to detect the presence of problematic cellular entities in target 101 based on the analysis of the first image and the three-dimensional image. The analysis model may be trained to detect the presence of problematic cellular entities in a target.

[0177] The analysis model is trained to detect the presence of problematic cellular entities in the target. In particular, the analysis model can be trained using a plurality of fluorescence-based reference images to detect the presence of problematic cellular entities in the target. The analysis model can be trained to distinguish between fluorescence emitted from the problematic cellular entity in the fluorescence-based image and fluorescence emitted from areas other than the problematic cellular entity in the fluorescence-based image.

[0178] In an example, in addition to being trained using multiple fluorescence-based reference images, the analysis model can also be trained using multiple reference three-dimensional images of the target to detect the presence of problematic cellular entities in the target. In this regard, the analysis model can be trained to distinguish between fluorescence emitted from the problematic cellular entity in the fluorescence-based image and fluorescence emitted from areas other than the problematic cellular entity in the fluorescence-based image. Additionally, the analysis model can be trained to compensate for differences in the distance and curvature of the spatial region of the target 101 relative to the three-dimensional image capture sensor 120 by determining changes in the intensity of light emitted from the spatial region of the target 101. Changes in the intensity of light emitted from the spatial region of the target 101 can be determined based on changes in the distance of the spatial region of the target relative to the three-dimensional image capture sensor 120, changes in the curvature of the spatial region of the target 101, and the intensity measured at the spatial region of the target 101. Reference Figure 5-Figure 6 Explain the training of analytical models.

[0179] The processor 140 may create a composite image of the first image and the three-dimensional image of the target 101. The interface may display the result corresponding to the detection of the problematic cell entity and the composite image of the first image and the three-dimensional image of the target 101. Figure 7-11 Explain the detection of the presence of problematic cellular entities.

[0180] In an example, the device 300 may include a system-on-module (SOM). The SOM may include an imaging module 102, an interface module 104, and multiple light source drivers 150. The SOM may also include an FPGA module. The multiple light source drivers 150 may be configured to regulate the light sources in the multiple first light sources 130. In other words, the device 300 also utilizes an integrated circuit board, typically including a SOM 106. The SOM 106 includes the imaging module 102, the interface module 104, and the multiple light source drivers 150. In an example, the processor 140 may include a central processing unit (CPU) and a graphics processing unit (GPU). In addition, the SOM 106 may include a battery charging module. The SOM 106 may include an integrated circuit (also referred to as a "chip") that integrates all or most of the components of a computer or other electronic system. These components almost always include the processor 140, a memory interface, on-chip input / output devices 300s, an input / output interface, and an auxiliary storage interface, typically along with other components such as a modem, including a radio modem, all on a single substrate or microchip. The SOM 106 may contain digital, analog, mixed signal and typically RF signal processing functionality (otherwise it is considered just an application processor). Alternatively, the device 300 may include a system on a chip (SOC) instead of a SOM. The SOC may be similar to the SOM.

[0181] Processor 140 may also be configured to activate multiple first light sources 130 to emit light toward target 101 , and activate imaging sensor 122 in response to at least one or more of multiple first light sources 130 illuminating target 101 to capture light emitted by target 101 .

[0182] In an example, the device 300 may include an emission filter wheel 124 rotatably disposed within the imaging module. The emission filter wheel 124 may be operably coupled to a servo motor 128. The emission filter wheel 124 may include a plurality of first optical bandpass filters 126. It should be understood that based on a desired optical bandpass filter 126 from the plurality of first optical bandpass filters, the servo motor 128 may be actuated to position the desired optical bandpass filter between the target 101 and the imaging sensor 122. In this regard, the processor 140 may be configured to activate the servo motor 128 to rotate the emission filter wheel, thereby positioning one of the plurality of first optical bandpass filters 126 between the target 101 and the imaging sensor 122. In an example, the device 300 may include a ranging sensor 132 operable to determine a distance of the target 101 from the device 300, for positioning the device 300 at a predetermined position relative to the target 101.

[0183] The apparatus 300 may include a first set of excitation filters 142. Each of the first set of excitation filters 142 may be configured to filter excitation radiation of a predetermined wavelength range emitted by a light source in the plurality of first light sources 130, allowing the excitation radiation to pass through the excitation filter to illuminate the target 101. In addition, one or more excitation filters may be further configured to filter excitation radiation of a predetermined wavelength range emitted by a light source in the plurality of second light sources 156, allowing the excitation radiation to pass through the excitation filter.

[0184] In an example, the three-dimensional image capture sensor 120 can be used as the ranging sensor 132. In this regard, the three-dimensional image capture sensor 132 is operable to determine the distance of the target 101 from the device 100 for positioning the device 100 at a predetermined distance from the target 101. Additionally, in an example, the device 300 can be operated with a blackout curtain to reduce ambient light.

[0185] Although only a few examples of detecting problematic cellular entities are explained with reference to device 300, it is understood that device 300 may include other components similar to device 100, such as thermal sensors, etc. In addition, device 300 may also perform similar functions as device 300 and may perform the detection, classification, etc. of problematic cellular entities explained with reference to device 100.

[0186] In an example, in addition to the first image and the three-dimensional image of the target 101, the analysis model can also utilize white light images to detect problematic cellular entities. In this regard, in an example, at least one or more of the plurality of second light sources 156 can be configured to emit light with a wavelength in the visible region. The imaging sensor 122 can be configured to capture a plurality of white light images formed based on light reflected by the target 101 in response to illumination of the target by at least one or more of the plurality of second light sources 156. The processor 140 can be configured to analyze the white light images obtained from the plurality of white light images using the analysis model. The processor 140 can analyze the three-dimensional image of the target 101 using the analysis model to determine changes in the intensity of light reflected from the spatial region of the target 101 by compensating for changes in the distance of the spatial region of the target 101 from the three-dimensional image capture sensor 120 and compensating for changes in the curvature of the spatial region of the target 101.

[0187] The processor can be configured to use the analysis model to detect the presence of problematic cellular entities in the target based on an analysis of the first image, the white light image, and the three-dimensional image of the target 101. The processor 140 can be configured to create a composite image of the target 101 using the first image, the white light image, and the three-dimensional image. The interface 108 can be configured to display a result corresponding to the detection of the problematic cellular entity and a composite image of the first image, the white light image, and the three-dimensional image of the target 101. It should be understood that in this case, the analysis model can be trained using multiple fluorescence-based reference images, multiple reference white light images, and multiple reference three-dimensional images to detect the presence of problematic cellular entities in the target 101.

[0188] Device 300 may include multiple polarizers. For example, device 300 may include a first polarizer positioned between multiple first light sources 130 and target 101 to allow excitation radiation of a first polarization from multiple first light sources 130 to pass through the first polarizer. Device 300 may include a second polarizer positioned between target 101 and imaging sensor 122 to allow light of a second polarization emitted by the target to pass through. In one example, the first polarization and the second polarization may be the same. In another example, the first polarization and the second polarization may be different. In one example, the first polarization and the second polarization may be the same. For example, in one example, the first polarization and the second polarization may be left-handed circular polarization (LHCP). In another example, the first polarization and the second polarization may be right-handed circular polarization (RHCP). In another example, the first polarization and the second polarization may be different. For example, the first polarization may be one of LHCP or RHCP, and the second polarization may be the other of LHCP or RHCP. Multiple polarizers may be combined with the first set of excitation filters 142 or the plurality of first emission filters 126, or both.

[0189] Figure 4aA perspective view of an apparatus 300 for inspecting a target 101 is shown according to an example embodiment of the present subject matter. Figure 4a ) and a portable stand 410. The portable stand 410 can easily position and reposition the device 300 to a desired location, particularly a desired local location within a given room or medical area (such as a hospital floor or triage space). The portable stand 410 can include a base 411, a telescopic arm 413 engaged with the base 411 at a first end 421, and an articulated arm 416 engaged with a second end 423. The base 411 can include one or more legs 418 with wheels 420, at least one of which can include a brake (not shown). The wheel 420 can be, for example, a caster or other similar wheel having a mounting member, a rod, and at least one wheel. Additional components may be present depending on the type of caster and its intended surface of use. The caster can be a plate caster mounted using a mounting plate and can have a single wheel or two wheels. As an alternative to the wheel 420, the floor-engaging end of one or more legs 418 can have a floor glide attached to the floor instead of a wheel. The base 411 may allow a user to roughly position the device 300 to approach a desired area for illumination and imaging, or to easily move it from one room to another.

[0190] The telescopic arm 413 can include a lower arm 422 and an upper arm 426 coupled by a collar 424. The collar 424 allows one of the arms 422, 426 to slide within the other and is held in place by a set screw 424a having a handle so that the set screw 424a can be easily tightened and loosened by hand. The telescopic arm 413 can allow the device 300 to be positioned at an appropriate height in the vertical direction (up / down direction) for use. It should be understood that the collar 424 can also be integral with one of the arms 422, 426.

[0191] Articulating arm 416 can be configured to allow for the precise positioning of device 300 necessary to illuminate and image a specific location on a target when using device 300 to capture images, without requiring the user to capture any images or to use device 300 while the user is holding the device 300. Typically, after base 411 and telescoping arm 413 are roughly positioned near the target, articulating arm 416 can be positioned. Articulating arm 416 can then be used to precisely position device 300 at that location to target the area of the patient or subject to be scanned or otherwise assessed using device 300. Articulating arm 416 can include a lower arm 430, an upper arm 432, and a connecting bracket 434, which engages and connects device 300 to portable stand 410. Connecting bracket 434 can include one or more generally D-shaped holes forming a handle 434a, which allows the user to move device 300 into position without applying stress to, or more typically, even touching, the device 300 itself during the positioning process. While a generally D-shaped aperture forms the handle 434a, the handle 434a may be formed by any aperture in the connecting bracket 434, such as a rectangular aperture or a circular aperture. Furthermore, it is contemplated that the handle 434a or handles may be formed by a separate knob or knobs that are coupled to the connecting bracket 434 via one or more fasteners, such as a screw or bolt system. The lower arm 430 and the upper arm 432 may include a hinge that allows them to rotate and move vertically relative to each other so that the device 300 is precisely positioned in the desired position by the practitioner using the device 300.

[0192] Bracket 428 can be inserted between telescoping arm 413 and articulating arm 416. Bracket 428 can be used to place or suspend any medical and / or dental instruments required by the practitioner. Furthermore, bracket 428 can also incorporate or include a portable power module 136, which can be fastened to bracket 428 or housed within a housing 437 formed on one or more sides of the bracket and sized to accommodate the portable power module 136. Housing 437 is typically an open-top portion of bracket 428 or a separate component. It typically has a front, a back, two sides, and a bottom, with a top that is open to air during use and sized to receive portable power module 136. The locations where the sides of the housing meet other surfaces can be partially or completely sealed. It is contemplated that housing 437 can be made of any material, such as cloth or leather, but is more typically a sterilizable medical-grade metal. In practice, portable stand 410 as a whole is typically made of a material that can be periodically sterilized using UV or other means as needed.

[0193] The portable power module 136 may include a rechargeable battery 46, as will be described in detail in the accompanying drawings. Figure 4d. It is contemplated that the portable power supply module 136 can be plugged into a typical electrical outlet between uses or during use if it becomes depleted before the next use or is likely to become depleted during the next use. The use of a rechargeable battery system(s) allows the entire assembly to be easily moved from one location to another without the need for a constantly plugged-in power cord that can make it difficult for a practitioner to move around. Multiple portable power supply modules 136 can be employed throughout the system of the present disclosure, including the device 300. For example, a "kit" or rechargeable battery-only of one or more portable power supply modules 136 can be provided in a travel case or housing before it is deployed for use.

[0194] Figure 4b A perspective view of a device 300 for inspecting a target is shown in accordance with an embodiment of the present subject matter. The interface between the articulated arm 416 and the device 300 is shown in greater detail. The distal end of the upper arm 432 may include a 2-axis hinge 38 that allows the device 300 to rotate in both vertical and lateral directions (i.e., left / right and up / down). The bracket 434 may be attached via an x-bracket 440 that allows the device 300 to be stably coupled to the articulated arm 416 by removing as much "tilt" as possible between the bracket 434 and the articulated arm 416. It should be understood that coupling the bracket 434 to the articulated arm 416 may be accomplished by any coupling known in the art.

[0195] The portable power module 136 may have a power cord 442 that passes through a hinge 437 and connects to a battery 446 (e.g., a battery pack) on the arms 430, 432. Figure 4d 437 and 436. The power cord 442 extends from the lower arm 430 to the upper arm 432 and extends to the device 300. It should be understood that the power cord 442 can be constrained within a cord retainer attached to any or all of the arms 430, 432 and hinge 437, or can hang freely from the portable power module 136 to the device 300, or any combination thereof. Typically, the power cord 442 is protected within the lower arm 430 and the upper arm 432 so that it cannot be damaged or removed from the joint of the lower arm 430 and the upper arm 432.

[0196] Figure 4cAn exploded view of an apparatus 300 for inspecting a target is shown in accordance with an embodiment of the present subject matter. The apparatus 300 may include an imaging module 102 and an interface module 104 joined by a bridge 438. The processor 140 and the interface 108 may each be fixedly, mechanically attached to the bracket 409 while being directly electrically coupled to each other via bus bars, serial wires, or any other wires known in the art from the processor to the interface 108. The bracket 409 may then be sandwiched between a rear frame 412 and a front frame 414, holding the interface module 104 together. The rear frame 412 and the front frame 414 may together form a second housing to house the interface module 104. The processor 140 may be, for example, random access memory (RAM), flash memory, a WiFi and / or cellular data antenna, Antennas, and other interfaces that allow various peripheral devices to be electronically attached. It is a short-range wireless technology standard used to exchange data between fixed and mobile devices over short distances and to build a personal area network (PAN). UHF radio waves in the ISM band from 2.402 GHz to 2.48 GHz are employed.

[0197] The processor 140 can be connected to a cloud server via WiFi or a cellular data antenna for uploading and downloading data for the orientation of the imaging sensor 122 and for further analysis of the captured images and three-dimensional point clouds. Typically, all hardware drivers for the device 300 are onboard along with one or more components of the imaging module 102 (such as the light sources 130, 156, the imaging sensor 122, the filters 142, 126, the filter wheel 124, the servo motor 128, the light source driver 150, the three-dimensional image capture sensor 120, the imaging sensor 122, the ranging sensor 132, etc.). The processor 140 allows for extremely fast switching / commands to activate the light sources 130, 156, which provides many advantages to the device 300.

[0198] The ability of the device 300 to obtain oxygenation and fluorescence data at varying distances is a significant advantage of the present system, which better allows less-trained or non-medical personnel to more easily use the device while obtaining accurate data. The components of the imaging module 102 can be held together by a rear frame 435 and a connecting bracket 436. The rear frame 435 and the connecting bracket 436 together form a first housing to enclose the imaging module 102. The imaging module 102 and the interface module 104 can be constructed to be substantially or completely waterproof. Although the device 300 is shown as employing a plurality of visible light filters and other filters, the device 300 can also be constructed without visible light filters or other filters.

[0199] Figure 4dAn exploded view of the portable power module 136 of the device 300 for inspecting a target according to an embodiment of the present subject matter is shown. The portable power module 136 can include a rechargeable battery 446 electrically coupled to a power printed circuit board (PCB) 444. The power PCB 444 and the rechargeable battery 446 can be sandwiched between a front cover 448 and a rear cover 450. The front and rear covers 448, 450 can form a third housing to enclose the portable power module 136. A power cord 442 can be electrically attached to the power PCB 444 and exit through the covers 448, 450. The power cord 442 typically extends up the arm of the portable stand 410 to the device 300, but it is contemplated that the power cord 442 can be free from the portable stand 10.

[0200] Figure 4e An exploded view of the interface module 104 of the apparatus 300 for inspecting a target according to an embodiment of the present subject matter is shown. Here, a power line 442 is shown as entering the interface module 104 upward through the bottom of the interface module 104. The imaging module 102 and the interface module 104 can be mechanically and electrically connected via a bridge 438, which stably holds the imaging module 102 and the interface module 104 together and allows electronic communication between the elements of the imaging module 102 and the interface module 104 via a camera serial interface (CSI), a serial management bus (such as an I2C interface), a system packet interface (SPI), a universal asynchronous receiver / transmitter (UART), a general purpose input / output (GPIO) interface, a universal serial bus (USB) interface, a pulse width modulation (PWM) interface, a display-serial interface (DSI), a high-definition multimedia interface (HDMI), or any other electronic connection known in the art.

[0201] Figure 5 A method 500 for training an analytical model to detect problematic cellular entities in a target is shown, in accordance with an embodiment of the present subject matter. The order in which the method blocks are described should not be construed as limiting, and some of the described method blocks can be combined in any order to implement method 500 or an alternative method. Additionally, some individual blocks can be deleted from method 500 without departing from the scope of the subject matter described herein. Herein, targets will be explained with reference to wounds, and problematic cellular entities will be explained with reference to pathogens. However, it will be understood that the target can be a tissue sample, an edible product, a laboratory device, a sanitary appliance, a sanitary fixture, a biochemical assay chip, a microfluidic chip, a medical device, a body fluid, or a combination thereof, and that the problematic cellular entity can be cancerous tissue, necrotic tissue, or the like.

[0202] At block 502, a fluorescence-based reference image, a reference white light image, and a reference three-dimensional image are labeled with various reference labels, such as the type of target (i.e., skin or wound), the type of wound area (i.e., slough, bone, etc.), the type of infecting pathogen, the Gram type, etc. In an example, various spatial features, such as the texture and porosity of the wound and adjacent areas, various spectral features, such as the hue of the fluorescence, or a combination thereof, are extracted. In an example, labeling may be performed only in the white light image.

[0203] At block 504, the labeled image is pre-processed. For example, the image is converted to grayscale, resized, and enhanced. Enhancing the image may include rotating the image, flipping the image, etc.

[0204] At box 506, various features are extracted from the image, such as spatial features, spectral features, or a combination thereof. In some instances, spatial features such as histogram of oriented gradients (HOG) features, entropy features, local binary patterns (LBP), scale-invariant feature transform (SIFT), etc. can be extracted from the image. Similarly, in some examples, spectral features can be extracted from white light images of RGB wavelengths and fluorescence images of various excitation wavelengths. For white light images and fluorescence images, spectral features are extracted using red, green, blue (RGB), hue saturation value (HSV) values, or any other color map values at each pixel / region. In an example, a machine learning model or a deep learning model can be used to extract spatial features and spectral features.

[0205] At block 508, the extracted spatial and spectral features and labels may be stored in the memory of processor 140 ( Figure 5 The extracted features are then passed to the analysis model for pathogen detection and spatial mapping, as described below. For example, for some pathogens, such as Pseudomonas aeruginosa, the pathogen can be detected by using spatial features and excitation wavelengths. For some pathogens, such as Escherichia coli (E-coli), Klebsiella, Staphylococcus, etc., detection can be performed by extracting a combination of spatial features and spectral features.

[0206] Steps 502-508 may be repeated for several fluorescence-based reference images, several white light images, and several three-dimensional images until a target predetermined target training accuracy is achieved.At block 510, the information in the database may be used to train an analysis model.

[0207] With the help of training, the analysis model becomes capable of identifying wounds in a given image based on the extracted spatial features, spectral features, or a combination thereof. In other words, the analysis model is capable of performing wound segmentation. In an example, after block 510, method 500 may include post-processing steps, such as connected component labeling, hidden Markov models, etc., which can be used to smooth the results of wound segmentation, thereby improving the accuracy of wound segmentation.

[0208] While the analysis model is being trained, the analysis model can be tested to verify that it can correctly identify wounds in an image. Thus, at box 512, a region of interest is selected in the test image. In an example, the region of interest can be selected automatically, such as by the analysis model. In another example, the region of interest can be selected manually, such as by a user. Furthermore, at box 514, the test image is preprocessed, and at box 516, spatial features of the test image are extracted. At box 518, the extracted features are fed to the analysis model to perform wound segmentation and problematic cell entity detection and classification. Subsequently, the results of the wound segmentation, problematic cell entity detection, and classification performed by the analysis model can be received.

[0209] In one embodiment, the analysis model used for wound segmentation can be different from the analysis model used for pathogen detection and classification. Thus, the output of wound segmentation can be provided by a first analysis model to a second analysis model. The second analysis model can then analyze the fluorescence from the wound region identified by the first analysis model and then detect and classify pathogens in the wound region. Alternatively, in an example, the second analysis model can also use spatial features, information about the wound, bone, tissue region, etc. from the first analysis model, combined with spectral features to detect and classify pathogens.

[0210] In one example, the analysis model can include an ANN model and an ML model, each performing different functions. For example, an ML model can be trained to perform wound segmentation, while an ANN model can be trained to detect and classify pathogens. In another example, an ANN model can generate a spectral image from a fluorescence-based image, and an ML model can detect and classify pathogens based on the generated spectral image. In another example, in addition to the fluorescence-based image, the ANN model can also generate a spectral image from the white light image and the ML model can generate a spectral image.

[0211] In an example, the analysis model can classify pathogens in the wound into Gram-positive (GP) and Gram-negative (GN) pathogens. In addition, the analysis model can identify the type of pathogen in the wound.

[0212] The analytical model and references explained in this article Figure 1-Figure 4e The analytical model for interpretation is the same.

[0213] Figure 6 An example of training an analytical model to detect problematic cellular entities in a target according to an embodiment of the present subject matter is shown. In the example depicted herein, image 602a depicts a white light image, and image 604a depicts an image labeled with a reference label.

[0214] Similarly, autofluorescence images 602b, 602c, 602d depict autofluorescence images of the target at different excitation wavelengths, such as 365nm, 395nm, and 415nm, respectively. Images 604b, 604c, 604d depict autofluorescence images labeled with reference labels. Images 604b, 604c, 604d correspond to images 602b, 602c, 602d with reference labels. All images 602a-602d and images 604a-604d are fed to an analysis model 606 for training. The analysis model can provide an output of a composite image 608 of autofluorescence images that is overlaid with a predicted distribution of bacterial species. Figure 6 In the figure, red corresponds to Staphylococcus aureus and green corresponds to Pseudomonas aeruginosa.

[0215] The analytical model and references explained in this article Figure 1-Figure 5 The analytical model for interpretation is the same.

[0216] Figure 7 A method 700 for detecting problematic cellular entities according to an embodiment of the present subject matter is shown. The order in which the method blocks are described should not be construed as limiting, and some of the described method blocks can be combined in any order to implement method 700 or an alternative method. In addition, some individual blocks can be deleted from method 700 without departing from the scope of the subject matter described herein. Herein, reference is made to a wound interpretation target 101. However, it should be understood that the target can be a tissue sample, an edible product, a laboratory device, a sanitary device, a medical device, a sanitary device, a biochemical assay chip, a microfluidic chip, a body fluid, or a combination thereof. Method 700 can be performed by device 100 or by device 300.

[0217] At block 702, a red-green-blue depth (RGBD) image is captured using the imaging sensor 122. At block 704, the depth of the image is measured by the three-dimensional image capture sensor 120. At block 708, a multispectral image is acquired using the imaging sensor 122 using different excitation wavelengths and different emission wavelengths.

[0218] At block 706, before capturing each image in the multispectral wavelengths, the transfer function of the imaging sensor 122 is frozen and an auto-exposure model is run, as will be described with reference to FIG. Figure 10As explained in method 1000, a transfer function is used to convert the raw red-blue-green (RGB) sensor values into a more realistic representation of the colors perceived by the human eye. The transfer function can be, for example, a 3*3 matrix. The transfer function is frozen before imaging so that the color mixture is known and reproducible during the imaging phase. After capturing the multispectral image, the practitioner selects a region of interest (ROI). A model (such as Kaze descriptors of features and K-nearest neighbor (KNN) matching) is then used to orient the image.

[0219] After the image is directed, in box 714, the directed image is sent for federated learning. For example, the directed image can be securely transmitted to a remotely located health professional via a cloud-based server system, an email system, or otherwise transmitted electronically. The analysis model can allow for continued improvement of the analyzed image. After medical professionals from all over the world use a device (such as device 100 or device 300) and provide input about the type of information being displayed, future users of the same or another device 100 or device 300, whether the future user is located near the previous user or remote from the previous user / medical professional, benefit from "learning" that is based on previous human input from the knowledge of medical professionals who previously used the system. This is called federated learning, which is a machine learning technique that trains a model across multiple distributed edge devices or servers that maintain local data samples without exchanging local data samples.

[0220] In block 716, the wound may be segmented for spatial and dimensional parameters to be populated in the final report. Spatial parameters may be, for example, the extent of granulation tissue, slough, necrotic tissue, maceration, etc. Dimensional parameters may be, for example, the length of the wound area, the width of the wound, the perimeter of the wound, the depth of the wound, the area of the wound, or a combination thereof. In blocks 718-722, the wound is then segmented into sub-regions of interest, which are then spatially segmented by connected components and passed through a sparsity filter. At block 724, the analysis model may classify the output as Gram-positive or Gram-negative, and at block 726, it is displayed on the report page, as shown in FIG. Figure 1 and Figure 3 Explained.

[0221] Figure 8A method 800 for detecting problematic cellular entities according to an embodiment of the present subject matter is shown. The order in which the method blocks are described should not be construed as limiting, and some of the described method blocks may be combined in any order to implement method 800 or an alternative method. Additionally, some individual blocks may be deleted from method 800 without departing from the scope of the subject matter described herein. Here, steps 718-724 of method 700 are explained. It should be understood that blocks 718a-718c correspond to Figure 7 Box 718, boxes 720a-720c correspond to Figure 7 Box 720, boxes 722a-722c correspond to Figure 7 722 of FIG. 1 , and blocks 724a-724c correspond to Figure 7 The method 800 may be performed by the device 100 or by the device 300.

[0222] At block 802, a 395 nm unfiltered image is selected for ROI subselection. In blocks 718a, 718b, and 718c, three hue-based filters are used to distinguish the different colors of fluorescence emitted from the target. The hue-based filter network includes cyan, green, and red filters, respectively. In blocks 720a, 720b, and 720c, the hue-filtered binary mask generated is passed through a connected component analyzer. Connected component analysis separates the disconnected components and labels them. At blocks 722a, 722b, and 722c, the labels are then passed individually through a sparsity filter. Any area less than approximately 1% or exactly 1% of the entire wound area is excluded from processing, and inference methods are used to find the edges of the wound at blocks 724a, 724b, and 724c.

[0223] Figure 9 A method 900 for detecting problematic cellular entities according to an embodiment of the present subject matter is shown. The order in which the method blocks are described should not be construed as limiting, and some of the described method blocks can be combined in any order to implement method 900 or an alternative method. In addition, some individual blocks can be deleted from method 900 without departing from the scope of the subject matter described herein. Method 900 can be performed by device 100 or by device 300.

[0224] At boxes 902-908, the red-blue-green (RGB) + depth map image is superpixelized into superpixels, for example, 8×8 superpixels. Then, at box 910, spatial and spectral features are extracted from the image. At box 912, the spatial features are passed through an analysis model alone that can predict the probability of whether a given superpixel is a wound or part of the skin. At box 914, a Gaussian blur of the image is performed and then the image is thresholded at box 916. At box 916, an outline is drawn on the image and the largest outline is selected as the wound outline. At box 920, the length, width, depth, and area of the wound are derived from the drawn outline, and the output is displayed using interface 108 at box 922.

[0225] Figure 10 A method 1000 for an automatic exposure process according to an embodiment of the present subject matter is shown. The order in which the method blocks are described should not be construed as limiting, and some of the described method blocks can be combined in any order to implement method 1000 or an alternative method. In addition, some individual blocks can be deleted from method 1000 without departing from the scope of the subject matter described herein. Method 1000 can be performed by device 100 or by device 300.

[0226] The brightness of the image captured by imaging sensor 122 may need to be optimal. In other words, the image brightness should be neither too low nor too saturated. If the image brightness is too high, the image captured by imaging sensor 122 may be saturated and appear white. If the image brightness is too low, the image captured by imaging sensor 122 may be too dark and appear black. Therefore, it may be necessary to set an optimal brightness for the image. The brightness may depend on the exposure of imaging sensor 122. In this regard, the automatic exposure model is used to control the optimal exposure of imaging sensor 122 by allowing the appropriate brightness of the image to be set. From steps 1002 to 1010, the automatic exposure model is used to set the optimal exposure of imaging sensor 122 by setting the optimal brightness of the image. In an example, the optimal brightness may be set to 100, 200, or similar values. While imaging target 101, the optimal brightness of the plurality of first light sources 130 or the plurality of second light sources 156 of device 100 or device 300 may be set to 100, 200, or the like. The automatic exposure model is an iterative model that runs until the brightness set point is met. To find the next exposure value, the secant method is used. Once the automatic exposure model has reached a set point, it will no longer run until and unless it is called again. The automatic exposure model is set for each of the plurality of first sets of excitation filters 142 because the brightness of each of the plurality of first light sources 130 can be different.

[0227] While in the above examples, the target is explained with reference to a wound, in other examples, the target may be an edible product, laboratory equipment, sanitary equipment, sanitary fixtures, biochemical assay chips, microfluidic chips, medical devices, bodily fluids, or combinations thereof.

[0228] In addition, in the reference Figure 7-11 The analytical model mentioned in the explanation corresponds to the reference Figure 1 Explanation of analytical models or references Figure 3 Analytical model for explanation.

[0229] Figure 11 A method 1100 for detecting problematic cellular entities according to an embodiment of the present subject matter is shown. The order in which the method blocks are described should not be construed as limiting, and some of the described method blocks can be combined in any order to implement method 1100 or an alternative method. In addition, some individual blocks can be deleted from method 1100 without departing from the scope of the subject matter described herein. Method 1100 can be performed by device 100 or by device 300.

[0230] Based on the 3D depth image captured at block 1102 and the white light image captured at block 1110, a point cloud is formed at block 604. Based on the point cloud image, a homography can be performed at block 1106 to overlay the depth image on the white light image captured by the CMOS visible light camera 122 at block 1108. A projective transformation is an isomorphic mapping of a projective space, arising from the isomorphism of the vector space from which the projective space is derived. It is a bijection that maps a line to a line and is therefore a collineation. In general, some collineations are not projective transformations, but the fundamental theorem of projective geometry asserts that this is not the case in the case of a real projective space of at least two dimensions.

[0231] Figure 12a A perspective view of an apparatus 1200 for inspecting a target is shown, according to an embodiment of the present subject matter. Figure 12b A perspective view of an apparatus 1200 for inspecting a target is shown, according to an embodiment of the present subject matter. Figure 12c A perspective view of an apparatus 1200 for inspecting a target is shown, according to an embodiment of the present subject matter. Figure 12d Shown is a top view of an apparatus 1200 for inspecting a target, according to an embodiment of the present subject matter. Figure 12e Shown is a top view of an apparatus 1200 for inspecting a target, according to an embodiment of the present subject matter. Figure 12f An exploded view of an apparatus 1200 for inspecting a target is shown, according to an embodiment of the present subject matter. Figure 12g Shown is a front view of an apparatus 1200 for inspecting a target, according to an embodiment of the subject matter. Figure 12hShown is a top view of an apparatus 1200 for inspecting a target, according to an embodiment of the present subject matter. Figure 12i A side view of an apparatus 1200 for inspecting a target according to an embodiment of the present subject matter is shown. For the sake of brevity, the following will be explained in conjunction with each other. Figure 12a-12i .

[0232] The device may inspect a target, such as the target 101. In addition, the device 1200 may correspond to the device 100 or the device 300. The device 1200 may perform similar functions to those of the device 100 or the device 300.

[0233] The device 1200 may include a front cover 1236 and a back cover 1234. A portable power module, such as power module 136 (similar to the power module in device 100 or device 300), may be connected to a practitioner's cellular phone 1250 via a USB cable 1238 or similar power and / or data cable. The practitioner's phone 1250 or other mobile computing device (such as a desktop computer, tablet computer, laptop computer, smart accessory (such as a smartwatch), etc.) has a touch-activated user input screen, wherein the mobile computing device is connected to a cloud server via a wired or wireless connection. It is contemplated that the mobile computing device may even be a virtual reality headset that enables the wearer to view real-time complex wound site imaging while viewing the patient, and even conceivably, to view tissue during a medical procedure, thereby enabling the surgeon to view wound-related data in real time while performing the procedure.

[0234] Mobile computing devices can be wireless (such as through The practitioner's phone or other device 1250 can then receive images and other output from methods 700-1100, as described in detail in the accompanying drawings. Figure 7-11 As explained, the composite image is then displayed to the user, who may then delete, save, or otherwise use the image and data generated by the device and transmitted to the practitioner's phone 1250.

[0235] In an example, the images can be securely transmitted to a remotely located health professional via a cloud-based server system, an email system, or other means of electronic transmission. The analytical model used in conjunction with device 1200 allows for continuous improvement of the images analyzed by device 1200. After medical professionals from around the world use device 1200 and provide input regarding the type of information being displayed, future users of the same or another device 1200, whether located near or remote from the previous user / medical professional, benefit from "learning" from the system, which provides previous human input based on the knowledge of medical professionals who previously used the system. This is called federated learning, a machine learning technique that trains a model across multiple distributed edge devices or servers that maintain local data samples without exchanging local data samples. Detailed cumulative analysis is typically done remotely from a single device using previous wound imaging data, which is stored in a non-patient-specific manner on a cloud-based computer system that communicates with the device in use via wired or wireless signals. Device 1200 allows for continuous improvement based on the knowledge of medical professionals from around the world to be used to improve the output of device 1200, even for users who may not have the same advanced level of training as some other previous users. Instead of or in addition to remote detailed analysis, analysis based on previous imaging can also be performed on device 1200 itself. The analysis models of the present disclosure can be executed faster, but potentially less detailed, using the graphics processor of device 1200, which provides faster inference. Faster inference provides essentially instantaneous assessment of image features, such as oxygenation, bioburden, and wound analysis. This essentially instantaneous data availability helps medical practitioners provide urgent and accurate care to patients.

[0236] Similar to devices 100 and 300, device 1200 includes an imaging sensor 1222, multiple first light sources 1230, a three-dimensional image capture sensor 1220, and a ranging sensor 1232. Multiple first light source shields 1224 are typically used to house the multiple first light sources 1230 and protect them within device 1200. They can also prevent light from one light source from crossing with light from another light source. Device 1200 can also include a charger plate 1240 and an optional on / off switch 1210.

[0237] Figure 13A device 1300 for inspecting a target, according to an embodiment of the present subject matter, is shown. Device 1300 may include a plurality of first optical bandpass filters or polarizers 1302, and a plurality of first light sources 1304, optionally integrated with polarizers or excitation filters, or a combination thereof. Furthermore, device 1300 may include a computing device 1308, such as a smartphone, a laptop, a desktop computer, a smart accessory (such as a smartwatch), or the like. In the example depicted herein, computing device 1308 is depicted as a smartphone. Computing device 1308 may be coupled using a clip 1330. Thus, in this example, device 1300 may utilize the plurality of first light sources 1304 to illuminate a target. Furthermore, device 1300 may include a power button 1306 to turn device 1300 on or off. Furthermore, computing device 1308 may include an imaging sensor or camera (such as imaging sensor or camera 122), a three-dimensional image capture sensor (such as three-dimensional image capture sensor 120), and a ranging sensor (such as ranging sensor 132). It should be understood that the three-dimensional image capture sensor may function as a ranging sensor.

[0238] Device 1300 may correspond to device 100, device 300, or device 1200, and may include other similar components for detecting problematic cellular entities, such as those described in reference Figure 1-Figure 4e and Figure 12a-12i The plurality of first optical bandpass filters 1302 may correspond to the plurality of first optical bandpass filters 126. The plurality of first light sources 1304 may correspond to the plurality of first light sources 130. In addition, the apparatus 1300 may detect problematic cellular entities, similar to the apparatus described in reference Figure 7-11 The device 100 or the device 300 is explained.

[0239] In addition, the image is captured and processed as in reference Figures 1-11 As explained, the processing may be performed by the computing device 1300. In some scenarios, the device 1300 may include a processor, such as the processor 140. The processor may process the image and transmit the detection results of the problematic cellular entity to the computing device 1308. In another example, part of the processing may be completed by the processor, and part of the processing may be performed by the computing device 1308. For example, the analysis of the image may be performed by the processor, and the detection of the problematic cellular entity based on the analysis may be performed by the computing device 1308. Alternatively, the analysis of the image may be performed by the computing device 1308, and the detection of the problematic cellular entity based on the analysis may be performed by the processor.

[0240] In some examples, when the target is a wound, the present subject matter is capable of detecting biofilms in the wound, as will be explained below.

[0241] Figure 14Detection of problematic cellular entities according to an embodiment of the present subject matter is shown. Here, a wound inspection device is described. In other words, the object is explained with reference to a wound. The device for inspecting a wound may include an imaging module, an interface module, and an interface. The device may correspond to device 100, device 300, device 1200, and / or device 1300. Therefore, the components mentioned herein may be similar to the components of device 100, device 300, device 1200, and / or device 1300. In addition to the functions mentioned herein, reference is made to Figure 14 The explained devices may perform similar functions as device 100 , device 300 , device 1200 , and / or device 1300 .

[0242] The imaging module may include a plurality of first light sources, a plurality of second light sources, an imaging sensor, and a three-dimensional image capture sensor. Each of the plurality of first light sources may emit excitation radiation within a predetermined wavelength range to cause one or more markers in the wound to fluoresce. The plurality of first light sources may be, for example, homogeneous light sources or inhomogeneous light sources.

[0243] Each of the plurality of second light sources can emit excitation radiation within a predetermined wavelength range without causing markers in the wound to fluoresce. The imaging sensor can directly receive light emitted by the wound in response to illumination thereof by at least one or more of the plurality of first light sources, and directly receive light reflected by at least one or more of the plurality of second light sources, without providing an optical bandpass filter between the imaging sensor and the wound. The imaging sensor can capture a plurality of first images based on light emitted by the wound, and can capture a plurality of second images based on light reflected from the wound. Here, the light is said to be directly received by the imaging sensor because the emitted light and the reflected light are not filtered by the optical bandpass filter before the image is captured.

[0244] The three-dimensional image capture sensor can illuminate the wound, and can receive light reflected from the wound in response to the three-dimensional image capture sensor illuminating the wound, and can generate a three-dimensional image of the wound based on the reflected light. To illuminate the target, the three-dimensional image capture sensor may include one or more light sources ( Figure 14 (not shown). However, in some examples, a separate light source may also be coupled to the 3D image capture sensor to illuminate the target, and light reflected from the target may be captured due to the illumination. In examples, the 3D image capture sensor may be a structured light sensor, a time-of-flight sensor, a stereo sensor, or a combination thereof.

[0245] The interface module can be coupled to the imaging module. The interface module can include a processor. The processor can be configured to analyze a first image from a plurality of first images using an analytical model, wherein the first image is a fluorescence-based image that includes fluorescence emitted from the wound. The processor can analyze a second image obtained from a plurality of second images using the analytical model. Furthermore, the processor can analyze the three-dimensional image of the wound using the analytical model to determine a change in the intensity of light emitted from the spatial region of the wound by compensating for a change in the distance of the spatial region of the wound from the three-dimensional image capture sensor and compensating for a change in the curvature of the wound relative to the three-dimensional image capture sensor. Furthermore, the processor can analyze the three-dimensional image of the wound using the analytical model to determine a change in the intensity of light reflected from the spatial region of the wound by compensating for a change in the distance of the spatial region of the wound from the three-dimensional image capture sensor and compensating for a change in the curvature of the wound relative to the three-dimensional image capture sensor.

[0246] In this regard, the processor may use an analysis model to detect the presence of a biofilm in a wound based on an analysis of the first image, the second image, and the three-dimensional image. The analysis model may be trained to detect the presence of a biofilm in a wound. The analysis model may create a composite image of the first image, the second image, and the three-dimensional image of the wound. The interface may display a result corresponding to the detection of the biofilm in the wound and the composite image of the first image, the second image, and the three-dimensional image of the wound.

[0247] To detect biofilms in wounds, an analysis model was trained using multiple fluorescence-based reference images with biofilms, multiple 3D images with biofilms, and multiple fluorescence-based reference images without biofilms. The analysis model was trained to distinguish fluorescence emitted from biofilms in the fluorescence-based images from fluorescence emitted from areas outside the biofilms in the fluorescence-based images.

[0248] The analysis model may include, for example, multiple neural networks. Each of the multiple neural networks can extract relevant parameters from each modality (such as from a first image, a second image, and a three-dimensional image). For example, a first neural network can extract relevant parameters from the first image, a second neural network can extract relevant parameters from the second image, and a third neural network can extract relevant parameters from the third image. In addition, a fourth neural network can perform a fusion of the parameters extracted by the three neural networks from the first image, the second image, and the three-dimensional image to detect biofilms in wounds. Alternatively, all images can be sent to a single neural network to identify spatial regions within the target containing biofilms.

[0249] In addition to the first image, the second image, and the three-dimensional image, the analysis model can also utilize a polarization image. Thus, the apparatus may include a first polarizer disposed between the plurality of first light sources and the target to pass excitation radiation from the plurality of first light sources of a first polarization. The apparatus may also include a second polarizer disposed between the target and the imaging sensor to pass light emitted by the target of a second polarization.

[0250] In one example, the first polarizer and the second polarizer can be arranged in a perpendicular configuration, aligned at 90 degrees to each other. Furthermore, in another example, the first polarizer and the second polarizer can be arranged in a parallel configuration. When using a polarizer, the analysis model can include another neural network to extract parameters from the polarized image. Furthermore, the neural network can perform a fusion of parameters extracted by the neural network from the first image, the second image, the three-dimensional image, and the polarized image to detect biofilm in the wound.

[0251] In an example, the first polarization and the second polarization may be the same. For example, in an example, the first polarization and the second polarization may be left-handed circular polarization (LHCP). In another example, the first polarization and the second polarization may be right-handed circular polarization (RHCP). In another example, the first polarization and the second polarization may be different. For example, the first polarization may be one of LHCP or RHCP, and the second polarization may be the other of LHCP or RHCP.

[0252] In some examples, the plurality of polarizers may include a third polarizer positioned between the plurality of second light sources and the target to pass excitation radiation from the plurality of second light sources of a third polarization. The plurality of polarizers may be combined with a first set of excitation filters. In some examples, if the device includes a plurality of first optical bandpass filters for use as emission filters and positioned between the target and the imaging sensor, the plurality of polarizers may be combined with the plurality of first optical bandpass filters.

[0253] The device may include multiple second light sources for illuminating the target without causing one or more markers in the target to fluoresce. One or more of the multiple second light sources are configured to emit light with a wavelength in the visible region. The imaging sensor may be configured to capture multiple second images formed based on light reflected from the target in response to illumination thereof by at least one or more of the multiple second light sources. The processor 140 may analyze the three-dimensional image of the wound using an analytical model to determine variations in the intensity of light reflected across the spatial region of the wound by compensating for variations in distance from the three-dimensional image capture sensor across the spatial region of the wound and for variations in wound curvature relative to the three-dimensional image capture sensor. The processor may be configured to analyze a second image obtained from the multiple second images using the analytical model. The processor may be configured to use the analytical model to detect the presence of a problematic cellular entity in the target based on an analysis of the first image, the second image, and the three-dimensional image. The processor may use the first image, the second image, and the three-dimensional image to create a composite image of the target. The interface may be configured to display results corresponding to the detection of the problematic cellular entity and a composite image of the first image, the second image, and the three-dimensional image of the target.

[0254] In an example, a device may include a first set of excitation filters. Each of the first set of excitation filters may be configured to filter excitation radiation within a predetermined wavelength range emitted by a light source in the plurality of first light sources, allowing the excitation radiation within the predetermined wavelength range to pass through the excitation filter to illuminate the target. Furthermore, one or more excitation filters may be further configured to filter excitation radiation within a predetermined wavelength range emitted by a light source in the plurality of second light sources, allowing the excitation radiation to pass through the excitation filter.

[0255] In the example depicted herein, a reflectance image 1402 and a fluorescence image 1404 corresponding to a wound are provided as input to an analysis model 1406 comprising a plurality of neural networks to detect biofilm in the wound, as depicted in image 1408. The analysis model 1406 is similar to the analysis models previously mentioned or referenced herein. Figures 1-11 The analytical model explained is the same. In the example, the device is capable of detecting autofluorescence signals emitted from the extracellular matrix (ECM) of the biofilm. The device can also detect autofluorescence from quorum sensing elements released into the ECM.

[0256] Furthermore, the device can distinguish biofilms in wounds from planktonic bacteria in wounds. For example, reflective scattering and fluorescence at multiple wavelengths may differ between biofilms and planktonic bacteria. Furthermore, biofilms may have characteristics, such as higher specular reflectance compared to planktonic bacteria, which can be captured from a reflective image. Thus, by analyzing multiple first images, multiple second images, and a three-dimensional image, the device can distinguish between planktonic bacteria and biofilms. Furthermore, multiple polarizers can be used to capture differences in reflectance to enable differentiation between planktonic bacteria and biofilms. Reflection can consist of specular and diffuse reflections. In reflective imaging, specular and diffuse reflections can be obtained by having a parallel polarization geometry between a polarizer positioned between the light source and the target and a polarizer positioned between the imaging sensor and the target. Diffuse reflections can be obtained by having a perpendicular polarization geometry between a polarizer positioned between the light source and the target and a polarizer positioned between the imaging sensor and the target.

[0257] Detection of biofilms can facilitate better and faster wound care management. For example, wounds with biofilms are resistant to antibiotics and may take longer to heal. Therefore, when the subject device is able to detect biofilms in wounds, treatment can be provided accordingly. For example, interventional procedures, such as wound debridement, can be performed to effectively remove the biofilm from the wound. This enables the wound to heal faster.

[0258] In the above example, the device is explained with reference to detecting biofilms in wounds by capturing the wound. As an alternative or supplement to the above-mentioned capture of wounds, the device can also capture blotting paper to detect biofilms. The blotting paper can be embedded with chemicals, such as ruthenium red, alcian blue, etc., and can be pressed against the wound. Subsequently, the blotting paper can be imaged by an imaging sensor. The analysis model can analyze the image of the blotting paper and detect the presence of biofilms. For example, polysaccharides in the exudate are collected by attaching a nitrocellulose membrane to the wound surface, and the biofilm is visualized by staining with ruthenium red or alcian blue. In another example, the presence of biofilms in wounds can be detected by charged blotting paper. For example, the charged blotting paper can be pressed against the wound and can be captured by an imaging sensor. The analysis model can analyze the image and detect the presence of biofilms in the wound.

[0259] Figure 15 A system 1500 for inspecting a target according to an embodiment of the present subject matter is shown. Processing device 1501 may be a computing device, such as a server, provided at a remote location, such as on the cloud. Processing device 1501 may include computer(s), server(s), cloud device(s), or any combination thereof. Device 100 may be connected to processing device 1501 via a communication network 1501. According to an embodiment, an analytical model is on processing device 1501. The analytical model may correspond to a reference Figure 1-2c and Figure 7-11 The analytical model can also correspond to the reference Figure 3-4e The aforementioned analysis model. Processing device 1501 may include processor 2402 that implements the analysis model. Processor 1502 may correspond to processor 140. Thus, device 100 may capture fluorescence-based images and white light images of a target and transmit them to processing device 1501. Upon detecting and classifying a pathogen, processing device 1501 may transmit the analysis results to device 100, which may then display the results on interface 108.

[0260] In an implementation, the device 100 may execute as shown in FIG. Figure 1-2c and Figure 7-11 Furthermore, the training can be similar to the reference Figure 5 Although the devices are explained with reference to device 100 in the examples depicted herein, the devices of system 1500 may also correspond to device 300 or device 1200 in some examples.

[0261] Figures 16a-16b A method for inspecting a target according to an embodiment of the present subject matter is shown. The order in which method 1600 is described should not be construed as limiting, and any numbers in the described method blocks can be combined in any order to implement method 1600 or an alternative method. In addition, method 1600 can be implemented by a processor or computing device using any suitable hardware, non-transitory machine-readable instructions, or a combination thereof. The method can be performed by device 100, device 300, device 1200, device 1300, and / or system 1500. Therefore, the components described with reference to method 1600 can correspond to corresponding components of device 100, device 300, device 1200, device 1300, and / or system 1500.

[0262] At step 1602, method 1600 can include illuminating a target using at least one or more light sources of a plurality of first light sources of a device. Light emitted by each of the plurality of first light sources has a wavelength band.

[0263] At step 1604, a plurality of first images may be captured by an imaging sensor. The imaging sensor may be configured to receive light emitted by an object in response to illumination thereof by at least one or more of the plurality of first light sources. The plurality of first images may be formed based on the light emitted by the object.

[0264] At step 1606, method 1600 may include capturing, by a 3D image capture sensor, a 3D image of a target. The 3D image capture sensor may be configured to illuminate the target, and may receive light reflected from the target in response to the 3D image capture sensor illuminating the target, and may generate a 3D image of the target based on the reflected light. To illuminate the target, the 3D image capture sensor may include one or more light sources integrated with the 3D image capture sensor. However, in some examples, a separate light source may also be coupled to the 3D image capture sensor to illuminate the target, and light reflected from the target may be captured as a result of the illumination.

[0265] A first image of the plurality of first images may be analyzed by the processor using the analysis model at step 1608. The first image may be a fluorescence-based image including fluorescence from the target in response to light emitted by at least one or more of the plurality of first light sources.

[0266] In step 1610, the three-dimensional image of the target can be analyzed by the processor to determine changes in the intensity of light emitted from the spatial region of the target by compensating for changes in the distance of the spatial region of the target from the three-dimensional image capture sensor and compensating for changes in the curvature of the spatial region of the target.

[0267] At step 1612, method 1600 includes, by the processor, detecting, based on the analysis of the first image and based on the three-dimensional image of the object, the presence of problematic cellular entities in the object using the analysis model. The analysis model can be trained to detect the presence of problematic cellular entities in the object.

[0268] At step 1614, a composite image of the first image and the three-dimensional image of the target may be created. At step 1616, a result corresponding to the presence of the problematic cellular entity and the composite image of the first image and the three-dimensional image may be displayed via the interface.

[0269] In an example, an analysis model can be trained using multiple fluorescence-based reference images and multiple reference three-dimensional images to detect the presence of problematic cellular entities in a target. The analysis model can be trained to distinguish between fluorescence emitted from the problematic cellular entity in the fluorescence-based image and fluorescence emitted from areas other than the problematic cellular entity in the fluorescence-based image.

[0270] The target may be a wound area. In this regard, the method includes, by a processor, extracting spatial features and spectral features of the wound area from the first image and the three-dimensional image using an analysis model. The location of the wound area may be identified by the processor using the analysis model based on the extraction of the spatial features and spectral features. The method includes, by the processor, determining the outline of the wound area based on the extraction of the spatial features and spectral features using the analysis model. Pathogens in the wound area may be detected by the processor using the analysis model based on the extraction of the spatial features and spectral features. The method includes, by the processor, using the analysis model, classifying the pathogen by at least one of the family, genus, species, or strain of the pathogen.

[0271] In an example, method 1600 may include determining, by the processor, a length of the wound area, a width of the wound, a perimeter of the wound, a depth of the wound, an area of the wound, or a combination thereof based on determining a contour of the wound area using an analytical model.

[0272] Furthermore, in an example, the target is one of: a wound area, an edible product, a laboratory device, a medical device, a sanitary appliance, a sanitary fixture, a biochemical assay chip, a microfluidic chip, a body fluid, or a combination thereof. Furthermore, the method may include determining, by the processor, in response to detecting the presence of a problematic cellular entity, at least one of: the extent of infection in the wound area, the spatial distribution of pathogens in the wound area, or the healing rate of the wound area when the target is a wound area. Furthermore, the method includes, when the target is a tissue, detecting, by the processor, the presence of a problematic cellular entity (such as at least one of cancerous tissue, necrotic tissue, or a combination thereof) in a tissue sample. Furthermore, the method may include determining, by the processor, the problematic cellular entity as a pathogen, and classifying, by the processor, the pathogen in the target when the target is one of: a sanitary appliance, a sanitary fixture, a medical device, a biochemical assay chip, a body fluid, or a microfluidic chip.

[0273] In an example, method 1600 may include filtering, using one of a plurality of first optical bandpass filters, light of a predetermined wavelength emitted by an object in response to illumination by at least one or more of the plurality of first light sources to pass the light. The optical bandpass filter may be positioned between the object and an imaging sensor. The imaging sensor may capture the filtered light from the optical bandpass filter.

[0274] Figure 17Results 1700 corresponding to the detection of problematic cellular entities are shown, according to an embodiment of the present subject matter. In the example depicted herein, fluorescence-based images, such as autofluorescence images, of a wound are captured by illuminating the wound with various UV-visible wavelengths (such as 365 nm, 395 nm, 415 nm, and 450 nm) from an appropriate light source after the light passes through an appropriate narrow-band bandpass filter and a linear polarizer. The autofluorescence image is captured after linearly polarizing the fluorescence response from the wound by placing a linear polarizer in front of the imaging sensor such that the polarization axis of the imaging sensor is perpendicular to the polarization axis of the polarizer in front of the light source. Image 1702 depicts an exemplary autofluorescence image of a wound at an excitation wavelength of 365 nm.

[0275] In addition, a three-dimensional image capture sensor, such as a depth camera, is used to obtain a 3D depth image and a white light image of the wound. The 3D depth image is depicted by image 1704 .

[0276] The autofluorescence image (such as image 1702) and the 3D depth image (such as image 1704) are fed into the analysis model 1706 along with the white light image of the wound, which predicts the area of the wound where the problematic cellular entity is present. The analysis model 1706 provides a depth image with an overlay of autofluorescence intensity indicating the presence of the problematic cellular entity, such as depicted by image 1708.

[0277] Figure 18 Results 1800 corresponding to the detection of problematic cellular entities are shown, according to an embodiment of the present subject matter.

[0278] In this example, images captured in multiple modalities, including fluorescence-based imaging (such as autofluorescence images), reflectance imaging (such as NIR reflectance imaging), and 3D depth imaging are fed into the analysis model.

[0279] Image 1802 depicts an autofluorescence image captured at an excitation wavelength of 365 nm. Image 1804 depicts an autofluorescence image captured at an excitation wavelength of 395 nm. Image 1806 depicts a frame from an NIR diffuse reflectance image captured at an excitation wavelength of 660 nm. Image 1808 depicts a frame from an NIR diffuse reflectance image captured at an excitation wavelength of 850 nm. Image 1810 depicts a 3D depth image. Images 1802, 1804, 1810 and videos 1806, 1808 are provided as input to an analysis model 1812. In this example, analysis model 1812 is a deep convolutional neural network. Analysis model 1812 predicts wound areas that show the presence of problematic cellular entities. For example, in image 1814, an autofluorescence image is overlaid with a mask that indicates the predicted area of ​​the presence of problematic cellular entities. The region labeled 1816 corresponds to the pathogen Pseudomonas aeruginosa.

[0280] Figure 19 Results corresponding to the detection of problematic cellular entities according to an embodiment of the present subject matter are shown 1900. In this example, images captured with multiple modalities, including fluorescence-based imaging (such as autofluorescence imaging), reflectance imaging (such as NIR reflectance imaging), and 3D depth imaging, are fed into an analysis model. The analysis model predicts wound areas that show the presence of problematic cellular entities and wound areas that exhibit low, medium, and high tissue oxygen saturation.

[0281] Image 1902 depicts an autofluorescence image captured at an excitation wavelength of 365 nm. Image 1904 depicts an autofluorescence image captured at an excitation wavelength of 395 nm. Image 1906 depicts an NIR diffuse reflectance image captured at an excitation wavelength of 660 nm. Image 1908 depicts a frame from an NIR diffuse reflectance video, captured at an excitation wavelength of 850 nm. Image 1910 depicts a 3D depth image. Images 1902, 1904, 1906, 1910, and video 1908 are provided as inputs to analysis model 1912. In this example, analysis model 1912 is a deep neural network. As depicted in image 1916, analysis model 1912 predicts wound regions that display the presence of the problematic cellular entity depicted in image 1914 and problematic cellular regions of the wound that exhibit low, moderate, and high tissue oxygen saturation. In image 1914, the autofluorescence image is overlaid with a mask indicating the predicted region of problematic cellular entities, which (represented by region 1915) are identified as the pathogen Pseudomonas aeruginosa.

[0282] Figure 20Results 2000 corresponding to tissue oxygenation saturation according to an embodiment of the present subject matter are shown. In this example, time-varying NIR reflectance images captured at different NIR excitation wavelengths (such as 660nm, 740nm, 850nm) are imaged as separate videos (such as the videos depicted by 2002, 2004, 2006), first passed through an image and video processing module 2008. The image and video processing module 2008 can be part of a processor, such as processor 140 including a GPU. The image and video processing module 2008 can obtain the target's heart rate from the video and can filter the video in the temporal direction so that only a narrow frequency band around the heart rate frequency is retained. As depicted in image 2012, the filtered frame group is now passed to the analysis model 2010, which predicts problematic cell areas with low, medium, and high tissue oxygen saturation. In this example, the analysis model 2010 used is a deep convolutional neural network.

[0283] Figure 21 Results 2100 corresponding to detecting a biofilm in a wound according to an embodiment of the present subject matter are shown. In this example, as shown in image 2112, a white light image 2102 and fluorescence-based images (such as autofluorescence images 2104, 2106, and 2108 of the wound captured at different illumination wavelengths (such as 365 nm, 395 nm, and 450 nm, respectively) are used to train an analysis model 2110 to predict areas of the wound with biofilm. Image 2112 is a white light image of the wound covered with a detected biofilm (area 2113 in image 2112). In this example, the analysis model is a deep neural network. Additionally, oxygenation or thermal images can be added to the analysis model to improve biofilm detection accuracy.

[0284] Figure 22 Results corresponding to the detection of problematic cellular entities are shown, according to an embodiment of the present subject matter. In this example, a multispectral camera is used to capture fluorescence-based images of the wound in different wavelength bands, such as autofluorescence images. In addition, multi-channel images (such as images 2202, 2204, 2206, 2208, 2210) are processed by an analysis model 2212 (such as a deep neural network, such as a convolutional neural network) to predict wound areas with specific problematic cellular entities. Image 2214, a white light image of the wound, overlaps with the predicted area of the problematic cellular entity. In image 2214, area 2215 corresponds to the pathogen Staphylococcus aureus.

[0285] Figure 23a Results 2300 corresponding to the detection of problematic cellular entities are shown, according to an embodiment of the present subject matter.

[0286] A multispectral camera is used to capture autofluorescence images of the wound across various wavelengths, including visible and UV wavelengths. For example, image 2302 depicts an image captured at an illumination wavelength of 365 nm. Image 2304 depicts an image captured at an illumination wavelength of 395 nm, and image 2306 depicts an image captured at an illumination wavelength of 450 nm. Furthermore, image 2304 depicts a three-dimensional image of the wound. All of these images are provided as input to an analysis model 2308. Analysis model 2308 may be referred to as a tissue detection network (TDN). TDN 2308 can process multi-channel and three-dimensional images to identify specific problematic tissue areas within the wound. TDN 2308 can predict the composition of wound tissue, including elements such as granulation tissue, slough tissue, and necrotic tissue. The projected images are superimposed on a white-light image of the wound, facilitating the layering of the wound healing trajectory, as shown in image 2312. Image 2310 is a white-light image of the wound overlaid with predicted areas of slough tissue 2314 and granulation tissue 2316.

[0287] Figure 23b Results 2300 corresponding to the detection of problematic cellular entities are shown according to an embodiment of the present subject matter. In this example, as shown in FIG. Figure 23a As explained, TDN 2310 of a white light image is predicted to characterize areas of problematic tissue 2312 for further examination. Image 2312 is fed into analysis model 2318. Analysis model 2318 can be, for example, a deep learning network and can be referred to as a tissue-aware oxygenation prediction deep learning network. Additionally, images 2320 and 2322, corresponding to NIR diffuse reflectance wavelength images at illumination wavelengths of 660 nm and 850 nm, respectively, are provided as inputs to tissue-aware oxygenation prediction deep learning network 2318. Tissue-aware oxygenation prediction deep learning network 2318 generates a tissue-aware oxygenation image that emphasizes areas of problematic tissue, as shown in image 2324. Image 2324 corresponds to tissue-aware deep learning network 2318 predicting an oxygenation image with highlighted areas of slough and granulation tissue.

[0288] Figure 24Results 2400 corresponding to the detection of problematic cellular entities, according to an embodiment of the present subject matter, are shown. In the example depicted herein, a multispectral camera in multimodal detection mode is used to capture autofluorescence images, and a thermal camera captures temperature distribution images of the wound. Image 2412 corresponds to an image of the wound captured using the multispectral camera. Image 2414 corresponds to an image of the wound captured using a thermal imaging sensor. Images 2412 and 2414 are fed into an analysis model 2416. Analysis model 2416 is a deep neural network. Based on autofluorescence and thermal signatures, deep neural network 2416 predicts wound areas indicating the presence of problematic cellular elements. Image 2418 corresponds to an autofluorescence image with the detected problematic cellular entity.

[0289] Figure 25 Results 2500 corresponding to the detection of problematic cellular entities are shown according to an embodiment of the present subject matter. In this example, the wound is excited by a set of pulsed UV LEDs at wavelengths of 395 nm and 365 nm, as shown in image 2502, which repeat at a rapidly periodic predetermined rate. Generally, ambient light can couple into the imaging process, resulting in a constant shift in the measured intensity across all red, green, and blue (R, G, B) channels of the imaging sensor or a 50 / 60 Hz (depending on geography) oscillating component in all (R, G, B) channels. To counteract this effect, the excitation is performed at a frequency f different from these frequencies. ex An image is captured (autofluorescence image captured with ambient light), as depicted by image 2504, and the resulting image is pre-processed by image processing block 2506, which specifically looks for f ex The pre-processed data is then fed into an analysis model 2508 (such as a deep neural network) that determines regions with problematic cellular entities based on their autofluorescence signatures, as depicted in image 2510. ex ), the same number of target autofluorescence frames required for subsequent detection can be acquired in a proportionally shorter duration. Although in the above examples, image preprocessing is explained separately from the analysis model 2508, in some examples, image preprocessing can be performed by the analysis model 2508. In this example, the faster pulses also allow images to be captured with shorter durations, thereby reducing the overall imaging time. In addition, the image can be captured with a single high-power pulse of the LED, thereby reducing the exposure time of the imaging sensor and reducing the contribution of ambient light relative to the emitted light due to the high power of the illuminating light. Therefore, the overall imaging time is significantly reduced. For example, if the pulse width is reduced from 1ms to 0.1ms, the overall imaging time is reduced by a factor of 10. Therefore, any noise from patient or device movement is significantly reduced.

[0290] Figure 26 Results 2600 corresponding to the detection of problematic cellular entities are shown in accordance with an embodiment of the present subject matter. In the example depicted herein, the wound is excited by a set of pulsed UV LEDs, depicted by 2602, which repeat at a periodic predetermined rate. Generally, ambient light can couple into the imaging process, resulting in a constant offset in the measured intensity across all R, G, B channels of the imaging sensor or a 50 / 60 Hz (depending on geography) oscillating component in all (R, G, B) channels. To counteract this effect, the excitation is performed at a frequency f different from these frequencies. ex Pulse. The resulting image with ambient noise (autofluorescence image frame 2604 captured with ambient light) is passed to an analysis model 2606, such as a long short-term memory (LSTM) detection model, which combines the image processing required to denoise the raw data from environmental interference and subsequently detect and present areas with problematic cellular entities. Image 2608 depicts a white light image overlaid with areas with problematic cellular entities. In this example, the faster pulse also allows images to be captured with shorter durations, thereby reducing the overall imaging time. In addition, the image can be captured with a single high-power pulse of the LED, thereby reducing the exposure time of the imaging sensor and reducing the contribution of ambient light relative to the emitted light due to the high power of the illuminating light. Therefore, the overall imaging time is significantly reduced. For example, if the pulse width is reduced from 1ms to 0.2ms, the overall imaging time is reduced by a factor of 5. Therefore, any noise to patient or device movement is significantly reduced.

[0291] Figure 27 Result 2700 corresponding to the detection of problematic cellular entities according to an embodiment of the present subject matter is shown. In the example depicted herein, the overall functionality of the device for inspecting a target in the architecture variant is split between the CPU and the GPU. The CPU is responsible for the excitation and detection processes 2704, 2706, which are performed by emitting light at different excitation wavelengths λ. ex The autofluorescence images of the wound are captured by excitation at a wavelength of 365 nm, as shown in images 2708, 2710, and 2712. nm , 395 nm, and 415 nm autofluorescence images. The GPU may include and execute an analysis model 2714, such as a deep neural network, responsible for edge reasoning functions to determine areas with problematic cellular entities. Image 2716 corresponds to a white light image of a wound with areas of problematic cellular entities marked.

[0292] The fact that the system of the present disclosure creates composite images has another significant benefit to the user(s) of such systems. In particular, the system can be used to capture one or more images at any angle and at any distance, while still creating an accurate composite rendering of the image and information related to the wound that is provided to a medical professional or other user. Thus, the user may not need as much or any significant training in the use of the device, but can simply use the device to capture images in a manner similar to that of capturing a standard portrait image. This allows non-medical professionals or medical professionals with less training to use the device while still obtaining accurate information. As discussed herein, a non-medical professional or medical professional can transmit the image or image sequence to a remotely located medical professional for additional consultation prior to treatment using the device(s) of the present disclosure.

[0293] The present subject matter is capable of providing faster image capture and processing to detect problematic cellular entities. Since in the present subject matter, the processor and imaging module are provided onboard, the present subject matter is capable of capturing and processing images faster. In particular, by using a combination of a CPU and a GPU and an optional FPGA, the present subject matter is capable of capturing and processing images at a frequency of more than 30 images per second. An analysis model is trained on several fluorescence-based reference images and several reference three-dimensional images to detect the presence of problematic cellular entities in the target, thereby improving the accuracy of detection. The present subject matter ensures that the light emission of the light source is at a different frequency from the ambient light source. Therefore, the present subject matter is capable of eliminating the interference of ambient light on the light emitted by the target. In addition, in the present subject matter, the pulsed LED can be actuated at a faster frequency (such as from 100 Hz to tens of MHz). Therefore, the present subject matter enables faster capture of multiple first images and three-dimensional images and reduces ambient light interference (background interference). Therefore, the present subject matter eliminates background information and improves the accuracy of detection.

[0294] Furthermore, in examples, the analysis model can ignore background light and excitation light in fluorescence-based images and can even pick up weak fluorescence information in fluorescence-based images. Thus, in examples, the present subject matter also eliminates the use of emission filters for filtering background light and excitation light, as well as the use of filter wheels. Consequently, the present subject matter apparatus is simple and cost-effective.

[0295] In the present subject matter, variations in the distance from the three-dimensional image capture sensor to the target's spatial region and variations in the curvature of the target's spatial region relative to the three-dimensional image capture sensor are compensated for. Consequently, the present subject matter can improve the accuracy of detecting problematic cellular entities, particularly for targets such as wounds. Because the device can transmit the resulting, composite images to a cloud server, non-medical professionals or medical professionals can transmit images or image sequences to a remotely located medical professional for additional consultation prior to treatment using the disclosed device.

[0296] Thus, the present subject matter provides rapid, optionally filter-free, non-invasive, automated, and in situ detection and classification of pathogens using "optical computational biopsy" technology, which uses multispectral imaging along with computational models (such as machine learning models, artificial neural network (ANN) models, deep learning models, etc.) for non-invasive biopsy to detect and classify problematic cellular entities.

[0297] The subject matter can be used to detect the presence of problematic cellular entities in diabetic foot ulcers, surgical site infections, burns, skin, and the interior of the body (such as the esophagus, stomach, and colon). The subject matter device can be used in the fields of dermatology, cosmetology, plastic surgery, infection management, photodynamic therapy monitoring, and antimicrobial susceptibility testing.

[0298] The device can be integrated into normal clinical procedures and can be used for telemedicine and telehealthcare. Furthermore, most clinically relevant pathogens can be detected and classified within minutes. Furthermore, data acquisition and analysis can occur automatically. Therefore, the device can be easily operated without the need for skilled technicians. This feature facilitates rapid decision-making on treatment options. The device can also be used to detect and classify pathogens in resource-limited settings. The subject device can also be used for endoscopy.

[0299] The device of the present subject can be used for quantifying various pathogens present in a target. The device can also be used to monitor wound healing and wound closure. The device can also be used to study antimicrobial sensitivity by exposing the target to various antibiotics, observing and analyzing the target. For example, the device can be used to study bacteria that grow on antibiotics and can record corresponding biomarker characteristics. This information can be used to obtain information about antibiotics to be prescribed based on the antimicrobial sensitivity of specific bacteria. It should be understood that the antimicrobial sensitivity of other pathogens (such as fungi) can also be studied. In addition, the dosage and concentration of the antibiotic can also be determined based on the dilution factor to determine the dosage of the antibiotic or antifungal agent to be administered.

[0300] The device can be configured to study the biomolecular composition and kinetic behavior of various pathogens based on their fluorescence signatures. The device can also be used for cosmetic purposes. For example, the device can be used to detect the presence of Propionibacterium acnes, which causes acne. The device can also be used during tissue transplants to ensure that the tissue is free of pathogens. The device can be used for forensic testing, for example, to detect pathogens in body fluids (such as saliva, blood, mucus, etc.). The device can be configured to study the effectiveness of disinfectants on various hospital surfaces (such as beds, walls, hands, gloves, bandages, dressings, catheters, endoscopes, hospital equipment, sanitary equipment, etc.).

[0301] The device can also be used to detect the presence of pathogens on hands and surfaces, for example, in hospitals and other places that need to be pathogen-free. The device can be used to detect pathogen contamination in edible products such as food, fruits and vegetables.

[0302] Although the examples and embodiments of the present subject matter have been described in language specific to structural features and / or methods, it should be understood that the present subject matter is not necessarily limited to the specific features or methods described. Instead, specific features and methods are disclosed and explained in the context of several example embodiments of the present subject matter.

Claims

1. A device for inspecting a target, the device comprising: Imaging module, including: a plurality of first light sources, wherein each light source is configured to emit excitation radiation at a predetermined wavelength range to cause one or more markers in the target to fluoresce; an imaging sensor configured to directly receive light emitted by the target in response to the target being illuminated by at least one or more of the plurality of first light sources, without an optical bandpass filter disposed between the imaging sensor and the target, and to capture a plurality of first images formed based on the emitted light; and a three-dimensional image capture sensor for illuminating the target and receiving light reflected by the target in response to the illumination of the target by the three-dimensional image capture sensor, and generating a three-dimensional image of the target based on the reflected light; an interface module coupled to the imaging module, the interface module comprising: The processor is configured to: analyzing a first image of the plurality of first images using an analysis model, wherein the first image is a fluorescence-based image including fluorescence from the target; analyzing the three-dimensional image of the target using the analytical model to determine changes in intensity of light emitted by the spatial region of the target by compensating for changes in the distance of the spatial region of the target from the three-dimensional image capture sensor and compensating for changes in the curvature of the spatial region of the target; detecting the presence of problematic cellular entities in the object based on the analysis of the first image and the three-dimensional image using the analysis model, wherein the analysis model is trained to detect the presence of problematic cellular entities in the object; and creating a composite image of the first image and the three-dimensional image of the target; and Interface for: Results corresponding to the detection of the problematic cellular entity and the composite image of the target are displayed.

2. The device according to claim 1, wherein The analysis model is trained using a plurality of fluorescence-based reference images and a plurality of reference three-dimensional images to detect the presence of the problematic cellular entity in the target, and wherein the analysis model is trained to distinguish between fluorescence emitted from the problematic cellular entity in the fluorescence-based images and fluorescence emitted from areas outside of the problematic cellular entity in the fluorescence-based images.

3. The device according to claim 1, further comprising a system module (SOM), wherein the SOM comprises: the imaging module; the processor; as well as A plurality of light source drivers, wherein each light source driver of the plurality of light source drivers is configured to adjust a corresponding light source of the plurality of first light sources.

4. The device according to claim 3, wherein One or more of the plurality of first light sources are pulsed light emitting diodes (LEDs), wherein the processor is configured to actuate one or more of the plurality of light source drivers to regulate the pulsed LEDs to emit excitation radiation pulses to achieve faster imaging and reduce ambient light interference in the light emitted by the target.

5. The apparatus according to claim 1, wherein The processor is configured to operate the imaging sensor to capture and process the plurality of first images at more than 30 frames per second.

6. The apparatus according to claim 1, wherein The imaging module further includes: a plurality of second light sources for illuminating the target without causing the one or more markers in the target to fluoresce, wherein each of the plurality of second light sources is configured to emit light with a wavelength in the near infrared (NIR) region or the visible region, wherein the imaging sensor is configured to capture a plurality of second images formed based on light reflected by the target in response to illumination of the target by at least one or more of the plurality of second light sources; and wherein the processor is configured to: analyzing, using the analysis model, second images obtained from the plurality of second images to identify oxygenation at a plurality of regions in the target; analyzing the three-dimensional image of the target using the analysis model to determine a change in the intensity of the light reflected from the spatial region of the target by compensating for a change in the distance of the spatial region of the target from the three-dimensional image capture sensor and compensating for a change in the curvature of the spatial region of the target; detecting the presence of problematic cellular entities in the target based on the analysis of the first image, the second image, and the three-dimensional image using the analysis model; and creating a composite image of the target using the first image, the second image, and the three-dimensional image; and the interface is configured to: Results corresponding to the detection of the problematic cellular entity and the composite image of the first image, the second image, and the three-dimensional image of the object are displayed.

7. The apparatus according to claim 6, wherein The processor is configured to: activating the plurality of first light sources to emit light toward the target; activating the plurality of second light sources to emit light toward the target; as well as The imaging sensor is activated to capture light emitted by the target in response to illumination of the target by at least one or more of the plurality of first light sources, and to capture light emitted by the target in response to illumination of the target by at least one or more of the plurality of second light sources.

8. The apparatus according to claim 1, comprising: a plurality of second light sources for illuminating the target without causing the one or more markers in the target to fluoresce, wherein at least one or more of the plurality of second light sources is configured to emit light with a wavelength in the visible region, wherein the imaging sensor is configured to capture a plurality of third images formed based on light reflected by the target in response to illumination of the target by the at least one or more light sources of the plurality of second light sources, wherein the plurality of third images are white light images; and wherein the processor is configured to: analyzing a third image obtained from the plurality of third images using the analysis model; analyzing the three-dimensional image of the target using the analysis model to determine a change in the intensity of the light reflected from the spatial region of the target by compensating for a change in the distance of the spatial region of the target from the three-dimensional image capture sensor and compensating for a change in the curvature of the spatial region of the target; detecting the presence of problematic cellular entities in the target based on the analysis of the first image, the third image, and the three-dimensional image using the analysis model; as well as creating a composite image of the target using the first image, the third image, and the three-dimensional image; And the interface is configured as: Results corresponding to the detection of the problematic cellular entity and the composite image of the first image, the third image, and the three-dimensional image of the object are displayed.

9. The apparatus according to claim 1, wherein The processor is configured to control the plurality of first light sources to illuminate at a frequency different from a frequency of an ambient light source.

10. The apparatus according to claim 1, further comprising: A thermal sensor for thermal imaging of the problematic cellular entity.

11. The apparatus according to claim 1 , comprising: A first housing, configured to accommodate the imaging module; A second housing, configured to accommodate the interface module; as well as A bridge is used to connect the imaging module and the interface module, and the bridge includes an electronic interface to achieve electronic communication between the processor and the imaging module.

12. The apparatus according to claim 11, wherein The electronic interfaces include: a camera serial interface CSI, a serial management bus such as an I2C interface, a system packet interface SPI, a universal asynchronous receiver-transmitter UART, a general-purpose input / output GPIO interface, a universal serial bus USB interface, a pulse width modulation PWM interface, a display-serial interface DSI, and a high-definition multimedia interface HDMI.

13. The apparatus of claim 1 , further comprising: a portable power module operable to supply power to components of the imaging module and the interface module; as well as The third housing is configured to accommodate the portable power module.

14. The apparatus of claim 1 , further comprising: A distance measuring sensor is operable to determine the distance of the target from the device for positioning the device at a predetermined distance from the target.

15. The apparatus according to claim 1, wherein The three-dimensional image capture sensor is operable to determine a distance of the target from the device for positioning the device at a predetermined distance from the target.

16. The apparatus of claim 1 , wherein the target is a wound area, and wherein the processor is further configured to: extracting spatial and spectral features of a wound area from the first image and the three-dimensional image by using the analysis model; identifying a location of the wound area based on the extraction of the spatial features and the spectral features by using the analysis model; determining a contour of the wound area based on the extraction of the spatial features and the spectral features by using the analysis model; detecting pathogens in the wound area based on the extraction of the spatial features and the spectral features by using the analysis model; and By using the analytical model, the pathogen is classified by at least one of its family, genus, species, or strain.

17. The apparatus according to claim 16, wherein The processor is further configured to determine a length of the wound area, a width of the wound, a depth of the wound, a perimeter of the wound, or an area of the wound based on the determination of the contour of the wound area.

18. The apparatus according to claim 1, wherein The target is one of the following: a wound area, an edible product, a laboratory device, a sanitary device, a hygienic device, a medical device, a biochemical assay chip, a microfluidic chip, or a body fluid, wherein: When the target is a wound area, the processor is configured to, in response to detecting the presence of the problematic cellular entity, determine at least one of: the extent of infection of the wound area, the spatial distribution of pathogens in the wound area, or the healing rate of the wound area, When the target is tissue, the processor is further configured to detect the presence of the problematic cellular entity, the problematic cellular entity being at least one of cancerous tissue or necrotic tissue in the tissue sample, and When the target is one of the following: sanitary equipment, sanitary equipment, laboratory equipment, medical equipment, biochemical assay chip, microfluidic chip or body fluid, the processor is configured to determine the problematic cellular entity as a pathogen and classify the pathogen in the target.

19. The apparatus according to claim 1, comprising: a first polarizer positioned between the plurality of first light sources and the target to pass excitation radiation of the plurality of first light sources of a first polarization; as well as A second polarizer is positioned between the target and the image sensor to pass light emitted from the target with a second polarization.

20. The apparatus of claim 1, wherein The processor is configured to: The results and the composite image of the first image and the three-dimensional image are transmitted to a remote system in electronic communication with the device.

21. The apparatus of claim 1, wherein The interface is configured as follows: In response to the input, by using the application programming interface, when the pathogen in the target is detected and classified, a result corresponding to the detection and classification of the pathogen is transmitted.

22. The apparatus of claim 1, wherein The device is a smartphone.

23. The apparatus of claim 1, wherein The imaging sensor is a charge coupled device (CCD) sensor, a CCD digital camera, a complementary metal oxide semiconductor (CMOS) sensor, a CMOS digital camera, a single photon avalanche diode (SPAD), a SPAD array, an avalanche photodetector (APD) array, a photomultiplier tube (PMT) array, a near infrared (NIR) sensor, a red, green, and blue (RGB) sensor, or a combination thereof.

24. The apparatus of claim 1, comprising: A lens is integrated with the imaging sensor to capture the image.

25. The apparatus of claim 1, wherein The imaging sensor is a multispectral camera configured to capture the light at multiple wavelengths emitted by the target.

26. The apparatus of claim 1, wherein The analysis model includes an artificial neural network ANN model, a machine learning model ML or a combination thereof.

27. The apparatus of claim 1, wherein The processor is configured to detect changes in fluorescence emitted from the target over time.

28. The apparatus of claim 1, wherein The fluorescence from the target is one of: autofluorescence or exogenous fluorescence.

29. The apparatus of claim 1, comprising: A first set of excitation filters, wherein each of the first set of excitation filters is configured to filter excitation radiation of a predetermined wavelength range emitted by a light source of the plurality of first light sources, and pass the excitation radiation of the predetermined wavelength range to illuminate the target.

30. A device for inspecting a target, the device comprising: Imaging module, including: a plurality of first light sources, wherein each light source is configured to emit excitation radiation at a predetermined wavelength range to cause one or more markers in the target to fluoresce; a plurality of first optical bandpass filters, wherein each optical bandpass filter is configured to filter light of a predetermined wavelength emitted by the target in response to illumination thereof by at least one or more of the plurality of first light sources to allow light of the predetermined wavelength to pass through the optical bandpass filter; an imaging sensor configured to capture the filtered light filtered by an optical bandpass filter of the plurality of first optical bandpass filters and to capture a plurality of first images formed based on the filtered light; a three-dimensional image capture sensor for illuminating the target and receiving light reflected by the target in response to the illumination of the target by the three-dimensional image capture sensor to generate a three-dimensional image of the target based on the reflected light; an interface module coupled to the imaging module, the interface module comprising: The processor is configured to: analyzing a first image of the plurality of first images using an analysis model, wherein the first image is a fluorescence-based image including fluorescence from the target; analyzing the three-dimensional image of the target using the analytical model to determine changes in intensity of light emitted by the spatial region of the target by compensating for changes in the distance of the spatial region of the target from the three-dimensional image capture sensor and compensating for changes in the curvature of the spatial region of the target; detecting the presence of problematic cellular entities in the object based on the analysis of the first image and the three-dimensional image using the analysis model, wherein the analysis model is trained to detect the presence of problematic cellular entities in the object; and creating a composite image of the target using the first image and the three-dimensional image; and Interface for: Results corresponding to the detection of the problematic cellular entity and the composite image are displayed.

31. The apparatus of claim 30, wherein: The analysis model is trained using a plurality of fluorescence-based reference images and a plurality of reference three-dimensional images to detect the presence of the problematic cellular entity in the target, and wherein the analysis model is trained to distinguish between fluorescence emitted from the problematic cellular entity in the fluorescence-based images and fluorescence emitted from areas outside of the problematic cellular entity in the fluorescence-based images.

32. The apparatus of claim 30, further comprising: An emission filter wheel is rotatably disposed within the imaging module and operably coupled to a servo motor, the emission filter wheel including the plurality of first optical bandpass filters.

33. The apparatus of claim 32, wherein: The processor is configured to: activating the servo motor to rotate the emission filter wheel to position one of the plurality of first optical bandpass filters between the target and the imaging sensor; activating the plurality of first light sources to emit light toward the target; as well as In response to illumination of the target by the at least one or more light sources of the plurality of first light sources, the imaging sensor is activated to capture light emitted by the target.

34. The device according to claim 30, comprising a system-on-module (SOM), wherein: The SOM includes: the imaging module; the processor; and A plurality of light source drivers are provided, wherein each light source driver of the plurality of light source drivers is configured to adjust a light source of the plurality of first light sources.

35. The apparatus of claim 34, wherein One or more of the plurality of first light sources are pulsed light emitting diodes (LEDs) configured to emit pulses of excitation radiation to enable faster imaging and reduce ambient light interference in light emitted by the target.

36. The apparatus of claim 30, comprising: A first set of excitation filters, wherein each of the first set of excitation filters is configured to filter excitation radiation of a predetermined wavelength range emitted by a light source of the plurality of first light sources, and pass the excitation radiation of the predetermined wavelength range to illuminate the target.

37. A system for inspecting a target, the system comprising: The processor is configured to: analyzing a first image of a plurality of first images using an analysis model, wherein the first image is a fluorescence-based image including fluorescence emitted from the object; analyzing the three-dimensional image of the target using the analytical model to determine a change in intensity of light emitted from the spatial region of the target by compensating for a change in the distance of the spatial region of the target from a three-dimensional image capture sensor and compensating for a change in the curvature of the spatial region of the target; detecting the presence of problematic cellular entities in the object based on the analysis of the first image and the three-dimensional image using the analysis model, wherein the analysis model is trained to detect the presence of problematic cellular entities in the object; as well as creating a composite image of the first image and the three-dimensional image of the target; as well as The result corresponding to the detection of the problematic cell entity and the composite image of the first image and the three-dimensional image of the target are transmitted to a device.

38. The system of claim 37, comprising the device, wherein the device comprises: Imaging module, including: a plurality of first light sources, wherein each of the plurality of first light sources is configured to emit excitation radiation at a predetermined wavelength range to cause one or more markers in the target to fluoresce; an imaging sensor configured to directly receive light emitted by the target in response to the target being illuminated by one or more of the plurality of first light sources, without an optical bandpass filter disposed between the imaging sensor and the target, and to capture a plurality of first images formed based on the emitted light; and A three-dimensional image capture sensor configured to illuminate the target and receive light reflected by the target in response to the illumination of the target by the three-dimensional image capture sensor, and generate a three-dimensional image of the target based on the reflected light; wherein the target is one of the following: a wound area, an edible product, a laboratory device, a medical device, a sanitary device, a sanitary device, a biochemical assay chip, a microfluidic chip or a body fluid, wherein the analysis model is trained using multiple fluorescence-based reference images and multiple reference three-dimensional images to detect the presence of problematic cellular entities in the target, and the analysis model is further trained to distinguish between fluorescence emitted from the problematic cellular entity in the fluorescence-based image and fluorescence emitted from areas other than the problematic cellular entity in the fluorescence-based image.

39. A device for examining a wound, the device comprising: Imaging module, including: a plurality of first light sources, wherein each of the plurality of first light sources is configured to emit excitation radiation at a predetermined wavelength range to cause one or more markers in the wound to fluoresce; a plurality of second light sources, wherein each light source of the plurality of second light sources is configured to emit excitation radiation at a predetermined wavelength range without causing one or more markers in the wound to fluoresce; an imaging sensor configured to directly receive light emitted by the wound in response to illumination of the wound by at least one or more of the plurality of first light sources, and directly receive light reflected by at least one or more of the plurality of second light sources, without an optical bandpass filter disposed between the imaging sensor and the wound, wherein the imaging sensor is configured to capture a plurality of first images based on light emitted by the wound and a plurality of second images based on light reflected by the wound; and a three-dimensional image capture sensor for illuminating the wound, receiving light reflected from the wound in response to the illumination of the wound by the three-dimensional image capture sensor, and generating a three-dimensional image of the wound based on the reflected light; an interface module coupled to the imaging module, the interface module comprising: The processor is configured to: analyzing a first image of the plurality of first images using an analysis model, wherein the first image is a fluorescence-based image that includes fluorescence from the wound; analyzing a second image of the plurality of second images using the analysis model; analyzing the three-dimensional image of the wound using the analysis model to determine changes in the intensity of the reflected light and the emitted light in the spatial region of the wound by compensating for changes in the distance of the spatial region of the wound from the three-dimensional image capture sensor and compensating for changes in the curvature of the spatial region of the wound; detecting the presence of a biofilm in the wound based on an analysis of the first image, the second image, and the three-dimensional image using the analysis model, wherein the analysis model is trained to detect the presence of the biofilm in the wound; and creating a composite image using the first image, the second image, and the three-dimensional image of the wound; and The interface is configured as: Results corresponding to the detection of the biofilm in the wound and the composite image of the wound are displayed.

40. A method for inspecting a target, the method comprising: illuminating the target using at least one or more light sources of a plurality of first light sources of an apparatus, wherein light emitted by each of the plurality of first light sources has a wavelength band; capturing a plurality of first images by an imaging sensor based on light emitted by the object, wherein the imaging sensor is configured to receive light emitted by the object in response to illumination of the object by the at least one or more light sources of the plurality of first light sources; a three-dimensional image capture sensor capturing a three-dimensional image of the target, wherein the three-dimensional image capture sensor is configured to illuminate the target, receive light reflected by the target in response to the illumination of the target by the three-dimensional image capture sensor, and generate the three-dimensional image of the target based on the reflected light; a processor analyzing a first image of the plurality of first images using an analysis model, wherein the first image is a fluorescence-based image comprising fluorescence from the target in response to light emitted by the at least one or more light sources of the plurality of first light sources; the processor analyzing the three-dimensional image of the target to determine changes in intensity of light emitted by the spatial region of the target by compensating for changes in the distance of the spatial region of the target from the three-dimensional image capture sensor and compensating for changes in curvature of the spatial region of the target; the processor detecting, using the analysis model, the presence of problematic cellular entities in the object based on the analysis of the first image and based on the three-dimensional image of the object, wherein the analysis model is trained to detect the presence of problematic cellular entities in the object; creating a composite image of the first image and the three-dimensional image of the target; The interface displays a result corresponding to the presence of the problematic cellular entity and a composite image of the first image and the three-dimensional image.

41. The method according to claim 40, wherein The analysis model is trained using a plurality of fluorescence-based reference images and a plurality of reference three-dimensional images to detect the presence of the problematic cellular entity in the target, and the analysis model is trained to distinguish between fluorescence emitted from the problematic cellular entity in the fluorescence-based images and fluorescence emitted from areas outside the problematic cellular entity in the fluorescence-based images.

42. The method of claim 40, wherein the target is a wound area, and wherein the method comprises: The processor extracts spatial features and spectral features of the wound area from the first image and the three-dimensional image using the analysis model; the processor identifying a location of the wound area based on the extraction of the spatial and spectral features using the analytical model; determining, by the processor, a contour of the wound area based on the extraction of the spatial features and the spectral features using the analytical model; the processor detecting pathogens in the wound region based on the extraction of the spatial features and the spectral features using the analytical model; as well as The processor uses the analytical model to classify the pathogen by at least one of its family, genus, species, or strain.

43. The method of claim 42, comprising: The processor determines a length of the wound area, a width of the wound, a depth of the wound, a perimeter of the wound, and / or an area of the wound based on the determination of the contour of the wound area using the analytical model.

44. The method of claim 40, wherein The target is one of: a wound area, an edible product, a laboratory device, a medical device, a sanitary device, a sanitary device, a biochemical assay chip, a microfluidic chip, or a body fluid, wherein the method comprises: When the target is a wound area, the processor determines, in response to detecting the presence of the problematic cellular entity, at least one of: an infection level of the wound area, a spatial distribution of pathogens in the wound area, or a healing rate of the wound area, When the target is tissue, the processor detects the presence of the problematic cellular entity, the problematic cellular entity being cancerous tissue, necrotic tissue, or a combination thereof, and When the target is one of: sanitary equipment, laboratory equipment, hygiene equipment, biochemical assay chip, medical equipment, microfluidic chip, or body fluid, the processor determines that the problematic cellular entity is a pathogen, and the processor classifies the pathogen in the target.

45. The method of claim 40, comprising: An optical bandpass filter among the plurality of first optical bandpass filters filters light of a predetermined wavelength emitted by the target in response to illumination of the target by the at least one or more light sources among the plurality of first light sources to pass through the optical bandpass filter, wherein the optical bandpass filter is located between the target and the imaging sensor, and the imaging sensor is configured to capture the filtered light from the optical bandpass filter.

Images