System and method for wound imaging

Abstract

A device for characterizing a wound of an individual. The device includes a non- infrared-filtered imaging camera operable to capture images using visible and near infrared (NIR) wavelengths; an illumination source having one or more red / green sources and one or more NIR sources, wherein the one or more red / green sources are addressable independently from the one or more NIR sources; a controller in electronic communication with the imaging camera and the illumination source, the controller configured to: selectively activate the one or more red / green sources or the one or more NIR sources to illuminate the wound; capture at least one image of the wound illuminated by the activated sources; process the at least one captured image to provide one or more of oxygenation, pulsation, vascular imaging, and photoplethysmography information of the wound.

Description

SYSTEM AND METHOD FOR WOUND IMAGINGStatement Regarding Federally Sponsored Research

[0001] This invention was made with government support under grant numbers 2106996 awarded by National Science Foundation and EB028978 awarded by National Institutes of Health. The government has certain rights in the invention.Cross-Reference to Related Applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 657,963, filed on June 9, 2024, now pending, the disclosure of which is incorporated herein by referenceBackground of the Disclosure

[0003] Diabetic foot ulcers, which are exposed wounds that develop on the foot, may sound harmless but are stunningly lethal; with 5-year mortality rates exceeding 25% in individuals that experience complications according to one study, many types of cancer present better odds for survival. Contributing to these bleak outcomes foot ulcers, are commonly related to nephropathy disorders which impact the mobility of these diabetics making them less inclined to seek out regular treatment and monitoring. Outcomes for individuals with diabetic foot ulcers, like many other medical conditions, are often better when interventions are prompt. For those with limited mobility getting to a clinic may be impractical and home visits from clinicians can be expensive, leading to delayed discover}' of infections and complications in the healing process as well as causing a greater burden to medical professionals.

[0004] In many other medical specialties telehealth has been introduced as a way of addressing these concerns for people with limited mobility' or motivation. However, for telehealth to be an adequate alternative to in person visits sensors that can properly and reliably illuminate the condition of the patient are necessary. Scientific studies have previously show how poor oxygenation and blood flow in tissues ultimately cause foot ulcers to develop and, in many cases, then continue to impede their healing, for this reason mobile sensors that are able to assess the blood perfusion are of particular importance to telehealth efforts. In hospitals and specialized clinics computed tomography (CT) angiogram tests are commonplace but require X-rays which bring with them large expense and area requirements as well as causing repeated exposure to be concerning. As alternatives to those prohibitively expensive CT technologiesresearchers have proposed using surrogate biomarkers to ascertain blood perfusion quality, such as capillary refill time which is obtained using a blood pressure meter. Thermals has also been proposed to sense poor blood perfusion by detection of abnormal center-to-toe temperature differences, it has the advantage of being less involved than measuring blood pressure. But these whole-body biomarkers all lack the necessary locality to provide strong assurance that any wound blood perfusion issues have been detected.

[0005] Optical blood perfusion monitoring is a promising technology, it is noninvasive and can determine local issues like peripheral vascular disease by directly measuring tissue oxygenation and blood flow at and around the site of the ulcer unlike whole body surrogate markers which may read normal. Existing optical devices make use of broadband light sources and spectroscopy which drive up device expense and size causing such devices to be inaccessible to those who practice primary care and at home care, which is important for ulcer patients with limited mobility. A recent effort to make blood perfusion cheaper uses a narrowband green light source which can be observed using a conventional camera, but it does not provide oxy genation information or vascular imagery which can be useful for medical professionals in determining a prognosis.

[0006] Furthermore, most optical devices for wound monitoring tend to be single function, with the vast majority of devices focusing only on tissue oxygenation monitoring. Other devices now monitor the bacteria content of wounds using florescence and NIRS cameras can be used to obverse surface blood vessels. However, there exists no such device that integrates all these capabilities together allowing for the complete monitoring of wounds and their surroundings in the cases of both ischemia and infection. Moreover, no commercial devices image the dynamic markers of blood perfusion such as the pulse image choosing to instead focus on surface tissue oxygenation exclusively. This fragmented ecosystem of cameras causes complete adoption of Non-Invasive Diabetic Foot Monitoring technology to be prohibitively expensive and undermines the value proposition of using a camera in the diagnostic arena in the first place.

[0007] There is a need for a compact, affordable, non-invasive device that can deliver rich information for wound characterization in applications including telehealth, enabling clinicians and patients to monitor wound healing effectively.Brief Summary of the Disclosure

[0008] The present disclosure provides a portable device and method for characterizing wounds, such as diabetic foot ulcers, using imaging with selective illumination to deliver information including, for example, oxygenation, pulsation, vascular imaging, photoplethysmography information. The device employs a non-infrared-filtered imaging camera, such as a CMOS device, a co-planar multispectral illumination source with independently addressable red / green and near-infrared (NIR) light sources, and a controller to capture and process images. Orthogonal polarization enhances image clarity by rejecting specular skin reflections. The device is compact, lightweight, and suitable for telehealth, primary care, and home use, addressing the limitations of existing technologies by providing affordable, non- invasive, and precise wound assessment.

[0009] WoundSight provides an integrated hardware and software solution for providing informed wound care in a variety of settings such as, for example, at the point of care and at home. Using a multispectral sensing solution, we are able to integrate several distinct vascular tools into one easy to use solution providing the capabilities of a pulse oximeter, photoplethysmography (PPG) sensor, oxygenation camera, and pulsation camera into one lightweight handheld device.

[0010] The document addresses the need for a portable and affordable device that allows health professionals to assess wound healing prospects through visualization of microvascular blood flow dynamics. The device, called WoundSight, aims to make tracking vascular health indicators more accessible and clinically impactful for telehealth wound care.

[0011] The sensing scheme may be a feed-forward scheme. Using this feed-forward sensing scheme, the system can produce microvascular images and video for determining if there is damage to veins, tissue oxygenation images and video to evaluate blood perfusion into the tissue and proving nutrients, and pulsation image and video to observe if blood is being impeded in a particular part of the vasculature.

[0012] The presently disclosed technology can be incorporated into a standalone professional use device for care facilities, incorporated into a consumer device that can work with a phone (similar to FLIR One camera), or other embodiments for professional and / or consumer use. In various embodiments, the devices may work with a cloud-based service toprovide storage of images and allow easy management of health data as well as remote real time processing if the devices are not powerful enough to process the images on their own.

[0013] Some embodiments of the presently disclosed technology have several advantages over existing solutions. Firstly, it is spatially precise, unlike whole-body solutions that are unable to distinguish the area of issue. Secondly, it is affordable for at-home care, and even individual diabetics and their families should be able to afford devices based on this technology, unlike broadband imaging solutions that require complex lighting and advanced non-standard cameras. Lastly, the device is non-invasive — being optical means requiring no discomfort or direct contact with the patient, unlike blood pressure monitoring cuffs that may need to be applied to several limbs to calculate metrics related to blood perfusion. The technology also allows for the extraction of vascular structure and blood perfusion / oxygenation images, and it uses a dual wavelength system for oxygenation images leveraging polarization filters and camera channel orthogonality. Embodiments may have a suite of imaging algorithms that share imaging requirements, enabling the use of one device for several capabilities.

[0014] To address the drawbacks of these expensive optical single-function devices, we introduce a novel whole-condition sensor designed to address the needs of a medical practitioner who wishes to evaluate the quality of all aspects of wound healing. In various embodiments, the presently disclosed multi-spectral solution has several advantages compared to the cunent state of the art wound sensing cameras.• Able to sense both dynamic and static indicators of blood perfusion such as the dynamic pulsation and both the dynamic and static tissue oxygenation. Allowing for practitioners to take multiple views and avoid potential pitfalls associated with a single assessor of blood perfusion quality'.• Able to sense through transparent wound dressings — providing comfort to patients who may have pain or discomfort w hen wound dressings are removed.• Able to sense bacteria using ultraviolet (UV) florescence in order to effectively diagnose wound infections which may have a fast onset.• Able to image surface blood vessels using NIR allowing practitioners to both sense blood perfusion and make assessments of a major related cause of poor perfusion.

[0015] Additionally, embodiments of the presently disclosed device and method provide all these capabilities while maintaining desirable properties for telehealth deployment:

[0016] (1) Affordable for at-home care, nurses can afford to have these devices in their practice and even individual diabetics and their families should be able to afford devices based on this technology, unlike broadband imaging solutions which require complex lighting and advanced non-standard cameras.

[0017] (2) Non-invasive being optical means requiring no discomfort or direct contact with the patient, unlike blood pressure monitoring cuffs which may need to be applied to several limbs to calculate metrics related to blood perfusion

[0018] All this by recognizing that all the capabilities listed above can be achieved by just six different narrowband illuminates, four of which can be provided by just commercial off the shelf RGBW LEDs and computer vision methods like those employed by PulseCam [Kumar et al. 2020] or by hyperspectral reconstruction. Along with the fact that an affordable RGB camera with a removed NIR filter can observe all the reflected / remitted light needed to support all of the disclosed capabilities without any modification. We also note that transparent wound dressings have little to no effect on the operation of the disclosed active illumination based wound camera.Description of the Drawings

[0019] For a fuller understanding of the nature and objects of the disclosure, reference should be made to the following detailed description taken in conjunction with the accompanying drawings.

[0020] Figure 1 A. A perspective view of a device according to an embodiment of the present disclosure.

[0021] Figure IB. A diagram of the device shown in Figure 1 A.

[0022] Figure 2. Embodiments of the present system use optical polarization imaging to see though the skin and produce images for informed wound care.

[0023] Figure 3. The data flow steps in an example pipelined suite of methods. Based on the selected mode, certain lights are turned on and the captured image can be processed in accordance with the corresponding pipeline.

[0024] Figure 4. Components linkage diagram for an embodiment of the presently disclosed device.

[0025] Figure 5. Various views of an example device according to the present disclosure and highlighting the handle, the circular lighting system, and connectivity.

[0026] Figure 6. Views of a PCB and light-emitting diodes (LEDs) with circular cutout for a camera. The back of the board (bottom photograph) has power MOSFETs for switching, (a) Front of PCB. (b) Back of PCB.

[0027] Figure 7. Prototype jig holding both the WoundSight device and foot for stable consistent imaging / video.

[0028] Figure 8. Scotch tape basic test of pulsation capability — smartphone picture of foot with tape.

[0029] Figure 9. Scotch tape basic test of pulsation capability — pulsation image sequence.

[0030] Figure 10. Two possible smartphone case embodiments based on either a camera embedded in a circular LED array or an offset camera and rectangular LED array.

[0031] Figure 11. Correspondence between lighting bands and capabilities.

[0032] Figure 12. Software pipelines for each capability.

[0033] Figure 13. An example flow showing a user experience for an implementation of an embodiment of the presently disclosed system in a consumer product.

[0034] Figure 14. A chart showing a method according to another embodiment of the present disclosure.Detailed Description of the Disclosure

[0035] With reference to Figure 1. the present disclosure may be embodied as a device 10 for wound characterization. The wound may be, for example, a diabetic foot ulcer. However, applications are not necessarily limited to such ulcers or to diabetic patients. The device may be a portable, handheld device. For example, the device may be a portable imaging system designed for telehealth wound care.

[0036] The device 10 includes an imaging camera 20 having the ability to capture infrared and visible light images (operable to capture images using visible and / or near infrared (NIR) wavelengths). Suitable imaging cameras are sometimes referred to herein as non-infrared- filtered cameras, though it should be noted that cameras filtering some wavelengths of infrared light may nonetheless be used. In particular, the imaging camera may be configured to capture images using near-infrared light or portions of the near infrared spectrum. For example, the imaging camera may be configured to capture images using light having a wavelength in the range of 700-900 nm, inclusive and sub-ranges thereof.

[0037] The device has an illumination source 12 having one or more red / green sources 13,15 and one or more NIR sources 14. The red / green sources may be red sources 13 and green sources 15 that are configured for simultaneous activation. The one or more NIR sources are addressable independently from the one or more one or more red / green sources (and vice versa — the one or more red / green source(s) are addressable independently from the one or more one or NIR sources). For example, the red / green source(s) may be activated separate from activation of the NIR source(s). In some embodiments, the illumination source includes one or more UV sources 16. In some embodiments, the illumination source includes one or more white sources 18. Each of the various source types is independently addressable from the others (can be selectively activated without (necessarily) activating others).

[0038] A controller 30 is in electronic communication with the imaging camera 20 and the illumination source 12. The controller 30 is configured to operate the imaging camera 20 and the illumination source 12 to capture images and imaging sequences (time-series sequences) using various lighting modalities. The controller 30 may be configured to selectively activate the one or more red / green sources 13,15 or the one or more NIR sources 14 to illuminate the wound.

[0039] The controller 30 is configured to capture at least one image of the wound illuminated by the activated source(s). The controller 30 is configured to process the at least one captured image to provide one or more of oxygenation, pulsation, vascular imaging, and photoplethysmography information of the wound. In embodiments having UV sources 16, the controller may be configured to provide bacteria imaging. In embodiments having white sources 18, the controller may be configured to provide hyperspectral imaging and / or static oxygenation imaging.

[0040] The system may provide oxygenation, pulsation, vascular imaging, photoplethysmography information, bacteria imaging, hyperspectral imaging, etc. to assess blood perfusion and wound healing prospects. Orthogonal polarization filters enhance image quality byrejecting specular skin reflections, and advanced image processing algorithms deliver precise, localized diagnostic data.

[0041] With reference to Figure 14, in another aspect, the present disclosure may be embodied as a method 100 for characterizing a wound of an individual. The method 100 includes selectively activating 103 a subset of light sources in a multispectral illumination source. The light sources may include one or more red / green sources and one or more near-infrared (NIR) sources. In some embodiments, the light sources additionally include one or more UV sources. In some embodiments, the light sources additionally include one or more white sources.

[0042] At least one image of the wound is captured 106 as illuminated by the activated subset of light sources. The at least one captured image is processed 109 to provide one or more of oxygenation, pulsation, vascular imaging, and photoplethysmography information of the wound. In embodiments having UV light sources, the at least one captured image may be processed to provide bacteria imaging. In embodiments having white light sources, the at least one captured image may be processed to provide hyperspectral imaging and / or static oxygenation imaging.

[0043] It should be noted that some embodiments of the method 100 (and of the device described above) may include activating 112 another light source of the subset of light sources (a second subset of light source(s), capturing 115 at least one additional image, and processing 118 the at least one additional image. The process may be repeated to provide additional modalities described herein. In this way, a medical professional may have the benefit of several or all of the ty pes of imaging and / or information described herein.

[0044] 2 BACKGROUND

[0045] Several fundamental physical components can be associated with human system functionality such as hemoglobin, vascular structure, and vascular change. Hemoglobin carries oxygen and nutrients throughout the human cycle, responsible to ensure human organs receive all substances when needed. Vascular structure, help identify underlying risk of abnormal in vessel structure, potential indication of help.

[0046] 2. 1 Absorption Spectroscopy

[0047] Using an ABG machine, researchers have performed Absorption Spectroscopy on samples of oxygenated and deoxygenated hemoglobin separated from blood samples. The resulting spectroscopy allows for the determination of the amount of light by wavelength that would be absorbed by either hemoglobin or its bound variants. The spectral response of hemoglobin taken with the Beer-Lambert law which states that the absorbance A of a material at a wavelength 2 is linearly related to its relative concentration C in the sample as when comparing sample concentrations the optical path length I would be identical.A = E X)bC (1)Which allows for the absorption spectroscopy process to be taken in reverse using the spectral information and light response of the sample to determine the quantities of hemoglobin variants in the sample.

[0048] 2.2 Polarization Imaging

[0049] When linearly polarized light passes through human tissue, the non-homogeneous tissue causes the light to lose its original polarization direction becoming randomly polarized. Direct reflections from the skin maintain their original polarization, so systems arranged to reject the original polarization can be used to reject specular skin reflections, allowing reflections from the tissue to be more clearly observed. An example of a system that leverages this effect is [Li et al. 2022],

[0050] 2.3 Hyperspectral Reconstruction

[0051] Hyperspectral reconstruction is a powerful technology that can be used to transform images (linearly) into a more meaningful physical space for observation orcomposition analysis or with the aid of deep learning or dictionary learning can convert spatial information into more valuable spectral information not already present in the image [Cai et al. 2022], The most commonly studied method for blood perfusion static imaging is the wiener filter [Nishidate et al. 2013] which assumes that the following linear transformation between the observed camera vector v = [r, g, b] and a hyperspectral estimation vector h' = [400nm, 410nm, . . . ,700nm] can be made using a matrix W. h' = Wv (2)Where W is calculated by minimizing the minimum square error between the ground truth hyperspectral vector which yields the following expression for W where V and H are a full RGB calibration image (N, 3) and a full corresponding ground truth HSI image (N, b) where b is the number of desired bands.W = (HBlXVVt;r1(3)

[0052] Where is the ensemble-averaging operator. In order to obtain ground truth HSI vectors to populate H a hyperspectral camera is typically not used instead opting for a color calibration target with prior known colors. This can be done for either normal reflected light with a simple color calibration target or for florescent emissions by using a florescent calibration target [He et al. 2021],

[0053] 3 WOUNDSIGHT SYSTEM

[0054] Some embodiments of the WoundSight system enable telehealth for monitoring diabetic foot ulcers and may do so inexpensively. This can be accomplished using integrated optical hardware and computer vision algorithms. An integrated camera and narrowband light source are used to collect data at specific wavelengths of interest which are then processed into various vascular, bacteria, and perfusion images and signals for diagnostic usage.

[0055] Each capability may share the same data collection pipeline and may leverage the same hardware as described in Section 6:

[0056] (1) Relevant narrowband lights are turned on.

[0057] (2) Camera settings are applied.

[0058] (3) Data is captured and sent to processor.

[0059] (4) Images / Video are processed in accordance with the selected function.

[0060] From there depending on the capability the reverent data processing steps are applied as described in Section 7

[0061] The WoundSight system aims to enable telehealth for monitoring diabetic foot ulcers and do it on the cheap. It accomplishes this by using integrated optical hardware and computer vision algorithms. The specific narrowband wavelengths that relate to generating vascular and oxygenation images are those that the device can collect images at. In addition to this the unique hardware software co-design enables new capabilities like full-rate oxygenation video. All at a reduced size.

[0062] The sensing scheme is entirely feed-forward and goes as follows, the lights corresponding to the required wavelengths are turned on, the camera captures frames which are encoded and sent over USB, a computer program accesses the frames using an operating system service, then our computer vision algorithms process the frames into the needed image, video, or signal and display it to the operator. All captured frames leverage the polarization imaging technique outlined in 2.2. Using this feed-forward sensing scheme our system can produce microvascular images and video for determining if there is damage to veins, tissue oxygenation images and video to evaluate blood perfusion into the tissue and proving nutrients, and pulsation image and video to observe if blood is being impeded in a particular part of the vasculature. Together these capabilities build a blood perfusion toolbox for medical professionals from just one compact handled device.

[0063] 4 IMAGING

[0064] 4. 1 Single Camera Imaging

[0065] An aspect of the system’s hardware is its single camera imaging setup. Optical systems used in labs tend to utilize two cameras for imagining involving IR and color sources. This is because typical color cameras have IR blocking high-pass filterers. Color cameras possess this to prevent noise from ambient IR sources (the largest of which is the sun) from perturbing the image as every' color channel has a sensitivity' to IR in normal commercial cameras. So, IR cameras are used alongside filtered color cameras to capture IR images, but thiscauses the additional expense of an IR camera to be necessary'. The present system provides a different, application-specific solution. Since vascular images require IR while oxygenation images do not, the device’s operation can be separated into two modes. A vascular capture mode which has only the IR LEDs powered on for sensing veins; and an oxygenation mode where the red and green LEDs are powered on for sensing tissue oxygenation; and with both modes using a single, non-IR-filtered color camera. This separation is aided by the reasonable assumptions accompanying the use of the device, that being (1) the user is not outdoors as nearly half the suns light is in the IR band; and (2) the ambient lighting is dimmed indoors as some indoor lights also generate IR as a byproduct. Any stray IR can appear as color distorting noise when in the oxygenation mode, but environmental IR is at a minimal when the use assumptions are met.

[0066] Additionally since a camera without a IR filter still blocks UV light the signal camera setup naturally can also be used for florescence imaging which again leverages a normal color camera for operations.

[0067] 4.2 Single Frame Oxygen Image

[0068] Oxygen imaging using hyperspectral imaging is commonplace in the literature now [Saiko et al. 2020], There is a drawback however in that the frames eliminated with each wavelength of light are collected separately as that is normal spectroscopy practice. This introduces the need to align the frames together which can get complex as optical flow algorithms do not assume frames will have completely different illumination. It also greatly reduces the frame rate if every frame requires a new light source. To alleviate these concerns, embodiments of the present device make use of a new technique in which red and green lights are on at the same time during every oxygenation frame. As these wavelengths are nearly orthogonal in the color space of our camera (and even further that the true red and green can be recovered with a linear system using the pixels sensitivity to each wavelength and observed response), the device’s imaging system can in this mode be regarded as two distinct ones, a red sensing camera and light, and a green sensing camera and light. These two ''virtual” cameras capture both the frames needed for a dual-wavelength oxygen image at once.

[0069] 4.3 Fluorescence Imaging

[0070] Fluorescence imaging can typically be done by illuminating the target with an even narrowband UV light such as 405 nm [Rennie et al. 2019], It is somewhat common forfilters to be used to block unwanted violet / blue light from being emitted when LEDs are used although this may not be necessary depending on the LED. In practice filters are used to allow for the isolation of different excitation bands (light from the target) in order to increase the performance, especially when two or more excitation bands fall under the same camera band or if a mono channel camera is used [Rennie et al. 2019], Since we are using a tri-band camera we would likely not use any filters in order to implement florescence imaging as bacteria from diabetic wounds tend to glow red / cyan and other tissue glow shades of green [Armstrong et al. 2023] so there is hkely manageable overlap. A longwave UV light source is used to excite the target into emitting light. However, in some embodiments, one or more filters may be used to isolate the desired excitation bands.

[0071] 5 THROUGH-DRESSING WOUND MONITORING

[0072] Specialized tapes by 3M and others provide wound dressings that are virtually see through (transparent) for medical observation purposes. However, even though these wound dressings are intended for use by doctors in order to both treat and view wound healing progress. No attempt at optical monitoring of wounds has been made leveraging the transparency of these dressings. Other product lines of transparent dressings intended for ulcers exist as well including DermaView’s extensive line of products. Furthermore, most tapes do not naturally (at least completely) block UV or IR making achieving a material / tape that is transparent to all three major light bands UV / IR / visible highly achievable with most commercially available tapes. Some drawbacks of this approach can include scattering effects such as specular reflections but the use of orthogonal polarizers in some embodiments of the present device is able to cancel out such specular reflections (e.g., a cancel out a sufficient amount of such reflections so as to provide a useful result).

[0073] A preliminary test of such through-dressing imaging was performed using an embodiment of the present device. Running the pulsation function with a horizontal band of 3M scotch tape, we were not able to distinguish the region covered by the tape from the rest of the foot within the pulsation image. Furthermore, the dressing was hardly visible in the raw images taken by the WoundSight system used to produce the pulsation image. As such, the present device may be used through transparent (at least partially transparent) dressings.

[0074] 6 DEVICE

[0075] 6. 1 Design Considerations

[0076] Full Illumination. In order to ensure that ulcers are fully illuminated regardless of location in the frame so that a diagnosis can be rendered, the device may advantageously have a powerful and even lighting system. Our system utilizes high power LEDs arranged in a circle to achieve this.

[0077] Compactness. The device may be advantageously sized to be able to fit into a cabinet or on a desk in a typical primary care doctor’s office. Our system has the lighting and camera co-planer to save space. Some embodiments of the device can be miniaturized so as to attach to the back of a smartphone or similar device.

[0078] Real Time Streaming. To provide insights into conditions that are temporal and allow doctors to line up images of patients, the device may advantageously be able to support a real time feed. Our device incorporates a powerful single board computer to translate raw frames from the camera to be sent over USB.

[0079] 6.2 Device Construction

[0080] Possible device embodiments will integrate a multispectral light source composed of various narrowband LEDs, such as, for example (but not limited to):

[0081] (1) Deep NIR: used for vascular sensing in order to enable higher penetration depth for sensing.

[0082] (2) RGBW: Red, Green and Blue used to enable pulse image and dynamic pulse oximetry along with rPPG. White can be used for hyperspectral reconstruction for static estimation of oxygenation.

[0083] (3) Longwave UV used for florescence sensing.

[0084] Figure 10 depicts two different embodiments which can effectively integrate the multispectral lighting source, camera and orthogonal polarization filters (used to block specular reflections):

[0085] (1) Coaxial Circular Layout (Figure 10, left): In this embodiment, a circular custom printed circuit board (PCB) houses one or more concentric rings of LEDs. For example,the LEDs may be RGBW, UV, and NIR. In some embodiments, the LEDs alternate between these. The imaging camera is positioned at the center of this ring array, co-planar with the lighting array, allowing the system to be compact and symmetrical. Orthogonal polarizing films are placed on the front surface of the device to reduce surface glare by cross-polarization. For example, the polarizing filters may be films. In some embodiments, the polarizing filters are circularly polarized. A first polarizing filter may be disposed on the illumination source and a second polarizing filter may be disposed on the imaging camera. The circular configuration of this embodiment minimizes the device footprint and enables straightforward alignment of optical components but may limit the uniformity of illumination as the imaging camera requires a cutout in the array.

[0086] (2) Rectangular Layout (Figure 10. right): This embodiment repositions the camera at a fixed angle next to the illumination source, rather than embedding it within. The offset placement enables the use of a rectangular PCB populated with a larger number of LEDs arranged in, for example, a checkerboard pattern to distribute various spectral bands more evenly — this includes RGBW, UV, and NIR. This configuration may allow for more powerful and uniform illumination across the imaging area and may simplify the electrical and mechanical design of the lighting system. Despite the physical separation, orthogonal polarizing filters may be used over both the camera and lighting surface to preserve the ability to reject specular reflections. This embodiment may sacrifice some compactness and may introduce off-axis viewing drawbacks for the camera depending on the offset.

[0087] In all cases, embodiments of the device feature a camera capable of capturing images under NIR illumination (e.g., with a removed NIR filter) to enable single camera imaging. A wireless / USB interface to connect to a smartphone, PC. or other device. The device can be set down on the table using a kickstand. From there by either a laser rangefinder or a physical instrument the distance can be established to the foot. The foot may be held by a 3D printed holder designed to conform to most heals (see, for example, Figure 7). Such a 3D holder can be set on a table or floor and is used to keep the foot stable for imaging.

[0088] In an example embodiment, the device is made up of a commercially available CMOS camera and a circular lighting system contained in a lightweight 3D printed enclosure. A single board computer manages the system and processes raw frames from the camera. In addition, it also configures the camera’s autofocus and gain settings. On the back of the devicethere is a button which is used to set the mode of the device. Video is streamed from the device though the USB type A port on the side which provides a UVC (USB video class) device that any host device can access without requiring proprietary’ software. The single board computer also has wireless capabilities and space enough to store video and images that can be transmitted over the internet. The lighting system is comprised of 8 high power LEDs arranged in a circle alternating between RGB and Near IR. All the LEDs and their supporting power and switching circuitry is contained on a custom PCB also shaped in a circle. The LEDs are attached to power MOSFET transistors which allow them to be switched with the normal logic level signal emitted by the GPIO pins on the connected single board computer.

[0089] The lighting system and camera are co-planer, and both housed at the front of the device with the lens of the camera pointing forward. The camera is located at the center of a circular cutout made in the lighting PCB. The handle for the device is located at its top above the housing for the single board computer and all the devices’ ports and wiring. The handle is slightly longer than the housing. All exterior facing device edges are filleted to make griping the device for long periods of time comfortable. Two orthogonal polarizing films are affixed to the front of the device. Both circular in shape and one is a circular cutout of the other, the polarizing films are used to reject skin reflections leveraging the principal outlined in section 2.1.

[0090] 7 IMAGE PROCESSING

[0091] After illuminated images are streamed from the camera they are processed into into the relevant product images / signals in accordance with the procedures / bands specified in each of the following subsections.

[0092] 7. 1 Remote PPG

[0093] To get the PPG in a non-contact way, we utilize red and green light. Images are first segmented, then all pixel intensity values corresponding to a channel within the region of interest are averaged and the blue channel is discarded, yielding two time-series signals by tracking this ROI mean. These signals are then filtered using a bandpass filter to remove noise and the effects of the respiration signal. Lastly, independent component analysis is used with the fully filtered red and green signals to further refine the signal. While some papers use 1CA method with constraints [Macwan et al. 2018] to remove the interference from the skin, our polarization imaging method accomplishes that already.

[0094] 7.2 Pulsation Image

[0095] To ascertain the pulsation strength in the foot surface tissue a green image stream is used. After the images have been aligned to each other to compensate motion using an optical flow algorithm. An averaging filter is convolved with each of the images to reduce the noise from quantization and random camera noise. At each observed pixel, small fluctuations in the reflected light from the surface of the foot form a time series signal that is correlated with the cardiovascular dynamics in the tissue corresponding to that pixel. To extract these dynamics, the image stream is regarded as a collection of timeseries signals observed by each pixel in the camera P (x,y, t ) from frames between time between time Toand Trsliced from the original stream, these signals are then normalized in a way that makes the image invariant to uneven lighting and non-cardiovascular dynamics in the tissue. This is done by using two band-pass filters one from 0.5 Hz - 5 Hz to get an “AC” signal and the other from 0 to 0.3 Hz to get the “DC” signal. The AC signal is then normalized by the DC signal to get the skin response signal R.(4)(5)

[0096] The pulsation strength at each geometric pixel is then extracted by computing the inner product between the skin response signal R and an out-of-channel PPG signal captured using a contact optical PPG sensor or by averaging the skin signal R over the entire visible tissue for a non-contact option, these PPG signals are filtered from 0.5 Hz to 5 Hz then normalized using their Hilbert transform before being used. The inner product result captures the correlation between the PPG signal which is the pulsation of the entire tissue and each individual skin response signal R signals that have higher magnitude correlated components have higher pulsation magnitudes. The pulsation magnitudes are arranged back into a frame F to provide the visualization of pustule dynamics throughout the tissue. The inner product can be computed using a sliding window of source frames to enable a real time pulsation view.

[0097] 7.3 Dynamic Oxygenation Image

[0098] Using our single-camera oxygenation imaging technique we obtain both the necessary red and green frames for computing the oxygenation saturation at each location in the visible tissue. That is the ratio of the oxygenated hemoglobin HbCh to the non-oxygenated hemoglobin Hb. Just as in the pulsation image, we first align a group of successive pairs of green and red images using optical flow, perform a spatially averaging convolution to reduce noise, and regard the values associated with a pixel (x,y) as a time-series PRedand Pcreen- We then also AC / DC normalize both of these images using band-pass filters to obtain the normalized reflectivity R for each channel as the blood flow dynamics are still present in both the channels. Then, to obtain the SpCh value, we fit a quadratic model to the ratio of the two normalized images [rim Park et al. 2023], To determine the coefficients of the model, a pulse oximeter is used for ground truth and the model is optimized such that the spatial average of a ROI near the oximeter has the minimum error. The model can be fit for various exposure settings to enable the use of Auto Exposure as well.

[0099] 7.4 Static Oxygenation Image

[0100] If the illumination is spatially even and consistent between images and environments, then it stands to reason that images taken of a subject at a consistent distance the hyperspectral reconstructed images can be normalized to a physically meaningful image known as a reflectance image R^”'^ which is an image where a value of 1 in any channel represents 100% reflection of emitted light and 0 conversely 0%. From there the image can be converted into an absorbance image A^^l by the Beer Lambert law [Nishidate et al. 2013],A = ln R = pEC (7)Where p is the optical path length (reasonable values can be given by previous works for skin or a path length of 1 can be assumed) and Elz,,c' is the matrix of the moral absorption coefficients which are known prior for the c = 3 chromophore present in skin (melanin, oxyhemoglobin, and deoxyhemoglobin) whose quantity are captured by the estimated chromophore matrixwhich through the above relation can be obtained using least squares. From the matrix C we can obtain oxygenation pixiewise using the following expression.[HbO2]SpO2= (8)[HbO2] + [Hb]

[0101] 7.5 Vascular Image

[0102] Near-infrared (NIR) illumination and imaging techniques can be employed for non-invasive visualization of superficial human vasculature. NIR light between 700-900 nm wavelengths can penetrate sufficiently deep into skin tissue to reach subcutaneous vessels while remaining unaltered by absorption from melanin. However, contrast and resolution of raw vascular images acquired through NIR photography tend to be poor owing to significant optical scattering events and limitations of digital camera sensors. To address these factors, postprocessing of the NIR images can be performed through a vessel enhancement algorithm known as Sato’s method [Sato et al. 1998],

[0103] Sato’s method calculates the eigenvectors v and eigenvalues of the Hessian matrix H which captures the all the second derivatives at a given pixel and uses a detector function which is maximized when the eigenvalues indicate that in one direction the direction of the tube there is minimal change and that in the other direction the direction orthogonal to the tube the change is maximized. That is the function show n below captures the likeness of a given neighborhood around a pixel to a tube. Capturing all the likeness values for each pixel into an image reveals the vasculature for clinical evaluation.(9)where Anis the nth smallest eigenvalue of H(x, y)7

[0104] 7.6 Bacteria Image

[0105] Bacteria will be visible using excitation light in the ultraviolet (UV) band (e.g., ■‘longwave” UV light, which may comprise wavelengths between 315 and 400 nm, inclusive) to capture florescence images. Optionally, special visualizations (e.g., techniques to improve the visibility of bacteria under fluorescence microscopy, such as, for example, false-color imaging, contrast enhancement, digital processing, etc.) may make it easier to observe bacteria.Hyperspectral reconstruction can be used to make bacteria images more physically meaningful (e.g., potentially revealing characteristics of the bacteria not easily visible in standard florescence imaging).

[0106] 7 INNOVATIONS

[0107] Summarized here for clarity we review several of the innovative aspects of embodiments of the present WoundSight system:• A small handheld device with a multi-spectral lighting system and camera.• A device consolidating both infection and perfusion imaging capabilities.• Various methods enabling single camera imaging under different wavebands.• A suite of imaging algorithms that share imaging requirements enabling the use of one device for several capabilities.• Through dressing optical wound imaging for the first time.

[0108] 8 COMMERCIALIZATION

[0109] In some embodiments, the present device may be configured to attach directly to a smartphone (for example, to the back of a smartphone). The device may be configured to integrate with a cloud service for storing and processing images without requiring a powerful computer to be local to the device. The device may communicate with a telehealth service for enabling remote health care, which would be beneficial for patients with poor mobility. The device may use a rangefinder, such as a laser range finder, to provide consistent usage of the device by inexperienced users. In some embodiments, the rangefinder may be a LiDAR sensor incorporated in a smartphone device. For example, the device may be configured to attach to a smartphone and use the LiDAR sensor of the smartphone (if equipped) to aid in positioning the device relative to the foot.

[0110] The present disclosure may be embodied as a system and a method. For example, a digital camera system designed to measure tissue oxygenation and blood flow around wounds. The system is designed to be portable and inexpensive.

[0111] In some embodiments, the device includes:

[0112] 1. A non-infrared filtered CMOS camera.

[0113] 2. A ring of LED lights alternating between red / green (R / G) and near infrared (NIR) positioned around the camera aperture and co-planer to it.

[0114] 3. The R / G and NIR lights alternate in time.

[0115] 4. A single board computer connected to the camera by, for example, USB orBluetooth connection. The computer controls the settings of the camera concerning autofocus and gain. It also controls the LED power by MOSFET transistors via GPIO pins.

[0116] 5. The reflected light passes through two orthogonal polarizing films preventing light in the original polarization from being directly reflected off the skin. This is background noise.

[0117] 6. Image is captured by the camera and stored on the camera and / or transmitted to a single board computer via, for example. USB cord or Bluetooth.

[0118] 7. System may be used in dim light to avoid ambient infrared light interfering with NIR signal.

[0119] 8. The Red / Green lights are on simultaneously eliminating the need to align the frames. Since they are orthogonal, the signals can be separated.

[0120] 9. Resulting images are processed to produce images or videos highlighting:

[0121] a. Remote Photoplethysmography (rPPG) - uses R / G signal filtered to remove noise and respiration signal.

[0122] b. Pulsation - uses green signal stream. It uses two band-pass filters and theDC signal to normalize the AC signal. Algorithms are then applied to calculate the pulsation strength.

[0123] c. Oxygenation - uses R / G frames with algorithms to produce the blood oxygen levels (SpO2). The SpO2 is not measured directly to avoid the need for additional lighting bands.

[0124] d. Vascular Image - uses the NIR signal (700-900 nm) with processing by Sato’s method to compensate for scattering and common limitations.

[0125] It should be noted that the term controller or processor used herein should be interpreted broadly. In some instances, the controller includes one or more modules and / or components. Each module / component executed by the controller can be any combination of hardware-based module / component (e.g., a field-programmable gate array (FPGA), an application specific integrated circuit (ASIC), a digital signal processor (DSP)), software-based module (e.g., a module of computer code stored in the memory and / or in the database, and / or executed at the processor), and / or a combination of hardware- and software-based modules. Each module / component executed by the controller is capable of performing one or more specific functions / operations as described herein. In some instances, the modules / components included and executed in the controller can be, for example, a process, application, virtual machine, and / or some other hardware or software module / component. The controller can be any suitable processor configured to run and / or execute those modules / components. The controller can be any suitable processing device configured to run and / or execute a set of instructions or code. For example, the controller can be a general purpose processor, a central processing unit (CPU), an accelerated processing unit (APU), a field-programmable gate array (FPGA), an application specific integrated circuit (ASIC), a digital signal processor (DSP), and / or the like. In some instances, the controller may have a module local to the device (e.g., to operate the illumination source and the imaging camera) and a module remote from the device (e.g., a cloud service to process the captured image(s)).

[0126] Although the present disclosure has been described with respect to one or more particular embodiments, it will be understood that other embodiments of the present disclosure may be made without departing from the spirit and scope of the present disclosure. While the device is described herein as being used for imaging a foot ulcer, this is merely an exemplary' application, and the device may be employed for imaging other types of wounds.

Claims

We claim:

1. A device for characterizing a wound of an individual, comprising: an imaging camera operable to capture images using visible and near infrared (NIR) wavelengths; an illumination source having one or more red / green sources and one or more NIR sources, wherein the one or more red / green sources are addressable independently from the one or more NIR sources; a controller in electronic communication with the imaging camera and the illumination source, the controller configured to: selectively activate the one or more red / green sources or the one or more NIR sources to illuminate the wound; capture at least one image of the wound illuminated by the activated source(s); and process the at least one captured image to provide one or more of oxygenation, pulsation, vascular imaging, photoplethysmography information of the wound.

2. The device of claim 1 , wherein the one or more red / green sources, the one or more NIR sources, and the imaging camera are co-planar.

3. The device of claim 1. wherein the one or more red / green sources and the one or more NIR sources are arranged around a circumference of the imaging camera.

4. The device of claim 1, wherein the one or more NIR sources have a wavelength in the range of 700-900 nm, inclusive.

5. The device of claim 1, wherein the steps of illumination and image capture are repeated to provide a time series of images.

6. The device of claim 1, wherein the illumination source has a first polarization, and the imaging camera has a second polarization orthogonal to the first polarization.

7. The device of claim 1. wherein the at least one captured image is captured using red / green illumination, and the at least one captured image is processed to determine the ratio of oxygenated hemoglobin to non-oxygenated hemoglobin to provide oxygenation information.

8. The device of claim 1, wherein the at least one captured image is processed using a dualwavelength oxygenation imaging technique, wherein red / green light images are captured and processed to compute oxygen saturation (SpCh) using a quadratic model fitted to a ratio of normalized red and green reflectivity.

9. The device of claim 1, wherein the at least one captured image is captured using red / green illumination, and the at least one captured image is processed to provide photoplethysmography information using independent component analysis.

10. The device of claim 1, wherein the at least one captured image includes a plurality7of images captured using red / green illumination, and the plurality of images is processed to provide pulsation information using a green channel and analyzing the green channel for fluctuations in light reflected from a surface of the wound.

11. The device of claim 10, wherein the controller provides pulsation information by: aligning the images of the plurality of images to one another to compensate motion; applying, to the aligned plurality of images, a first band-pass filter to extract an AC signal and a second band-pass filter to extract a DC signal; normalizing the AC signal using the DC signal to obtain a normalized skin response signal; and computing pulsation strength by correlating the normalized skin response signal with a photoplethysmography signal.

12. The device of claim 1, wherein the at least one captured image is captured using NIR illumination, and the at least one captured image is processed to generate a vascular image.

13. The device of claim 12, wherein the controller is configured to generate the vascular image by: applying a vessel enhancement algorithm based on a Hessian matrix to compute eigenvectors and eigenvalues; and generating the vascular image by maximizing a detector function indicating tubular structures in the image.

14. The device of claim 1, wherein the at least one captured image includes a plurality of images captured using red / green illumination, and the controller is configured to process the plurality of images to perform remote photoplethysmography (rPPG) by: segmenting each image of the plurality’ of images; averaging pixel intensities in a region of interest to generate a red time-series signal and a green time-series signal; filtering each of the red time-series signal and the green time-series signal using a bandpass filter to remove noise and effects caused by a respiration signal; and applying independent component analysis to refine the time-series signals as an rPPG signal.

15. The device of claim 1, wherein the illumination source further comprises one or more ultraviolet light sources.

16. The device of claim 15, wherein the at least one captured image is captured of fluorescence excited using ultraviolet illumination.

17. The device of claim 15, wherein the illumination source further comprises one or more white sources.

18. The device of claim 17, wherein the at least one captured image is captured using white illumination, and the at least one captured image is processed to provide a hyperspectral reconstruction.

19. The device of claim 18, wherein the at least one captured image is processed using a neural network trained with deep learning network or using dictionary learning.

20. A method for characterizing a wound of an individual, comprising: selectively activating a subset of light sources in a multispectral illumination source, the light sources including one or more red / green sources and one or more near-infrared (NIR) sources; capturing at least one image of the wound illuminated by the activated subset of light sources; processing the at least one captured image to provide one or more of oxygenation, pulsation, vascular imaging, and photoplethysmography information.

21. The method of claim 20, wherein the one or more NIR sources have a wavelength in the range of 700-900 nm, inclusive.

22. The method of claim 20, wherein the illumination source has a first polarization, and the images are captured using a second polarization orthogonal to the first polarization.

23. The method of claim 20, wherein the at least one captured image is captured using red / green illumination and the at least one captured image is processed to determine the ratio of oxygenated hemoglobin to non-oxygenated hemoglobin to provide oxygenation information.

24. The method of claim 20, wherein the at least one captured image is captured using red / green illumination and the at least one captured image is processed to provide photoplethysmography information using independent component analysis.

25. The method of claim 20, wherein the at least one captured image is captured using red / green illumination and the at least one captured image is processed to provide pulsation information using a green channel and analyzing the green channel for fluctuations in light reflected from a surface of the wound.

26. The method of claim 25, wherein pulsation information is provided by: aligning the images of the plurality of images to one another to compensate motion; applying, to the aligned plurality' of images, a first band-pass filter to extract an AC signal and a second band-pass filter to extract a DC signal; normalizing the AC signal using the DC signal to obtain a normalized skin response signal; and computing pulsation strength by correlating the normalized skin response signal with a photoplethysmography signal.

27. The method of claim 20, wherein the at least one captured image is captured using NIR illumination and the at least one captured image is processed to generate a vascular image.

28. The method of claim 27, wherein the vascular image is generated by: applying a vessel enhancement algorithm based on a Hessian matrix to compute eigenvectors and eigenvalues; and generating the vascular image by maximizing a detector function indicating tubular structures in the image.

29. The method of claim 20, wherein the at least one captured image includes a plurality of images captured using red / green illumination, and the at least one captured image is processed to perform remote photoplethysmography (rPPG) by: segmenting each image of the plurality of images; averaging pixel intensities in a region of interest to generate a red time-series signal and a green time-series signal; filtering each of the red time-series signal and the green time-series signal using a bandpass filter to remove noise and effects caused by a respiration signal; and applying independent component analysis to refine the time-series signals as an rPPG signal.

30. The method of claim 20, wherein the illumination source further comprises one or more ultraviolet light sources and the at least one captured image is captured of fluorescence excited using ultraviolet illumination.

31. The method of claim 20, wherein the illumination source further comprises one or more white sources and the at least one captured image is captured using white illumination.

32. The method of claim 31, wherein the at least one captured image is processed to provide a hyperspectral reconstruction.

33. The method of claim 32, wherein the at least one captured image is processed using a neural network trained with deep learning network or using dictionary learning.

Images