Computer-implemented method for wound analysis

Abstract

A computer-implemented method for wound analysis is provided. The wound characteristic of the wound emits fluorescence with a unique spectral signature when the wound is subjected to excitation light of a known wavelength or range of wavelengths. An image captured therefrom is subjected to analysis of pixels of the image, where a plurality of training images on which known wound sizes and characteristics are marked are used to generate training data, which is then used to identify wound characteristics from the test image in real time. Wound size, boundaries, bacterial presence, and other characteristics may be quantified and graphically represented as overlays on an original wound image as well as documents related to the wound.

Description

[0001] This application is a divisional application of the PCT application with an application date of January 15, 2019, application number 201980011067.6, and invention name “Wound Imaging and Analysis” entering the Chinese national phase application. The international application number of the PCT application is PCT / CA2019 / 000002.

[0002] This application claims the benefit of priority to U.S. Provisional Application No. 62 / 625,611, filed on February 2, 2018, the entire contents of which are incorporated herein by reference. Technical Field

[0003] A fluorescence-based imaging and monitoring device and method is disclosed. Specifically, the device and method may be suitable for both human and animal applications, such as monitoring biochemical and / or biological and non-biological substances in wound assessment and wound care management. Background Art

[0004] Wound care is a major clinical challenge. Healing and chronic non-healing wounds are associated with many biological tissue changes, including inflammation, necrosis, exudate production, bleeding, proliferation, connective tissue remodeling, and the common major problem, bacterial presence, growth, and infection. Some of the wound infections are not clinically obvious and lead to increased personal, emotional, and economic burdens associated with wound care, especially in the elderly population. For example, Pseudomonas aeruginosa and Staphylococcus aureus are bacterial genera prevalent in hospital settings and are common causes of bacterial infections. Currently, the clinical gold standard for wound assessment includes direct visual inspection of the wound site under white light illumination for classic signs and symptoms of infection. This is usually combined with swab culture or tissue biopsy samples for laboratory testing.

[0005] However, these results are usually delayed, expensive, and can produce insensitive bacterial results. This may affect the timing and effectiveness of treatment. Quantitative and subjective visual assessment only provides a gross view of the wound site, but does not provide information related to the potential biological, biochemical and molecular changes that occur at the tissue and cellular levels. In addition, bacteria are invisible to the naked eye, resulting in undesirable wound sampling, and the changes in bacterial growth at the wound site cannot be properly tracked. This may hinder healing and timely selection of the best antimicrobial treatment. A relatively simple and complementary method is expected in clinical wound management, which utilizes biological and molecular information to improve the early identification of this hidden change in the wound site. Early identification of high-risk wounds (e.g., including clinically significant bacterial presence or "load") can promote early treatment, guide therapeutic interventions, and provide therapeutic response monitoring over time, thereby greatly reducing the morbidity and mortality caused in particular due to chronic wounds. Summary of the invention

[0006] The present disclosure solves the problems identified above by presenting devices, systems, and computer-implemented methods that identify in real time spectral wavelength signatures and other information indicative of wound characteristics and changes thereto, perform analysis on the identified information, and output the results to a user of a wound monitoring device or system. Wound characteristics include wound size, wound boundaries, wound depth, wound temperature, changes in tissue and cellular wound composition, vascularization, necrosis, and the presence of bacteria therein. Other characteristics identified include characteristics of resected tissue such as cancerous tissue (e.g., lumpectomy for breast cancer surgery). When used with resected tissue, the devices and methods can be used to identify characteristics such as tissue composition, tumor size, tumor margins, tumor boundaries, and tissue vascularization.

[0007] In one exemplary embodiment, the present disclosure provides a computer-implemented method for wound analysis, the computer-implemented method stored on a computer-readable medium and including logic instructions, the logic instructions being executed by a processor to perform operations, the operations including: receiving an image of a wound, the image including a plurality of pixels; determining at least one region of interest in the image based at least on applying a chromaticity mask to the plurality of pixels, the chromaticity mask being based on a histogram of pixel values; determining one or more contours of the at least one region of interest; and generating an output image including the one or more contours superimposed on the image. The region of interest includes one or more wound characteristics.

[0008] In another exemplary embodiment, the present disclosure provides a system, comprising: an imaging device; a processor coupled to the imaging device; and a memory coupled to the processor. The memory may be configured to store computer-readable instructions, which, when executed by the processor, cause the processor to perform operations including: acquiring an image of a wound using the imaging device, the image comprising a plurality of pixels; applying a chromaticity mask to the plurality of pixels, the chromaticity mask being based on a histogram of pixel values; generating a binary mask based on the application of the chromaticity mask, the binary mask identifying at least one region of interest on the image; detecting one or more contours of the at least one region of interest to define the region of interest; superimposing the one or more contours on the image to form a composite image identifying the at least one region of interest; and outputting the composite image to a user of the imaging device in real time, and saving the image in an original or compressed format.

[0009] In yet another exemplary embodiment, the present disclosure provides a tangible, non-transitory computer-readable medium for storing computer-readable code, the computer-readable code being executed by a processor to perform operations, the operations comprising: acquiring a plurality of red, green, and blue (RGB) images; marking a known region of interest on each of the plurality of images using a computer interface, the known region of interest comprising at least one of bacterial presence, wound boundary, collagen proliferation, and wound size; converting each of the plurality of RGB images to an alternative color space. Non-limiting examples of color spaces include: CIELAB color space used in computer vision applications, hue saturation value (HSV), hue saturation brightness (HSL), hue saturation darkness (HSD), lightness chroma hue (LCH), CMYK, cylindrical transform, lightness plus chroma / chroma, YCbCr: https: / / en.wikipedia.org / wiki / YCbCr, LUV: https: / / en.wikipedia.orq / wiki / CIELUV, XYZ: https: / / en.wikipedia.org / wiki / CIE1931 color space、 YUV: https: / / en.wikipedia.org / wiki / YUV, Munsell color system, Natural Color System (NCS), Pantone Matching System (PMS), RAL, Aerospace Material Specification - Standard 595A (replacing (U.S.) Federal Standard 595C), (U.S.) Federal Standard 595C (Archive.org), British Standard Color (BS) 381C, BS2660, BS5252 and BS4800, LMS color space (long, medium, short), a perceptual color space based on the response function of cones in the retina of the eye, and rg chromaticity space. After converting the image to the alternative color space, the operation includes: determining a histogram of values ​​in the alternative color space for each of the plurality of RGB images, the histogram of values ​​identifying a unique spectral signature for each of the known regions of interest, and generating a synthetic histogram based on the histogram of values ​​in the alternative color space for each of the plurality of RGB images. The synthetic histogram is used to identify unknown regions of interest from at least one wound image in real time using a wound imaging device based on one or more unique spectral signatures.

[0010] In yet another exemplary embodiment, the present disclosure provides a system including a processor and a memory coupled to the processor. The memory may be configured to store computer-readable instructions that, when executed by the processor, cause the processor to perform operations including: receiving an image of a wound, the image comprising a plurality of pixels; applying a chromaticity mask to the plurality of pixels, the chromaticity mask being based on a histogram of pixel values ​​and identifying at least one region of interest on the image; detecting one or more contours surrounding the at least one region of interest; superimposing the one or more contours on the image to form a composite image identifying the at least one region of interest; outputting the composite image on a display device coupled to the processor and saving the image in an original or compressed format.

[0011] Additional objects and advantages will be set forth in part in the following description, and in part will be apparent from the description, or may be learned by practicing the present teachings. The objects and advantages of the present disclosure will be realized and obtained by the elements and combinations specifically pointed out in the appended claims. It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limitations on the subject matter claimed. The accompanying drawings, which are incorporated in and constitute part of this specification, illustrate exemplary embodiments of the present disclosure, and the drawings and descriptions are used to explain the principles of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] At least some features and advantages of the present teachings will be apparent from the following detailed description of exemplary embodiments consistent with the present teachings, which description should be considered with reference to the accompanying drawings, in which:

[0013] Figure 1 Depicted are exemplary methods for wound imaging and analysis.

[0014] FIG. 2A to FIG. 2C Schematic diagram depicting an exemplary apparatus for wound imaging, analysis, and output of wound imaging analysis and documentation.

[0015] Figure 3 An exemplary system for wound imaging, analysis, and output of wound documentation is depicted.

[0016] FIG. 4A to FIG. 4D Depicts exemplary histograms for training images.

[0017] FIG. 5A to FIG. 5D Depicted are exemplary composite histograms for a plurality of training images.

[0018] Figure 6 An exemplary method for chrominance masking is depicted.

[0019] Figure 7 An exemplary method for contour detection is depicted.

[0020] Figure 8 Depicted are exemplary methods for image restoration, analysis, and output of wound documentation.

[0021] FIG. 9A to FIG. 9B Depicting exemplary output images of a wound imaging and analysis operation.

[0022] Fig.10 An exemplary method for color analysis of wound images is depicted.

[0023] Fig.11 Depicting exemplary output images and documentation of wound images via an exemplary user interface.

[0024] FIG. 12A to FIG. 12C An exemplary image depicting a wound having a user-defined boundary and foreground and background regions determined based thereon.

[0025] FIG. 13A to FIG. 13B Depicting an exemplary method for identifying stickers in wound images and illustrations thereof.

[0026] Although the following detailed description refers to exemplary illustrative embodiments, many alternatives, modifications, and variations thereof will be apparent to those skilled in the art. Therefore, it is intended that the claimed subject matter be viewed broadly. DETAILED DESCRIPTION

[0027] Reference will now be made in detail to various exemplary embodiments, examples of which are illustrated in the accompanying drawings. The various exemplary embodiments are not intended to limit the present disclosure. On the contrary, the present disclosure is intended to cover alternatives, modifications, and equivalents of the exemplary embodiments. In the drawings and descriptions, similar elements are provided with similar reference numerals. It should be noted that the features explained separately in the description can be combined with each other in any technically advantageous manner, and additional embodiments of the present disclosure are disclosed.

[0028] The present disclosure provides devices, systems, and computer-implemented methods that identify spectral signatures and other information indicative of wound characteristics and changes thereto in real time, perform analysis on the identified information, and output the results to a user or operator of a wound monitoring device or system. Wound characteristics may include, for example, wound size, wound boundaries, wound depth, changes in tissue and cellular wound composition, vascularization, necrosis, wound temperature, and changes in wound temperature, as well as the presence, distribution, and load of bacteria. Although described herein with respect to use with wounds, the devices and methods disclosed herein may also be used to identify characteristics of resected tissue, such as cancerous tissue (e.g., for lumpectomy in breast cancer surgery). When used with resected tissue, for example, the devices and methods may be used to identify characteristics such as tissue composition, tumor size, tumor margins, tumor boundaries, and tissue vascularization.

[0029] The exemplary wound monitoring device described herein includes a handheld / portable optical digital imaging device having a specific excitation light source and an optical bandpass filter attached thereto. Using the imaging device and system further described herein, the fluorescence generated by the components in the wound due to exposure to the excitation light can be imaged and analyzed. For example, in a wound with the presence of bacteria, such as Pseudomonas aeruginosa or caused by the inclusion of Pseudomonas aeruginosa, when subjected to excitation light, Pseudomonas aeruginosa fluoresces with a specific spectral signature (i.e., one or more wavelength bands with known peaks). The excitation light may include any light with a known wavelength or any light with a range of wavelengths with a known peak, such as a peak at 405nm. Capturing and analyzing this data allows identification of ubiquitous bacteria and also identification of specific types of bacteria present. In order to identify, classify, and quantify the presence of bacteria and additional characteristics of the wound, the device and system must be trained.

[0030] Spectral information and wound size information from a plurality of training images are used to generate training data, the plurality of training images being labeled with wound size and bacterial presence and / or load. The training data is then applied to real-time analysis of images of new wounds on a pixel-by-pixel basis, thereby enabling identification of wound characteristics. Wound boundaries, bacterial presence, and other wound characteristics can be quantified and graphically represented as overlays on white light images of wounds and surrounding healthy tissue. In addition, specific types of bacteria (e.g., Pseudomonas aeruginosa) and / or other wound characteristics can be identified, quantified, and highlighted, or otherwise indicated or superimposed on an image of the wound or an image of the wound obtained over time. Other characteristics can be identified, such as characteristics of resected tissue, such as cancerous tissue (e.g., lumpectomy for breast cancer surgery), characteristics such as tissue composition, tumor size, tumor margins, tumor boundaries, and tissue vascularization. For the purposes of this disclosure, "real-time" operation refers to nearly instantaneous processing that occurs simultaneously with the use of a wound imaging device or system. For example, a user who uses a device or system described herein to acquire an image of a patient's wound is provided with analysis results on a display of the same device or a display communicatively coupled to the imaging device. The wound analysis results can be output in real time, or near real time (i.e., upon command of a user), without having to perform any additional steps and without having to wait for a processing period. In addition, the wound analysis results can be digitally stored for future access or printed as part of a clinical documentation program. For purposes of this disclosure, the term "image" may refer to any representation of a wound, including raw pixel data or information, or any input received at a light sensor such as a camera described herein. In addition, the analyses described herein can be performed on a series of images (including frames of a video) captured over time or in rapid succession. These operations and additional operations are relative to the following Figures 1 to 13B The illustrated embodiment is further described.

[0031] Figure 1 Depicted are exemplary methods for wound imaging and analysis. Further references Figures 2A to 3 Description for execution Figure 1 However, it should be noted that with the necessary adjustments that will be apparent to one of ordinary skill in the art in light of the present disclosure, Figure 1The operations described in can be performed by any device or system. At operation 101, a histogram is generated based on a training image with a known region of interest marked thereon. This step includes collecting or acquiring a database of clinical wound images or clinical tissue specimens (e.g., excised tissue or pathological tissue specimens). The images can be acquired using the same device / system components used for real-time imaging of wounds, or at least acquired using common imaging conditions such as excitation (or irradiation) light type and frequency, filters, etc. Further, for the purposes of this disclosure, a frame of a wound image or video depicts one or more wounds, surrounding tissue surfaces, and their characteristics. For example, a wound may include any injury or damage to the surface of an organism (such as a cut, burn, scrape, surgical incision, surgical cavity, ulcer, etc.). The wound may expose areas including blood, connective tissue, adipose tissue, nerves, muscles, bones, etc., beneath the skin. Thus, exemplary properties of wounds that can be analyzed include the size of the wound, the depth and / or volume of the wound (including the depth and / or volume of the surgical cavity), the edges (borders) of the wound, the presence and amount of different types of bacteria and other organisms, the amount of connective tissue (e.g., collagen and elastin, exudate, blood, bone, etc.), which are detected based on how they absorb, scatter, reflect white light, and / or emit fluorescence due to intrinsic fluorescence (autofluorescence emission) and fluorescence from exogenous contrast agents intended to detect wound components. Exemplary properties of resected tissue samples that can be analyzed include the size of the tumor (any tumor that can be sensed / visualized by FL tumor can be partially buried, exposed to the surface, completely resected, or sectioned), the edges (borders) of the tumor in the wound, the amount of connective tissue (e.g., collagen and elastin, fat, and blood), which are detected based on how they absorb, scatter, reflect white light, and / or emit fluorescence due to intrinsic fluorescence (autofluorescence emission) and fluorescence from exogenous contrast agents intended to detect tissue components including the tumor. Exemplary methods for causing tumors to fluoresce to enable use of the methods and devices disclosed herein can be found in U.S. Provisional Patent Application No. 62 / 625,983, filed on February 3, 2018, entitled “Devices, Systems, and Methods for Tumor Visualization and Removal,” the entire contents of which are incorporated herein by reference.

[0032] Therefore, training images are labeled with specific regions of interest by experts (such as medical professionals / clinicians / scientists / technicians) with prior knowledge related to these characteristics. Regions of interest can indicate general areas such as wound boundaries / edges, or specific areas, such as areas containing the presence of specific types of bacteria or other organisms, the number or "load" of bacteria / organisms within or in the wound region of interest, or areas known to contain another wound characteristic of interest. Prior knowledge of bacterial presence, colonies and / or their load can be based on swabs and / or tissue biopsy analysis with positive results for specific bacterial strains. Therefore, images of each type of region of interest can be acquired and classified separately according to the target characteristics or information (including the presence of known bacterial types and quantities or concentrations).

[0033] Continuing with operation 101, the pixel information of the "marked" image is then processed and analyzed to generate a histogram. Depending on the type of analysis being performed (wound size relative to bacterial load or any other target information and its changes over time), the histogram may include white light and / or fluorescence data, RGB color data, and other pixel-based image information / values. FIG. 4A to FIG. 4D and FIG. 5A to FIG. 5D An exemplary histogram is further described. Typically, the histogram defines pixel data as a target in a predefined region of interest that contrasts with pixel data outside the region of interest based on the spectral signature of the pixel and classifies it. In addition, the training (marked) image may include multiple images of the same wound but with different saturation / hue / intensity values ​​and under varying lighting conditions to enhance the histogram. Such multiple training images can be used to generate a first synthetic histogram based on a combination of the histograms of each training image. The first synthetic histogram enables the region of interest to be distinguished from the region of no interest for a specific characteristic, and the region is classified according to the target characteristic. A second synthetic histogram can be generated based on multiple first synthetic histograms. The second synthetic histogram can be used to detect multiple different target characteristics in a test image, or similar target characteristics across multiple test images.

[0034] Each histogram includes a plurality of parameters that are subsequently used in real-time processing of new images in which a priori knowledge of the region of interest is not available. The parameters may be stored as a spreadsheet, a lookup table, or other structures known in the art. Ultimately, and as further described herein, the real-time processing operation includes outputting a processed image that includes a highlighted region of interest and quantified biological and / or non-biological data such as bacterial load or wound size.

[0035] At operation 102, operation 102 is substantially at any point after training operation 101, scanning a test image for real-time analysis. The test image can be acquired in real time using imaging hardware coupled to the analysis module described herein. Alternatively or additionally, the test image can be acquired from the imaging hardware and sent to a computer that performs the disclosed operations. Alternatively or additionally, the test image can be acquired from an external source such as a database or a network. Typically, the test image is initially acquired using an RGB camera or sensor, thereby obtaining an RGB raw image. Other systems for acquiring images in various formats are also possible. For example, when excited by short wavelength light (e.g., ultraviolet or short visible wavelengths) or illuminated with monochromatic light, most endogenous biological components of tissues (e.g., connective tissues such as collagen and elastin, metabolic coenzymes, proteins, etc.) produce, for example, longer wavelength fluorescence in the ultraviolet, visible, near infrared, and infrared wavelength ranges. Tissue autofluorescence imaging provides a unique means to obtain biologically relevant information and changes therein between normal and diseased tissues in real time and over time. Biologically relevant information includes, for example, the presence of bacteria, changes in the presence of bacteria, changes in tissue composition, and other factors that can distinguish between normal and diseased tissue states. This is based in part on the inherently different light-tissue interactions (e.g., absorption and scattering of light), changes in tissue morphology, and changes in the blood content of the tissue that occur at the bulk tissue and cellular levels. In tissue, blood is the major light-absorbing tissue component (i.e., chromophore). This type of technology is suitable for imaging diseases in hollow organs (e.g., Gl tract, oral cavity, lungs, bladder) or exposed tissue surfaces (e.g., skin). Therefore, autofluorescence imaging devices can be used for rapid, non-invasive, and non-contact real-time imaging of wounds to detect and utilize the rich biological information of wounds, thereby overcoming current limitations and improving clinical care and management. Reference FIG. 2A to FIG. 2C and Figure 3 Exemplary imaging devices and systems are further described. Exemplary devices that may be particularly useful for surgical cavities, hollow organs, and resected tissue specimens are also disclosed in U.S. Provisional Patent Application No. 62 / 625,983, entitled “Devices, Systems, and Methods for Tumor Visualization and Removal,” filed on February 3, 2018, the entire contents of which are incorporated herein by reference.

[0036] At operation 103, chroma masking is performed on the image acquired at operation 102. Chroma masking enables identification of whether each pixel in the image is within or outside a region defined as a region of interest based on a spectral signature of the region. The spectral signature may be based on selectable color space values ​​of training image pixels from a synthetic histogram generated during training operation 101. Thus, chroma masking may be performed on a pixel-by-pixel basis, and relies on the general assumption that a pixel has a higher probability of being in the region of interest if other pixels in its vicinity are also in the region of interest. The output of the chroma masking operation is a binary mask that identifies "blobs" or relatively uniform regions of pixels. Some blobs may be of interest, while others may not be; therefore, additional filtering operations are performed as part of the chroma masking operation 103, such as filtering out isolated outlier pixels (erosion), and shifting toward clusters of pixels (dilation). Reference Figure 6 The chroma masking operation is described in further detail.

[0037] At operation 104, contour detection is performed on the mask generated in operation 103. Contour detection is applied to find an envelope surrounding each blob detected from the mask. This enables subsequent enumeration of regions of interest and ranking of regions of interest based on the enumeration. Contour detection may also be subject to additional steps such as discarding blobs below a certain region threshold, or picking the top 2-3 based on size. Figure 7 An exemplary method for contour detection is described in further detail. FIG. 12A to FIG. 12C Another exemplary method for contour detection is described in further detail.

[0038] At operation 105, inpainting and analysis are performed on the contours detected in operation 104. The inpainting and analysis may be further based on the database of pixel data collected during the training operation 101 in order to identify specific tissues (such as unnatural portions of the contour or envelope of the region of interest). This may be based on a general assumption that specific biological features such as wounds, bacterial presence, etc. will not have artificial edges and will be more convex rather than concave in shape. Therefore, the inpainting and analysis evaluates the performance of the chromatic mask and contour detection features and corrects any deficiencies thereof. The method ends with the output of one or more images, which may include contours and other biological information superimposed on the original image of the wound. For example, a single output image may include multiple color-coded overlays. Multiple images taken over time may be superimposed where a registration algorithm and markers or stickers are used to find co-located features, align images, identify distances, and redirect images.

[0039] In general, although the sequence of operations described above is based on specific experiments performed by the applicant using the hardware described herein, other sequences of these operations may be contemplated by one of ordinary skill in the art in light of this disclosure, particularly if different hardware is used. The use of different hardware may include simple changes, such as changing the wavelength of the excitation light or a filter used to block or remove the wavelength of light directed to the device. As will be understood and expected by one of ordinary skill in the art, such changes will require similar changes in the training process.

[0040] FIG. 2A to FIG. 2C Depicting different views of an exemplary device for wound imaging and analysis. Figure 2A , depicts a schematic diagram of an exemplary device for wound imaging and analysis. The device is shown positioned to image a target object 10 or a target surface (such as a wound on a patient). In the example shown, the device has a digital image acquisition device 1 (such as a digital camera, a video recorder, a video camera, a cellular phone with a built-in digital camera, a "smart" phone with a digital camera, a personal digital assistant (PDA), a laptop / PC with a digital camera, or a webcam). The digital image acquisition device 1 has a lens 2 that can be aligned to point to the target object 10 and can detect light signals emitted from the object 10 or surface. The device has a filter holder 3 that can accommodate one or more filters 4. Each filter 4 can have a different discrete spectral bandwidth and can be a bandpass or longpass filter. These filters 4 can be selected and moved into the digital camera lens to selectively detect specific light signals based on the wavelength of the light. The digital imaging detector device can be, for example, a digital camera having at least ISO800 sensitivity, but more preferably ISO3200 sensitivity, and can be combined with one or more light emission filters, or other equally effective (e.g., miniaturized) mechanized spectral filtering mechanisms (e.g., acousto-optically tunable filters or liquid crystal tunable filters).

[0041] The apparatus may include a light source 5 that produces excitation light or illumination (e.g., monochromatic light or white light with a wavelength peak of 400-450 nm), or any other combination of single or multiple wavelengths (e.g., wavelengths in the UV / visible / near infrared / infrared range) to illuminate the object 10 to elicit an optical signal (e.g., fluorescence). For example, the excitation / illumination light source may be a blue or violet LED array that emits light at approximately 405 nm (e.g., + / - 5 nm), and may be coupled with an additional bandpass filter centered at approximately 405 nm to remove / minimize side spectral bands of light from the LED array output so as not to leak light into an imaging detector having its own filter. The light source 5 may further include laser diodes and / or filtered light arranged in various geometric shapes. The apparatus may include methods or devices 6 (e.g., heat sinks or cooling fans) for dissipating heat and cooling the illumination light source 5. The apparatus may include a system or device 7 (e.g., an optical bandpass filter) to remove any undesirable wavelengths of light from the light source 5 used to illuminate the object 10 being imaged.

[0042] The device may include a system or device 8 such as a rangefinder or other device (e.g., the use of a compact micro-laser diode that emits a collimated light beam) for measuring and determining the distance between the imaging device and the object 10. For example, the device may use two light sources such as two laser diodes as part of a triangulation device to maintain a constant distance between the device and the object 10. Other light sources are also possible. The device may also use ultrasound or a physical measurement such as a ruler to determine the constant distance to be maintained. The device may also include a structure 9 (e.g., a pivot) to allow manipulation and orientation of the excitation light sources 5, 8, so as to position these light sources 5, 8 to change the angle of illumination of the light that illuminates the object 10 for varying distances.

[0043] The target object 10 may be marked with a marker 11 to allow multiple images of the object to be taken at one time or over a period of time and then co-registered for analysis. The co-registration may be a spatio-temporal co-registration (i.e., the images may be correlated over time and with the size of the marker in order to track changes or growth in a particular characteristic. The marker 11 may involve, for example, the use of an exogenous fluorescent dye of different colors which, when illuminated by the light source 5, may generate a plurality of different light signals and be detectable within the image of the object 10. This may allow the orientation of multiple images (e.g., taken over time) of the same region of interest by co-registering the different colors and the distances between them. The device itself may further include software that allows a user to control the device, including control of imaging parameters, visualization of images, storage of image data and user information, transmission of images and / or related data, and / or related image analysis (e.g., detection and or diagnostic algorithms).

[0044] The digital image acquisition device 1 may further include one or more of the following: an interface 12 for a head mounted display; an interface 13 for an external printer; an interface 14 for a tablet, laptop, desktop or other computer device; an interface 15 for a device allowing wired or wireless transmission of imaging data to a remote site or another device; an interface 16 for a global positioning system (GPS) device; an interface 17 for a device allowing the use of additional memory; and an interface 18 for a microphone. The device may include a power source 19 such as an AC / DC power supply, a compact battery pack, or a rechargeable battery pack. Alternatively, the device may be adapted to be connected to an external power source. The device may have a housing 20 that houses all components in one entity. The housing 20 may be equipped with a device to secure any digital imaging device therein. The housing 20 may be designed to be handheld, compact and / or portable. The housing 20 may be one or more housings.

[0045] refer to Figure 2B , depicting different views of an exemplary wound imaging and analysis device 200. Device 200 may be, for example, a MolecuLight i:X (RTM) device developed by MolecuLight (RTM). Device 200 allows clinical professionals to quickly, safely, and easily visualize bacterial presence and distribution in skin and wounds in real time, including but not limited to the point of care. Device 200 is contactless and does not require imaging contrast agents for white light and / or fluorescence imaging. Device 200 is depicted as a handheld portable medical device that includes a high-resolution color LCD display and touch-sensitive screen 208 with integrated optical and microelectronic components and an internal battery power source. The device 200 further includes a power button 201 for turning the device on and off, a display power button 202 for turning the display 208 on and off, a system status LED 203 indicating overall device performance, a battery status LED 204 indicating the battery level of the device, a rangefinder LED system 205 indicating the optimal distance to the wound being targeted or imaged, an ambient light status LED 206 for indicating the optimal lighting environment for fluorescence mode imaging, a heat sink 207 for dissipating heat if the device 200 becomes warm after prolonged use, a home button 209 for providing access to the image and video capture functions of the device 200, and a port 210 for charging and data transfer. The port 210 can be used with any common or proprietary cable such as a USB, or MolecuLighti:X (RTM) connection cable.

[0046] The device 200 further includes a rocker switch 211 capable of switching between a standard imaging mode and a fluorescence imaging mode. For example, the device 200 uses both the standard imaging mode and the fluorescence imaging mode to capture real-time images (e.g., JPG format) and videos (e.g., MOV format). The standard imaging mode is typically used for standard photography (i.e., capturing RGB images and videos of a target illuminated with standard white light). The fluorescence imaging mode is used to capture RGB images and videos of a target illuminated with light having a known peak wavelength, and is intended to generate fluorescence from a specific target stimulated by the light. Therefore, the device 200 further includes an LED 212 having a specific wavelength or range of wavelengths for illuminating a target when in the fluorescence imaging mode, and a camera lens 213 capable of capturing images and videos, a range sensor 214 for detecting the optimal distance from the wound or surrounding skin, and an ambient light sensor 215 for detecting the optimal lighting conditions for the fluorescence imaging mode. Further, the device 200 includes a holding profile 217 for allowing a user to firmly grip the device, and a charging port 218 for enabling the device to be charged using a standard or proprietary power adapter.

[0047] refer to Figure 2C , the device 200 is depicted as being used to image a wound on a patient's foot 220. Two high-efficiency LEDs with a specific wavelength or range of wavelengths on the device 200 illuminate the wound and surrounding healthy skin for high-resolution and real-time fluorescence imaging of bacteria and tissue, and the resulting image is depicted on a display 208. The imaging relies on the fact that bacteria and tissue produce different levels of red and green (i.e., intrinsic) fluorescence emission wavelengths under illumination with a specific wavelength. Unlike healthy skin, which is primarily composed of connective tissue and adipose tissue, bacteria produce distinct colors such as red or green, which are primarily caused by endogenous molecules called porphyrins that are excited to fluoresce under illumination. The device 200 captures the fluorescence emitted from both bacteria and tissue and creates a composite image on a high-resolution color LCD display 208. A user of the device 200 can easily and instantly visualize the presence and location of bacteria in and around the wound (e.g., as depicted by overlay 221) and record the data.

[0048] The device can be used in a common wound care facility and integrated into a conventional wound care practice to allow real-time imaging of the patient. The device can be used to image under white light illumination and / or take fluorescent images of the wound under dimmed room lights. The device can be used in a telemedicine / telehealth infrastructure, for example, a fluorescent image of the patient's wound can be sent to a wound care specialist via an electronic mail via a wireless communication device (such as a smartphone in another hospital using a wireless / WiFi internet connection). Using the device, high-resolution white light and / or fluorescent images can be sent to a wound care specialist as an electronic mail attachment from a remote wound care site for immediate consultation with a clinical specialist, microbiologist, etc. at a dedicated clinical wound care and management center. Exemplary wound imaging devices, their features, structures, and uses thereof are described in further detail in U.S. Pat. No. 9,042,967, entitled "Device and Method for Wound Imaging and Monitoring," issued May 26, 2015, the entire contents of which are incorporated herein by reference.

[0049] Figure 3 An exemplary system for wound imaging and analysis is depicted. The system includes a memory 300 storing a plurality of processing modules or logic instructions that are executed by a processor 301 in communication with a computer 303. The computer 303 may communicate with the memory 301 via a network or a direct communication link. For example, the memory 300 and the processor 301 and the image acquisition system 1 may be as shown in FIG. FIG. 2A to FIG. 2C 300 and the processor 301 are directly coupled to the computer 303. In general, in addition to the processor 301 and the memory 300, the computer 303 may also include user input and output devices such as a keyboard, a mouse, a stylus, and a display / touch screen. As will be explained in the following discussion, the processor 301 executes the logic instructions stored on the memory 300, performs the image analysis operations, and outputs the quantitative / graphical results to the user operating the computer 303.

[0050] Image acquisition 1 including reference FIG. 2A to FIG. 2C Any of the above imaging components, including a camera or light sensor, a light or excitation source, and appropriate filters or filtering mechanisms. Other excitation and emission wavelengths can be used with different devices and different pixel signatures are detected. Typically, image acquisition 1 is performed, for example, by using FIG. 2A to FIG. 2C The device shown provides images or image data of the wound in real time to obtain images or videos (including multiple image frames) of the patient's wound. The images and associated data are received by modules 310 - 350 and can be stored in database 305 .

[0051] The database 305 further includes training image data from images labeled with specific regions of interest by experts (such as medical professionals / clinical experts / scientists / technicians) with prior knowledge related to these regions of interest. The training image data can be classified according to target characteristics, including images with known bacterial presence, images with known wound sizes, images with known collagen values, etc. The training image data can include fluorescence data, RGB color data, and histograms of other pixel values ​​depicting training images with known wound boundaries and bacterial presence. Reference FIG. 4A to FIG. 4D and FIG. 5A to FIG. 5D Exemplary histograms are further described.

[0052] The chroma masking module 103 is performed on the image acquired from the image acquisition 1. Chroma masking enables identification of each pixel in the image as being within a region of color space defined as a region of interest, or as being outside of a region of interest. This determination uses the pixel values ​​from the synthetic histogram generated during the training operation (i.e., the image data stored on the database 305). The output of the chroma masking operation is a binary mask that identifies "blobs" or relatively uniform regions of pixels. Figure 6 The chroma masking operation is described in further detail.

[0053] The feature contour detection module 320 is performed on the mask generated by the chroma masking module 310. Contour detection is applied to find the envelope surrounding each blob detected in the mask. This enables the regions of interest to be subsequently enumerated and sorted based on the enumeration. Contour detection also requires additional filtering, such as discarding blobs below a certain region threshold, or picking the top 2-3 based on size. Figure 7 and FIG. 12A to FIG. 12C Contour detection is described in further detail.

[0054] The image restoration and analysis module 330 is performed on the contour and may also be based on the image data 305, which may include specific issues that occurred during training (such as identifying unnatural parts of the contour and correcting deficiencies of previous modules). The restoration and analysis operations will refer to Figure 8 Further description.

[0055] The color analysis and overlay module 340 generates a composite image of biological information superimposed on the original image of the wound or bacterial presence, as well as color intensities based on user-defined thresholds. For example, a single output image may include multiple color-coded overlays. In some embodiments, the intensity of red fluorescence (or fluorescence with peaks at one or more specific wavelengths, i.e., spectral signatures) can be quantified and used to indicate the presence of bacteria within a given wound area. In some embodiments, this includes determining whether the intensity of a particular wavelength meets a threshold and triggering a determination of bacterial presence when the threshold is reached. Similarly, different intensities can be associated with different levels of bacterial presence, so higher thresholds can be used to trigger a determination of a severe infection. Reference Fig.10 Color analysis is further described.

[0056] Additional analysis modules 350 include operations such as determining the percentage of wound area to be normalized, tracking the progression of wounds, comparing multiple images taken over time, registering markers and / or stickers to find co-located features and redirected images, etc. In some embodiments, for specific bacteria such as Pseudomonas or other target characteristics, excitation / emission maps may be stored on database 305. The map may, for example, define the excitation wavelength range that will elicit fluorescence of the target characteristic, and the emission wavelength range that will be used to detect the target characteristic. The target characteristic information may be input by a user of computer 303 (coupled to the device of image acquisition 1), or as part of the image data provided by image acquisition 1. Therefore, additional analysis may include retrieving the correct filter and pixel information (i.e., histogram) from database 305, or instructing the operator of the imaging device to set the device in a specific configuration that is ideal for imaging the target characteristic. Such excitation and emission information may be used for many types of target characteristics, as shown in Table 1 below.

[0057]

[0058]

[0059] Table 1 - Fluorescence results of 9 target bacterial species

[0060] The target characteristics may further include: the presence of at least one of bacteria, fungi, yeasts, and other microorganisms present in the irradiated portion of the wound and periwound area; at least one of the location, population, number, distribution, colonization, contamination, critical colonization, infection, and extent when at least one of bacteria, fungi, yeasts, and other microorganisms is present in the irradiated portion of the wound and periwound area; and at least one of the presence, location, distribution, and extent of collagen, elastin, connective tissue, blood, bone, exudate, matrix tissue, granulation tissue, and other tissues, cells, molecules, and fluids indicative of wound infection and / or healing present in the irradiated portion of the wound and periwound area. In some embodiments, in addition to Pseudomonas aeruginosa, the presence of the following bacteria is detected: Staphylococcus aureus, Escherichia coli, Enterococcus (i.e., species in the genus Enterococcus), Proteus, Klebsiella pneumoniae, coagulase-negative Staphylococci, β-hemolytic Streptococcus (Group B), and Enterobacter. All of these bacteria emit fluorescence between 600nm-660nm when excited under light having a wavelength peak of 405nm, thereby eliminating the need for additional imaging hardware or spectral filtering. Other characteristics identified include characteristics of resected tissue such as cancerous tissue (e.g., lumpectomy for breast cancer surgery). When used with resected tissue, the devices and methods can be used to identify characteristics such as tissue composition, tumor size, tumor margins, tumor boundaries, and tissue vascularization.

[0061] In some embodiments, a large number of pixels can indicate the saturation of a particular color or color combination. This may cause errors in the conversion from RGB to an alternative color space. For example, when the green channel is saturated (i.e., the value caused by emission is greater than the maximum value of 255), this may cause the hue to shift unnaturally from a narrow band of hue values ​​of unsaturated colors during conversion. Therefore, additional imaging steps can discard pixels with low saturation values. In some embodiments, this can be solved by quickly acquiring sequential images under varying light intensities and selecting images with minimum saturation to improve the detection of target characteristics or colors of interest. In other embodiments, the information lost due to saturation is still useful in determining specific markers of a particular type of region of interest. In other words, the fact that saturation is occurring for a particular type of wound or bacteria can be recorded and used in subsequent determination of the particular type of wound or bacteria.

[0062] As described above, the module includes logic executed by the processor 301. As used herein and throughout this disclosure, "logic" refers to any information in the form of instruction signals and / or data that can be applied to the operation of the processor. Software is an example of such logic. Examples of processors are computer processors (processing units), microprocessors, digital signal processors, controllers, and microcontrollers, etc. Logic can be formed by signals stored on a computer-readable medium such as a memory 300, which can be a random access memory (RAM), a read-only memory (ROM), an erasable / electrically erasable programmable read-only memory (EPROMS / EEPROMS), a flash memory, etc. in an exemplary embodiment. Logic can also include digital and / or analog hardware circuits, for example, hardware circuits including logical AND, OR, XOR, NAND, NOR, and other logical operations. Logic can be formed according to a combination of software and hardware. On a network, logic can be programmed on a server or a complex of servers. A specific logic unit is not limited to a single logical location on the network. Moreover, modules do not need to be executed in any particular order. Each module can call another module when it needs to be executed.

[0063] FIG. 4A to FIG. 4D Describe the exemplary histogram of training image.As described herein, histogram is used to identify the exemplary hue saturation and color distribution of standard wound shape.For example, the image of fluorescence emission and / or white light reflection from known wound and bacterial load can be marked with known information, pixel value of the described image converted to HSV (hue, saturation, brightness) from RGB (red, green, blue) or other alternative color space as described above, and the 2D histogram of the pixel inside and outside the region of interest can be generated.In addition, the different histogram groups of wound size relative to bacteria can be generated individually or in parallel. Figure 4A depicts a histogram of the pixels inside the region of interest for a single training image, while Figure 4B A histogram depicting pixels outside the region of interest is plotted. The histogram shown is plotted on the x-axis with saturation values ​​from 0 to 255 and on the y-axis with hue values ​​from 0 to 179. These ranges are exemplary only and may vary depending on the sensitivity of the imaging instrument and / or the type of image being analyzed.

[0064] also, Figure 4A and Figure 4BThe histogram of is presented from a top view with the population density of each hue and saturation "bins" indicated by a color scale. A bin is simply a unique combination of saturation and hue values. Bins drawn in orange and yellow contain a large number of pixels. In order to draw the histogram of pixels inside and outside the ROI using the same population density scale, each bin frequency from the histogram inside the ROI is multiplied by the maximum bin frequency value from the histogram outside the ROI. This process is called data normalization. Figure 4C and Figure 4D The same histogram is depicted from different angles (inside the AOI and outside the AOI, respectively). It is clear from these histograms that the pixels inside the region of interest have a tightly grouped range of hue and saturation values ​​relative to the pixels outside the region of interest.

[0065] As described further herein, after a suitable sample of images having identified regions of interest have been processed, a composite histogram may be generated. FIG. 5A to FIG. 5D Describe and FIG. 4A to FIG. 4D An exemplary composite histogram of multiple training images corresponding to the histogram of . The composite histogram is used to generate a suitable first pass chroma mask as described herein. In addition, out-of-bounds behavior (such as image saturation identified above) can be visualized through the histogram, and real-time image analysis programs can be developed to minimize these effects.

[0066] Figure 6 Describes an exemplary method for chroma masking. The method can be FIG. 2A to FIG. 2C and Figure 3, or in any suitable manner. Chroma masking begins with removing unwanted data at operation 601 using a low-pass spatial filter that removes noise and insignificant outlier pixels. At operation 602, the image is converted from an RGB (red / green / blue) color space to an alternative color space to facilitate the subsequent generation of a histogram. Whether the wound is excited with white light or with light of a specific wavelength or range thereof, the color space conversion uses the RGB input image sensed at the camera. At operation 603, a binary image mask is generated based on a predetermined threshold from an earlier training operation. In other words, the threshold is applied to the alternative color space value of the current image, thereby producing a binary mask. Subsequently, at operation 604, a spatial filter is applied to the binary color mask, which has the effect of removing unwanted pixels such as outliers and sparse segments. This is based on the theory that pixels of interest will tend to be surrounded by other pixels of interest. However, erosion may remove pixels that are actually within the region of interest, so operation 605 is performed to apply a dilation spatial filter that counteracts some of the negative effects of the erosion in operation 604 and has the effect of rejoining smaller clusters that survived the erosion.

[0067] Figure 7 Depicted in Figure 6 An exemplary method for contour detection is performed after a chroma masking operation of . The method can be performed by FIG. 2A to FIG. 2C and Figure 3 , or in any suitable manner. The method begins at operation 701 with a low-pass filter, which removes some details in the mask, resulting in blurring. Blurring is combined with subsequent operation 702 (i.e., a high-pass edge detection filter (Canny filter)), which finds the edge of the area identified in the chroma masking operation. Subsequently, at operation 703, contour detection is used to detect continuous closed edges. The continuous closed edge defines the boundary between the pixels inside and outside the region of interest. This causes a large number of closed contours of various sizes. Subsequently, in step 704, the contours are analyzed to find the contours that surround the largest area (i.e., those that are more likely to carry significant information). For example, closed contours can be arranged in order of region as described herein, and the contours that surround the largest 2-3 regions can be selected to define the region of interest. This method outputs one or more most significant regions of interest.

[0068] generally, Figure 7Contour detection may fail to detect all relevant contours, or may end up eroding contours until they are convex in shape, thereby losing useful information. For example, as erosion occurs, sometimes wound boundaries are eroded, resulting in concave contours. Since the actual shape of a wound can be very irregular, with many concave areas, the image inpainting operation identifies features of specific extreme concavities that may be considered unnatural. This can be further applied to the presence of bacteria. Since erosion can discard pixels that are part of areas with bacteria, abnormal contours may result. In addition, referring to FIG. 12A to FIG. 12C Another exemplary method for contour detection is described in further detail.

[0069] Figure 8 An exemplary method for image restoration and analysis is described. The method can be performed by FIG. 2A to FIG. 2C and Figure 3 The method begins at operation 801 where a concave contour is detected and a convex hull of the wound is determined. The contour is analyzed to ensure that the shape of the closed contour (enclosing the region of interest) is relatively convex in nature. If the contour exhibits concave features, this may be an indicator that part of the contour detection may be wrong. The concept is based on the theory that many detected biological features are generally more convex in shape. This information can be programmed into the system based on training information. Therefore, at 802, as Fig. 9B As depicted, erroneous concave features can be reshaped by bringing them closer to a convex hull, thereby providing a more overall convex shape for the wound boundary. Finally, at 803, 804, and 805, the final analysis provides a graphical overlay on the raw data to highlight the region of interest, and performs a final quantification of the metric of interest (such as bacterial load or wound size), and outputs a composite image with the overlay.

[0070] FIG. 9A to FIG. 9B Described wound imaging and analysis operation exemplary output image.This figure shows the wound boundary detected and marked with white border, and the wound convex hull depicted by the cyan overlay around the wound.The wound area calculation presented at the top of each image is the count of the number of pixels within the wound.A target marker (or a sticker of known size, shape, color and / or pattern, or a known image, mark or graphic thereon) can be attached to the patient, so that the actual wound area can be calculated using a simple ratio between the known target pixel count and the detected wound pixel count. Fig. 9A Describe the white outline measured from the original wound border. Figure 8As shown, the cyan overlay represents the convex hull around the wound, which is used as an intermediate calculation. The image is then repaired by detecting significant concave features, which are obtained by comparing the white contour with the vector points of the convex hull. If a significant concave feature is detected, the vector points of the wound boundary in the convex area are replaced with the vector points of the convex hull. Fig. 9B The resulting reshaped wound borders are depicted.

[0071] As described herein, these operations are used to determine a number of target characteristic information and changes thereto, such as wound size, bacterial load, type and presence of bacteria, and / or infection. Despite the fact that wound images typically include only one wound, the same (or different) images may include several areas of bacterial presence / growth / range / colony, and the described modules may be applied to both wound size, depth, and bacterial detection. For example, the detected wound boundary may be a continuous perimeter (i.e., a single line), and Pseudomonas bacteria may be present as various islands within and around the wound boundary. Therefore, erosion operations may be applicable to both the perimeter of the wound and the perimeter of bacterial presence. For example, when determining the contour, the operation of marking the perimeter around the region of interest may be repeated for multiple regions of interest, and ultimately sorted by the size of the area of ​​the final filter that may be adjusted for different applications.

[0072] In addition, additional color and intensity determination operations can be performed on the wound image. For example, some bacteria produce a red fluorescent signal when illuminated and imaged by the devices and systems described herein. In order to analyze the fluorescent signal in the captured image, a bacterial load quantification operation can be used to identify and quantify the bacterial fluorescent signal. Although described herein with reference to red fluorescence, it should be understood that the described methods and analyses can be used to analyze fluorescence of other colors or spectral wavelengths to identify bacterial load or other parameters associated with a given fluorescence wavelength.

[0073] Fig.10Describe an exemplary method for spectral analysis of a wound image. The method begins at 1001, which receives an optional input of a fluorescent image and a wound size input. The wound size input can be used to determine wound progression by processing color information as a function of wound size. In either case, the image can be an RGB (red, green, blue) color image based on an additive color model in which red, green, and blue color channels are added together to produce a wide array of colors. Each pixel in the digital image has three 8-bit values ​​(0-255) corresponding to the intensity of each individual RGB color channel, where 0 represents no color and 255 represents a true RGB color. In order to identify red areas in the image and create an image mask as a visual representation, the boundary must contain all three color channels. This is achieved by defining thresholds on the RGB channels, and then using these thresholds to create boundaries that are considered fluorescent red and are not considered fluorescent red.

[0074] At operation 1002, the threshold of the boundary function is defined using a (pre-set) default value or by the user's input. The boundary function will represent a boundary on the RGB color cube that separates the accepted red from the rest of the colors. The boundary will be centered on RGB red (255, 0, 0), but the distance from RGB red to the boundary will not be equal in all directions. There will be a greater acceptable distance along the red channel than the blue or green channels to give the red channel a greater weight. In other words, the threshold identifies the accepted intensity of the color channel to be accepted in the resulting mask. Therefore, in order to detect red fluorescence, a minimum threshold is set for the red channel, and a maximum threshold is set for the green and blue channels. In addition, there must be a separate threshold for each color channel to give the red channel a greater weight when determining whether a pixel is red. Since various variables (brightness, saturation, hue) can affect the color of the image, these thresholds can also be adjusted by the user to allow the best mask to be generated for the image. The resulting boundary function will be a 3D quadratic form defined by three initial conditions (user-defined thresholds) representing the minimum red intensity, the maximum green intensity, and the maximum blue intensity. Additionally, other combinations of color thresholds can be defined to generate masks of specific colors.

[0075] Operations 1003 to 1007 select individual pixels and determine whether the pixel meets the boundary conditions or thresholds defined in operation 1002. As long as there continue to be pixels to be analyzed, based on determination 1004, pixels continue to be "binned" (i.e., included in the red fluorescence signal output (step 1006) or excluded from the red fluorescence signal output (step 1007). Ultimately, when there are no more pixels, a mask image is output, along with an optional RGB histogram, fluorescence signal data, and a defined wound size. For example, as 4A to 5DAs shown, a histogram generated based on the intensity of each RGB channel can be used to visually guide the user to select an appropriate threshold level. In other words, the operation can be an iterative process to allow the user to adjust the threshold in real time while viewing the output until they are satisfied.

[0076] Additionally, similar to the RGB histogram, the individual RGB color channels can provide valuable information for additional image analysis. A color channel is represented by a grayscale image of the same color image, which consists of only one of the RGB colors. Dark areas (black) represent low intensities in the channel, while bright areas (white) represent high intensities in the channel. These grayscale images are generated by outputting only one color channel of interest when displaying the image.

[0077] As mentioned above Figure 3 Further description, clearly defining the wound size enables additional operations. For example, this enables the area of ​​red or other fluorescent signals to be calculated as a percentage of the wound size. At 1001, the wound size can be defined as an input, which is selected by the user via a user interface, for example, to select the periphery of the wound. The output can be normalized as a percentage of the wound size and can be used to track the progress of healing. For example, the change in bacterial load / red fluorescence can be monitored over a period of time and determined as the percentage or rate of change in pixels per unit area of ​​the wound. In addition, the bacterial load (or red) can be outside the wound size, so it is possible to use the wound size as a fixed measurement value and determine the change in the relative amount of red, thereby indicating the growth of the bacterial load. In addition to the wound size, any other fixed quantity such as a percentage of the image size can be used.

[0078] In addition, the intensity of the red fluorescent signal can be used to quantify the bacterial load. The intensity can also be used to quantify other fluorescent elements / compounds / ingredients of the target in the wound or in the surgical area. Given the same threshold and the same imaging conditions of a series of images, the histogram values ​​of each image can be compared over time to track the intensity changes of red, which is directly related to the bacterial load. Therefore, the output 1008 can include the minimum, maximum, and average intensity of the signal, and a histogram of the distribution for visual representation.

[0079] As described herein, the output may be used to determine the effectiveness of a treatment via a labeled image that highlights regions of interest and / or is superimposed on the original / starting image. Fig.11 Depicts an exemplary user interface (GUI) for color analysis of wound images. The GUI demonstrates Fig.10The inputs and outputs of the bacterial load quantification operation described in . The "Original Image" is used by the user to define the wound perimeter and perform the load quantification operation. A mask of red pixels is displayed as superimposed on the "Mask Image". The metric displayed in "Red Fluorescent Signal Intensity" is calculated based on the pixels included in the mask. The pixels included in the mask are used to calculate the percentage of the wound size that is covered by bacteria. In addition, a lookup table (LUT) can be used to superimpose colors on the mask pixels to indicate the corresponding fluorescence intensity. Fig.11 Depicts an example application of a LUT on an image, with the intensity of bacteria identified in the LUT being Fig.11 The right side of is shown in grayscale.

[0080] In addition, as described above, spatiotemporal co-registration can be performed to correlate multiple images, thereby providing a more detailed analysis of a specific wound, characteristic, or patient, such as tracking changes or growth in a specific characteristic. For example, a device equipped with white light, fluorescence, and thermal sensors can be used to obtain synchronized images of each type from the same target wound or characteristic. In an embodiment, a white light reflectance image, a fluorescence image, and a thermal image of the same wound can be subjected to their own respective analysis and then used as input to generate a composite image with all three images and the analysis superimposed thereon. The combined or super-composite output image can be used to determine additional analysis or diagnosis of a specific wound. For example, a wound (or area thereof) with a large number of bacteria present (i.e., a significant bacterial load) and depicted with a high temperature or "hot spot" (including when used in conjunction with standard clinical practice guidelines) can be determined to be infected. In other words, analysis data from different types of images of the same wound can be viewed concurrently (i.e., in a single analyzed super-synthetic image) to determine additional information about the wound that may not be obtained or immediately displayed from viewing separate white light, fluorescence, or thermal images alone.

[0081] Even deeper analysis can be performed by viewing hyper-synthetic images generated over a period of time for the same wound or patient, for example by using registration markers / stickers or co-located features. In addition, synchronously acquired and spatially co-located images acquired using, for example, an imaging device with multiple sensors can be used to track changes in the bacterial load of a particular wound over time. For the same wound, the total bacterial load and the difference in wound temperature relative to the surrounding surface temperature can be determined over time. Observing the relationship between changes in bacterial load and temperature can be used to trigger the determination of infection. For example, knowing that bacterial load increases before temperature increases, a relationship can be determined and used to predict the occurrence or risk of infection in various situations.

[0082] Although these operations have been described with respect to red fluorescence, other colors may be used to determine other target properties, such as proliferation of collagen which may provide measurements of wound healing, blood, bone, etc. Target properties may also be determined, such as the density of collagen, elastin, and other fluorescent compounds including those in diseased tissues such as tumors.

[0083] In other embodiments, the results of the spectral analysis can be used to distinguish between living tissue and non-living tissue, for example, Fig.11 Many of the operations described herein can be combined in different ways, such as to determine and output wound size, and then determine or quantify the presence of bacteria or other characteristics within the boundaries of the wound size.

[0084] Furthermore, these operations may be applied to a 3D stereoscopic image comprising two simultaneously acquired and longitudinally displaced 2D images. This is accomplished by generating two histograms corresponding to each of the two stereoscopic images and performing the above analysis performed on each of the two subsequently acquired stereoscopic images. In some embodiments, the histograms for the 2D images may be used to process a pair of stereoscopic (or 3D) images without substantially affecting the output.

[0085] In an exemplary embodiment, detection and measurement of wound boundaries (e.g. Figure 7 , Figure 8 ,as well as FIG. 9A to FIG. 9B The method may be facilitated by receiving user input corresponding to an approximate wound boundary and performing operations based thereon to identify the boundary and obtain a measurement thereof. Figure 7 This exemplary embodiment for identifying and measuring wound boundaries is performed in addition to the contour detection described and described below and with reference to FIG. 12A to FIG. 13BDescribed. Typically, a user interface is provided that enables a user to define the approximate boundary of a wound on an image of the wound via an input device. The boundary may include any shape and need not correspond precisely to the shape of the wound depicted in the image. As described above, the user may optionally further indicate a region of interest. The operations subsequently performed by an imaging device or computer provided with a user interface include marking pixels outside the approximate user-defined boundary as background, and marking pixels within the boundary as foreground. Other pixels may be marked as background or foreground. For example, pixels outside the user-defined boundary may be marked as obvious background (BG), and pixels within the user-defined boundary may be classified into three categories including possible background (PBG), possible foreground (PFG), and obvious foreground (FG). A combination of processing techniques is used to identify boundaries, including image segmentation by iterative minimization, boundary matting, foreground estimation, and other operations including those performed in the GrabCut method (https: / / cvg.ethz.ch / teaching / cvl / 2012 / grabcut-siggraph04.pdf).

[0086] FIG. 12A to FIG. 12C An exemplary image depicting a wound having a user-defined boundary and foreground and background regions determined based thereon. Fig. 12A An exemplary image 1200 depicting a wound 1220. The image 1200 may be acquired using the same device / system components as described herein for real-time imaging of wounds, or at least using common imaging conditions such as excitation (or illumination) light type and frequency, filters, etc. The image 1200 may be acquired in real time using imaging hardware coupled to an analysis module described herein. Alternatively or additionally, the image 1200 may be acquired from the imaging hardware and sent to a computer performing the disclosed operations, or acquired from an external source such as a database or network. Typically, the image 1200 is initially acquired using an RGB camera or sensor, thereby generating an RGB raw image. Other systems for acquiring images in various formats are also possible. In addition, the image 1200 depicts one or more wounds 1220, surrounding tissue surfaces, and their characteristics. For example, the wound 1220 may include any damage or injury to the surface of an organism, such as a cut, burn, scrape, surgical incision, ulcer, etc. The wound may expose an area including blood, connective tissue, muscle, bone, etc. beneath the skin. In an exemplary embodiment, the wound 1220 may include a surgical cavity.

[0087] Fig. 12BA user-defined boundary 1222 provided by a user of a device having a user interface disposed thereon for receiving touch-based input is depicted. For example, the user-defined boundary 1222 and other characteristics of the wound 1220 (such as fluorescence data, RGB color data, and other pixel values ​​and bacterial presence) may be provided as part of the training image data included. Alternatively or additionally, the user-defined boundary 1222 may be provided during real-time imaging of a wound on a patient and may be input via a touch-sensitive screen of a wound imaging and analysis device (such as the MolecuLight i:X (RTM) device developed by MolecuLight (RTM) described above). In either case, as described herein, the user-defined boundary 1222 need not conform to the shape of the wound 1220 and may simply be an approximation of the area of ​​the image 1200 containing the wound 1220.

[0088] Fig. 12C Depicts foreground and background regions determined based on analysis of image 1200 and user-defined boundaries 1222. Fig. 12C As shown, pixels outside the user-defined boundary 1222 can be marked as obvious background 1224, while pixels inside the user-defined boundary 1222 can be segmented into three segments including possible background 1226, possible foreground 1228, and obvious foreground 1230. These areas can be detected using a combination of processing techniques, including image segmentation by iterative minimization, boundary matting, foreground estimation, and other methods described in the grab cutting algorithm cited above, so that the user-defined boundary 1222 is irregular or incomplete. In addition, an adjustment mechanism can be provided via a user interface to enable the user to adjust the thickness or position of each of the segments 1224, 1226, 1228, and 1230. For example, a slider can be provided to adjust the segmentation variable, which causes the expansion or contraction of the segments 1224, 1226, 1228, and 1230 until an ideal or precise level is reached. Therefore, facilitating such boundary detection can enhance other operations such as detecting regions of interest, quantifying metrics of interest, etc. described above.

[0089] In an additional exemplary embodiment, the size of the wound boundary determined by the above segmentation can be determined via a series of operations performed on the determined boundary. For example, in order to determine the length of the wound, a bounding box is first drawn around the determined wound boundary. Subsequently, one or more intersections are determined between the wound boundary and the bounding box. The one or more intersections correspond to the extreme points of the length. The distance between each intersection or extreme point is measured, and the maximum value of these distances is determined to be the length of the wound. Then, for the width, the vertical slope is determined based on the two intersections of the defined length, and the contour points along the vertical slope are iterated from the first extreme point to the last extreme point. At each iteration, a vertical line is constructed, and a bitwise operation is performed on each vertical line and the wound boundary. The resulting multiple lines are determined using one or more line search techniques, the width of each line is determined as a vector, and the maximum value is found from multiple vectors. The maximum value corresponds to the width of the wound. In addition, the area of ​​the wound can be calculated using a line integral technique such as Green's theorem. Furthermore, although the length, width, and area values ​​are determined in pixels based on the image itself, they can also be converted into physical values ​​(e.g., mm, mm) by detecting two stickers placed around the wound and calculating the ratio of pixels to mm. 2 ).

[0090] As described herein, a marker or sticker placed on the patient's body can be used to orient the camera's field of view to facilitate registration, find co-located features, align images, identify distances, and redirect images. For example, an image is acquired after two different markers or stickers are placed at opposite ends of a wound, the acquired image is processed to detect the stickers and their diameters (i.e., the pixel / mm ratio of each sticker obtained by dividing the diameter of each sticker measured in pixels by its physical length), and the pixel / mm ratio of the wound is determined as the average of the pixel / mm ratios of the two stickers. In further exemplary embodiments, a combination of one or more of the color of the sticker, the size of the sticker, the shape of the sticker, the image or mark on the sticker, and different combinations of stickers can be used to indicate different types of wounds or patients, or to trigger different types of co-registration and analysis thereof (such as, for example, automatic file association and storage of images containing certain stickers). In an exemplary embodiment, a sticker of known size, shape, color, and / or pattern, or a known image, mark, or graphic is utilized.

[0091] However, because the lighting may change even when the same imaging device is used, relying solely on the color of the sticker or marker may produce unpredictable results. Therefore, additional properties such as shape, roundness, elongation, area, etc. of the sticker can be used to distinguish the sticker from other objects in the image or field of view. Typically, these properties may depend on how the sticker is displayed in the image. Therefore, the operation described herein includes using a data set of known images to adjust or train how to analyze these properties. In an exemplary embodiment, stickers of various types and shapes are manually segmented, and the properties of each sticker fragment are measured and input into the training algorithm. Similar to the training data set of the above-mentioned wound image, this manual segmentation promotes the generation of ground truth by carefully isolating the sticker from their background. Subsequently, the performance of the data set can be objectively determined. In addition, these operations can be performed in real time (i.e., during the visualization and analysis of the wound using the above-mentioned imaging device), so that real-time feedback can be provided to improve the efficiency of the imaging device and determine the size and area of ​​the wound.

[0092] Fig.13A A method for identifying stickers in a wound image according to an exemplary embodiment is described. Figures 2A to 3 Description includes apparatus and system for performing Fig.13A However, it should be noted that with the necessary adjustments that would be apparent to one of ordinary skill in the art in light of the present disclosure, Fig.13A The described operations can be performed by any device or system. At operation 1301, an image of a wound is received. The same device / system components as described herein for real-time imaging of wounds may have been used, or at least common imaging conditions such as excitation (or irradiation) light type and frequency, filters, etc. are used to acquire images. Imaging hardware coupled to the analysis module described herein can be used to acquire images in real time. Alternatively or additionally, the image can be acquired from the imaging hardware and transmitted to a computer that performs the disclosed operations, or acquired from an external source such as a database or network. In addition, the image depicts one or more wounds, surrounding tissue surfaces and their characteristics. For example, a wound can include any damage or injury to the surface of an organism (such as a cut, burn, scrape, surgical incision, ulcer, etc.). The wound can expose an area including blood, connective tissue, muscle, bone, etc., below the skin. In an exemplary embodiment, the wound can include a surgical cavity.

[0093] At 1302, the image is converted to one or more binary images by applying several thresholds ranging from a minimum inclusion threshold to a maximum exclusion threshold, and a distance thresholding step performed between adjacent thresholds. In an exemplary embodiment, as described above with reference to Figure 6As described above, the binary image can be generated using a chroma masking operation. In addition, at 1303, an additional binary image is added to the binary image generated in 1302, wherein the additional binary image is based on a thresholding operation using the color of the sticker. At 1304, erosion and dilation operations are performed to remove noise. Similar to Figure 6 This step involves applying a spatial filter to the binary image to remove unwanted pixels, such as outliers and sparse fragments, and applying a dilation spatial filter to counter some of the negative effects of erosion and rejoin smaller clusters that survived the erosion.

[0094] At 1305, the binary image is filtered using multiple criteria to extract the blob-shaped object from the background. In other words, the sticker is correctly identified by filtering out the detected blob based on the blob shape. Therefore, the filtering operation includes calculating all moments up to the third order, and then performing several filtering on the returned blob based on multiple criteria adjusted to accurately and reliably detect the sticker. In an exemplary embodiment, the multiple criteria include area, roundness, ratio of minimum inertia to maximum inertia, convexity, compactness, binary color, and / or ellipticity. For example, the extracted block may be required to have an area between the minimum value (inclusive) and the maximum value (exclusive); roundness between the minimum value and the maximum value (e.g., calculated using the arc length formula); ratio of minimum inertia to maximum inertia between the minimum value and the maximum value (which provides a measure of elongation); the area of ​​the blob divided by the area of ​​the convex hull (i.e., convexity) of the blob between the minimum value and the maximum value, and the compactness between the minimum value and the maximum value. In addition, the intensity of each binary image can be compared with the color value of the spot at the center of the spot, and different values ​​can be filtered out (because this is a binary image, the color filtering process is different from filtering the image based on RGB / CIELAB / HSV color space values ​​as described above). Finally, the area measured by the first moment is compared with the area of ​​the ellipse, and spots with values ​​greater than the maximum value are filtered out.

[0095] Then, at 1306, connected components are extracted from each binary image and their centers are calculated. At 1307, the centers from several binary images are grouped based on their coordinates, where close centers form a group corresponding to a spot. This can be determined using the minimum distance parameter between spots or other techniques such as the open resource OpenCVsimpleblobdetector (https: / / docs.opencv.org / 3.3.1 / d0 / d7a / classcv_1_1SimpleBlobDetector.html). Each of these parameters can be adjusted by the operator according to the desired results. For example, the parameters can be adjusted to effectively separate stickers from other spot-shaped objects. One or more standardized images can be generated to enable all parameters to be measured from a single image. Such an image can be referred to as a parameter-adjusted image, in which stickers in the parameter-adjusted image are manually segmented and the above-mentioned techniques are used to measure intensity, roundness, inertia, area, convexity, ellipticity, compactness, and minimum distance. The minimum and maximum values ​​of these measurements can be stored and used as optimal values ​​for detecting stickers in subsequent images. Furthermore, the stored control parameters can be continuously adapted as the base true value database becomes larger.

[0096] The result of this method provides a set of two-dimensional points (i.e., contours) for each spot. In addition, the set of contours can be merged and displayed on the source image. Each merged contour represents the boundary of a single sticker, and given the set of 2D points for each spot, the final contour can be approximated by approximating a curve with another curve having fewer vertices so that the distance between them is less than or equal to a specific accuracy, thereby determining each merged contour. The final center of each spot has been determined, and a loop operation is performed to loop through the contour and fit an ellipse around it to return a rotated rectangle inscribed with an ellipse. In addition, the major and minor axes of each spot are obtained and stored. Finally, ellipses and rectangles are drawn around each sticker using the rotated rectangle calculated in the previous step. Fig. 13B An image 1300 of a wound 1320 is shown, including a boundary 1322 determined as described above, and a major axis 1331 and a minor axis 1332 displayed on the wound image 1300 .

[0097] To enable real-time processing and visualization of, for example, a video comprising a series of images (i.e., frames), each frame may be retrieved from a storage buffer and processed in real time. For example, the entire first frame may be processed to locate stickers, and to reduce consumption of computing resources, a small region of interest may be defined around each sticker detected in the first frame, and subsequent frames may be processed based on the defined region of interest rather than processing the entire frame.

[0098] As described above, the validation dataset can be used to evaluate the performance of the above sticker detection method. For example, the validation process for detecting stickers can include manually segmenting the image to create a ground truth value for quantitatively evaluating the detection method. Metrics such as dice metrics, area, and Hausdorff distance are useful in verifying the accuracy of the segmentation. The dice coefficient is determined based on the following formula to measure the spatial overlap range between two binary images, and its value can range between 0 (no overlap) and 1 (complete consistency):

[0099]

[0100] Where TP, FP, and FN refer to true positives, false positives, and false negatives, respectively. If the segmented region is labeled as 1 and the background is labeled as 0, then the true positives represent the total number of pixels with a value of 1 in both the segmented image and the ground truth image, the false positives represent the total number of pixels that appear as 1 in the segmented image but appear as 0 in the ground truth, and the false negatives represent the total number of pixels that appear as 0 in the segmented image but appear as 1 in the ground truth.

[0101] In addition, the region similarity (AS) operation utilizes the following equation:

[0102]

[0103] Here, for perfect segmentation, AS=1, and for poor segmentation, AS~0.

[0104] Furthermore, the average Hausdorff distance (AHD) between two finite point groups of A and B can be defined by the following equation:

[0105] AHD(AB)=mil x(d{A,B)d(BA))

[0106] in,

[0107] A Sobel edge detection operation can be used to define Group A and Group B as points on the edges of stickers in both the ground truth and the automatically segmented image.

[0108] Therefore, the above method facilitates sticker detection by utilizing a combination of sticker color, sticker shape, and sticker size to facilitate determination of the size and orientation of the wound in the wound image. In addition, a camera (e.g., a surgical camera) can be co-registered with the anatomical location based on the sticker. The gyroscope and self-orientation software incorporated into the surgical camera can be used to co-register the camera field of view with the surgical field to spatially identify features of the surgical cavity or wound and enhance the real-time view provided to the surgeon or other operator of such a camera. In addition, each of these methods can be adjusted to execute at a speed of approximately 27 frames per second to provide real-time feedback to the surgeon / operator. In an exemplary embodiment, the method is adjusted to a minimum of 27 frames per second, and potentially the frame rate is higher than 27 frames per second.

[0109] The foregoing disclosure of exemplary embodiments of the present disclosure has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. In view of the foregoing disclosure, many variations and modifications of the embodiments described herein will be apparent to those of ordinary skill in the art. The scope of the present disclosure is limited only by the appended claims and their equivalents.

[0110] In addition, when describing representative embodiments of the present disclosure, the specification may present the method and / or process of the present disclosure as a specific sequence of steps. However, to the extent that the method or process does not rely on the specific sequence of steps set forth herein, the method or process should not be limited to the specific sequence of steps described. As will be appreciated by those of ordinary skill in the art, other sequences of steps are also possible. Therefore, the specific sequence of steps set forth in the specification should not be interpreted as a limitation on the claims. In addition, the claims for the method and / or process of the present disclosure should not be limited to performing its steps in the order written, and those skilled in the art can easily understand that these sequences can be changed and still remain within the spirit and scope of the present disclosure.

[0111] It will be apparent to those skilled in the art that various modifications and variations may be made to the apparatus and method of the present invention without departing from the scope of the teachings of the present invention. Other embodiments of the present invention will be apparent to those skilled in the art based on the description and practice of the teachings disclosed herein. The description and embodiments described herein are intended to be considered as exemplary only.

[0112] For the purpose of this specification and the appended claims, unless otherwise indicated, all numerals and other values ​​used in the specification and claims indicating amounts, percentages, or ratios should be understood to be modified in all cases according to the term "approximately" as long as they have not been so modified. Therefore, unless otherwise indicated, the numerical parameters set forth in the following specification and the appended claims are approximate values ​​that can vary according to the desired properties sought to be obtained. At least, and not attempting to limit the application of the doctrine of equivalents to the scope of the claims, each numerical parameter should at least be interpreted in view of the number of reported significant figures and by applying ordinary rounding techniques.

[0113] Although the numerical ranges and parameters setting forth the broad scope of the present teachings are approximate, the numerical values ​​set forth in the specific examples are reported as precisely as possible. However, any numerical value inherently contains certain errors necessarily resulting from the standard deviation found in the corresponding testing measurements of the numerical value. In addition, all ranges disclosed herein are understood to include any and all subranges subsumed therein.

Claims

1. A computer-implemented method for wound analysis, the computer-implemented method being stored on a computer-readable medium and comprising logic instructions executed by a processor to perform operations comprising: receiving an image of a wound, the image comprising a plurality of pixels; determining at least one region of interest in the image based at least on applying a chrominance mask to the plurality of pixels, the chrominance mask being based on a histogram of pixel values; determining one or more contours of at least one of the regions of interest; and generating an output image including the one or more contours superimposed on the image; Wherein, the region of interest includes one or more wound characteristics.

2. The method according to claim 1, wherein: The one or more wound characteristics include wound borders, wound size, wound depth, presence of bacteria, bacterial load, wound temperature, presence of connective tissue, presence of blood, presence of bone, changes in tissue or cellular wound components, vascularization, or necrosis.

3. The method according to any one of claims 1 and 2, further comprising generating the histogram of pixel values ​​based on a plurality of training images of one or more wounds, each of the plurality of training images containing at least one known region of interest, wherein The histogram of pixel values ​​identifies a unique spectral signature of the one or more wound characteristics.

4. The method according to claim 3, wherein: At least one of the known regions of interest is based at least in part on a swab or tissue biopsy analysis of the wound in a corresponding one of the plurality of training images.

5. The method according to any one of claims 3 and 4, further comprising classifying the plurality of training images based on at least one of the known regions of interest.

6. The method according to any one of claims 3, 4 and 5, wherein: The histogram includes a composite histogram based on a plurality of known regions of interest corresponding to the plurality of training images.

7. The method of any one of claims 1 to 3, further comprising repairing the one or more contours from a concave shape to a convex shape.

8. The method of any one of claims 1 to 3, further comprising acquiring the image of the wound using a wound imaging device.

9. The method according to any one of claims 1 to 3, wherein: Acquiring the image of the wound further comprises using an imaging device of a mobile communication device, the mobile communication device forming part of a handheld wound imaging and analysis device.

10. The method according to claim 9, wherein: Receiving the image of the wound includes transmitting image data from the imaging device of the mobile communication device to a processor of the handheld wound imaging and analysis device.

Images