Systems, methods and devices for three-dimensional imaging, measurement and display of wounds and tissue specimens

The method of aligning two-dimensional white light and fluorescence images using fiducial markers generates precise three-dimensional representations of wounds and tissues, addressing the limitations of two-dimensional imaging in surgical contexts and enhancing surgical precision.

JP2024167233APending Publication Date: 2024-12-03UNIV HEALTH NETWORK
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
JP2024134236
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2019-01-17
Filing Date
2024-08-09
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Current wound and tissue characterization methods, particularly in surgical contexts like breast-conserving surgery and melanoma treatment, are inadequate for accurately identifying tumor margins and residual tissue, often leading to suboptimal surgical outcomes due to the limitations of two-dimensional imaging.

Method used

A method and device for generating three-dimensional images by aligning two-dimensional white light and fluorescence images using fiducial markers, enabling precise visualization of tissue structures and lesions through a combination of white light and fluorescence imaging techniques.

Benefits of technology

Enhances surgical precision by providing detailed three-dimensional representations of wounds and tissues, allowing for more accurate tumor margin identification and targeted tissue removal, thereby improving surgical outcomes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024167233000001_ABST
    Figure 2024167233000001_ABST
Patent Text Reader

Abstract

To provide methods, systems and devices for registration of imaging data to form three-dimensional superimposed images of a biological target such as a wound, a tumor, or a surgical bed.SOLUTION: A three-dimensional map can be generated by projecting infrared radiation at a target area, receiving reflected infrared radiation, and measuring depth of the target area. A three-dimensional white light image can be created from a captured two-dimensional white light image and the three-dimensional map. A three-dimensional fluorescence image can be created from a captured two-dimensional fluorescence image and the three-dimensional map. The three-dimensional white light image and the three-dimensional fluorescence image can be aligned using one or more fiducial markers to form a three-dimensional superimposed image. The superimposed image can be used to track wound healing and to excise cancerous tissues, for example, breast tumors. The images can be in the form of videos.SELECTED DRAWING: Figure 26D
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 62 / 793,837, filed January 17, 2019, the entire contents of which are incorporated herein by reference.

[0002] Technical Field The present disclosure relates to systems and methods for registering multi-modal image data to generate a three-dimensional representation. [Background technology]

[0003] Background technology Wounds appear in a wide range of patient populations. Although wounds can be the result of a myriad of different causes, from sudden traumatic injuries to more gradual causes such as prolonged bed rest and metabolic conditions, one of their main commonalities and problems may be accurate characterization. Traditionally, wound characterization has been performed by vision alone, which provides relatively little information regarding the number and types of cells present in the wound bed. Even with imaging devices, the images obtained are typically two-dimensional. Because wounds can undergo significant changes in depth while maintaining a relatively constant planar area, two-dimensional images can be quite limited in assessing wound healing. It is believed that improved wound characterization will aid clinicians in better understanding whether a given treatment is working and better identifying which treatment has the greatest efficacy for a particular wound. Summary of the Invention [Problem to be solved by the invention]

[0004] Accurate identification of tumor margins in lumpectomy or mastectomy (breast) surgical samples and the corresponding surgical bed remains a major problem in breast-conserving surgery. Identifying the exact location of the remaining tumor tissue in the surgical bed when a positive margin is found in the surgical sample also remains a challenge. Traditional approaches are suboptimal and are not performed in real time in the operating room. For example, generally, when a surgical sample has a positive margin, the surgeon removes the corresponding tumor in the surgical bed by scraping off a fairly thick and laterally wide layer of tissue to achieve a negative margin. This imprecise procedure reduces the chance of preserving sufficient healthy breast tissue and makes preservation efforts difficult. Similar problems exist for other cancers and procedures, such as melanoma removal. Characterization of tissue damage and lesions, such as wounds, and removal of tissue generally require improved tools and techniques for greater precision and accuracy.

[0005] Summary of the Invention The present disclosure may solve one or more of the problems set forth above and / or demonstrate one or more of the desirable features set forth above. Other features and / or advantages may become apparent from the following description. [Means for solving the problem]

[0006] According to the present disclosure, there is provided a method for generating a three-dimensional image of a target using two-dimensional images. The method may include, for example: generating a three-dimensional map of a target area having one or more associated fiducial markers; capturing a two-dimensional white-light image of the target area and the fiducial markers; capturing the two-dimensional white-light image and From the three-dimensional map, a three-dimensional white light image can be generated. Two-dimensional fluorescent images of the target area and the fiducial markers can be captured. From the two-dimensional fluorescent image and the three-dimensional map, a three-dimensional fluorescent image can be generated. The three-dimensional white light image and the three-dimensional fluorescent image can be aligned using the fiducial markers to form a three-dimensional superimposed image.

[0007] According to one aspect of the present disclosure, an imaging device is provided. The imaging device may include one or more of the following components: An excitation light source may be configured to emit a first radiation capable of exciting a fluorophore. A filter may be configured to prevent the passage of reflected excitation light and allow the passage of fluorescence emitted by the fluorophore. An imaging lens may be configured to collect the radiation. A visible light source may be configured to emit a second radiation. An infrared light source may be configured to emit a third radiation. At least one image sensor may be configured to detect the radiation. A processor may be configured to receive the detected radiation and output data related to the detected radiation. The imaging device may be configured to be handheld. The imaging device may be used to perform any of the methods described herein.

[0008] According to one aspect of the disclosure, an imaging device is provided. The imaging device may include one or more of the following components: An excitation light source may be configured to emit a first radiation capable of exciting a fluorophore. A filter may be configured to prevent the passage of reflected excitation light and allow the passage of fluorescence emitted by the fluorophore. An imaging lens may be configured to collect the radiation. A visible light source may be configured to emit a second radiation. An infrared light source may be configured to emit a third radiation. At least one image sensor may be configured to detect the radiation. A processor may be configured to receive the detected radiation and output data related to the detected radiation. The imaging device may be configured to be handheld.

[0009] According to one aspect of the disclosure, a system for three-dimensional fluorescence-based imaging of a target is provided. The system may include one or more of the following components, one or more of the components described for the imaging device, or both: At least one excitation light source may be configured to uniformly illuminate the target surface with a uniform field of excitation light during fluorescence imaging; At least one white light source may be configured to illuminate the target surface during white light imaging; At least one infrared radiation source may be configured to emit infrared radiation toward the target surface; An image sensor may detect fluorescence, reflected radiation, or both; A filter may be configured to allow an optical signal in response to illumination of the target surface with the excitation light, the optical signal having a wavelength corresponding to one or more of bacterial autofluorescence, tissue autofluorescence, and exogenously induced fluorescence to pass through the filter and reach the image sensor; A processor may be configured to perform, for example, one or more of the following: The processor may receive an optical signal in response to illumination of the target with infrared light, and based on the received signal in response to illumination of the target with infrared light, the processor may generate a three-dimensional map of the target surface. The processor can receive the detected optical signals responsive to illumination of the target surface with the excitation light and generate a two-dimensional fluorescent image of the target surface. The processor can receive the optical signals responsive to white light illumination of the target surface and generate a two-dimensional white light image of the target surface, and the processor can combine the three-dimensional map, the fluorescent image, and the white light image to generate a three-dimensional image of the target surface.

[0010] According to one aspect of the present disclosure, a computer program product for use with an imaging device is provided. The computer program product may include a non-transitory computer readable medium. The non-transitory computer readable medium may store computer program code for image processing. The computer program code may be executable by a processor in an imaging device to perform one or more of the methods described herein.

[0011] Additional objects and advantages of the disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the disclosure. The objects and advantages of the disclosure may be attained by means of the elements and combinations particularly pointed out in the appended claims.

[0012] Both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure, as claimed.

[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate one or more embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure. [Brief description of the drawings]

[0014] [Figure 1] White light photograph of a tissue phantom marked with fluorescein fiducial markers on the surface and fluorescent PPIX injected just below the surface. The tissue phantom is a porcine tissue piece used to simulate a breast lumpectomy tissue sample. [Figure 2A] 3D white light surface image of the tissue phantom from the first angle of view. [Figure 2B] 3D white light surface image of the tissue phantom from a second angle of view. [Figure 3A] 3D white light surface image of tissue phantom+mesh from the first field of view. [Figure 3B] 3D white light surface image of tissue phantom+mesh from the second field of view. [Figure 4A] 3D mesh + white light overlay image of tissue phantom from the first angle of view. [Figure 4B] 3D mesh + white light overlay image of the tissue phantom from the second field of view. [Figure 5A] Raw mesh / surface image of the tissue phantom from the first view angle. [Figure 5B] Raw mesh / surface image of the tissue phantom from the second view angle. [Figure 6A] 3D fluorescent surface image of a tissue phantom from a first angle of view. [Figure 6B] 3D fluorescent surface image of the tissue phantom from a second angle of view. [Figure 7A] 3D fluorescent surface image of tissue phantom + mesh from the first field of view. [Figure 7B] 3D fluorescent surface image of tissue phantom + mesh from the second field of view. [Figure 8A] 3D mesh + fluorescence overlay image of a tissue phantom from the first field of view. [Figure 8B] 3D mesh + fluorescence overlay image of a tissue phantom from a second angle of view. [Figure 9A] Raw mesh / surface image of the tissue phantom from the first view angle. [Figure 9B] Raw mesh / surface image of the tissue phantom from a second view angle. [Figure 10A] FIG. 1 is a first view of a tissue phantom with CT fiducials overlaid on fluorescent fiducials. [Figure 10B] FIG. 2 is a second view of the tissue phantom with CT fiducials overlaid on the fluorescent fiducials. [Figure 11A] Cone beam CT 3D volumetric reconstruction of a tissue phantom from the first field of view. [Figure 11B] Fluorescence 3D rendering of a tissue phantom from the first angle of view. [Figure 11C]13 is a registered CT+fluorescence image of a tissue phantom from a first angle of view. [Figure 12A] Cone beam CT 3D volumetric reconstruction of the tissue phantom from the second field of view. [Figure 12B] Fluorescence 3D rendering of the tissue phantom from a second angle of view. [Figure 12C] 13 is a registered CT+fluorescence image of the tissue phantom from a second angle of view. [Figure 13A] Cone beam CT 3D volumetric reconstruction of the tissue phantom from the third field angle. [Figure 13B] Fluorescence 3D rendering of a tissue phantom from the first angle of view. [Figure 13C] 13 is a registered CT+fluorescence image of the tissue phantom from the third view angle. [Figure 14A] Cone beam CT 3D volumetric reconstruction of a tissue phantom from the fourth field angle. [Figure 14B] Fluorescence 3D rendering of a tissue phantom from the fourth view angle. [Figure 14C] 13 is a registered CT+fluorescence image of the tissue phantom from the fourth view angle. [Figure 15A] is the first image of a second tissue phantom for the tumor model being created from a pig shoulder with skin similar to human skin. [Figure 15B] 13 is a second image of a second tissue phantom for the tumor model being created. [Figure 16] FIG. 11 is an image of a second tissue phantom showing a model PPIX tumor, with the fluorescein fiducial markers also visible. [Figure 17A] FIG. 13 is an image of a perspective view of a second tissue phantom being exposed to violet light to excite fluorescence. [Figure 17B] FIG. 13 is a top view image of an emission image of a second tissue phantom exposed to violet light to excite fluorescence. [Figure 18A] 11 is a 3D white light surface image of a second tissue phantom from a first angle of view. [Figure 18B] 13 is a 3D white light surface image of a second tissue phantom from a second angle of view. [Figure 19A] 3D white light surface image of the second tissue phantom+mesh from the first angle of view. [Figure 19B] 3D white light surface image of the second tissue phantom+mesh from a second angle of view. [Figure 20A] 3D mesh + white light overlay image of the second tissue phantom from the first field of view. [Figure 20B] 3D mesh+white light overlay image of a second tissue phantom from a second angle of view. [Figure 21A] 13 is a raw mesh / surface image of the second tissue phantom from the first field of view. [Figure 21B] 13 is a raw mesh / surface image of a second tissue phantom from a second angle of view. [Figure 22A] 11 is a 3D fluorescent surface image of a second tissue phantom from a first angle of view. [Figure 22B] 13 is a 3D fluorescent surface image of a second tissue phantom from a second angle of view. [Figure 23A] 3D fluorescent surface image of the second tissue phantom+mesh from the first field of view. [Figure 23B] 3D fluorescent surface image of the second tissue phantom+mesh from a second angle of view. [Figure 24A] 3D mesh + fluorescence overlay image of the second tissue phantom from the first field of view. [Figure 24B] 3D mesh+fluorescence overlay image of a second tissue phantom from a second angle of view. [Figure 25A] 13 is a raw mesh / surface image of the second tissue phantom from the first field of view. [Figure 25B] 13 is a raw mesh / surface image of a second tissue phantom from a second angle of view. [Figure 26A] FIG. 13 is a first image of the third tissue phantom for the wound model being created. [Figure 26B] 13 is a second image of a third tissue phantom for the wound model being created. [Figure 26C] FIG. 13 is a third image of a third tissue phantom for the wound model under construction showing a deep wound to which PPIX has been applied to simulate the presence of bacteria within the wound. [Figure 26D] FIG. 4 is a fourth image of the third tissue phantom for the wound model under development showing both deep and superficial wounds, as well as fluorescein fiducial markers. [Figure 27A] 11 is a 3D white light surface image of the third tissue phantom from a first angle of view. [Figure 27B] 13 is a 3D white light surface image of a third tissue phantom from a second angle of view. [Figure 28A] 3D white light surface image of the third tissue phantom+mesh from the first field angle. [Figure 28B] 3D white light surface image of the third tissue phantom+mesh from a second angle of view. [Figure 29A] 3D mesh + white light overlay image of the third tissue phantom from the first field of view. [Figure 29B] 3D mesh + white light overlay image of a third tissue phantom from a second angle of view. [Figure 30A] 13 is a raw mesh / surface image of the third tissue phantom from the first field of view. [Figure 30B] 4 is a raw mesh / surface image of a third tissue phantom from a second angle of view. [Figure 31A] 13 is a 3D fluorescent surface image of a third tissue phantom from a first angle of view. [Figure 31B] 13 is a 3D fluorescent surface image of a third tissue phantom from a second angle of view. [Figure 32A] 3D fluorescent surface image of the third tissue phantom+mesh from the first field of view. [Figure 32B] 3D fluorescent surface image of the third tissue phantom+mesh from a second angle of view. [Figure 33A] 3D mesh + fluorescence overlay image of the third tissue phantom from the first field of view. [Figure 33B] 3D mesh + fluorescence overlay image of a third tissue phantom from a second angle of view. [Figure 34A] 13 is a raw mesh / surface image of the third tissue phantom from the first field of view. [Figure 34B] 4 is a raw mesh / surface image of a third tissue phantom from a second angle of view. [Figure 35A] 13 is an image of a third tissue phantom prepared with CT fiducial markers for cone beam CT scanning. [Figure 35B] 13 is a cone beam CT scan image of the third tissue phantom. [Figure 35C] 13 is an image of a third tissue phantom created for cone beam CT scanning. [Figure 35D] 13 is an image of a third tissue phantom during a cone beam CT scan. [Figure 36A] FIG. 13 is an image of a third tissue phantom created by spatially registering the CT fiducials to the fluorescein fiducials for the CT scan. [Figure 36B] 13 is an image of a third tissue phantom during a CT scan. [Figure 37] 1 is a photograph of a wound topology measured in accordance with the present disclosure. [Figure 38] 1 is a flow chart illustrating the method of the present disclosure. [Figure 39] 1 is a flow chart illustrating the method of the present disclosure. [Diagram 40] 1 is a flow chart illustrating the method of the present disclosure. [Diagram 41] 1 is a flow chart illustrating the method of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] The present disclosure allows for the generation of three-dimensional images of objects that aid users in better characterizing and understanding the object to achieve goals that were not possible before or to achieve those goals more effectively than before. Several types of images carrying unique information can be combined in a way that aids the user in visualizing the problem and finding a solution more easily. A fluorescent image carrying information about the location of diseased or cancerous tissue, when placed in the direct background of a white light image, can present the information carried by the fluorescence to the surgeon in a more familiar context, aiding the surgeon in more accurate removal of diseased tissue and minimizing the need for reoperation. Furthermore, these images can be converted from two-dimensional to three-dimensional images for overlay to guide the user in a model environment that better approximates the real object. A two-dimensional image can be converted to a three-dimensional image by wrapping it around a three-dimensional map of the object. The three-dimensional map can take many forms, such as a mesh, and can be generated using a variety of different techniques, such as using infrared radiation and its interaction with the object.

[0016] The imaged object may be a target area of ​​a biological target, but the visualized object is by no means limited to a medical context. Rather, the disclosed method, apparatus, system, and program are applicable to a wide range of technical fields where the information provided by fluorescence is meaningful, for example, in food safety, hygiene, security, cosmetics, agriculture, horticulture, pharmaceutical areas, agriculture, veterinary medicine, border customs, quality control, and forensics. The images can include one or more of a single image, a set of consecutive images, a set of discontinuous images, a set of images from a common viewpoint (angle), a set of images from different viewpoints (angles), a set of images using the same imaging technique, a set of images using different imaging techniques, a set of time-lapse images, and a video. A video can include one or more sets of images.

[0017] According to the present disclosure, a method is provided for generating a three-dimensional image of a target using two-dimensional images. The method may include, for example: A three-dimensional map of a target area associated with one or more fiducial markers may be generated; A two-dimensional white light image of the target area and the fiducial markers may be captured; The white light image may be replaced with a single wavelength or a combination of wavelengths that constitute a subset of visible light wavelengths; A three-dimensional white light image may be generated from the two-dimensional white light image and the three-dimensional map; A two-dimensional fluorescent image of the target area and the fiducial markers may be captured; A three-dimensional fluorescent image may be generated from the two-dimensional fluorescent image and the three-dimensional map; The three-dimensional white light image and the three-dimensional fluorescent image may be aligned using the fiducial markers to form a three-dimensional overlapping image. An example of such steps, shown as 510, 520, 530, 540, 550, and 560, is shown in FIG. 38. The order of the steps may be changed; The two-dimensional images may be overlapped and then collectively converted into a three-dimensional overlapping image using the three-dimensional map. The three-dimensional map may be in the form of a mesh.

[0018] Capturing the two-dimensional fluorescent image of the target area and the fiducial markers can be performed using any applicable technique. For example, capturing can include illuminating the target area and the fiducial markers with an excitation light and receiving at least one fluorescent emission in response to illumination of the target area with the excitation light. Examples of such steps, designated as 542 and 544, are shown in FIG. 39. The excitation light can include one or more wavelengths. For example, the excitation light can be between about 400 nm and about 450 nm. A wavelength of about 405 nm is such an example. Capturing the two-dimensional fluorescent image of the target area and the fiducial markers can include capturing the emission of at least one fluorescent molecule. The at least one fluorescent molecule can include an endogenous molecule capable of emitting fluorescence. The at least one fluorescent molecule can include an endogenous molecule capable of emitting fluorescence upon induction by an exogenous molecule. The induction can result, for example, from accumulation, modification, or both of the endogenous molecule. The at least one fluorescent molecule can include an exogenous molecule capable of fluorescing, a molecule that includes an exogenously added moiety capable of fluorescing, or both. For example, the at least one fluorescent molecule can include an aminolevulinic acid (ALA)-derived porphyrin.

[0019] The three-dimensional map can be generated using any suitable technique or combination of techniques. For example, the three-dimensional map can be generated using infrared light. The three-dimensional map can be generated using near-infrared light. For example, generating the three-dimensional map can include projecting infrared radiation onto a target area, receiving infrared radiation reflected by the target area, and measuring the depth of the target area based on the reflected infrared radiation to generate the three-dimensional map. Examples of such steps, shown as 512, 514, and 516, are shown in FIG. 40. The infrared radiation can be projected as a beam that is split into a light pattern. The reflected infrared radiation can include a distortion of the light pattern, and the depth can be measured based on the distortion of the light pattern. The light pattern can be formed by a diffraction grating. The light pattern can include multiple dots of any size, shape, or intensity, or combinations thereof. Alternative methods of measuring depth can be used. For example, the depth can be measured by time of flight based on a phase shift between the projected and reflected infrared radiation.

[0020] According to the present disclosure, the following steps may be performed: One or more fiducial markers may be positioned within, along, or / and outside a perimeter of a target area on a surface of a biological target. A three-dimensional map of the target area may be generated by projecting infrared radiation onto the target area, receiving infrared radiation reflected by the target area, and measuring the depth of the target area based on the reflected infrared radiation to generate the three-dimensional map. A two-dimensional white light image of the target area and one or more fiducial markers may be captured. The white light image may be replaced or generated with a single wavelength of light or a combination of wavelengths of light that constitute a subset of visible light wavelengths. A three-dimensional white light image may be generated from the two-dimensional white light image and the three-dimensional map. A two-dimensional fluorescent image of the target area and one or more fiducial markers may be captured. Capturing the fluorescent two-dimensional image may include exposing the target area and one or more fiducial markers to at least one wavelength capable of exciting at least one fluorescent molecule in the target area and receiving at least one fluorescent emission from at least one fluorescent molecule in the target area through a filter. A three-dimensional fluorescent image can be generated from the two-dimensional fluorescent image and the three-dimensional map. The three-dimensional white light image and the three-dimensional fluorescent image can be aligned using one or more fiducial markers to form a three-dimensional superimposed image. The order of steps can be changed. After the two-dimensional images are superimposed, The three-dimensional map may then be used to convert the three-dimensional image into a three-dimensional overlay image. The three-dimensional map may be in the form of a mesh.

[0021] The two-dimensional white light image, two-dimensional fluorescence image, or any other type of image described herein can be captured using any suitable camera, imager, or image sensor. For example, the MolecuLight i:X imager, available from MolecuLight Inc., Toronto, Ontario, Canada, includes a 5 megapixel camera, emits at 405 nm, and includes fluorescence emission filters of 500-545 nm and 600-665 nm, respectively. The imager and related methods described in U.S. Pat. No. 9,042,967, the entire contents of which are incorporated by reference, can be used. Alternatively, the imaging devices and related methods described in U.S. Pat. No. 9,042,967, the entire contents of which are incorporated by reference herein, may be used. Alternatively, the imaging devices and related methods described in U.S. Pat. No. 9,042,967, the entire contents of which are incorporated by reference herein, may be used. No. 62 / 625,967, filed Feb. 3, 2018, entitled “Devices, Systems, and Methods for Tumor Visualization and Removal,” U.S. Provisional Patent Application No. 62 / 625,983, filed Feb. 1, 2019, entitled “Devices, Systems, and Methods for Tumor Visualization and Removal,” and / or International Application No. PCT / CA2019 / 000015, filed Feb. 1, 2019, entitled “Devices, Systems, and Methods for Tumor Visualization and Removal,” and published Aug. 8, 2019 as WO 2019 / 148,268, may be used to capture white light and / or fluorescent images.

[0022] Charge-coupled displays (CCDs), complementary metal-oxide semiconductors (CMOS), N-type metal-oxide semiconductors (NMOS), quantum image sensors (QIQ), or other image sensors, or combinations thereof, can be used to capture two-dimensional white-light images, two-dimensional fluorescent images, or any other type of image described herein. Images can be captured using artificial and / or natural ambient light, one or more dedicated light sources, or combinations thereof. For example, images can be captured using flash photography.

[0023] The source of white light may be full spectrum or partial spectrum visible light. For example, the white light spectrum may be about 380 nm to about 740 nm, about 400 nm to about 700 nm, about 425 nm to about 690 nm, about 450 nm to about 680 nm, or any intermediate ranges thereof. The source of white light may be generated directly or initially as white light, and such white light may be formed from light sources of different wavelengths, for example, a light source of a given wavelength or wavelength range shifted to multiple wavelengths or a broader spectrum of wavelengths using quantum dots (QDots), or any combination thereof. The white image may be replaced or supplemented, for example, with a monochromatic image, an infrared image, or an ultraviolet image, or any combination thereof.

[0024] Infrared radiation may be generated, projected, and received using any suitable technique, device, or combination thereof. The infrared radiation may be projected as a beam that is split into a light pattern, and the reflected infrared radiation may include a distortion of the light pattern, and depth may be measured based on the distortion of the light pattern. Thus, the infrared radiation may include structured light. The light pattern may be formed by a diffraction grating, and the light pattern may include a plurality of dots. Depth measurements may be determined using triangulation. The infrared depth imaging described above may be implemented using a Microsoft Kinect I system. Infrared depth imaging can be used in a Microsoft Kinect II system (Microsoft Corporation, Redmond, Wash.). Unstructured infrared radiation can be used in addition to, or instead of, structured infrared radiation. Additionally or alternatively, depth can be measured by one or more time-of-flight (ToF) sensors, for example, based on the phase shift between projected and reflected infrared radiation. This form of infrared depth imaging can be used in a Microsoft Kinect II system. Any suitable infrared camera, sensor, or imager, or combination thereof, can be used.

[0025] The projected and / or reflected infrared radiation may be from about 700 nm to about 1.0 mm, from about 750 nm to about 1.0 μm, from about 1.0 μm to about 1.25 μm, from about 1.25 μm to about 1.5 μm, from about 1.5 μm to about 5.0 μm, from about 5.0 μm to about 15.0 μm, from about 15.0 μm to about 50 μm, from about 50 μm to about 150 μm, from about 150 μm to about 250 μm, from about 250 μm to about 500 μm, from about 500 μm to about 750 μm, or from about 750 μm to about 1.0 mm, or any intermediate ranges thereof, or any combination thereof. Infrared imaging can be performed in the near infrared (e.g., about 0.75 μm to about 1.4 μm), short wavelength infrared (e.g., about 1.4 μm to about 3.0 μm), mid wavelength infrared (e.g., about 3.0 μm to about 8.0 μm), long wavelength infrared (e.g., about 8.0 μm to about 15 μm), far (very long wave) infrared (e.g., about 8.0 μm to about 1.0 mm), or any combination thereof.

[0026] The white light source, fluorescent excitation light source, infrared light source, or any other relevant light source for use in the present disclosure may be of any suitable design or combination of designs. The light source may be coherent or non-coherent, parallel or non-parallel, focused or non-focused, polarized or non-polarized, or any combination thereof. Light beam angles of less than about 1 degree, about 1 degree to about 5 degrees, about 5 degrees to about 15 degrees, about 15 degrees to about 25 degrees, about 25 degrees to about 35 degrees, about 35 degrees to about 50 degrees, about 50 degrees to about 75 degrees, about 75 degrees to about 90 degrees, about 90 degrees to about 120 degrees, about 120 degrees to about 150 degrees, and about 150 degrees to about 180 degrees can be used as white light or other light sources. One or more lasers and / or light emitting diodes (LEDs) can be used. Incandescent, thermal, arc, incandescent, fluorescent, semiconductor-based, sodium vapor, or mercury vapor, or any combination or number of these, can be used as the light source. Single or multiple light sources can be used. An array of light sources may be used.

[0027] The distance of the light source to the target being imaged and / or measured may be, for example, about 1.0 mm to about 10 m, about 0.5 cm to about 5.0 m, about 1.0 cm to about 2.5 m, about 2.5 cm to about 1.0 m, about 5.0 cm to about 0.5 m, about 10.0 cm to about 0.25 m, or about 25 cm to about 100 cm from the target area, or any intermediate distances thereof, or any combination thereof. Any number or type of light source may be used. The light sources may be fixed or movable. The light sources may be integrated into the same device that houses the camera, detector, or other imaging device, or even external to such device. The light sources may be located inside or outside the target area or target volume. One or more light sources may be articulated (e.g., manually) to vary the illumination angle and spot size at the imaging plane, for example, by using an incorporated pivot, and may be operated via electrical connection, for example, to a wall outlet and / or a separate portable rechargeable battery pack.

[0028] The fluorescence excitation wavelength can be matched to the emission wavelength of one or more fluorophores targeted for imaging. For example, the excitation wavelength can be about 300 nm to about 325 nm, about 325 nm to about 350 nm, about 350 nm to about 375 nm, about 375 nm to about 400 nm, about 400 nm to about 425 nm, about 425 nm to about 450 nm, about 450 nm to about 475 nm, about 475 nm to about 500 nm, about 500 nm to about 525 nm, about 525 nm to about 550 nm , about 550 nm to about 575 nm, about 575 nm to about 600 nm, about 600 nm to about 625 nm, about 625 nm to about 650 nm, about 675 nm to about 700 nm, about 750 nm to about 775 nm, about 775 nm to about 800 nm, about 800 nm to about 825 nm, about 825 nm to about 850 nm, about 850 nm to about 875 nm, about 875 nm to about 900 nm, or about 900 nm to about 1.0 mm, or any intermediate or overlapping ranges thereof, or any combination thereof.

[0029] At least one fluorescence excitation wavelength can include, for example, wavelengths such as 405 nm with a spread of about 0.0 nm, about 0.01 nm to about 0.05 nm, about 0.5 nm to about 1.0 nm, about 1.0 nm to about 2.5 nm, about 2.5 nm to about 7.5 nm, about 10 nm to about 25 nm, or about 15 nm to about 30 nm, or intermediate spreads, or combinations thereof. The imaging device can use, for example, two violet / blue light (e.g., 405 nm + / - 10 nm emission, narrow emission spectrum) LED arrays (Opto Diode Corporation, Newbury Park, Calif.) located on each side of the imaging detector assembly as excitation or illumination sources, respectively. These arrays can have, for example, a power output of about 1 watt with a 70 degree illumination beam angle emanating from a source of 2.5 x 2.5 cm2 each. The LED array can be used to illuminate the tissue surface from a distance of about 10 cm, which means that the total light power density at the skin surface can be about 0.08 W / cm2.

[0030] The light signals generated by the light source can be detected by an imaging device using one or more optical filters that exclude the excitation light but allow detection of selected wavelengths of emitted light from the tissue, thus forming an image on a display. Bandpass filters can be selected and aligned in front of a digital camera lens or other image detector or sensor to selectively detect specific light signals from a target based on the desired wavelengths of light. Spectral filtering of the detected light signals (e.g., absorption, fluorescence, and / or reflection) can also use liquid crystal tunable filters (LCTFs) or acousto-optical tunable filters (AOTFs), which can be electronically tunable solid-state spectral bandpass filters. Spectral filtering can also include the use of continuously variable filters and / or manual bandpass optical filters. These devices can be placed in front of the imaging detector to produce multispectral, hyperspectral, and / or wavelength-selective imaging of the target area.

[0031] Optical or variable orientation polarizing filters (e.g., linear or circular polarizing filters in combination with the use of optical waveplates) can be attached to one or more light sources and / or imagers, sensors, or cameras. These filters can enable imaging with polarized illumination and unpolarized detection, or vice versa, or polarized illumination and polarized detection, for either white light reflectance and / or fluorescence imaging.

[0032] PrimeSense cameras (technology available from Apple Inc., Cupertino, Calif.), components thereof, or other sensors capable of three-dimensional imaging can be used in the technology of this disclosure. PrimeSense cameras include Carmine 1.08, Carmine 1.09 (short-range), and Capri 1.25 (embedded) sensors. Depth acquisition uses light-encoding technology. The scene can be scanned with near-infrared light. An infrared dot pattern can be projected outward toward the target area. When the light contacts the target area, it can become distorted, and this distortion can be measured by a camera or other sensor, including, for example, a CMOS image sensor. At the same time that the infrared camera sensor images the dot pattern on the target, a white light RGB camera (adjacent to the infrared camera) captures a regular white light image of the target. The CMOS image sensor works in conjunction with a visible video sensor to produce a color image. The PrimeSense system-on-chip (SoC) generates depth maps provided by the Carmine (PS 1080) and Capri (PS 1200) sensors, which can be merged with the Carmine sensor to generate depth maps.

[0033] As an example, the PrimeSense device emits an infrared dot pattern (not visible to the naked eye) outwards towards a target. An infrared camera sensor images the dot pattern on the target while a white light RGB camera (adjacent to the infrared camera) captures a regular white light image of the target. Embedded software generates a "mesh" from the infrared dot pattern of the target topology. It then wraps this mesh topology with a white light image using image (transformation function) morphing software code. The result is a white light image that can be transformed from two-dimensional to three-dimensional. The software used to perform this process may be an open source software development kit (SDK) called OpenNI. Any suitable software can be used, such as OpenKinect or Microsoft Kinect software development kit. Meshes or point clouds can also be generated and manipulated using MeshLab.

[0034] Alternatively or additionally, according to the present disclosure, the color image may be a fluorescent image. A registration process may be performed to align the color image (RGB) and depth (D) information. The light-encoded infrared pattern may be decoded to generate a VGA-sized depth image of the target area. The PrimeSense camera includes built-in software for image processing. Additional or alternative software may be used. For example, an open-source SDK called OpenNI may be used. The built-in software generates a "mesh" from the infrared dot pattern of the target topology. Then, an image (transformation function) morphing software code is used to wrap this mesh topology with a white light image. The result is a white light image that can be transformed from two-dimensional to three-dimensional. The PrimeSense camera may provide visible video, depth, and audio information in a synchronized manner via a USB 2.0 interface. Alternatively or additionally to the PrimeSense camera, the Xtion PRO LIVE camera (including an infrared dot projector, an RGB camera, and an infrared camera) available from ASUS Computer International (Fremont, Calif.) may be used. A camera / sensor / projector system such as that described in U.S. Patent Application Publication No. 2017 / 0054966, which is incorporated by reference in its entirety, can be used. Time-of-flight infrared imaging can be used instead of or in addition to structured light based infrared imaging for depth measurement to obtain three-dimensional coordinate information of the target area.

[0035] Prior to imaging, fiducial markers (e.g., using indelible fluorescent ink pens) can be placed on the surface of the skin or other relevant surface near the edge or perimeter of the biological target. For example, four spots of different fluorescent ink colors, each from separate indelible fluorescent ink pens that can be supplied as a kit to the clinical technician, can be placed near the margin or border of the target area on the normal skin surface. These colors can be imaged by the device using excitation light and a multispectral bandpass filter that matches the emission wavelengths of the four ink spots. Image analysis can then be performed by aligning the fiducial markers for image-to-image alignment. This technique can facilitate long-term time-series imaging of the target area, thus allowing the clinical technician to image the target area over time without having to align the imaging device for each image acquisition.

[0036] To aid in the intensity calibration of fluorescent images, simple disposable fluorescent standard "strips" are available. The strip can be placed in the field of view (e.g., by using a weak adhesive that temporarily affixes the strip to the skin) during imaging of the target area. The strip can be impregnated with one or more different fluorescent dyes in various concentrations that can generate a predetermined calibrated fluorescent intensity when illuminated by an excitation light source that can have a single (e.g., 405 nm) or multiple fluorescent emission wavelengths or wavelength bands for image intensity calibration. Additionally, the disposable strip can have the four spots mentioned above (e.g., each of a different diameter or size, each of a different fluorescent ink color, with a unique black spot placed next to each other) from separate indelible highlighter ink pens. With the strip positioned near the margin or border of the target area of ​​normal skin surface, the device can be used to take white light and fluorescent images. The strip can provide a convenient way to take multiple images of a given target area over time and then align the images using image analysis. Additionally, the fluorescent "intensity calibration" strip can include an additional linear measurement device, such as a ruler of fixed length to aid in measuring spatial distances of the target area. Such a strip may be one example of a calibration target that may be used with a device to assist in calibrating or measuring image parameters (e.g., target area size, fluorescence intensity, etc.) Additionally or alternatively, other similar or functionally equivalent calibration targets may be used.

[0037] The one or more reference markers can include a fluorescent reference marker. For example, the fluorescent reference marker can include fluorescein. The at least one fluorescent molecule can include an endogenous molecule capable of fluorescing, an exogenous molecule capable of fluorescing, or both. The at least one fluorescent molecule can include an endogenous molecule capable of fluorescing induced by an exogenous molecule. The induction can be caused, for example, by accumulation, modification, or both of the endogenous molecule. The at least one fluorescent molecule can include an exogenous molecule capable of fluorescing, or a molecule including an exogenously added moiety capable of fluorescing. In one example, indocyanine green (ICG) can be excited at about 760 nm, about 780 nm, or both. A filter including a notch from about 657 nm to about 825 nm can be used for excitation at 760 nm. A filter including a notch from about 690 nm to about 840 nm can be used for excitation at 780 nm.

[0038] The techniques described herein can detect some, most, or essentially all tissue autofluorescence (AF). For example, a multispectral bandpass filter can be used to measure tissue autofluorescence arising from various tissue biomolecules such as collagen (type I, II, III, IV, V, and others), which appears green, elastin, which appears greenish yellow-orange, reduced nicotinamide adenine dinucleotide (NADH), flavin adenine dinucleotide (FAD), which emits a blue-green autofluorescence signal, and bacteria / microbes, which can appear to have mostly broad (e.g., green and red) area fluorescence emissions, as well as blood-related light absorption under, for example, 405 nm excitation. Image analysis can include calculating the ratio of red-green AF in the image. Intensity calculations can be obtained from regions of interest in the image of the target area. The pseudocolor image can be mapped to a white light image of the target area.

[0039] The techniques of the present disclosure can be used in conjunction with exogenous "pro-drug" agents, such as, but not limited to, ALA, to increase the endogenous production of porphyrins in bacteria / microorganisms, thereby enhancing the intensity of the inherent "porphyrin" fluorescent signal arising from these bacteria and improving the detection sensitivity and specificity of the device. These techniques can thus be used to conveniently image the photosensitizer-induced fluorescence (e.g., PpIX) of bacteria growing in culture or in a patient's wound for subsequent image-guided targeted swabbing / biopsy or treatment with, for example, photodynamic therapy (PDT) or hyperbaric oxygen therapy (HOT). These techniques, when used with, for example, consumable, commercially available fluorescent contrast agents, can provide improved signal-to-background for sensitive detection of bacteria in and around a target area.

[0040] The at least one fluorescent molecule may include a derivatized endogenous fluorescent molecule, such as, for example, an aminolevulinic acid (ALA) derivatized porphyrin. ALA may be administered locally to the target area, and imaging may be performed 1-3 hours later for enhanced red fluorescence of bacteria in the target area. The prodrug aminolevulinic acid (ALA) causes the formation of porphyrins in almost all living cells. Many bacterial species exposed to ALA can cause the fluorescence of protoporphyrin IX (PpIX). Use of ultra-low doses of ALA can cause the formation of PpIX in bacteria, thus increasing the red fluorescence emission and, as a result, enhancing the red-green fluorescence contrast of bacteria imaged by the device. ALA does not fluoresce by itself, but PpIX fluoresces at approximately 630 nm, 680, and 710 nm, with the emission strongest at 630 nm. The imaging device may then be used to image the green and red fluorescence from the target area and surrounding tissue.

[0041] ALA, usually available commercially in lyophilized form, can be premixed by the clinical technician at a given dose with saline or other types of commercially available creams / ointments / hydrogels / dressings, etc., and administered topically by spraying, pouring, or carefully applying the agent to the target area prior to imaging. After approximately 10-30 minutes (although this time can vary), fluorescence imaging can be performed in a dimly lit or darkened room. Bacteria that are probably poorly autofluorescent under white light can appear as bright red fluorescent areas in and around the target area. The fluorescence image can be used to guide targeted swabbing, biopsy, and / or fine needle aspiration of the target area for bacterial culture based on the inherent bacterial fluorescence signal. This procedure can be performed at various depths for superficial and deep wounds.

[0042] Suitable probes directed to exogenous optical molecules can be generated using commercially available fluorescent labeling kits such as Alexa Fluor active esters and kits (e.g., Zenon antibody labeling kit and / or EnzChek protease assay kit, Invitrogen) for labeling proteins, monoclonal antibodies, nucleic acids, and oligonucleotides (Invitrogen). For example, these fluorescent dye bioconjugates cover wavelength ranges as follows: Alexa Fluor 350, Alexa Fluor 405, Alexa Fluor 430, Alexa Fluor 488, Alexa Fluor 500, Alexa Fluor 514, Alexa Fluor 532, Alexa Fluor 546, Alexa Fluor 555, Alexa Fluor 568, Alexa Fluor 594, Alexa Fluor 610, Alexa Fluor 633, Alexa Fluor 635, Alexa Fluor 647, Alexa Fluor 660, Alexa Fluor 680, Alexa Fluor 700, and Alexa Fluor 750 dyes (where the numbers refer to the excitation wavelengths of the dyes). These kits can provide well-differentiated fluorescence emission spectra and provide multiple options for multicolor fluorescence detection and fluorescence resonance energy transfer based on appropriate selection of fluorescence emission filters in imaging devices. Fluorescent dyes can provide high absorbance at the wavelengths of maximum output of common excitation sources. They can be bright and remarkably photostable to aid in achieving fluorescence of their bioconjugation reactions. The dyes can provide good water solubility for ease of conjugation in the clinical laboratory and resistance to precipitation and aggregation of the resulting conjugation reactions. The fluorescence spectrum of the dyes can be pH insensitive over a wide range, making them particularly advantageous for imaging wounds, where the pH of the wounds can vary. Additionally, there are other commercially available or non-commercially available fluorescent agents that may be suitable for biological imaging of wounds and that can be combined with the above-mentioned devices, such as fluorescent blood pool agents, as well as various wound-enzyme or protease-activated probes from VisEn Medical (Boston, Massachusetts, USA).

[0043] These targeted fluorescent bioconjugate reactants can be made using such labeling kits prior to clinical examination of the target area using the imaging device in a fluorescence mode. They can be stored in a light-proof container to avoid photobleaching. Such fluorescent bioconjugate reactants can be made in a solution of known appropriate concentration prior to fluorescent imaging of the target area using the device, and then administered / applied directly to the target area by one or more means, such as topically (e.g., by aerosol or spray), orally (e.g., by drinking or washing), or systemically (e.g., by intravenous injection).Such dyes can target specific biological components depending on the area of ​​interest, such as bacteria, fungi, yeast, spores, viruses, microorganisms, parasites, exudates, pH, blood vessels, reduced nicotinamide adenine dinucleotide (NADH), flavin adenine dinucleotide (FAD), microorganisms, certain types of connective tissue (e.g., collagen, elastin), tissue components, vascular endothelial growth factor (VEGF), endothelial growth factor (EGF), epidermal growth factor, epithelial cell membrane antigen (ECMA), hypoxia inducible factor (HIF-1), carbonic anhydrase IX (CAIX), laminin, fibrin, fibronectin, fibroblast growth factor, transforming growth factor (TGF), fibroblast activation protein (FAP), enzymes (e.g., caspases, matrix metalloproteinases (MMPs), etc.), tissue inhibitors of metalloproteinases. (e.g., TIMPs), nitrogen monoxide synthase (NOS), inducible and endothelial NOS, intracellular lysosomes, macrophages, neutrophils, lymphocytes, hepatocyte growth factor (HGF), anti-neuropeptides, neutral endopeptidase (NEP), granulocyte-macrophage colony-stimulating factor (GM-CSF), neutrophil elastase, cathepsins, arginase, fibroblasts, endothelial cells, keratinocytes, keratinocyte growth factor (KGF), macrophage inflammatory protein-2 (MIP-2), macrophage chemoattractant protein-1 (MCP-1), polymorphonuclear neutrophils (PMN) and macrophages, myofibroblasts, interleukin-1 (IL-1) and tumor necrosis factor (TNF), nitrogen monoxide (NO) (Calbiochem kit, model DAF-2 Exogenous optical agents can include one or more of: DA, c-myc, beta-catenin, and circulating endothelial progenitor cells (EPCs) from bone marrow. Exogenous optical agents can include, for example, one or more of activating molecular beacons (e.g., targeted), nanoparticles having fluorescent agents (e.g., labeled at the surface and / or containing or carrying fluorescent agents), and scattering or absorbing nanoparticles (e.g., gold, silver, etc.).

[0044] Commercially available organic fluorophores have properties that depend on hydrogen ion concentration and are therefore useful as probes for measuring pH, and typically have pH-sensitive UV / visible absorption properties. Commercially available pH-sensitive fluorescent dyes used for intracellular studies may not yield much fluorescent signal in acidic media, or the pKa of the dye may be outside the intracellular pH window between 5-8 pH units. However, other pH-sensitive fluorescent agents respond by increasing their fluorescence intensity. For example, Invitrogen Molecular Probes (Thermo Fisher Scientific) offers a variety of fluorescent pH indicators, their conjugates, and other reagents for pH measurement in biological systems. Among these, there are several probes with unique optical responses and special localization properties, for example, visible light excitable SNARF pH indicators allow researchers to determine intracellular pH in the physiological range using dual emission or dual excitation ratiometric techniques, thus providing a useful tool for confocal laser scanning microscopy and flow cytometry. LysoSensor probes, as well as indicators based on the Oregon Green fluorophore, have been used to measure the acidic organelles of cells. The pH of the neura can be estimated. There are also fluorescent pH indicators bound to dextran that can be used. The indicator dextran can be well retained after introduction into the cells. Such fluorescent agents can be pre-made in solution at known and appropriate concentrations prior to fluorescent imaging of the target area with the device, and then administered / applied directly to the target area and surrounding normal tissue by one or more means, such as topically (e.g., by aerosol or spray), orally (e.g., by drinking or washing), or systemically (e.g., by intravenous injection).

[0045] The target area may include at least one wound. The wound may be any type of wound. For example, the wound may include an abrasion, laceration, puncture, dehiscence, bruise, contusion, bite, burn, rash, frostbite, boil, mole, pimple, cyst, ulcer (such as diabetic ulcer), bed sore, or any combination thereof. The wound may be deep or superficial. The wound may be external on or within the skin or internal on or within the intima. The wound may include the epidermis, dermis, or subcutaneous tissue, or any combination thereof. The wound may be infected or non-infected. An infected wound may be infected with any type, number, or combination of organisms. The infection may include one or more bacteria, one or more fungi, one or more protozoa, or one or more viruses, or any combination thereof. The organism may be unicellular, multicellular, or non-cellular.

[0046] The method can further include determining one or both of the surface area and volume of the wound. The determination can be performed any suitable number of times. For example, the method can be performed at least twice, the two runs including a first run and a second run separated by a selected period of time, such as at least 1 hour, at least 3 hours, at least 1 day, at least 1 week, at least 1 month, etc. The interval between runs can be, for example, 1 minute to about 10 minutes, about 5 minutes to about 30 minutes, about 30 minutes to about 3 hours, about 1 hour to about 6 hours, about 6 hours to about 12 hours, about 12 hours to about 1 day, about 1 day to about 2 days, about 2 days to about 1 week, about 1 week to about 2 weeks, about 2 weeks to about 1 month, about 1 month to about 6 weeks, about 6 weeks to about 3 months, about 3 months to about 6 months, or about 6 months to about 1 year, or any intervening or overlapping period, or combinations thereof. Image captures at very short (less than 1 second) intervals can be performed and combined to generate a video. Time-lapse videos spanning longer intervals of image capture can also be generated. The three-dimensional superimposed image of the first run can be a first three-dimensional superimposed image, and the three-dimensional superimposed image of the second run can be a second three-dimensional superimposed image. The method can further include comparing the first and second three-dimensional superimposed images to determine a wound healing state. The wound progression state can include a worsening wound, and the method can further include administering at least one wound improvement adjuvant. The comparison can further include tracking the wound topography over time. For example, three-dimensional fluorescent imaging can be used to assist in debridement of the wound to remove bacterial load. Imaging can further be used to track the results of debridement over time in three dimensions at the topographical level. Using this approach, a clinician can debride a wound in one visit and track the effect of surgical or curettage debridement in three and two dimensions over time.

[0047] Wound healing can be assessed by planimetric measurements of wound area at multiple time points (e.g., at clinic visit) until wound healing. The time course of wound healing can be compared to the expected healing time calculated by measuring the reduction in the radius of the wound at multiple time points using the formula R=√A / π (R is radius; A is planar wound area; π is a constant 3.14). This quantitative information about the wound can be used to track and monitor changes in the appearance of the wound over time to assess and determine the degree of wound healing resulting from natural means or any therapeutic intervention. This data can be stored electronically in the patient's health record for future reference. Wound imaging can be used to measure tissue / bacteria fluorescence, measured wound area, thermal map of the wound, and other parameters. , and infrared imaging of blood flow may be captured, calculated, and / or combined.

[0048] By changing excitation and emission wavelengths accordingly, the imaging device can interrogate tissue components (e.g., connective tissue and bacteria in a wound) located at the surface and at specific depths within the tissue (e.g., wounds). For example, by changing from violet / blue (about 400 nm to about 500 nm) to green (about 500 nm to about 540 nm) wavelengths of light, excitation of deeper tissue / bacterial fluorescent sources can be achieved. This can be achieved, for example, in wounds. Similarly, by detecting longer wavelengths, fluorescent emissions from tissue and / or bacterial sources deeper within the tissue can be detected at the tissue surface. With respect to wound evaluation, the ability to interrogate surface and / or subsurface fluorescence can be useful, for example, in the detection and possible identification of bacterial contamination, colonization, significant colonization, and / or infection that may occur at the surface and often occurs deeper within the wound (e.g., in chronic non-healing wounds). The at least one fluorescent molecule detected by imaging can include at least one bacterial associated fluorescent molecule.

[0049] The biological target can include tissues excised from a subject organism. For example, the tissues can include pre-cancerous or cancerous tissues. The cancerous tissues can include tumors. For example, the tumors can be breast tumors and the excised tissues can include lumpectomies. The breast cancer can include any type or combination of types of breast cancer. For example, the breast cancer can be luminal A breast cancer that expresses cytokeratins 8 and 18 and high levels of estrogen receptor expression. The breast cancer can be luminal B breast cancer. The breast cancer can resemble a normal breast in terms of gene expression. The breast cancer can be HER 2 amplified (amplification of the HER 2 gene on chromosome 17q). The type of breast cancer can be basal type, which can be negative for certain receptors (estrogen, progesterone, and HER 2) and have markers characteristic of basal / myoepithelial cells. The breast cancer can be characterized by one or more mutations in the BRCA 1 gene, the BRCA 2 gene, or both.

[0050] Cancerous tissue may be a pre-malignant growth, a malignant growth, or a tumor caused by abnormal, uncontrolled cell division, which may be metastatic or non-metastatic. The cancer can be, for example, breast cancer, prostate cancer, lung cancer, colon cancer, rectal cancer, bladder cancer, non-Hodgkin's lymphoma, melanoma, kidney cancer, pancreatic cancer, oral cancer, pharyngeal cancer, ovarian cancer, thyroid cancer, gastric cancer, brain cancer, multiple myeloma, esophageal cancer, liver cancer, cervical cancer, laryngeal cancer, intrahepatic bile duct cancer, acute myeloid leukemia, soft tissue cancer, small intestine cancer, testicular cancer, chronic lymphocytic leukemia, Hodgkin's lymphoma, chronic myeloid carcinoma, acute lymphocytic carcinoma, anal, anal canal, or anorectal cancer, vulva cancer or neck, gallbladder, or pleura cancer, malignant mesothelioma, osteosarcoma, joint cancer, hypopharyngeal cancer, eye cancer, nose, nasal cavity, neck, or middle ear cancer, nasopharyngeal cancer, ureteral cancer, peritoneum, omentum, or mesentery cancer, or gastrointestinal carcinoid tumor, or any combination thereof. The resected tissue may include fluorescent molecules associated with a probe targeting a tumor receptor, an enzyme-activated fluorescent molecule, or a genetically modified oncolytic virus-induced fluorescence, or any combination thereof. For example, the tumor receptor may include HER2, folate receptor, CXCR4, hormone receptor, EGFR, or VEGF, or a combination thereof. Examples of hormone receptors include estrogen receptor and progesterone receptor. The enzyme may include, for example, proteases, carbohydrases, lipases, transferases, oxidoreductases, matrix metalloproteinases (MMPs), caspases, cathepsins, kallikreins, serine proteases, isocitrate dehydrogenase, or enzymes overexpressed by tumor cells, or a combination thereof. The biological target may include a surgical bed from which tissue was resected. The surgical bed and the resected tissue may include cancer tissue. The cancer tissue may be benign, malignant, or both. The tissue to be removed may include, for example, a cytoplasmic endothelial cell. It is not necessarily cancerous and the techniques of the present disclosure can be used in other situations, such as plastic surgery, reconstructive surgery, organ transplant surgery, skin grafts, and cosmetic surgery.

[0051] The method may be performed any suitable number of times. For example, the method may be performed at least two times in any order, the two runs including a first run and a second run. The first run may be performed on a biological target. For example, the biological target may be a first biological target including resected tissue. The second run may be performed on a second biological target including a surgical bed from which tissue has been resected. The three-dimensional overlap image of the first run may be a first three-dimensional overlap image, and the three-dimensional overlap image of the second run may be a second three-dimensional overlap image. The method may further include comparing the first and second three-dimensional overlap images and determining a fluorescence continuity between the resected tissue and the surgical bed based on an orientation of the resected tissue relative to the surgical bed. The fluorescence continuity may include, for example, one or more of bacterially infected tissue, virally infected tissue, burn wound, cancerous tissue, connective tissue, muscle tissue, blood vesicles, and skin features. The fluorescence continuity may correspond to damaged tissue, and the method may further include resecting at least a portion of the damaged tissue from the surgical bed.

[0052] The method may include capturing a companion three-dimensional image of the target area and one or more fiducial markers using imaging techniques including, for example, one or more of computed tomography (CT), magnetic resonance imaging (MRI), photoacoustic imaging, ultrasound, and optical coherence tomography. The three-dimensional overlapping image may be overlapped with the companion three-dimensional image to form a second three-dimensional overlapping image. An example of such a step, shown as 570 and 580, is shown in FIG. 41. The companion three-dimensional image capture may include a first and a second set of fiducial markers. The companion three-dimensional image may be captured using computed tomography, and the one or more fiducial markers may include at least one fluorescent molecule and at least one CT contrast agent. The companion three-dimensional image may be captured using photoacoustic imaging, and the target area may include a breast tumor, and the at least one fluorescent molecule may include an anti-HER2 dual fluorescent-photoacoustic probe.

[0053] Any suitable number or type of fiducial markers can be used. Any suitable number of sets of fiducial markers can be used. The number of fiducial markers used can include, for example, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 15, 20, or 25 or more fiducial markers. The number of sets of fiducial markers used can be 1, 2, 3, 4, or 5 or more sets. The sets can differ in one or more of the following: the location of the fiducial markers, the size, the fluorescence spectrum, the fluorescence intensity, and the specificity with respect to the imaging technique. Different types of fiducial markers can be co-localized.

[0054] The present disclosure allows for digital integration of pathology data from histology samples, such as samples using hematoxylin and eosin staining, in a three-dimensional data cube of multiple imaging modalities. The data cube allows for spatially localized digital archiving and documentation of location-specific histopathology or other data from tissue surfaces of either ex vivo surgical samples or the surgical bed of a given patient over time. Sites of biopsies can be digitally cataloged over time with corresponding pathology results or other relevant information such as suture sites, anastomosis sites, implant sites, etc. Cataloging allows clinical teams to track where tissues were sampled and the pathology of those sites, which can be compared before and after treatment. In case of tissue deformation in the ex vivo surgical sample, an image deformation correction model can be applied to compensate for the deformation and align to the topographical surface of the surgical bed.

[0055] The present disclosure provides a method for spatially-temporally registering multimodal imaging 2D and 3D data sets of biological tissues. Additionally, fluorescence and photoacoustic imaging techniques are provided that work in conjunction with exogenously applied tumor contrast agents to enhance the accuracy of tumor cell detection at surgical margins. The present disclosure provides a clinically useful approach for hybrid surface and volumetric imaging techniques to improve surgical planning, intraoperative guidance, and margin assessment. Alternatively, occult tumor margins can be identified. Cancer diagnosis can be improved, for example, by combining optical imaging data sets from one or more of photoacoustic, fluorescence, optical coherence tomography (OCT), and Raman imaging with traditional magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET), and / or ultrasound (US) scans for tumor diagnosis and response assessment. Ultrasound and other imaging techniques can be 2D or 3D. Doppler imaging (e.g., of blood flow) and / or thermal imaging can also be combined with such other imaging techniques. The disclosed methods, devices, and systems also enable endoscopic imaging and general research.

[0056] According to the present disclosure, an imaging device is provided. The imaging device may include one or more of the following components: An excitation light source may be configured to emit a first radiation capable of exciting a fluorophore. A filter may be configured to prevent the passage of reflected excitation light and allow the passage of fluorescence emitted by the fluorophore. An imaging lens may be configured to collect the radiation. A visible light source may be configured to emit a second radiation. An infrared light source may be configured to emit a third radiation. At least one image sensor may be configured to detect the radiation. A processor may be configured to receive the detected radiation and output data related to the detected radiation. The imaging device may be configured to perform one or more of the methods described herein, or portions thereof. The imaging device may be configured to be handheld. For example, the imaging device may be configured to be held in one hand.

[0057] The imaging device can be configured to visualize any suitable target area of ​​a biological target, other types of targets, or combinations thereof. The imaging device can be configured to visualize a wound. The imaging device can be configured to visualize one or more of precancerous cells, cancer cells, and peripheral lesions within the surgical margin. The excitation light source can be further configured to excite autofluorescence emission of tissue cells of the surgical margin and fluorescence emission of induced porphyrins in the tissue cells. The filter can be further configured to prevent the passage of reflected excitation light and allow the passage of emission having a wavelength corresponding to the autofluorescence emission of the tissue cells and the fluorescence emission of the induced porphyrins in the tissue cells. The image sensor can be further configured to detect the filtered autofluorescence emission of the tissue cells and the fluorescence emission of the induced porphyrins in the tissue cells of the surgical margin. The processor can be further configured to receive the detected emission and output data related to the detected filtered autofluorescence emission of the tissue cells and the fluorescence emission of the induced porphyrins in the tissue cells of the surgical margin.

[0058] The detected radiation can include one or more of fluorescent light, reflected visible light, and reflected infrared light. The detected radiation can include fluorescent light, reflected visible light, and reflected infrared light. The first radiation can include fluorescent light. The second radiation can include, for example, white light. The second radiation can include monochromatic visible light. The third radiation can include infrared radiation, for example, near infrared radiation, for example, near infrared radiation. The at least one image sensor can be configured to detect radiation including one or more of fluorescent light, reflected white light, and reflected infrared light. The at least one sensor can include any number and / or types of sensors, for example, at least two sensors. The at least two sensors can include a first sensor configured to detect fluorescent light and a second sensor configured to detect reflected visible light. and a second detector configured to detect light. The at least one sensor can include at least three sensors including a first sensor configured to detect fluorescent light, a second detector configured to detect reflected visible light, and a third sensor configured to detect reflected infrared light.

[0059] At least one sensor may include a thermal sensor. The device may combine the thermal sensor with one or more other sensors described herein for three-dimensional imaging. The thermal sensor may be used to provide a thermal map to indicate areas of high temperature that may correlate with the location of bacteria when the wound is examined. The imaging may be combined with three-dimensional ultrasound imaging of the wound, including, for example, Doppler imaging of blood flow. The thermal mapping may be performed separately from the three-dimensional mapping or may be combined with the three-dimensional mapping. For example, thermal imaging may be performed simultaneously with the three-dimensional imaging by coupling a thermal imaging sensor to a three-dimensional imaging device. The captured thermal image or video may then be topographically overlaid on the visible light / fluorescence image using one or more reference points. The camera used for three-dimensional imaging may, for example, capture tissue / bacteria fluorescence, measure the area of ​​the wound, capture a thermal map of the wound, and / or image infrared of the blood flow.

[0060] The imaging device may further include a common radiation source configured to cooperate with or act as at least one of the light sources. At least one light source may include a converter that converts source radiation emitted from the common radiation source into a first radiation, a second radiation, or a third radiation, or a combination thereof. The converter may include, for example, a filter, a lens, a prism, a diffractor, or a quantum dot, or a combination thereof. The excitation light source may include a first converter, the visible light source may include a second converter, and the infrared light source may include a third converter.

[0061] The imaging device may further include a display unit. Any display unit may be used, such as, for example, a liquid crystal display (LCD), a light emitting display (LED), an organic light emitting display (OLED), a plasma, or a cathode ray, or any combination thereof. The display unit may be configured to display data output by the processor. For example, the data may include a three-dimensional image. The display unit may include a touch screen and / or any other type of graphical user interface. Alternatively or additionally, the display unit may be located in a device other than the imaging device.

[0062] The imaging device can be configured to visualize a target area of ​​a biological target. The processor can be configured to generate a three-dimensional map of the target area. The three-dimensional map can be generated from infrared light reflected from the target area. The processor can be configured to capture a two-dimensional visible light image of the target area based on the detected radiation. The processor can be configured to generate a three-dimensional visible light image of the target area based on the three-dimensional map and the two-dimensional visible light image. The processor can be configured to capture a two-dimensional fluorescent image of the target area based on the detected radiation. The processor can be configured to generate a three-dimensional fluorescent image of the target area based on the three-dimensional map and the two-dimensional fluorescent image. The processor can be configured to align the three-dimensional visible light image of the target area to the three-dimensional fluorescent image of the target area to form a three-dimensional superimposed image of the target area. The alignment can be performed based on alignment of fiducial markers associated with the target area.

[0063] In accordance with one or more embodiments, the imaging device may be configured as follows: The imaging device may be configured to be handheld; The excitation light source may be configured to excite autofluorescence emission of tissue cells and fluorescence emission having a wavelength between about 600 nm and about 660 nm in precancerous cells, cancer cells, and surrounding lesions of the surgical margin after exposure to an imaging or contrast agent; The filter may be configured to allow passage of emission having a wavelength corresponding to the autofluorescence emission of tissue cells and the fluorescence emission between about 600 nm and about 660 nm in the tissue cells of the surgical margin; The at least one image sensor may be configured to detect the filtered autofluorescence emission of tissue cells and the fluorescence emission between about 600 nm and about 660 nm in the tissue cells of the surgical margin; The processor may be configured to receive the detected emission and output data related to the detected filtered autofluorescence emission of tissue cells and the fluorescence emission between about 600 nm and about 660 nm in the tissue cells of the surgical margin to generate a three-dimensional image of one or more of the precancerous cells, cancer cells, and surrounding lesions of the surgical margin.

[0064] According to one aspect of the disclosure, a system for three-dimensional fluorescence-based imaging of a target is provided. The system may include one or more of the following components, one or more of the components described for the imaging device, or both: At least one excitation light source may be configured to illuminate the target surface with an excitation light field during fluorescence imaging. The illumination and / or field may be uniform, partially uniform, or non-uniform. At least one white light source may be configured to illuminate the target surface during white light imaging. At least one infrared radiation source may be configured to emit infrared radiation toward the target surface. An image sensor may detect fluorescence, reflected radiation, or both. A filter may be configured to allow an optical signal responsive to illumination of the target surface with the excitation light and having a wavelength corresponding to one or more of bacterial autofluorescence and tissue autofluorescence to pass through the filter and reach the image sensor. A processor may be configured to perform, for example, one or more of the following: The processor may receive an optical signal responsive to illumination of the target with infrared light. Based on the received signal responsive to illumination of the target with infrared light, the processor may generate a three-dimensional map of the target surface. The processor can receive the detected optical signals responsive to illumination of the target surface with the excitation light and generate a two-dimensional fluorescent image of the target surface. The processor can receive the optical signals responsive to white light illumination of the target surface and generate a two-dimensional white light image of the target surface, and the processor can combine the three-dimensional map, the fluorescent image, and the white light image to generate a three-dimensional image of the target surface.

[0065] According to one aspect of the present disclosure, a computer program product is provided for use with an imaging device. The computer program product may include a non-transitory computer readable medium. The non-transitory computer readable medium may store computer program code for image processing. The computer program code may be executable by a processor in the imaging device to perform one or more of the methods described herein. EXAMPLES

[0066] Example 1: A tissue phantom was used to demonstrate the imaging technique of the present disclosure. To image the entire surface of a breast lumpectomy sample, a scan was performed under white light using the PrimeSense camera, and then a second scan was performed after converting the CCD sensor on the PrimeSense device to a fluorescence camera. The two imaging scans can be aligned by using fiducial markers placed externally on the sample surface. Fluorescence was excited using a 405 nm LED array that shines light on the tissue surface. An optical filter was attached to the PrimeSense system to image the entire surface of the resected tissue (tumor) specimen, specifically a breast lumpectomy. Real-time fluorescence images were generated as 3D surface renderings of the tissue. PrimeSense detected specific fluorescence signatures from both autofluorescence of the resected tissue and fluorescence induced by an exogenous contrast agent (ALA-derived porphyrin), although fluorescence imaging can be modified for any number or type of contrast agent.

[0067] FIG. 1 is a white light photograph of an experimental model 100 including a tissue phantom 110 marked with fluorescein fiducial markers 130 on the surface 112 of the tissue phantom. The fluorescein fiducial markers may appear yellow / orange under white light and green / yellow under UV illumination. A simulated tumor 120 is created by fluorescent PPIX injected just below the surface. Fluorescent PPIX typically appears red under UV illumination. The tissue phantom 110 is a porcine tissue section used to simulate a breast lumpectomy tissue sample.

[0068] FIG. 2A is a 3D white light surface image of tissue phantom 110 from a first angle of view. FIG. 2B is a 3D white light surface image of tissue phantom 110 from a second angle of view. FIG. 3A is a 3D white light surface image of tissue phantom 110+mesh from a first angle of view. FIG. 3B is a 3D white light surface image of tissue phantom 110+mesh from a second angle of view. FIG. 4A is a 3D mesh+white light overlay image of tissue phantom 110 from a first angle of view. FIG. 4B is a 3D mesh+white light overlay image of tissue phantom 110 from a second angle of view. FIG. 5A is a raw mesh / surface image of tissue phantom 110 from a first angle of view. FIG. 5B is a raw mesh / surface image of tissue phantom 110 from a second angle of view.

[0069] The red fluorescence in the following figures generally indicates a simulated "tumor" using added PPIX. The simulated tumor is a schematic representation of what a breast lumpectomy looks like with positive (red fluorescence) margins. FIG. 6A is a 3D fluorescence surface image of tissue phantom 110 from a first view angle. FIG. 6B is a 3D fluorescence surface image of tissue phantom 110 from a second view angle. FIG. 7A is a 3D fluorescence surface image of tissue phantom 110+mesh from a first view angle. FIG. 7B is a 3D fluorescence surface image of tissue phantom 110+mesh from a second view angle. FIG. 8A is a 3D mesh+fluorescence overlay image of tissue phantom 110 from a first view angle. FIG. 8B is a 3D mesh+fluorescence overlay image of tissue phantom 110 from a second view angle. FIG. 9A is a raw mesh / surface image of tissue phantom 110 from a first view angle. FIG. 9B is a raw mesh / surface image of tissue phantom 110 from a second angle of view.

[0070] Multimodal (optical, CT) fiducial markers using mixed fluorescent and CT dyes for aligning 3D surface-rendered white light reflectance and fluorescence images with 3D surface-rendered computed tomography of the phantom were used. Excitation light was 405 nm. Emission filters were dual band 500-550 nm and 600-650 nm emission filters. This filter was placed in a long plastic filter holder that could be moved in and out of position in front of the image sensor of the RGB camera to change the RGB camera from a white light imaging camera to a fluorescence imaging camera. The camera was connected to a laptop running PrimeSense data acquisition software. An external light source was used to illuminate the target area of ​​the biological target (a pig leg with a skin wound) with 405 nm light from two LEDs. Cone beam CT scan image processing utilized InVesalius 3 software.

[0071] FIG. 10A is a first view of the tissue phantom 110 with CT fiducials 140 overlaid on the fluorescent fiducials 130. FIG. 10B is a second view of the tissue phantom with CT fiducials overlaid on the fluorescent fiducials. The red fluorescence in the following figures generally indicates a "tumor" that has been simulated using added PPIX. The simulated tumor is a tumor that has been simulated after a breast lumpectomy with positive (red fluorescent) margins. 11A is a cone beam CT 3D volume reconstruction of the tissue phantom 110 from a first view angle. FIG. 11B is a fluorescence 3D rendering of the tissue phantom 110 from the first view angle. FIG. 11C is a registered CT+fluorescence image of the tissue phantom 110 from the first view angle. FIG. 12A is a cone beam CT 3D volume reconstruction of the tissue phantom 110 from a second view angle. FIG. 12B is a fluorescence 3D rendering of the tissue phantom 110 from the second view angle. FIG. 12C is a registered CT+fluorescence image of the tissue phantom 110 from the second view angle. FIG. 13A is a cone beam CT 3D volume reconstruction of the tissue phantom 110 from a third view angle. FIG. 13B is a fluorescence 3D rendering of the tissue phantom 110 from the first view angle. Figure 13C is a registered CT+fluorescence image of the tissue phantom 110 from a third view angle. Figure 14A is a cone beam CT 3D volumetric reconstruction of the tissue phantom 110 from a fourth view angle. Figure 14B is a fluorescence 3D rendering of the tissue phantom 110 from the fourth view angle. Figure 14C is a registered CT+fluorescence image of the tissue phantom 110 from the fourth view angle.

[0072] These figures show that the techniques of the present disclosure can generate three-dimensional images of tissues and other objects that serve as valuable tools for better characterization and understanding of tissues and other objects. For example, surgeons can better visualize the extent of cancerous tissue in a patient and plan resections to efficiently remove the cancerous tissue while sparing adjacent healthy tissue. Thus, Example 1 demonstrates the effectiveness of the techniques of the present disclosure. EXAMPLES

[0073] Example 2 In this example, essentially the same procedure as in Example 1 was carried out using a second experimental model 200 including a second tissue phantom modeling a simulated tumor. FIG. 15A is a first image of a second tissue phantom 210 for a simulated tumor 220 being created from a pig's shoulder with a skin (surface) 212 similar to human skin. FIG. 15B is a second image of the second tissue phantom for a tumor model being created. FIG. 16 is an image of the second tissue phantom 210 showing a simulated (model PPIX) tumor 220, and the fluorescein fiducial marker 230 can also be seen. A ruler 250 can be used to clarify the scale. FIG. 17A is an image of a perspective view of the second tissue phantom 210 being exposed to violet light by a handheld imaging device to excite fluorescence. FIG. 17B is an image of a top view of an image of the fluorescence emission from the second tissue phantom 210 exposed to violet light to excite fluorescence. FIG. 18A is a 3D white light surface image of the second tissue phantom 210 from a first angle of view. FIG. 18B is a 3D white light surface image of the second tissue phantom 210 from a second angle of view. FIG. 19A is a 3D white light surface image of the second tissue phantom 210+mesh from a first angle of view. FIG. 19B is a 3D white light surface image of the second tissue phantom 210+mesh from a second angle of view. FIG. 20A is a 3D mesh+white light overlay image of the second tissue phantom 210 from a first angle of view. FIG. 20B is a 3D mesh+white light overlay image of the second tissue phantom 210 from a second angle of view. FIG. 21A is a raw mesh / surface image of the second tissue phantom 210 from a first angle of view. FIG. 21B is a raw mesh / surface image of a second tissue phantom 210 from a second angle of view.

[0074] The red fluorescence in the figure generally indicates "bacteria" surrounding the "wound" that was simulated using added PPIX. The simulated wound is a schematic representation of what a bacterially infected concave wound looks like with a positive (red fluorescence) margin. FIG. 22A is a 3D fluorescence surface image of the second tissue phantom 210 from a first angle of view. FIG. 22B is a 3D fluorescence surface image of the second tissue phantom 210 from a second angle of view. FIG. 23A is a 3D fluorescence surface image of the second tissue phantom 210 from a first angle of view. FIG. 23B is a 3D fluorescent surface image of the second tissue phantom 210+mesh from a second view angle. FIG. 24A is a 3D mesh+fluorescence overlay image of the second tissue phantom 210 from a first view angle. FIG. 24B is a 3D mesh+fluorescence overlay image of the second tissue phantom 210 from a second view angle. FIG. 25A is a raw mesh / surface image of the second tissue phantom 210 from a first view angle. FIG. 25B is a raw mesh / surface image of the second tissue phantom 210 from a second view angle.

[0075] These figures show that the techniques of the present disclosure can generate three-dimensional images of tissues and other objects that serve as valuable tools for better characterization and understanding of tissues and other objects. For example, surgeons can better visualize the extent of cancerous tissue in a patient and plan resections to efficiently remove the cancerous tissue while sparing adjacent healthy tissue. Thus, Example 2 demonstrates the effectiveness of the techniques of the present disclosure. EXAMPLES

[0076] Example 3 In this example, essentially the same procedure as in Example 1 was performed using a third tissue phantom modeling deep and superficial wounds. Cone beam CT and CT scan image processing utilized InVesalius 3 software. FIG. 26A is a first image of a third tissue phantom 310 for a wound model 300 in preparation for skin 312 to include both a deep wound 360 and a superficial wound 370. A ruler 350 can be incorporated as a scale. FIG. 26B is a second image of a third tissue phantom 310 in preparation. FIG. 26C is a third image of a third tissue phantom 310 in preparation showing a deep wound 360 including a wound margin 362 where PPIX was applied to simulate the presence of bacteria in the wound. FIG. 26D is a fourth image of a third tissue phantom 310 in preparation showing both a deep wound 360 and a superficial wound 370, as well as fluorescein fiducial markers 330. FIG. 27A is a 3D white light surface image of the third tissue phantom 310 from a first angle of view. FIG. 27B is a 3D white light surface image of the third tissue phantom 310 from a second angle of view. FIG. 28A is a 3D white light surface image of the third tissue phantom 310+mesh from a first angle of view. FIG. 28B is a 3D white light surface image of the third tissue phantom 310+mesh from a second angle of view. FIG. 29A is a 3D mesh+white light overlay image of the third tissue phantom 310 from a first angle of view. FIG. 29B is a 3D mesh+white light overlay image of the third tissue phantom 310 from a second angle of view. FIG. 30A is a raw mesh / surface image of the third tissue phantom 310 from a first angle of view. FIG. 30B is a raw mesh / surface image of the third tissue phantom 310 from a second angle of view. FIG. 31A is a 3D fluorescent surface image of the third tissue phantom 310 from a first angle of view. FIG. 31B is a 3D fluorescent surface image of the third tissue phantom 310 from a second angle of view. FIG. 32A is a 3D fluorescent surface image of the third tissue phantom 310+mesh from a first angle of view. FIG. 32B is a 3D fluorescent surface image of the third tissue phantom 310+mesh from a second angle of view. FIG. 33A is a 3D mesh+fluorescence overlay image of the third tissue phantom 310 from a first angle of view.Figure 33B is a 3D mesh+fluorescence overlay image of the third tissue phantom 310 from a second angle of view. Figure 34A is a raw mesh / surface image of the third tissue phantom 310 from a first angle of view. Figure 34B is a raw mesh / surface image of the third tissue phantom 310 from a second angle of view.

[0077] FIG. 35A is an image of a third tissue phantom 310 prepared with CT fiducial markers for cone beam CT scanning. FIG. 35B is a cone beam CT scan image of the third tissue phantom 310. FIG. 35C is a cone beam CT scan image in a CT device 380. FIG. 35D is an image of the third tissue phantom 310 prepared for CT scanning with the CT fiducials 340 spatially aligned to the fluorescein fiducials 330. FIG. 36A is an image of the third tissue phantom 310 prepared for CT scanning with the CT fiducials 340 spatially aligned to the fluorescein fiducials 330. FIG. 36B is an image of the third tissue phantom 310 during CT scanning. FIG. 37 is a photograph overlaid with a wound topology grid 490 of a wound 460 on skin 410 measured in accordance with the present disclosure. The measurements determined a length of about 4 cm, a width of about 9 cm, a depth of about 3 cm, and a volume of about 99.5 cm3.

[0078] These figures show that the techniques of the present disclosure can generate three-dimensional images of tissues and other objects that serve as valuable tools for better characterization and understanding of tissues and other objects. For example, a physician can better visualize the nature of a patient's wound and its progression over time to aid in determining whether a treatment plan is effective or needs modification to promote wound healing. Additionally, a physician can better visualize the location or size of an infection in a wound (or the location / amount of bacteria present) and adjust the treatment plan accordingly. Thus, Example 3 demonstrates the effectiveness of the techniques of the present disclosure.

[0079] The present disclosure includes the following aspects / embodiments / features in any order and / or in any combination.

[0080] 1. A method for generating a three-dimensional image of a target using two-dimensional images, the method comprising: generating a three-dimensional map of a target area having one or more fiducial markers associated therewith; capturing a two-dimensional white-light image of the target area and the one or more fiducial markers; generating a three-dimensional white-light image from the two-dimensional white-light image and the three-dimensional map; capturing a two-dimensional fluorescent image of the target area and the one or more fiducial markers; generating a three-dimensional fluorescent image from the two-dimensional fluorescent image and the three-dimensional map; and aligning the 3D white-light image and the three-dimensional fluorescent image using the one or more fiducial markers to form a three-dimensional overlaid image.

[0081] 2. The method of any preceding or subsequent embodiment / feature / aspect, wherein capturing a two-dimensional fluorescent image of the target area and the one or more fiducial markers includes illuminating the target area and the one or more fiducial markers with an excitation light and receiving at least one fluorescent emission in response to illumination of the target area by the excitation light.

[0082] 3. The method of any preceding or subsequent embodiment / feature / aspect, wherein the excitation light is between about 400 nm and about 450 nm.

[0083] 4. The method of any preceding or subsequent embodiment / feature / aspect, wherein the excitation light has a wavelength of about 405 nm.

[0084] 5. The method of any preceding or subsequent embodiment / feature / aspect, wherein capturing a two-dimensional fluorescent image of the target area and the one or more fiducial markers comprises capturing the emission of at least one fluorescent molecule.

[0085] 6. The method of any preceding or subsequent embodiment / feature / aspect, wherein said at least one fluorescent molecule comprises an endogenous molecule capable of fluorescing.

[0086] 7. The at least one fluorescent molecule is an exogenous molecule capable of emitting fluorescence; or The method of any preceding or subsequent embodiment / feature / aspect comprising a molecule comprising an exogenously added moiety capable of emitting fluorescence.

[0087] 8. The method of any preceding or subsequent embodiment / feature / aspect, wherein said at least one fluorescent molecule comprises an aminolevulinic acid (ALA)-derivatized porphyrin.

[0088] 9. The method of any preceding or subsequent embodiment / feature / aspect, wherein the three-dimensional map is generated using infrared light.

[0089] 10. The method of any preceding or subsequent embodiment / feature / aspect, wherein the three-dimensional map is generated using near-infrared light.

[0090] 11. The method of any preceding or subsequent embodiment / feature / aspect, wherein generating the three-dimensional map includes projecting infrared radiation onto the target area, receiving infrared radiation reflected by the target area, and measuring a depth of the target area based on the reflected infrared radiation to generate the three-dimensional map.

[0091] 12. The method of any preceding or subsequent embodiment / feature / aspect, wherein the infrared radiation is projected as a beam that is split into a light pattern, the reflected infrared radiation includes a distortion of the light pattern, and the depth is measured based on the distortion of the light pattern.

[0092] 13. The method of any preceding or subsequent embodiment / feature / aspect, wherein the light pattern is formed by a diffraction grating, and the light pattern includes a plurality of dots.

[0093] 14. The method of any preceding or subsequent embodiment / feature / aspect, wherein the depth is measured by time of flight based on a phase shift between the projected infrared radiation and the reflected infrared radiation.

[0094] 15. The method of any preceding or subsequent embodiment / feature / aspect, wherein said target area comprises at least one wound.

[0095] 16. The method of any preceding or subsequent embodiment / feature / aspect, wherein said wound comprises at least one bacterium comprising at least one fluorescent molecule.

[0096] 17. The method of any preceding or subsequent embodiment / feature / aspect, further comprising determining one or both of a surface area and a volume of said wound.

[0097] 18. The method of any preceding or subsequent embodiment / feature / aspect, wherein the method is performed at least twice, the two runs including a first run and a second run separated by a time period of at least three hours, the three-dimensional superimposed image of the first run being a first three-dimensional superimposed image and the three-dimensional superimposed image of the second run being a second three-dimensional superimposed image, and the method further comprises comparing the first and second three-dimensional superimposed images to determine a healing status of the wound.

[0098] 19. The method of any preceding or subsequent embodiment / feature / aspect, wherein said time period is at least one day.

[0099] 20. The method of any preceding or subsequent embodiment / feature / aspect, wherein the wound healing status comprises a worsening wound, and the method further comprises administering at least one wound improvement adjunct.

[0100] 21. The method of any preceding or subsequent embodiment / feature / aspect, wherein said comparing further comprises tracking a topography of said wound.

[0101] 22. The method of any preceding or subsequent embodiment / feature / aspect, wherein said target area comprises tissue excised from a subject organism.

[0102] 23. The method of any preceding or subsequent embodiment / feature / aspect, wherein said tissue comprises cancer tissue.

[0103] 24. The method of any preceding or subsequent embodiment / feature / aspect, wherein said cancerous tissue comprises a tumor.

[0104] 25. The method of any preceding or subsequent embodiment / feature / aspect, wherein the tumor is a breast tumor and the excised tissue comprises a lumpectomy.

[0105] 26. The method of any preceding or subsequent embodiment / feature / aspect, wherein the excised tissue comprises fluorescent molecules associated with probes targeting tumor receptors, enzymatically active fluorescent molecules, or fluorescence caused by genetically engineered oncolytic viruses, or any combination thereof.

[0106] 27. The method of any preceding or subsequent embodiment / feature / aspect, wherein the tumor receptor comprises HER2, a folate receptor, CXCR4, a hormone receptor, EGFR, or VEGF, or a combination thereof, and the enzyme comprises a protease, a carbohydrase, a lipase, a transferase, an oxidoreductase, a matrix metalloprotease (MMP), a caspase, a cathepsin, a kallikrein, a serine protease, an isocitrate dehydrogenase, or an enzyme overexpressed by a tumor cell, or a combination thereof.

[0107] 28. The method of any preceding or subsequent embodiment / feature / aspect, wherein the target area comprises a surgical bed from which tissue has been removed.

[0108] 29. The method of any preceding or subsequent embodiment / feature / aspect, wherein the surgical bed and the resected tissue comprise cancerous tissue.

[0109] 30. The method of any preceding or subsequent embodiment / feature / aspect, wherein the method is performed at least two times in any order, the two runs including a first run and a second run, the first run being performed on the target area, the target area being a first target area including the resected tissue, and the second run being performed on a second target area including a surgical bed from which the tissue was resected, the three-dimensional overlay image of the first run being a first three-dimensional overlay image and the three-dimensional overlay image of the second run being a second three-dimensional overlay image, the method further comprising comparing the first and second three-dimensional overlay images to determine fluorescence continuity between the resected tissue and the surgical bed based on an orientation of the resected tissue relative to the surgical bed.

[0110] 31. The method of any preceding or subsequent embodiment / feature / aspect, wherein the continuum of fluorescence includes one or more of the following characteristics: bacterially infected tissue, virally infected tissue, burns, cancerous tissue, connective tissue, muscle tissue, blood vesicles, and skin.

[0111] 32. Any preceding or subsequent steps, wherein the continuum of fluorescence corresponds to diseased tissue, and the method further comprises the step of resecting at least a portion of the diseased tissue from the surgical bed. Method of embodiment / feature / aspect.

[0112] 33. The method of any preceding or subsequent embodiment / feature / aspect, further comprising: capturing a companion three-dimensional image of the target area and the one or more fiducial markers using imaging techniques including one or more of computed tomography (CT), magnetic resonance imaging (MRI), photoacoustic imaging, ultrasound, and optical coherence tomography; and overlaying the companion three-dimensional image, which is a first three-dimensional overlay image, with the companion three-dimensional image to form a second three-dimensional overlay image.

[0113] 34. The method of any preceding or subsequent embodiment / feature / aspect, wherein the one or more fiducial markers include a first set of fiducial markers and a second set of fiducial markers.

[0114] 35. The method of any preceding or subsequent embodiment / feature / aspect, wherein the companion three-dimensional image is captured using computed tomography and the one or more fiducial markers include at least one fluorescent molecule and at least one CT contrast agent.

[0115] 36. The method of any preceding or subsequent embodiment / feature / aspect, wherein the companion three-dimensional image is captured using photoacoustic imaging and the target area comprises a breast tumor and an anti-HER2 dual fluorescent-photoacoustic probe.

[0116] 37. An imaging device comprising: an excitation light source configured to emit a first radiation capable of exciting a fluorophore; a filter configured to block the passage of reflected excitation light and allow the passage of fluorescence emitted by the fluorophore; an imaging lens; a visible light source configured to emit a second radiation; an infrared light source configured to emit a third radiation; at least one image sensor configured to detect radiation; and a processor configured to receive the detected radiation and output data related to the detected radiation.

[0117] 38. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the imaging device is configured to be handheld.

[0118] 39. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the imaging device is configured to visualize a wound.

[0119] 40. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the imaging device is configured to visualize one or more of pre-cancerous cells, cancerous cells, and peripheral lesions at the surgical margins.

[0120] 41. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the excitation light source is further configured to excite autofluorescence emission of tissue cells and fluorescence emission of induced porphyrins in tissue cells of the surgical margin.

[0121] 42. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the filter is further configured to block the passage of reflected excitation light and allow the passage of emission having a wavelength corresponding to the autofluorescence emission of the tissue cells and the fluorescence emission of the induced porphyrins in the tissue cells.

[0122] 43. Any preceding or subsequent embodiment / feature / aspect, wherein the image sensor is further configured to detect the filtered autofluorescence emission of the tissue cells and the fluorescence emission of the derivatized porphyrins in the tissue cells of the surgical margin. Imaging device of the like.

[0123] 44. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the processor is further configured to receive the detected luminescence and output data related to the detected filtered autofluorescence luminescence of the tissue cells and the fluorescence luminescence of the induced porphyrins in the tissue cells of the surgical margin.

[0124] 45. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the second radiation comprises white light.

[0125] 46. ​​The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the second radiation comprises monochromatic visible light.

[0126] 47. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the third radiation includes infrared radiation.

[0127] 48. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the infrared radiation includes near infrared radiation.

[0128] 49. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the at least one image sensor is configured to detect radiation including the fluorescent light, reflected white light, and reflected infrared light.

[0129] 50. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the at least one sensor includes at least two sensors.

[0130] 51. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the at least two sensors include a first sensor configured to detect fluorescent light and a second sensor configured to detect reflected visible light.

[0131] 52. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the at least one sensor includes at least three sensors including a first sensor configured to detect fluorescent light, a second sensor configured to detect reflected visible light, and a third sensor configured to detect reflected infrared light.

[0132] 53. The imaging device of any preceding or subsequent embodiment / feature / aspect, further comprising a common radiation source configured to cooperate with at least one of the light sources.

[0133] 54. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the at least one light source comprises a converter for converting source radiation emitted from the common radiation source into the first radiation, the second radiation, or the third radiation, or a combination thereof.

[0134] 55. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the transducer comprises a filter, a lens, a prism, a diffractor, or a quantum dot, or a combination thereof.

[0135] 56. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the excitation light source comprises a first transducer, the visible light source comprises a second transducer, and the infrared light source comprises a third transducer.

[0136] 57. The imaging device of any preceding or following embodiment / feature / aspect, further comprising a display unit.

[0137] 58. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the display unit is configured to display the data output by the processor.

[0138] 59. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the data includes a three-dimensional image.

[0139] 60. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the display unit comprises a touch screen.

[0140] 61. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the detected radiation includes one or more of: fluorescent light, reflected visible light, and reflected infrared light.

[0141] 62. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the detected radiation includes fluorescent light, reflected visible light, and reflected infrared light.

[0142] 63. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the imaging device is configured to visualize a target area of ​​a biological target.

[0143] 64. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the processor is further configured to generate a three-dimensional map of the target area.

[0144] 65. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the 3D map is generated from infrared light reflected from the target area.

[0145] 66. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the processor is further configured to capture a two-dimensional visible light image of the target area based on the detected radiation.

[0146] 67. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the processor is further configured to generate a three-dimensional visible light image of the target area based on the three-dimensional map and the two-dimensional visible light image.

[0147] 68. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the processor is further configured to capture a two-dimensional fluorescent image of the target area based on the detected radiation.

[0148] 69. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the processor is further configured to generate a three-dimensional fluorescence image of the target area based on the three-dimensional map and the two-dimensional fluorescence image.

[0149] 70. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the processor is further configured to align a three-dimensional visible light image of the target area to a three-dimensional fluorescent image of the target area to form a three-dimensional superimposed image of the target area.

[0150] 71. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the alignment is performed based on alignment of fiducial markers associated with the target region.

[0151] 72. The imaging device of any preceding or subsequent embodiment / feature / aspect, wherein the imaging device is configured to perform a method of any preceding or subsequent embodiment / feature / aspect.

[0152] 73. The imaging device is further configured to be handheld, and the excitation light source is further configured to excite autofluorescence emission of tissue cells and fluorescence emission having a wavelength between about 600 nm and about 660 nm in precancerous cells, cancer cells, and surrounding lesions of the surgical margin after exposure to an imaging or contrast agent, and the filter is further configured to allow passage of emission having a wavelength corresponding to the autofluorescence emission of the tissue cells and the fluorescence emission between about 600 nm and about 660 nm in the tissue cells of the surgical margin, and the at least one image sensor is further configured to capture the filtered fluorescence of the tissue cells. and detecting autofluorescence emission and fluorescence emission between about 600 nm and about 660 nm in the tissue cells of the surgical margin, wherein the processor is further configured to receive the detected emissions and output data related to the detected filtered autofluorescence emission of the tissue cells and fluorescence emission between about 600 nm and about 660 nm in the tissue cells of the surgical margin, and generate a three-dimensional image of one or more of the precancerous cells, the cancer cells, and the surrounding lesions of the surgical margin.

[0153] 74. A system for fluorescence-based three-dimensional imaging of a target, comprising: at least one excitation light source configured to uniformly illuminate a target surface with a uniform excitation light field during fluorescence imaging; at least one white light source configured to illuminate the target surface during white light imaging; at least one infrared radiation source configured to emit infrared radiation toward the target surface; an image sensor; and a filter configured to allow an optical signal responsive to illumination of the target surface by the excitation light, the optical signal having a wavelength corresponding to one or more of bacterial autofluorescence and tissue autofluorescence, to pass to the image sensor. and a processor configured to: receive optical signals responsive to illumination of the target with the infrared light, generate a three-dimensional map of the target surface based on the received signals responsive to illumination of the target with the infrared light, receive detected optical signals responsive to illumination of the target surface with the excitation light and generate a two-dimensional fluorescence image of the target surface, receive optical signals responsive to white light illumination of the target surface and generate a two-dimensional white light image of the target surface, and combine the three-dimensional map, the fluorescence image, and the white light image to generate a three-dimensional image of the target surface.

[0154] 75. A computer program product for use with an imaging device of any preceding or subsequent embodiment / feature / aspect, comprising a non-transitory computer readable medium, the non-transitory computer readable medium storing computer program code for image processing, the computer program code being executable by the processor of the imaging device to perform the method of any preceding or subsequent embodiment / feature / aspect.

[0155] Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. It is intended that the specification and examples be considered as exemplary only. Various aspects of the disclosure are set forth in the following claims.

Claims

1. 1. A method for generating a three-dimensional image of a target using a two-dimensional image, comprising: generating a three-dimensional map of the target area having one or more fiducial markers associated therewith; capturing a two-dimensional white light image of the target area and the one or more fiducial markers; generating a three-dimensional white-light image from the two-dimensional white-light image and the three-dimensional map; capturing a two-dimensional fluorescent image of the target area and the one or more fiducial markers; generating a three-dimensional fluorescence image from the two-dimensional fluorescence image and the three-dimensional map; aligning the three-dimensional white light image and the three-dimensional fluorescent image using the one or more fiducial markers to form a three-dimensional superimposed image; The method includes:

2. Capturing a two-dimensional fluorescent image of the target area and the one or more fiducial markers comprises: illuminating the target area and the one or more fiducial markers with excitation light; receiving at least one fluorescent emission in response to illumination of the target area with the excitation light; The method of claim 1 , comprising:

3. The method of claim 2 , wherein the excitation light is between about 400 nm and about 450 nm.

4. The method of claim 2 , wherein the excitation light has a wavelength of about 405 nm.

5. The method of claim 1 , wherein capturing a two-dimensional fluorescent image of the target area and the one or more fiducial markers comprises capturing the emission of at least one fluorescent molecule.

6. The method of claim 5 , wherein the at least one fluorescent molecule comprises an endogenous molecule capable of emitting fluorescence.

7. 6. The method of claim 5, wherein the at least one fluorescent molecule comprises an exogenous molecule capable of fluorescing or a molecule that includes an exogenously added moiety capable of fluorescing.

8. The method of claim 7 , wherein the at least one fluorescent molecule comprises an aminolevulinic acid (ALA)-derived porphyrin.

9. The method of claim 1 , wherein the three-dimensional map is generated using infrared light.

10. The method of claim 9 , wherein the three-dimensional map is generated using near-infrared light.

11. The step of generating a three-dimensional map includes: projecting infrared radiation onto the target area; receiving infrared radiation reflected by the target area; determining a depth of the target area based on the reflected infrared radiation; and generating a The method of claim 1 , comprising:

12. 12. The method of claim 11, wherein the infrared radiation is projected as a split beam into a light pattern, the reflected infrared radiation comprises a distortion of the light pattern, and the depth is measured based on the distortion of the light pattern.

13. The method of claim 12 , wherein the light pattern is formed by a diffraction grating, and the light pattern includes a plurality of dots.

14. The method of claim 11 , wherein the depth is measured by time of flight based on a phase shift between the projected and reflected infrared radiation.

15. The method of claim 1 , wherein the target area comprises at least one wound.

16. The method of claim 15, wherein the wound contains at least one bacterium that contains at least one fluorescent molecule.

17. determining one or both of a surface area and a volume of the wound; The method of claim 15 further comprising:

18. the method is performed at least twice, the two executions including a first execution and a second execution separated by a time period of at least three hours, the three-dimensional superimposed image of the first execution being a first three-dimensional superimposed image and the three-dimensional superimposed image of the second execution being a second three-dimensional superimposed image; 16. The method of claim 15, further comprising comparing the first and second three-dimensional overlapping images to determine a healing status of the wound.

19. 20. The method of claim 18, wherein the period of time is at least one day.

20. The wound healing status includes a worsening wound; 20. The method of claim 18, further comprising administering at least one wound improvement adjunct.

21. The method of claim 18, wherein the comparing further comprises tracking a topography of the wound.

22. The method of claim 1 , wherein the target area comprises tissue excised from a subject organism.

23. 23. The method of claim 22, wherein the tissue comprises cancer tissue.

24. 24. The method of claim 23, wherein the cancerous tissue comprises a tumor.

25. 25. The method of claim 24, wherein the tumor is a breast tumor and the excised tissue comprises a lumpectomy.

26. 23. The method of claim 22, wherein the excised tissue comprises fluorescent molecules associated with probes targeting tumor receptors, enzymatically active fluorescent molecules, or fluorescence caused by genetically engineered oncolytic viruses, or any combination thereof.

27. 27. The method of claim 26, wherein the tumor receptor comprises HER2, a folate receptor, CXCR4, a hormone receptor, EGFR, or VEGF, or a combination thereof, and the enzyme comprises a protease, a carbohydrase, a lipase, a transferase, an oxidoreductase, a matrix metalloproteinase (MMP), a caspase, a cathepsin, a kallikrein, a serine protease, an isocitrate dehydrogenase, or an enzyme overexpressed by a tumor cell, or a combination thereof.

28. The method of claim 1 , wherein the target area comprises a surgical bed from which tissue has been removed.

29. 30. The method of claim 28, wherein the surgical bed and the resected tissue comprise cancerous tissue.

30. The method is performed at least twice in any order, the two runs including a first run and a second run, the first run being performed on the target area, the target area being a first target area including the resected tissue, the second run being performed on a second target area including a surgical bed from which the tissue was resected, the three-dimensional superimposed image of the first run being a first three-dimensional superimposed image, and the three-dimensional superimposed image of the second run being a second three-dimensional superimposed image; 10. The method of claim 1, further comprising comparing the first and second three-dimensional overlapped images to determine fluorescence continuity between the resected tissue and the surgical bed based on an orientation of the resected tissue relative to the surgical bed.

31. 31. The method of claim 30, wherein the continuum of fluorescence includes one or more of the following characteristics: bacterially infected tissue, virally infected tissue, burns, cancerous tissue, connective tissue, muscle tissue, blood follicles, and skin.

32. the continuum of fluorescence corresponds to diseased tissue; 31. The method of claim 30, further comprising the step of resecting at least a portion of the diseased tissue from the surgical bed.

33. capturing companion three-dimensional images of the target area and the one or more fiducial markers using imaging techniques including one or more of computed tomography (CT), magnetic resonance imaging (MRI), photoacoustic imaging, ultrasound, and optical coherence tomography; registering the first three-dimensional overlapping image with the companion three-dimensional image to form a second three-dimensional overlapping image; The method of claim 1 further comprising:

34. 34. The method of claim 33, wherein the one or more fiducial markers include a first set of fiducial markers and a second set of fiducial markers.

35. 34. The method of claim 33, wherein the companion three-dimensional image is captured using computed tomography, and the one or more fiducial markers include at least one fluorescent molecule and at least one CT contrast agent.

36. 34. The method of claim 33, wherein the companion three-dimensional image is captured using photoacoustic imaging and the target area includes a breast tumor and an anti-HER2 dual fluorescence-photoacoustic probe.

37. an excitation light source configured to emit a first radiation capable of exciting a fluorophore; a filter configured to block the passage of reflected excitation light and to allow the passage of fluorescent light emitted by said fluorophore; An imaging lens; a visible light source configured to emit a second radiation; and an infrared light source configured to emit a third radiation; at least one image sensor configured to detect radiation; a processor configured to receive the detected radiation and output data related to the detected radiation; An imaging device comprising:

38. 40. The imaging device of claim 37, wherein the imaging device is configured to be hand-held.

39. 38. The imaging device of claim 37, wherein the imaging device is configured to visualize a wound.

40. 40. The imaging device of claim 37, wherein the imaging device is configured to visualize one or more of pre-cancerous cells, cancerous cells, and peripheral lesions at surgical margins.

41. 41. The imaging device of claim 40, wherein the excitation light source is further configured to excite autofluorescence emission of tissue cells and fluorescence emission of induced porphyrins in tissue cells of the surgical margin.

42. 42. The imaging device of claim 41 , wherein the filter is further configured to block the passage of reflected excitation light and allow the passage of emission having a wavelength corresponding to the autofluorescence emission of the tissue cells and the fluorescence emission of the induced porphyrins in the tissue cells.

43. 43. The imaging device of claim 42, wherein the image sensor is further configured to detect the filtered autofluorescence emission of the tissue cells and the fluorescence emission of the derivatized porphyrins in the tissue cells of the surgical margin.

44. 44. The imaging device of claim 43, wherein the processor is further configured to receive the detected luminescence and output data related to the detected filtered autofluorescence luminescence of the tissue cells and the fluorescence luminescence of the induced porphyrins in the tissue cells of the surgical margin.

45. 38. The imaging device of claim 37, wherein the second radiation comprises white light.

46. 38. The imaging device of claim 37, wherein the second radiation comprises monochromatic visible light.

47. 38. The imaging device of claim 37, wherein the third radiation comprises infrared radiation.

48. 48. The imaging device of claim 47, wherein the infrared radiation comprises near infrared radiation.

49. 38. The imaging device of claim 37, wherein the at least one image sensor is configured to detect radiation including the fluorescent light, reflected white light, and reflected infrared light.

50. The imaging device of claim 37 , wherein the at least one sensor comprises at least two sensors.

51. The at least two sensors include a first sensor configured to detect fluorescence and a second sensor configured to detect reflection fluorescence. and a second detector configured to detect incident visible light.

52. 38. The imaging device of claim 37, wherein the at least one sensor comprises at least three sensors including a first sensor configured to detect fluorescent light, a second sensor configured to detect reflected visible light, and a third sensor configured to detect reflected infrared light.

53. 38. The imaging apparatus of claim 37, further comprising a common radiation source configured to cooperate with at least one of the light sources.

54. 54. The imaging device of claim 53, wherein the at least one light source comprises a converter for converting source radiation emitted from the common radiation source into the first radiation, the second radiation, or the third radiation, or any combination thereof.

55. 55. The imaging device of claim 54, wherein the transducer comprises a filter, a lens, a prism, a diffractor, or a quantum dot, or a combination thereof.

56. 55. The imaging apparatus of claim 54, wherein the excitation light source comprises a first transducer, the visible light source comprises a second transducer, and the infrared light source comprises a third transducer.

57. 38. The imaging device of claim 37, further comprising a display unit.

58. 58. The imaging device of claim 57, wherein the display unit is configured to display the data output by the processor.

59. 58. The imaging device of claim 57, wherein the data comprises a three-dimensional image.

60. 58. The imaging device of claim 57, wherein the display unit comprises a touch screen.

61. 38. The imaging device of claim 37, wherein the detected radiation includes one or more of fluorescent light, reflected visible light, and reflected infrared light.

62. 38. The imaging device of claim 37, wherein the detected radiation includes fluorescent light, reflected visible light, and reflected infrared light.

63. 40. The imaging device of claim 37, wherein the imaging device is configured to visualize a target area of ​​a biological target.

64. 64. The imaging device of claim 63, wherein the processor is further configured to generate a three-dimensional map of the target area.

65. 65. The imaging device of claim 64, wherein the three-dimensional map is generated from infrared light reflected from the target area.

66. 65. The imaging device of claim 64, wherein the processor is further configured to capture a two-dimensional visible light image of the target area based on the detected radiation.

67. 67. The method of claim 66, wherein the processor is further configured to generate a three-dimensional visible light image of the target area based on the three-dimensional map and the two-dimensional visible light image. The imaging device.

68. 65. The imaging device of claim 64, wherein the processor is further configured to capture a two-dimensional fluorescent image of the target area based on the detected radiation.

69. 69. The imaging device of claim 68, wherein the processor is further configured to generate a three-dimensional fluorescence image of the target area based on the three-dimensional map and the two-dimensional fluorescence image.

70. 64. The imaging device of claim 63, wherein the processor is further configured to align a three-dimensional visible light image of the target area with a three-dimensional fluorescent image of the target area to form a three-dimensional superimposed image of the target area.

71. 71. The imaging apparatus of claim 70, wherein the alignment is performed based on alignment of fiducial markers associated with the target region.

72. 38. The imaging device of claim 37, wherein the imaging device is configured to perform the method of claim 1.

73. The imaging device is further configured to be handheld; the excitation light source is further configured to excite autofluorescence emissions of tissue cells, as well as fluorescence emissions having wavelengths between about 600 nm and about 660 nm in precancerous cells, cancer cells, and surrounding lesions at the surgical margins after exposure to an imaging or contrast agent; the filter is further configured to allow passage of emission having a wavelength corresponding to autofluorescence emission of the tissue cells and fluorescence emission between about 600 nm and about 660 nm in the tissue cells of the surgical margin; the at least one image sensor is further configured to detect the filtered autofluorescence emission of the tissue cells and the fluorescence emission between about 600 nm and about 660 nm in the tissue cells of the surgical margin; 38. The imaging device of claim 37, wherein the processor is further configured to receive the detected emissions and output data related to the detected filtered autofluorescence emissions of the tissue cells and the fluorescence emissions between about 600 nm and about 660 nm in the tissue cells of the surgical margin, and generate a three-dimensional image of one or more of the pre-cancerous cells, the cancer cells, and the surrounding lesions of the surgical margin.

74. 1. A system for fluorescence-based three-dimensional imaging of a target, comprising: at least one excitation light source configured to uniformly illuminate the target surface with a uniform excitation light field during fluorescence imaging; at least one white light source configured to illuminate the target surface during white light imaging; at least one infrared radiation source configured to emit infrared radiation towards the target surface; An image sensor; a filter configured to allow an optical signal responsive to illumination of the target surface by the excitation light, the optical signal having a wavelength corresponding to one or more of bacterial autofluorescence and tissue autofluorescence to pass to the image sensor; Processor and It is equipped with The processor, receiving an optical signal responsive to illumination of the target with the infrared light; generating a three-dimensional map of the target surface based on the received signals in response to illumination of the target; receiving detected optical signals in response to illumination of the target surface with the excitation light to generate a two-dimensional fluorescent image of the target surface; receiving optical signals responsive to white light illumination of the target surface and generating a two-dimensional white light image of the target surface; Combining the 3D map, the fluorescent image, and the white light image to generate a 3D image of the target surface. The system is configured as follows:

75. 38. A computer program product for use with an imaging device according to claim 37, comprising a non-transitory computer readable medium storing computer program code for image processing, the computer program code being executable by the processor of the imaging device to perform the method of claim 1.

Citation Information

Cited By

  • Real-time positioning and puncture track control method and system in breast duct endoscope operation

    CN120859626A