System and method for single-sensor visible light and SWIR imaging

Abstract

A system for imaging biological tissue may include one or more optical sources configured to provide short-wave infrared (SWIR) illumination and visible light illumination; a sensor configured to sense the reflective portions of the SWIR illumination and the reflective portions of the visible light illumination; and a controller communicating with the sensor. The controller may be configured to receive first information from the sensor corresponding to the reflective portions of the SWIR illumination, receive second information from the sensor corresponding to the reflective portions of the visible light illumination, use the first and second information to generate at least one image of the biological tissue, and output at least one image to at least one of a display and / or memory.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the benefit of U.S. Provisional Application No. 63 / 495,023, filed Apr. 7, 2023, the entire contents of which are incorporated herein by reference.

[0002] The present disclosure generally relates to systems and methods for SWIR and visible - light - based medical imaging.

Background Art

[0003] The state of biological tissue can provide insights regarding the health state of the person or animal to which the tissue belongs. For example, the state of a subject's lymph nodes can indicate whether the subject has a disease or infection. For example, swollen lymph nodes can be indicative of a bacterial infection, a viral infection, or a sign of cancer. Determining the state of lymph nodes is thus extremely useful for disease diagnosis, prevention, and treatment.

Summary of the Invention

Means for Solving the Problems

[0004] The state of biological tissue can be checked by imaging the tissue. Various imaging modalities exist for imaging tissue. Specialized imaging modalities can be used to image specific types of tissue. For example, lymph nodes can be imaged using lymphangiography, which involves injecting a radioactive contrast agent into the patient and imaging the lymph nodes and lymphatic vessels via X-rays. However, lymphangiography is invasive, can cause significant discomfort, and requires the use of radiopharmaceuticals. Other more versatile imaging modalities such as computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, and positron emission tomography (PET) can also be employed to image specific types of tissue. While these imaging techniques can allow biological tissue to be identified and assessed with reasonable accuracy, they cannot provide ideal contrast for visualizing target biological tissue unless a specific contrast agent is injected. As a result, the image may show organs and tissues other than the target tissue with identical or better contrast compared to the target tissue, making it difficult to locate and examine the target tissue.

[0005] As described above, several imaging modalities exist for visualizing and examining lymph nodes and other biological tissues. However, known medical imaging techniques have several drawbacks, such as their inability to accurately and robustly identify target tissues and / or target features, and their reliance on contrast agents. Therefore, improved medical imaging methods and systems are needed.

[0006] Provided herein are techniques for single-sensor multi-wavelength imaging of biological tissue. The disclosed systems and methods may employ a single sensor to capture images of biological tissue at multiple wavelengths, for example, to capture both short-wave infrared (SWIR) and visible light images of biological tissue. The captured images may be displayed, stored, and / or used to generate composite images. Images produced using the described techniques exhibit high contrast, show target tissue (e.g., lymph nodes) and / or target features, and can facilitate the identification and examination of the target tissue without requiring the use of contrast agents.

[0007] A system provided for imaging biological tissue may include one or more optical sources configured to provide short-wave infrared (SWIR) illumination and visible light illumination; a sensor configured to sense the reflective portions of the SWIR illumination and the visible light illumination; and a controller communicating with the sensor. The controller may be configured to receive first information from the sensor corresponding to the reflective portions of the SWIR illumination and second information from the sensor corresponding to the reflective portions of the visible light illumination. Using the first and second information, the controller may be configured to generate at least one image of the biological tissue. The controller may then be configured to output at least one image to at least one of a display and / or memory.

[0008] One or more optical sources may include lasers or light-emitting diodes. The sensor may also include one or more cameras selected from the following group: silicon cameras, InGaAs cameras, black silicon cameras, germanium cameras, germanium-tin-on silicon cameras, quantum dot shortwave infrared cameras, and cadmium telluride mercury cameras.

[0009] In some embodiments, one or more optical sources are configured to alternately provide SWIR illumination and visible light illumination according to a time-pulsing scheme. In some embodiments, one or more optical sources include a first optical source configured to provide SWIR illumination and a second optical source configured to provide visible light illumination.

[0010] The system may further include a first filter configured to block at least a portion of the SWIR illumination, and a second filter configured to block at least a portion of the visible light illumination. The system may also include a filter support device configured to move at least one of the first filter and the second filter between a first position within the optical path of the system and a second position outside the optical path of the system.

[0011] In some embodiments, one or more optical sources are configured to provide SWIR illumination and visible light illumination simultaneously. The system may further include a filter array configured to spatially selectively block and transmit the reflective portions of the SWIR illumination and the reflective portions of the visible light illumination. In some embodiments, the filter array comprises a first portion that transmits some or all of the reflective portions of the visible light illumination and blocks the reflective portions of the SWIR illumination, and a second portion that transmits some or all of the reflective portions of the SWIR illumination and blocks the reflective portions of the visible light illumination.

[0012] In some embodiments, the filter array comprises: a first portion that transmits the green portion of reflected visible light illumination while blocking the blue and red portions of the reflected visible light illumination and blocking the reflected portion of SWIR illumination; a second portion that transmits the blue portion of reflected visible light illumination while blocking the green and red portions of the reflected visible light illumination and blocking the reflected portion of SWIR illumination; a third portion that transmits the red portion of reflected visible light illumination while blocking the blue and green portions of the reflected visible light illumination and blocking the reflected portion of SWIR illumination; and a fourth portion that blocks the green, blue, and red portions of the reflected visible light illumination and transmits the reflected portion of SWIR illumination. The first, second, third, and fourth portions may be arranged in a repeating 2x2 tile arrangement in the filter array, or in a repeating 4x4 tile arrangement in the filter array.

[0013] When the first, second, third, and fourth portions are arranged in a repeating 4x4 tile array, each repeating 4x4 tile portion in the repeating 4x4 tile array may have twice the number of green transparent space portions as blue transparent space portions, red transparent space portions, or SWIR transparent space portions. In some embodiments, the 4x4 tile portion does not include any lateral or vertically adjacent blue transparent portions, any lateral or vertically adjacent red transparent portions, any lateral or vertically adjacent green transparent portions, or any lateral or vertically adjacent SWIR transparent portions. In some embodiments, the 4x4 tile array comprises four 2x2 blocks, each 2x2 block comprising four space portions that transmit a single distinct wavelength range. In some embodiments, the 4x4 tile array comprises four 2x2 blocks, where the first 2x2 block comprises three red translucent space subportions and one SWIR translucent space subportion; the second 2x2 block comprises three green translucent space subportions and one SWIR translucent space subportion; the third 2x2 block comprises three blue translucent space subportions and one SWIR translucent space subportion; and the fourth 2x2 block comprises a red translucent space subportion, a blue translucent space subportion, a green translucent space subportion, and a SWIR translucent space subportion. In some embodiments, the filter array comprises a first portion that transmits some or all of the reflective portion of visible light illumination and blocks the reflective portion of SWIR illumination; a second portion that transmits a first wavelength range of the reflective portion of SWIR illumination and blocks a second wavelength range of the reflective portion of SWIR illumination and blocks the reflective portion of visible light illumination; and a third portion that transmits a second wavelength range of the reflective portion of SWIR illumination and blocks a first wavelength range of the reflective portion of SWIR illumination and blocks the reflective portion of visible light illumination.

[0014] SWIR illumination may have a first polarization. The reflective portion of the SWIR illumination, sensed by the sensor, may have a second polarization opposite to the first polarization. A polarizer may be arranged between the biological tissue and the sensor.

[0015] The sensor may include a stack of photosensors. The stack of photosensors may comprise a first photosensor configured to detect the blue portion of reflected visible light illumination, a second photosensor configured to detect the green portion of reflected visible light illumination, a third photosensor configured to detect the red portion of reflected visible light illumination, and a fourth photosensor configured to detect the reflective portion of SWIR illumination.

[0016] The imaged biological tissue may include a first region having a first water content (e.g., volume density of water) and a second region having a second water content lower than the first water content. The biological tissue may be contrast-enhanced. The controller may be configured to generate at least one image without reference light and / or information from ambient light surrounding the sensor.

[0017] A method provided for imaging biological tissue may include: providing short-wave infrared (SWIR) illumination and visible light illumination by one or more optical sources; sensing the reflective portions of the SWIR illumination and the visible light illumination by a sensor; receiving first information from the sensor corresponding to the reflective portions of the SWIR illumination by a controller communicating with the sensor; receiving second information from the sensor corresponding to the reflective portions of the visible light illumination by the controller; generating at least one image of the biological tissue using the first and second information by the controller; and outputting at least one image to at least one of a display and / or memory by the controller.

[0018] A non-transient computer-readable storage medium may store instructions for imaging biological tissue, the instructions being executed by one or more processors of the system, causing the system to provide short-wave infrared (SWIR) illumination and visible light illumination by one or more optical sources, causing sensors to detect the reflective portions of the SWIR illumination and the visible light illumination, a controller communicating with the sensor receiving first information from the sensor corresponding to the reflective portions of the SWIR illumination, the controller receiving second information from the sensor corresponding to the reflective portions of the visible light illumination, the controller generating at least one image of the biological tissue using the first and second information, and the controller outputting at least one image to at least one of a display and / or memory. [Brief explanation of the drawing]

[0019] The following figures illustrate various systems and methods for single-sensor visible light and SWIR imaging. The systems and methods shown in the figures may have one or more of the features described herein.

[0020] [Figure 1] Figure 1 shows exemplary systems for single-sensor visible light and SWIR imaging according to several embodiments.

[0021] [Figure 2] Figure 2 shows another exemplary system for single-sensor visible light and SWIR imaging according to several embodiments.

[0022] [Figure 3] Figure 3 shows another exemplary system for single-sensor visible light and SWIR imaging according to several embodiments.

[0023] [Figure 4A] Figure 4A shows filter arrays for single-sensor visible light and SWIR imaging systems according to several embodiments.

[0024] [Figure 4B] Figure 4B shows another filter array for a single-sensor visible light and SWIR imaging system, according to some embodiments.

[0025] [Figure 4C] Figure 4C shows another filter array for a single-sensor visible light and SWIR imaging system, according to some embodiments.

[0026] [Figure 4D] Figure 4D shows another filter array for a single-sensor visible light and SWIR imaging system, according to some embodiments.

[0027] [Figure 4E] Figure 4E shows another filter array for a single-sensor visible light and SWIR imaging system, according to some embodiments.

[0028] [Figure 5] Figure 5 shows a sensor stack for a single-sensor visible light and SWIR imaging system, according to some embodiments.

[0029] [Figure 6] Figure 6 shows another exemplary system for single-sensor visible light and SWIR imaging, according to some embodiments.

[0030] [Figure 7] Figure 7 shows a method for single-sensor visible light and SWIR imaging, according to some embodiments.

[0031] [Figure 8] Figure 8 shows a computer system, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0032] Detailed explanation Provided is a technique for single-sensor multi-wavelength imaging of biological tissue. The disclosed system and method may employ a single sensor to capture images of biological tissue at multiple wavelengths, for example, to capture both short-wave infrared (SWIR) and visible light images of biological tissue. The captured images may be displayed, stored, and / or used to generate composite images. Images produced using the described technique may exhibit high contrast, show target tissue (e.g., lymph nodes) and / or target features, and can facilitate the identification and examination of the target tissue without requiring the use of contrast agents, without the use of reference light, and without information from ambient light surrounding the sensor.

[0033] An exemplary imaging system 100 for SWIR and visible light imaging of biological tissue is shown in Figure 1. The system 100 may include one or more light sources 102 configured to provide SWIR illumination and visible light illumination, one or more lenses (104, 106) for directing light onto the biological tissue and collecting light reflected by the biological tissue, and a sensor 108 for detecting light within the visible wavelength range and the SWIR wavelength range. The system 100 may also be a component of a laparoscope.

[0034] The light source 102 can be implemented using any suitable light-emitting device, such as one or more laser light sources, one or more LED light sources, or a combination thereof. In some embodiments, the light source 102 is a single light source that emits in both SWIR and visible light ranges. In other embodiments, the light source 102 comprises a first light source that emits in the SWIR range and a second set of one or more light sources that emit in the visible light range. Visible light illumination may be provided by multiple light sources that emit in different visible light wavelength ranges. For example, as shown in Figure 1, visible light illumination may be provided by separate light sources that emit in the blue, green, and red visible light ranges. SWIR illumination and visible light illumination may be delivered to and incident on the biological tissue to be imaged.

[0035] The light source 102 may provide SWIR illumination within the wavelength range of approximately 800-2,600 nm, 800-1,700 nm, 900-2,000 nm, 1,000-2,600 nm, 1,000-1,700 nm, 1,500-1,700 nm, or any sub-part of any one or more of these wavelength ranges. For example, light source 102 emits light at approximately 800nm, 850nm, 900nm, 950nm, 1,000nm, 1,050nm, 1,100nm, 1,150nm, 1,200nm, 1,250nm, 1,300nm, 1,350nm, 1,400nm, 1,450nm, 1,500nm, 1,550nm, 1,600nm, 1,650nm, 1,700nm, and 1,750nm. SWIR illumination may be provided with wavelengths of 1,800 nm, 1,850 nm, 1,900 nm, 1,950 nm, 2,000 nm, 2,050 nm, 2,100 nm, 2,150 nm, 2,200 nm, 2,250 nm, 2,300 nm, 2,350 nm, 2,400 nm, 2,450 nm, 2,500 nm, 2,550 nm, or 2,600 nm. In some embodiments, the light source 102 provides SWIR illumination at multiple SWIR wavelengths, for example, SWIR illumination with a wavelength of about 800 nm, SWIR illumination with a wavelength of about 1,700 nm, and SWIR illumination with a wavelength of about 2,600 nm.

[0036] Visible light illumination may be provided by light sources within the wavelength range of approximately 380–750 nm, 450–625 nm, 485–590 nm, 500–565 nm, 450–750 nm, 500–750 nm, or 565–750 nm, or any sub-part of any one or more of these wavelength ranges. For example, light source 102 emits light at approximately 450nm, 455nm, 460nm, 465nm, 470nm, 475nm, 480nm, 485nm, 490nm, 495nm, 500nm, 505nm, 510nm, 515nm, 520nm, 525nm, 530nm, 535nm, 540nm, 545nm, 550nm, 555nm, 560nm, 565nm, 570nm, 575nm, 580nm, 585nm, 590nm, 600nm, 60 The light source 102 may provide visible light illumination with wavelengths of 5nm, 610nm, 615nm, 620nm, 625nm, 630nm, 635nm, 640nm, 645nm, 650nm, 655nm, 660nm, 665nm, 670nm, 675nm, 680nm, 685nm, 690nm, 700nm, 705nm, 710nm, 715nm, 720nm, 725nm, 730nm, 735nm, 740nm, 745nm, or 750nm. In some embodiments, the light source 102 provides visible light illumination at multiple visible wavelengths, for example, visible light illumination with wavelengths of 450-485nm, visible light illumination with wavelengths of 500-565nm, and visible light illumination with wavelengths of 625-750nm.

[0037] The rod lens 104 may deliver SWIR illumination and visible light illumination provided by the light source 102 to the biological tissue to be imaged. The rod lens 104 may also be the rod lens of a laparoscope, of which system 100 is a component. The light source 102 may transmit light to the rod lens 104 through a fiber coupler 110 or any other suitable optical coupler.

[0038] Light reflected by the biological tissue to be imaged may be collected by the rod lens 104 and subsequently guided to the VIS / SWIR sensor 108 by the imaging lens 106. The reflected light collected by the system 100 may include both the reflective portion of SWIR illumination and the reflective portion of visible light illumination. The reflective portion of SWIR illumination may be one or more of the same SWIR wavelength ranges listed above with respect to SWIR illumination, or any sub-parts thereof. Similarly, the reflective portion of visible light illumination may be one or more of the same visible light wavelength ranges listed above with respect to visible light illumination, or any sub-parts thereof.

[0039] The sensor 108 may be positioned at the proximal end of the laparoscope behind the imaging lens 106 and the rod lens 104. The sensor 108 may detect some or all of the reflected SWIR light, or some or all of the reflected visible light. The sensor 108 may be sensitive to SWIR illumination within one or more of the same SWIR wavelength ranges listed above with respect to SWIR illumination, or within any sub-part thereof, and may be sensitive to visible light illumination within one or more of the same visible light wavelength ranges listed above with respect to visible light illumination, or within any sub-part thereof. In some embodiments, the sensor 108 is monochromatic within the visible light wavelength range, and in other embodiments, the sensor 108 is a color-sensitive sensor.

[0040] Sensor 108 may be any device or combination of devices configured to detect both SWIR light and visible light. In some embodiments, sensor 108 includes one or more cameras, such as a silicon camera, an InGaAs camera, a black silicon camera, a germanium camera, a germanium-tin-on silicon camera, a quantum dot shortwave infrared camera, a cadmium telluride mercury camera, or a combination thereof. In some embodiments, sensor 108 comprises one or more color filter arrays or one or more photosensor stacks.

[0041] The light source 102 and / or sensor 108 may be electronically coupled to and controlled by one or more controllers 112. The controller 112 may be a computer system such as a laptop, tablet, desktop computer, or microcontroller. When the sensor 108 detects reflected visible light and reflected SWIR light, information corresponding to the reflected visible light and reflected SWIR light may be transmitted by the sensor 108 to the controller 112. The controller 112 may use the information received from the sensor 108 to generate one or more images or one or more videos. The images / videos may be separate images / videos corresponding to one or more wavelength bands (e.g., separate visible light and SWIR images) or a composite image / video. The controller 112 may then output the generated images and / or videos to a display (e.g., a computer monitor) and / or memory (e.g., the memory of the controller 112).

[0042] The controller 112 may be configured to synchronize functionality between the sensor 108 and the light source 102, for example, through one or more electrical triggers 114 connecting the light source 102 and the sensor 108. In some embodiments, the controller 112 may be configured to control the light source 102 so that it emits SWIR illumination and visible light illumination simultaneously. In other embodiments, the controller 112 may be configured to control the light source 102 so that it emits SWIR illumination and visible light illumination simultaneously in a pulsed timing scheme so that the visible light illumination is emitted in alternating temporal pulses with the SWIR illumination. When a pulsed timing scheme is used, the controller 112 may synchronize the sensor 108 with pulses from the light source 102 so that SWIR image frames and visible light image frames are captured. In some embodiments, white light images with red, green, and blue illumination are captured, and in other embodiments, images with illumination in specific visible light colors are captured. The captured visible and SWIR image frames may be used (e.g., by controller 112) to generate separate images or videos, and / or to generate composite images or videos.

[0043] In embodiments where the light source 102 provides continuous and simultaneous SWIR and visible light illumination, a filter wheel or other mechanical filter swapper 216 may be used to move one or more filters in and out of the optical path of the sensor 108, as shown in Figure 2. The filter swapper 216 may be positioned in front of the sensor 108, for example, between the imaging lens 106 and the sensor 108, and may be used to selectively block reflected light of different wavelengths. The controller 112 may control the movement of the filter swapper 216 and coordinate the functionality of the sensor 108 with the movement of the filter swapper 216 so that SWIR image frames and visible light image frames are captured. The captured frames may be used to generate separate images or videos and / or to generate a composite image or video.

[0044] The filter swapper 216 may selectively position one or more of the following filters within the optical path of the sensor 108: a SWIR-transmitting filter that transmits SWIR light and blocks visible light; a visible light-transmitting filter that transmits visible light and blocks SWIR light; a red-transmitting filter that transmits red light and blocks blue, green, and SWIR light; a green-transmitting filter that transmits green light and blocks blue, red, and SWIR light; a blue-transmitting filter that transmits blue light and blocks green, red, and SWIR light; and a yellow-transmitting filter that transmits yellow light and blocks red, green, blue, and SWIR light.

[0045] As an alternative to, or in addition to, a filter wheel or mechanical filter swapper such as a filter swapper 216, in embodiments in which the light source 102 provides continuous simultaneous SWIR and visible light illumination, a color filter 318 may be used to select with respect to the wavelength of light that is allowed to be incident on the sensor 108, as shown in Figure 3.

[0046] The filter array 318 may be implemented using visible and SWIR filter mosaics. A filter mosaic may allow spatial arrays of visible (e.g., blue, green, and red) and SWIR pixels to be captured based on the simultaneous capture of reflected SWIR illumination and reflected visible light illumination. Any spatial array of any combination of visible and SWIR color filters may constitute a filter mosaic. Exemplary filter mosaics include (but are not limited to) RGB (red, green, blue) + SWIR filter mosaics, RYB (red, yellow, blue) + SWIR filter mosaics, CYGM (cyan, yellow, green, magenta) + SWIR filter mosaics, RGBE (red, green, blue, emerald) filter mosaics, and RGBW (red, green, blue, white) + SWIR filter mosaics. In some embodiments, the filter mosaic may include multiple SWIR filters, each configured to transmit different SWIR wavelength bands (for example, an RGB (red, green, blue) + SWIR1 + SWIR2 filter mosaic (where the SWIR1 filter transmits SWIR light in the first SWIR wavelength band, and the SWIR2 filter transmits SWIR light in the second SWIR wavelength band)). In addition, the filter mosaic may have any dimensions; for example, the filter mosaic may be a 2x2 array of filters, a 3x3 array of filters, a 4x4 array of filters, etc.

[0047] Schematic diagrams of various exemplary filter mosaics are provided in Figures 4A-4E. As shown, in some embodiments, the filter array comprises a first spatial portion that transmits some or all of the reflective portion of visible light illumination and blocks the reflective portion of SWIR illumination, and a second spatial portion that transmits some or all of the reflective portion of SWIR illumination and blocks the reflective portion of visible light illumination. For example, the filter array may include: a first portion that transmits the green portion of reflected visible light illumination while blocking the blue and red portions of the reflective portion of visible light illumination and blocking the reflective portion of SWIR illumination; a second portion that transmits the blue portion of reflected visible light illumination while blocking the green and red portions of the reflective portion of visible light illumination and blocking the reflective portion of SWIR illumination; a third portion that transmits the red portion of reflected visible light illumination while blocking the blue and green portions of the reflective portion of visible light illumination and blocking the reflective portion of SWIR illumination; and a fourth portion that blocks the green, blue, and red portions of the reflective portion of visible light illumination and transmits the reflective portion of SWIR illumination.

[0048] The first, second, third, and fourth portions may be arranged in a repeating 2x2 tile array in the filter array (for example, as shown in Figure 4A) or in a repeating 4x4 tile array in the filter array (for example, as shown in Figures 4B-4E). If the spatial portions are arranged in a repeating 4x4 tile array, each repeating 4x4 tile portion in the repeating 4x4 tile array may have twice the number of green transparent spatial portions as blue transparent spatial portions, red transparent spatial portions, or SWIR transparent spatial portions (for example, as shown in Figure 4B). The 4x4 tile portion may not contain any laterally or vertically adjacent blue transparent portions, any laterally or vertically adjacent red transparent portions, any laterally or vertically adjacent green transparent portions, or any laterally or vertically adjacent SWIR transparent portions (for example, as shown in Figure 4B). In other embodiments, a 4x4 tile array comprises four 2x2 blocks, each 2x2 block having four spatial portions that transmit a single distinct wavelength range (for example, as shown in Figure 4C).

[0049] In other embodiments, a 4x4 tile array comprises four 2x2 blocks. The first 2x2 block may comprise three red translucent space subparts and one SWIR translucent space subpart. The second 2x2 block may comprise three green translucent space subparts and one SWIR translucent space subpart. The third 2x2 block may comprise three blue translucent space subparts and one SWIR translucent space subpart. The fourth 2x2 block may comprise a red translucent space subpart, a blue translucent space subpart, a green translucent space subpart, and a SWIR translucent space subpart (as shown, for example, in Figure 4D).

[0050] In some embodiments, the filter array comprises a first portion that transmits some or all of the reflective portion of visible light illumination and blocks the reflective portion of SWIR illumination; a second portion that transmits a first wavelength range of the reflective portion of SWIR illumination and blocks a second wavelength range of the reflective portion of SWIR illumination and blocks the reflective portion of visible light illumination; and a third portion that transmits a second wavelength range of the reflective portion of SWIR illumination and blocks a first wavelength range of the reflective portion of SWIR illumination and blocks the reflective portion of visible light illumination. For example, as shown in Figure 4E, the filter array may include a first spatial portion that is red-transmitting, a second spatial portion that is green-transmitting, a third spatial portion that is blue-transmitting, a fourth spatial portion ("SWIR1") that is transparent to a first wavelength range of the reflective portion of the SWIR illumination, a fifth spatial portion ("SWIR2") that is transparent to a second wavelength range of the reflective portion of the SWIR illumination, a sixth spatial portion ("SWIR3") that is transparent to a third wavelength range of the reflective portion of the SWIR illumination, and a seventh spatial portion ("SWIR4") that is transparent to a fourth wavelength range of the reflective portion of the SWIR illumination.

[0051] In some embodiments, a microlens array may be placed between the filter array 318 and the sensor 108.

[0052] In some embodiments, where the light source 102 provides continuous and simultaneous SWIR and visible light illumination, the sensor 108 may include a photosensor stack comprising two or more layers of photosensors. For example, the sensor stack may be a blue + green + red + SWIR sensor stack, a blue + yellow + red + SWIR sensor stack, a cyan + yellow + green + magenta + SWIR sensor stack, a red + green + blue + emerald sensor stack, a red + green + SWIR sensor stack, or a blue + green + SWIR sensor stack. An exemplary sensor stack is illustrated in Figure 5. The optical throughput to the sensor 108 may be higher when a sensor stack is used rather than a filter array, and therefore, an embodiment of system 100 in which the sensor 108 comprises a sensor stack may be ideal for imaging biological tissue in low-light conditions.

[0053] In some embodiments, the light emitted by the light source 102 is polarized. For example, the SWIR illumination may have a first polarization, and the reflective portion of the SWIR illumination, detected by the sensor, may have a second polarization opposite to the first polarization. Cross-polarization imaging modalities may be used. One or more polarizers 622 in the optical path of the system may polarize the illumination light and / or the reflective portion of the illumination light, for example, as shown in Figure 6.

[0054] An exemplary method 700 for single-sensor visible light and SWIR imaging of biological tissue is provided in Figure 7. Method 700 may be performed using a system for imaging biological tissue using visible and SWIR light, for example, one of the embodiments of system 100 shown in Figures 1-3 and 6. The biological tissue imaged using Method 700 may include one or more different types of biological tissue and / or one or more different regions of biological tissue. In some embodiments, the biological tissue may include lymphatic components.

[0055] In some embodiments, the biological tissue may include one or more different types of biological tissue and / or one or more different regions of biological tissue having different water contents. Since water is absorbent within the SWIR range (e.g., about 1,550 nm), imaging using SWIR illumination may be effective in distinguishing biological tissue with a higher water content from biological tissue with a lower water content. For example, lymph nodes with a high water content may be distinguished from fat with a low water content.

[0056] To image the biological tissue, SWIR illumination and visible light illumination may be provided to the biological tissue by one or more optical sources (e.g., light source 102 shown in Figure 1-3) (step 702 of Method 700). The SWIR illumination and visible light illumination may be delivered to the biological tissue by a lens, such as a rod lens of a laparoscope (e.g., rod lens 104 shown in Figures 1-3 and 6). The reflective portions of the SWIR illumination and visible light illumination may then be sensed using a sensor (e.g., sensor 108 shown in Figures 1-3 and 6) (step 704 of Method 700). The reflective portions of the SWIR illumination and visible light illumination may be guided from the biological tissue to the sensor by one or more lenses (e.g., rod lens 104 and / or imaging lens 106 shown in Figures 1-3 and 6). The sensor may transmit first information corresponding to the reflective portion of SWIR illumination and second information corresponding to the reflective portion of visible light illumination to a controller communicating with the sensor (e.g., controller 112 shown in Figures 1-3 and 1-6) (steps 706-708 of Method 700). Using the first and second information, the controller may generate at least one image of biological tissue (step 710 of Method 700). The controller may then output at least one image to at least one of a display and / or memory (step 712 of Method 700).

[0057] Figure 8 shows an exemplary computer system 800 that may be used in a provided visible and SWIR imaging system (e.g., system 100) to generate images and / or videos of biological tissue based on information received from sensors. In other words, computer system 800 may be used to implement a controller in a system for imaging biological tissue using visible and SWIR light, such as system 100. Computer system 800 may be any suitable type of microprocessor-based device, such as a personal computer, workstation, server, or handheld computing device (portable electronic device) such as a telephone or tablet, or a dedicated device. As shown in Figure 8, computer system 800 may include one or more processors 802, an input device 804, an output device 806, a storage device 808 for storing software 810, and a communication device 812.

[0058] The input device 804 and the output device 806 can be connected to or integrated with system 102. The input device 804 may be any suitable device that provides input, such as a touchscreen, keyboard or keypad, mouse, or voice recognition device. Similarly, the output device 806 may be any suitable device that provides output, such as a display, touchscreen, haptic device, or speaker.

[0059] The storage device 808 may be any suitable device that provides storage, such as electrical, magnetic, or optical memory, including RAM, cache, hard drive, removable storage disk, or other non-transient computer-readable media. The communication device 812 may include any suitable device capable of transmitting and receiving signals over a network, such as a network interface chip or device. The components of the computer system 800 may be connected in any suitable manner, such as via a physical bus or via a wireless network.

[0060] The processor 802 is any suitable processor or combination of processors, including any one of a central processing unit (CPU), a field-programmable gate array (FPGA), and an application-specific integrated circuit (ASIC), or any combination thereof. The software 810, stored in the storage device 808 and run by the processor 802, may include, for example, programming to embody the functionality of the Disclosure. The software 810 may be stored in and / or transported in any non-transient computer-readable storage medium for use by or related to an instruction execution system, device, or device, which can fetch instructions associated with the software from the instruction execution system, device, or device and execute those instructions. In the context of the Disclosure, the computer-readable storage medium may be any medium, such as the storage device 808, that contains or stores programming for use by or related to an instruction execution system, device, or device.

[0061] The software 810 may also be propagated in any transport medium for use by or related to an instruction execution system, apparatus, or device, such as those described above, which can fetch instructions associated with the software from an instruction execution system, apparatus, or device and execute those instructions. In the context of this disclosure, the transport medium may be any medium that can communicate, propagate, or transport programming for use by or related to an instruction execution system, apparatus, or device. The transport-readable medium may include, but is not limited to, wired or wireless propagating media of electronic, magnetic, optical, electromagnetic, or infrared.

[0062] The computer system 800 may be connected to a network, which may be any preferred type of interconnected communication system. The network may implement any preferred communication protocol and may be secured by any preferred security protocol. The network may include any preferred array of network links that may implement the transmission and reception of network signals, such as wireless network connections, T1 or T3 lines, cable networks, DSL, or telephone lines.

[0063] The computer system 800 can implement any operating system suitable for running over a network. The software 810 can be written in any suitable programming language such as C, C++, Java®, or Python. In various embodiments, application software embodying the functionality of this disclosure can be deployed in different configurations, for example, in a client / server array, or through a web browser as a web-based application or web service.

[0064] The foregoing description is illustrated with reference to specific embodiments and / or examples for illustrative purposes. However, the above illustrative discussion is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations can be considered as possible in light of the above teachings. The embodiments have been selected and described to illustrate in the most detail the principles of the technique and their practical applications. Those skilled in the art will be able to make the best use of the technique and its various embodiments, along with various modifications suitable for specific intended uses.

[0065] As used herein, the singular forms “a,” “an,” and “the” include plural nouns unless the context clearly determines otherwise. References of “about” values ​​or parameters or “approximately” values ​​or parameters herein include (and describe) variations relating to the value or parameter itself. For example, a description referring to “about X” includes a description of “X.” It should be understood that aspects and variations of the invention described herein include aspects and variations consisting of and / or essentially consisting of.

[0066] When a range of values ​​or values ​​are provided, it should be understood that each intermediate value between the upper and lower limits of that range, and any other stated or intermediate values ​​within that stated range, are included within the scope of this disclosure. If the stated range includes an upper or lower limit, the range excluding either of those limits is also included within this disclosure.

[0067] While this disclosure and examples are fully described with reference to the accompanying drawings, it should be noted that various changes and modifications will be obvious to those skilled in the art. Such changes and modifications are understood to be within the scope of this disclosure and examples as defined by the claims. Finally, all disclosures of the patents and publications referenced in this application are incorporated herein by reference.

[0068] Any system, method, technique, and / or feature disclosed herein may be combined, in whole or in part, with any other system, method, technique, and / or feature disclosed herein.

Claims

1. A system for imaging biological tissue, wherein the system is One or more optical sources configured to provide short-wave infrared (SWIR) illumination and visible light illumination, A sensor configured to detect the reflective portion of the SWIR illumination and the reflective portion of the visible light illumination, A controller that communicates with the aforementioned sensor, the controller is The sensor receives first information corresponding to the reflective portion of the SWIR illumination, The sensor receives second information corresponding to the reflective portion of the visible light illumination, Using the first and second pieces of information, generate at least one image of the biological tissue, Outputting the aforementioned at least one image to at least one of the display and / or memory. A controller and A system that includes these features.

2. The system according to claim 1, wherein the one or more optical sources are configured to alternately provide the SWIR illumination and the visible light illumination according to a temporal pulsation scheme.

3. The system according to claim 1 or 2, wherein the one or more optical sources comprises a first optical source configured to provide the SWIR illumination and a second optical source configured to provide the visible light illumination.

4. The system according to any one of claims 1 to 3, wherein the one or more optical sources comprises a first optical source configured to provide SWIR illumination, a second optical source configured to provide red light illumination, a third optical source configured to provide green light illumination, and a fourth optical source configured to provide blue light illumination.

5. A first filter configured to block at least a portion of the SWIR illumination, A second filter configured to block at least a portion of the visible light illumination, The system according to any one of claims 1-4, further comprising the above.

6. The system according to claim 5, further comprising a filter support device configured to move at least one of the first filter and the second filter between a first position within the optical path of the system and a second position outside the optical path of the system.

7. The system according to claim 1 or 2, wherein one or more optical sources are configured to simultaneously provide the SWIR illumination and the visible light illumination.

8. The system according to any one of claims 1 to 7, further comprising a filter array configured to spatially selectively block and transmit the reflective portion of the SWIR illumination and the reflective portion of the visible light illumination.

9. The system according to claim 8, wherein the filter array comprises a first portion that transmits some or all of the reflective portion of the visible light illumination and blocks the reflective portion of the SWIR illumination, and a second portion that transmits some or all of the reflective portion of the SWIR illumination and blocks the reflective portion of the visible light illumination.

10. The aforementioned filter array is A first portion that transmits the green portion of the reflected visible light illumination while blocking the blue and red portions of the reflected portion of the visible light illumination, and blocks the reflected portion of the SWIR illumination, A second portion that transmits the blue portion of the reflected visible light illumination while blocking the green and red portions of the reflective portion of the visible light illumination, and also blocks the reflective portion of the SWIR illumination, A third portion that transmits the red portion of the reflected visible light illumination while blocking the blue and green portions of the reflective portion of the visible light illumination, and also blocks the reflective portion of the SWIR illumination, A fourth portion that blocks the green, blue, and red portions of the reflective portion of the visible light illumination and transmits the reflective portion of the SWIR illumination. The system according to claim 8, comprising:

11. The system according to claim 10, wherein the first, second, third, and fourth portions are arranged in a repeating 2x2 tile arrangement in the filter array.

12. The system according to claim 10, wherein the first, second, third, and fourth portions are arranged in a repeating 4x4 tile arrangement in the filter array.

13. The system according to claim 12, wherein each repeating 4x4 tile portion in the repeating 4x4 tile arrangement comprises twice the number of green transparent space portions as blue transparent space portions, red transparent space portions, or SWIR transparent space portions.

14. The system according to claim 12, wherein the 4x4 tile portion does not include any lateral or vertically adjacent blue translucent portion, any lateral or vertically adjacent red translucent portion, any lateral or vertically adjacent green translucent portion, or any lateral or vertically adjacent SWIR translucent portion.

15. The system according to claim 12, wherein the 4x4 tile arrangement comprises four 2x2 blocks, each 2x2 block comprising four spatial portions that transmit a single, distinct wavelength range.

16. The aforementioned 4x4 tile arrangement comprises four 2x2 blocks, The first 2x2 block comprises three red transparent space sub-parts and one SWIR transparent space sub-part, The second 2x2 block comprises three green permeable space sub-parts and one SWIR permeable space sub-part, The third 2x2 block comprises three blue-transparent space sub-parts and one SWIR-transparent space sub-part, The system according to claim 12, wherein the fourth 2x2 block comprises a red transparent space sub-part, a blue transparent space sub-part, a green transparent space sub-part, and a SWIR transparent space sub-part.

17. The aforementioned filter array is A first portion that transmits part or all of the reflective portion of the visible light illumination and blocks the reflective portion of the SWIR illumination, A second portion that transmits a first wavelength range of the reflective portion of the SWIR illumination, blocks a second wavelength range of the reflective portion of the SWIR illumination, and blocks the reflective portion of the visible light illumination, A third portion that transmits the second wavelength range of the reflective portion of the SWIR illumination, blocks the first wavelength range of the reflective portion of the SWIR illumination, and blocks the reflective portion of the visible light illumination. The system according to claim 8, comprising:

18. The system according to any one of claims 1 to 17, wherein the biological tissue comprises a first region having a first water content and a second region having a second water content lower than the first water content.

19. The system according to any one of claims 1 to 18, wherein the one or more optical sources comprises lasers.

20. The system according to any one of claims 1 to 19, wherein the one or more optical sources include light-emitting diodes.

21. The system according to any one of claims 1 to 20, wherein the sensor comprises one or more cameras, and the one or more cameras are selected from the following group: a silicon camera, an InGaAs camera, a black silicon camera, a germanium camera, a germanium-tin-on silicon camera, a quantum dot shortwave infrared camera, and a cadmium telluride mercury camera.

22. The SWIR illumination has a first polarization, according to any one of claims 1 to 21.

23. The system according to claim 22, wherein the reflective portion of the SWIR illumination sensed by the sensor has a second polarization opposite to the first polarization.

24. The system according to any one of claims 1-23, further comprising polarizers arranged between the biological tissue and the sensor.

25. The system according to any one of claims 1 to 24, wherein the biological tissue is free of contrast agent.

26. The system according to any one of claims 1 to 25, wherein the controller is configured to generate the at least one image without a reference light.

27. The system according to any one of claims 1 to 26, wherein the controller is configured to generate the at least one image without information from ambient light surrounding the sensor.

28. The system according to any one of claims 1 to 27, wherein the sensor comprises a stack of photosensors.

29. The stack of the aforementioned photosensor is A first photosensor configured to detect the blue portion of the reflected visible light illumination, A second photosensor configured to detect the green portion of the reflected visible light illumination, A third photosensor configured to detect the red portion of the reflected visible light illumination, A fourth photosensor configured to detect the reflective portion of the SWIR illumination, The system according to claim 28, comprising:

30. A method for imaging biological tissue, wherein the method is To provide short-wave infrared (SWIR) illumination and visible light illumination using one or more optical sources, The sensor detects the reflective portion of the SWIR illumination and the reflective portion of the visible light illumination, The controller communicating with the sensor receives first information from the sensor that corresponds to the reflective portion of the SWIR illumination, The controller receives second information from the sensor that corresponds to the reflective portion of the visible light illumination, The controller generates at least one image of the biological tissue using the first information and the second information, The controller outputs the at least one image to at least one of the display and / or memory. Methods that include...

31. A non-transient computer-readable storage medium for storing instructions for imaging biological tissue, wherein the instructions are executed by one or more processors of the system, To provide short-wave infrared (SWIR) illumination and visible light illumination using one or more optical sources, The sensor detects the reflective portion of the SWIR illumination and the reflective portion of the visible light illumination, The controller communicating with the sensor receives first information from the sensor that corresponds to the reflective portion of the SWIR illumination, The controller receives second information from the sensor that corresponds to the reflective portion of the visible light illumination, The controller generates at least one image of the biological tissue using the first information and the second information, The controller outputs the at least one image to at least one of the display and / or memory. A non-transient, computer-readable storage medium configured to perform the following action.

Images