Systems and methods for single sensor visible and SWIR imaging

Abstract

A system for imaging biological tissue may include one or more light sources configured to provide short wave infrared (SWIR) illumination and provide visible light illumination; a sensor configured to sense a reflected portion of the SWIR illumination and a reflected portion of the visible light illumination; and a controller in communication with the sensor. The controller may be configured to receive, from the sensor, first information corresponding to a reflected portion of the SWIR illumination, receive, from the sensor, second information corresponding to a reflected portion of the visible light illumination, generate at least one image of the biological tissue using the first information and the second information, and outputting the at least one image to at least one of a display and / or a memory.

Description

Cross Reference to Related Applications

[0001] This application claims priority to U.S. Provisional Application No. 63 / 495,023, filed April 7, 2023, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates generally to systems and methods for SWIR and visible light based medical imaging. BACKGROUND

[0003] The condition of biological tissue can provide insight into the health of the person or animal to which the tissue belongs. For example, the condition of a subject’s lymph node can indicate whether the subject has a disease or infection. For example, lymph node enlargement can be a sign of a bacterial infection, a viral infection, or cancer. Thus, determining the condition of a lymph node can be very useful for the diagnosis, prevention, and treatment of disease. SUMMARY

[0004] The condition of biological tissue can be examined by imaging the tissue. There are various imaging modalities for imaging tissue. Specialized imaging modalities can be used to image specific types of tissue. For example, lymphangiography can be used to image lymph nodes, which includes injecting a radiographic contrast agent into a patient and imaging the lymph nodes and lymphatic vessels by X-ray. However, lymphangiography is invasive, can cause significant discomfort, and requires the use of a radiographic contrast agent. Other more general imaging modalities, such as computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, and positron emission tomography (PET) can also be used to image specific types of tissue. While these imaging techniques can enable biological tissue to be identified and assessed with reasonable accuracy, they do not provide ideal contrast for viewing the target biological tissue unless certain contrast agents are injected. As a result, the images can show organs and tissues other than the tissue of interest with the same or better contrast compared to the target tissue, making it challenging to find and examine the target tissue.

[0005] As noted above, there are many imaging modalities for visualizing and examining lymph nodes and other biological tissue. However, known medical imaging techniques have several drawbacks, for example, they do not accurately and robustly identify target tissue and / or target features and their reliance on contrast agents. Accordingly, there is a need for improved medical imaging methods and systems.

[0006] Techniques for single-sensor multi-wavelength imaging of biological tissue are provided herein. The disclosed systems and methods can employ a single sensor to capture images of biological tissue in multiple wavelengths, such as to capture short-wave infrared (SWIR) images and visible light images of biological tissue. The captured images can be displayed, stored, and / or used to generate composite images. Images generated using the techniques can display target tissue (e.g., lymph nodes) and / or target features with high contrast, thereby facilitating identification and examination of the target tissue without the need to use contrast agents.

[0007] A system provided for imaging biological tissue can include one or more light sources configured to provide short-wave infrared (SWIR) illumination and to provide visible light illumination, a sensor configured to sense a reflected portion of the SWIR illumination and a reflected portion of the visible light illumination, and a controller in communication with the sensor. The controller can be configured to receive first information from the sensor corresponding to the reflected portion of the SWIR illumination and to receive second information from the sensor corresponding to the reflected portion of the visible light illumination. Using the first information and the second information, the controller can be configured to generate at least one image of the biological tissue. The controller can be configured to subsequently output the at least one image to at least one of a display and / or a memory.

[0008] The one or more light sources can include a laser or a light-emitting diode. The sensor can include one or more cameras selected from the group consisting of a silicon camera, an InGaAs camera, a black silicon camera, a germanium camera, a germanium-on-silicon tin camera, a quantum dot short-wave infrared camera, and a mercury cadmium telluride camera.

[0009] In some embodiments, the one or more light sources are configured to alternately provide the SWIR illumination and the visible light illumination according to a temporal pulsing scheme. In some embodiments, the one or more light sources include a first light source configured to provide the SWIR illumination and a second light source configured to provide the visible light illumination.

[0010] The system can further include a first optical filter configured to block at least some of the SWIR illumination and a second optical filter configured to block at least some of the visible light illumination. The system can include an optical filter support device configured to move at least one of the first optical filter and the second optical filter between a first position in an optical path of the system and a second position outside of the optical path of the system.

[0011] In some embodiments, one or more light sources are configured to simultaneously provide SWIR illumination and visible light illumination. The system may also include a filter array configured to spatially selectively block and transmit reflective portions of both SWIR illumination and visible light illumination. In some embodiments, the filter array includes a first portion that transmits some or all of the reflective portions of visible light illumination and blocks the reflective portions of SWIR illumination, and a second portion that transmits some or all of the reflective portions of SWIR illumination and blocks the reflective portions of visible light illumination.

[0012] In some embodiments, the filter array includes: a first portion that transmits a green portion of reflected visible light illumination while blocking blue and red portions of the reflected visible light illumination and blocking a portion of SWIR illumination; a second portion that transmits a blue portion of reflected visible light illumination while blocking green and red portions of the reflected visible light illumination and blocking a portion of SWIR illumination; a third portion that transmits a red portion of reflected visible light illumination while blocking blue and green portions of the reflected visible light illumination and blocking a portion of SWIR illumination; and a fourth portion that blocks the green, blue, and red portions of the reflected visible light illumination and transmits a portion of SWIR illumination. The first, second, third, and fourth portions may be arranged in repeating configurations within the filter array. In the block arrangement, or in the repeating arrangement in the filter array Block layout in progress.

[0013] If the first, second, third, and fourth parts are arranged in a repetitive manner... Block layout, then repetition. Each repeating element in the block layout The block portion may include a green transmission space portion that is twice the size of the blue transmission space portion, red transmission space portion, or SWIR transmission space portion it comprises. In some embodiments, The block portion does not include any horizontally or vertically adjacent blue transmissive portions, any horizontally or vertically adjacent red transmissive portions, any horizontally or vertically adjacent green transmissive portions, or any horizontally or vertically adjacent SWIR transmissive portions. In some embodiments, The block layout includes four Block, each The block comprises four spatial portions, each of which transmits a single corresponding wavelength range. In some embodiments, The block layout includes four Block: First The block includes three red transmissive spatial sub-sections and one SWIR transmissive spatial sub-section, the second The block includes three green transmissive spatial sub-sections and one SWIR transmissive spatial sub-section, the third The block includes three blue transmissive spatial sub-sections and one SWIR transmissive spatial sub-section, and the fourth The block includes a red transmissive spatial sub-section, a blue transmissive spatial sub-section, a green transmissive spatial sub-section, and a SWIR transmissive spatial sub-section. In some embodiments, the filter array includes: a first portion that transmits some or all of the reflected portion of the visible light illumination and blocks the reflected portion of the SWIR illumination; a second portion that transmits a first wavelength range of the reflected portion of the SWIR illumination, blocks a second wavelength range of the reflected portion of the SWIR illumination, and blocks the reflected portion of the visible light illumination; and a third portion that transmits the second wavelength range of the reflected portion of the SWIR illumination, blocks the first wavelength range of the reflected portion of the SWIR illumination, and blocks the reflected portion of the visible light illumination.

[0014] The SWIR illumination can have a first polarization. The reflected portion of the SWIR illumination sensed by the sensor can have a second polarization opposite the first polarization. A polarizer can be disposed between the biological tissue and the sensor.

[0015] The sensor can include a stack of photosensors. The stack of photosensors can include: a first photosensor configured to detect the reflected blue portion of the visible light illumination; a second photosensor configured to detect the reflected green portion of the visible light illumination; a third photosensor configured to detect the reflected red portion of the visible light illumination; and a fourth photosensor configured to detect the reflected portion of the SWIR illumination.

[0016] The biological tissue being imaged can include a first region having a first water content (e.g., a volumetric density of water) and a second region having a second water content lower than the first water content. The biological tissue can be free of contrast agent. The controller can be configured to generate at least one image without information from a reference light and / or without information from ambient light surrounding the sensor.

[0017] A method for imaging biological tissue can include providing, by one or more light sources, short wave infrared (SWIR) illumination and visible light illumination, sensing, by a sensor, a reflected portion of the SWIR illumination and a reflected portion of the visible light illumination, receiving, by a controller in communication with the sensor, first information from the sensor corresponding to the reflected portion of the SWIR illumination, receiving, by the controller from the sensor, second information corresponding to the reflected portion of the visible light illumination, generating, by the controller, at least one image of the biological tissue using the first information and the second information, and outputting, by the controller, the at least one image to at least one of a display and / or a memory.

[0018] A non-transitory computer-readable storage medium can store instructions for imaging biological tissue, the instructions configured to be executed by one or more processors of a system to cause the system to provide, by one or more light sources, short wave infrared (SWIR) illumination and visible light illumination, sense, by a sensor, a reflected portion of the SWIR illumination and a reflected portion of the visible light illumination, receive, by a controller in communication with the sensor, first information from the sensor corresponding to the reflected portion of the SWIR illumination, receive, by the controller from the sensor, second information corresponding to the reflected portion of the visible light illumination, generate, by the controller, at least one image of the biological tissue using the first information and the second information, and output, by the controller, the at least one image to at least one of a display and / or a memory. BRIEF DESCRIPTION OF DRAWINGS

[0019] The following drawings illustrate various systems and methods for single sensor visible and SWIR imaging. The systems and methods shown in the drawings can have any one or more of the features described herein.

[0020] Figure 1 An example system for single sensor visible and SWIR imaging is shown in accordance with some embodiments.

[0021] Figure 2 Another example system for single sensor visible and SWIR imaging is shown in accordance with some embodiments.

[0022] Figure 3 Another example system for single sensor visible and SWIR imaging is shown in accordance with some embodiments.

[0023] Figure 4A A filter array for a single sensor visible and SWIR imaging system is shown in accordance with some embodiments.

[0024] Figure 4B Another filter array for a single sensor visible and SWIR imaging system is shown in accordance with some embodiments.

[0025] Figure 4CAnother filter array for a single sensor visible and SWIR imaging system is shown in accordance with some embodiments.

[0026] Figure 4D Another filter array for a single sensor visible and SWIR imaging system is shown in accordance with some embodiments.

[0027] Figure 4E Another filter array for a single sensor visible and SWIR imaging system is shown in accordance with some embodiments.

[0028] Figure 5 A sensor stack for a single sensor visible and SWIR imaging system is shown in accordance with some embodiments.

[0029] Figure 6 Another example system for single sensor visible and SWIR imaging is shown in accordance with some embodiments.

[0030] Figure 7 A method for single sensor visible and SWIR imaging is shown in accordance with some embodiments.

[0031] Figure 8 A computer system is shown in accordance with some embodiments. DETAILED DESCRIPTION

[0032] Techniques are provided for single sensor, multi-wavelength imaging of biological tissue. The disclosed systems and methods can employ a single sensor to capture images of biological tissue in multiple wavelengths, such as to capture short wave infrared (SWIR) images and visible light images of biological tissue. The captured images can be displayed, stored, and / or used to generate composite images. The images produced using the techniques can show target tissue (e.g., lymph nodes) and / or target features with high contrast, facilitating identification and examination of the target tissue without the use of contrast agents, without the use of reference light, and without the need for information from ambient light surrounding the sensor.

[0033] Figure 1 An exemplary imaging system 100 for SWIR and visible light imaging of biological tissue is shown. The system 100 can include one or more light sources 102 configured to provide SWIR illumination and visible light illumination, one or more lenses (104, 106) for directing light to a biological tissue and for collecting light reflected by the biological tissue, and a sensor 108 for detecting light in a visible light wavelength range and in a SWIR wavelength range. The system 100 can be a component of a laparoscope.

[0034] The light source 102 can be implemented using any suitable light emitting device, such as one or more laser light sources, one or more LED light sources, or a combination thereof. In some embodiments, the light source 102 is a single light source that emits in the SWIR and visible light ranges. In other embodiments, the light source 102 includes a first light source that emits in the SWIR range and a second set of one or more light sources that emit in the visible light range. Visible light illumination can be provided by multiple light sources that emit different visible light wavelength ranges. For example, as shown in FIG. 3, visible light illumination can be provided by separate light sources that emit in the blue, green, and red visible light ranges. The SWIR illumination and visible light illumination can be delivered and incident on the biological tissue to be imaged. Figure 1

[0035] The light source 102 can provide SWIR illumination in a wavelength range of about 800-2600 nm, 800-1700 nm, 900-2000 nm, 1000-2600 nm, 1000-1700 nm, 1500-1700 nm, 1500-2600 nm, or in any sub-portion of any one or more of the wavelength ranges. For example, the light source 102 can provide SWIR illumination having a wavelength of about 800 nm, 850 nm, 900 nm, 950 nm, 1000 nm, 1050 nm, 1100 nm, 1150 nm, 1200 nm, 1250 nm, 1300 nm, 1350 nm, 1400 nm, 1450 nm, 1500 nm, 1550 nm, 1600 nm, 1650 nm, 1700 nm, 1750 nm, 1800 nm, 1850 nm, 1900 nm, 1950 nm, 2000 nm, 2050 nm, 2100 nm, 2150 nm, 2200 nm, 2250 nm, 2300 nm, 2350 nm, 2400 nm, 2450 nm, 2500 nm, 2550 nm, or 2600 nm. In some embodiments, the light source 102 provides SWIR illumination at multiple SWIR wavelengths, such as SWIR illumination having a wavelength of about 800 nm, SWIR illumination having a wavelength of about 1700 nm, and SWIR illumination having a wavelength of about 2600 nm.

[0036] ​The visible illumination can be provided by a light source in a wavelength range of about 380-750 nm, 450-625 nm, 485-590 nm, 500-565 nm, 450-750 nm, 500-750 nm, or 565-750 nm, or in any sub-portion of any one or more of the wavelength ranges. For example, the light source 102 can provide visible illumination at a wavelength of about 450 nm, 455 nm, 460 nm, 465 nm, 470 nm, 475 nm, 480 nm, 485 nm, 490 nm, 495 nm, 500 nm, 505 nm, 510 nm, 515 nm, 520 nm, 525 nm, 530 nm, 535 nm, 540 nm, 545 nm, 550 nm, 555 nm, 560 nm, 565 nm, 570 nm, 575 nm, 580 nm, 585 nm, 590 nm, 600 nm, 605 nm, 610 nm, 615 nm, 620 nm, 625 nm, 630 nm, 635 nm, 640 nm, 645 nm, 650 nm, 670 nm, 675 nm, 680 nm, 685 nm, 690 nm, 700 nm, 705 nm, 710 nm, 715 nm, 725 nm, 730 nm, 735 nm, 740 nm, 745 nm, or 750 nm. In some embodiments, the light source 102 provides visible illumination at multiple visible wavelengths, such as visible illumination at a wavelength between 450 nm and 485 nm, visible illumination at a wavelength between 500 nm and 565 nm, and visible illumination at a wavelength between 625 nm and 750 nm.

[0037] The rod lens 104 can deliver the SWIR illumination and the visible illumination provided by the light source 102 to the biological tissue to be imaged. The rod lens 104 can be a rod lens of a laparoscope, and the system 100 can be a component of that laparoscope. The light source 102 can transmit light to the rod lens 104 through a fiber coupler 110 or any other suitable optical coupler.

[0038] Light reflected by the biological tissue to be imaged can be collected by the rod lens 104 and subsequently directed by the imaging lens 106 to the VIS / SWIR sensor 108. The reflected light collected by the system 100 can include both a reflected portion of the SWIR illumination and a reflected portion of the visible illumination. The reflected portion of the SWIR illumination can be in any one or more of the same SWIR wavelength ranges described above for the SWIR illumination, or in any sub-portion thereof. Likewise, the reflected portion of the visible illumination can be in any one or more of the same visible wavelength ranges described above for the visible illumination, or in any sub-portion thereof.

[0039] The sensor 108 can be disposed at the proximal end of the laparoscope, behind the imaging lens 106 and the rod lens 104. The sensor 108 can detect some or all of the reflected SWIR light, and can detect some or all of the reflected visible light. The sensor 108 can be sensitive to SWIR illumination within any one or more of the same SWIR wavelength ranges described above for SWIR illumination, or within any sub-portion thereof, and can be sensitive to visible light illumination within any one or more of the same visible light wavelength ranges described above for visible light illumination, or within any sub-portion thereof. In some embodiments, the sensor 108 is monochromatic within the visible light wavelength range; in other embodiments, the sensor 108 is a color-sensitive sensor.

[0040] The sensor 108 can be any device or combination of devices configured to detect both SWIR light and visible light. In some embodiments, the sensor 108 includes one or more cameras, such as a silicon camera, an InGaAs camera, a black silicon camera, a germanium camera, a germanium-on-silicon tin camera, a quantum dot short-wave infrared camera, and a mercury cadmium telluride camera, or combinations thereof. In some embodiments, the sensor 108 includes one or more color filter arrays or one or more photosensor stacks.

[0041] The light source 102 and / or the sensor 108 can be electrically coupled to and controlled by one or more controllers 112. The controller 112 can be, for example, a computer system such as a laptop, a tablet, a desktop computer, or a microcontroller. When the sensor 108 detects reflected visible light and reflected SWIR light, information corresponding to the reflected visible light and the reflected SWIR light can be transmitted by the sensor 108 to the controller 112. The controller 112 can use the information received from the sensor 108 to generate one or more images or one or more videos. The images / videos can be separate images / videos corresponding to one or more wavebands (e.g., separate visible light and SWIR images) or composite images / videos. The controller 112 can then output the generated images and / or videos to a display (e.g., a computer monitor) and / or a memory (e.g., a memory of the controller 112).

[0042] Controller 112 can be configured to synchronize the functions between sensor 108 and light source 102, for example, by means of one or more electronic triggers 114 connecting light source 102 and sensor 108. In some embodiments, controller 112 can be configured to control light source 102 such that light source 102 simultaneously emits SWIR illumination and visible light illumination. In other embodiments, controller 112 can be configured to control light source 102 such that light source 102 simultaneously emits SWIR illumination and visible light illumination in a pulse timing scheme, such that visible light illumination is emitted in time pulses alternating with SWIR illumination. When using a pulse timing scheme, controller 112 can synchronize sensor 108 with pulses from light source 102 to capture SWIR image frames and visible light image frames. In some embodiments, white light images are captured under red, green, and blue illumination conditions; in other embodiments, images are captured under illumination conditions of a specific visible light color. The captured visible light and SWIR image frames can be used (e.g., by controller 112) to generate individual images or videos and / or to generate composite images or videos.

[0043] In embodiments where light source 102 provides continuous, simultaneous SWIR and visible light illumination, a filter wheel or other mechanical filter exchanger 216 can be used to move one or more filters into and out of the optical path of sensor 108, such as... Figure 2 As shown. A filter exchanger 216 can be positioned in front of the sensor 108, for example, between the imaging lens 106 and the sensor 108, and can be used to selectively block reflected light of different wavelengths. The controller 112 can control the movement of the filter exchanger 216 and can coordinate the function of the sensor 108 with the movement of the filter exchanger 216, thereby capturing SWIR image frames and visible light image frames. The captured frames can be used to generate individual images or videos and / or to generate composite images or videos.

[0044] The filter exchanger 216 can selectively position one or more of the following filters in the optical path of the sensor 108: a SWIR-transmitting filter that transmits SWIR light and blocks visible light; a visible light transmission filter that transmits visible light and blocks SWIR light; a red transmission filter that transmits red light and blocks blue, green, and SWIR light; a green transmission filter that transmits green light and blocks blue, red, and SWIR light; a blue transmission filter that transmits blue light and blocks green, red, and SWIR light; and a yellow transmission filter that transmits yellow light and blocks red, green, blue, and SWIR light.

[0045] Alternatively, or in addition to filter wheels or mechanical filter changers such as filter changer 216, in embodiments where light source 102 provides continuous, simultaneous SWIR and visible light illumination, color filter 318 can be used to select the wavelength of light allowed to be incident on sensor 108, such as... Figure 3 As shown.

[0046] The filter array 318 can be implemented using a visible light and SWIR filter mosaic. The filter mosaic allows for the simultaneous capture of both reflected SWIR illumination and reflected visible light illumination to create a spatial array of visible light (e.g., blue, green, and red) and SWIR pixels. Any spatial arrangement of any combination of visible light and SWIR color filters can constitute a filter mosaic. Example filter mosaics include (but are not limited to) RGB (red, green, blue) + SWIR filter mosaic, RYB (red, yellow, blue) + SWIR filter mosaic, CYGM (cyan, yellow, green, magenta) + SWIR filter mosaic, RGBE (red, green, blue, emerald green) filter mosaic, and RGBW (red, green, blue, white) + SWIR filter mosaic. In some embodiments, the filter mosaic may include multiple SWIR filters, each configured to transmit a different SWIR band (e.g., an RGB (red, green, blue) + SWIR1 + SWIR2 filter mosaic, wherein SWIR1 transmits SWIR light in a first SWIR band, and SWIR2 transmits SWIR light in a second SWIR band). Additionally, the filter mosaic may have any size; for example, the filter mosaic may be... Filter array, Filter array, Filter arrays, etc.

[0047] exist Figure 4A to Figure 4EFigures are provided that illustrate various example filter mosaics. As shown, in some embodiments, a filter array includes a first spatial portion that transmits some or all of a reflected portion of visible light illumination and blocks a reflected portion of SWIR illumination and a second spatial portion that transmits some or all of a reflected portion of SWIR illumination and blocks a reflected portion of visible light illumination. For example, a filter array can include a first portion that transmits a green portion of reflected visible light illumination while blocking blue and red portions of the reflected portion of visible light illumination and blocking the reflected portion of SWIR illumination, a second portion that transmits a blue portion of reflected visible light illumination while blocking green and red portions of the reflected portion of visible light illumination and blocking the reflected portion of SWIR illumination, a third portion that transmits a red portion of reflected visible light illumination while blocking blue and green portions of the reflected portion of visible light illumination and blocking the reflected portion of SWIR illumination, and a fourth portion that blocks green, blue, and red portions of the reflected portion of visible light illumination and transmits the reflected portion of SWIR illumination.

[0048] The first portion, the second portion, the third portion, and the fourth portion can be arranged in a repeating tile arrangement in the filter array (e.g., as shown in Figure 4A ). If the spatial portions are arranged in a repeating tile arrangement, each repeating Figure 4B to Figure 4E tile portion in the repeating tile arrangement can include twice as many green transmissive spatial portions as blue transmissive spatial portions, red transmissive spatial portions, or SWIR transmissive spatial portions that it includes (e.g., as shown in ). The tile portion can not include any laterally or vertically adjacent blue transmissive portions, any laterally or vertically adjacent red transmissive portions, any laterally or vertically adjacent green transmissive portions, or any laterally or vertically adjacent SWIR transmissive portions (e.g., as shown in Figure 4B ). In other embodiments, the tile arrangement includes four Figure 4B blocks, each block including four spatial portions that are transmissive for a single respective wavelength range (e.g., as shown in ). Figure 4C In other embodiments, the tile arrangement includes four blocks. A first

[0049] block includes four spatial portions that are transmissive for a first wavelength range, a second block includes four spatial portions that are transmissive for a second wavelength range, a third block includes four spatial portions that are transmissive for a third wavelength range, and a fourth block includes four spatial portions that are transmissive for a fourth wavelength range (e.g., as shown in . ​The block can include three red transmissive spatial sub-sections and one SWIR transmissive spatial sub-section. The second The block can include three green transmissive spatial sub-sections and one SWIR transmissive spatial sub-section. The third The block can include three blue transmissive spatial sub-sections and one SWIR transmissive spatial sub-section. The fourth The block can include a red transmissive spatial sub-section, a blue transmissive spatial sub-section, a green transmissive spatial sub-section, and a SWIR transmissive spatial sub-section (e.g., as shown in Figure 4D

[0050] In some embodiments, the filter array includes a first portion of the reflective portion that transmits some or all of the visible light illumination and blocks the reflective portion of the SWIR illumination, a first wavelength range of the reflective portion of the SWIR illumination, a second wavelength range of the reflective portion of the SWIR illumination and a second portion of the reflective portion that blocks the visible light illumination, and a second wavelength range of the reflective portion of the SWIR illumination, a first wavelength range of the reflective portion of the SWIR illumination and a third portion of the reflective portion that blocks the visible light illumination. For example, as shown in Figure 4E The filter array can include a first spatial portion that transmits red, a second spatial portion that transmits green, a third spatial portion that transmits blue, a fourth spatial portion that transmits a first wavelength range of the reflective portion of the SWIR illumination (“SWIR1”), a fifth spatial portion that transmits a second wavelength range of the reflective portion of the SWIR illumination (“SWIR2”), a sixth spatial portion that transmits a third wavelength range of the reflective portion of the SWIR illumination (“SWIR3”), and a seventh spatial portion that transmits a fourth wavelength range of the reflective portion of the SWIR illumination (“SWIR4”).

[0051] In some embodiments, a microlens array can be disposed between the filter array 318 and the sensor 108.

[0052] In some embodiments in which the light source 102 provides continuous simultaneous SWIR and visible light illumination, the sensor 108 can include a photodetector stack that includes two or more photodetector layers. For example, the sensor stack can be a blue + green + red + SWIR sensor stack, a blue + yellow + red + SWIR sensor stack, a cyan + yellow + green + magenta + SWIR sensor stack, a red + green + blue + cyan green sensor stack, a red + green + SWIR sensor stack, or a blue + green + SWIR sensor stack. Figure 5 ​An example sensor stack is shown. When a sensor stack is used instead of a filter array, the light throughput to sensor 108 may be higher; accordingly, embodiments of system 100 in which sensor 108 includes a sensor stack may be ideal for imaging biological tissues under low light conditions.

[0053] In some embodiments, the light emitted by the light source 102 is polarized. For example, SWIR illumination may have a first polarization, and the reflected portion of the SWIR illumination detected by the sensor may have a second polarization opposite to the first polarization. A cross-polarization imaging mode can be used. One or more polarizers 622 in the optical path of the system can polarize the illumination light and / or the reflected portion of the illumination light, for example, as... Figure 6 As shown.

[0054] exist Figure 7 An exemplary method 700 for imaging biological tissue using visible light and SWIR with a single sensor is provided. Method 700 can be performed using a system for imaging biological tissue using visible light and SWIR light, such as... Figure 1 to Figure 3 and Figure 6 Any embodiment of the system 100 shown. Biological tissue imaged using method 700 may include one or more different types of biological tissue and / or one or more different regions of biological tissue. In some embodiments, biological tissue may include lymphatic components.

[0055] In some embodiments, biological tissue may include one or more different types of biological tissue and / or one or more different regions of biological tissue with different water contents. Because water is absorbent in the SWIR range (e.g., about 1550 nm), imaging using SWIR illumination can effectively distinguish biological tissue with higher water content from biological tissue with lower water content. For example, lymph nodes with high water content can be distinguished from fat with low water content.

[0056] To image biological tissues, one or more light sources (e.g., Figure 1 to Figure 3 The light source 102 shown provides SWIR and visible light illumination to the biological tissue (step 702 of method 700). The SWIR and visible light illumination can be provided by a rod-shaped lens, such as a laparoscope (e.g., Figure 1 to Figure 3 and Figure 6 The lens (as shown in the rod-shaped lens 104) is then delivered to the biological tissue. A sensor (e.g., Figure 1 to Figure 3 and Figure 6 The sensor 108 shown is used to sense the reflective portions of SWIR illumination and visible light illumination (step 704 of method 700). The reflective portions of SWIR illumination and visible light illumination can be formed by one or more lenses (e.g.,Figure 1 to Figure 3 and Figure 6 corresponding to the reflected portion of the SWIR illumination and second information corresponding to the reflected portion of the visible light illumination to a controller in communication with the sensor (e.g., controller 112 shown in FIGS. 1-3) (steps 706-708 of method 700). Using the first information and the second information, the controller can generate at least one image of the biological tissue (step 710 of method 700). The controller can then output the at least one image to at least one of a display and / or a memory (step 712 of method 700). Figure 8 and Figure 8 corresponding to the reflected portion of the SWIR illumination and second information corresponding to the reflected portion of the visible light illumination to a controller in communication with the sensor (e.g., controller 112 shown in FIGS. 1-3) (steps 706-708 of method 700). Using the first information and the second information, the controller can generate at least one image of the biological tissue (step 710 of method 700). The controller can then output the at least one image to at least one of a display and / or a memory (step 712 of method 700).

[0057] ​ An exemplary computer system 800 that can be used to generate images and / or videos of biological tissue based on information received from sensors in a provided visible and SWIR imaging system (e.g., system 100) is shown. In other words, the computer system 800 can be used to implement a controller in a system (e.g., system 100) that uses visible and SWIR light to image biological tissue. The computer system 800 can be any suitable type of microprocessor-based device, such as a personal computer, a workstation, a server, or a handheld computing device (portable electronic device) such as a phone or tablet, or a special purpose device. As shown, the computer system 800 can include one or more processors 802, input devices 804, output devices 806, storage 808 that stores software 810, and communication devices 812. ​

[0058] The input devices 804 and the output devices 806 can be connected or integrated with the system 102. The input devices 804 can be any suitable device that provides input, such as a touchscreen, a keyboard or keypad, a mouse, or a voice recognition device. Likewise, the output devices 806 can be any suitable device that provides output, such as a display, a touchscreen, a haptic device, or a speaker.

[0059] The storage 808 can be any suitable device that provides storage, such as an electrical, magnetic, or optical memory, including RAM, cache, a hard disk drive, a removable storage disk, or other non-transitory computer readable medium. The communication devices 812 can include any suitable device that is capable of transmitting and receiving signals over a network, such as a network interface chip or device. The components of the computer system 800 can be connected in any suitable manner, such as via a physical bus or via a wireless network.

[0060] ​The processor 802 can be or include any suitable processor or combination of processors, including any of a central processing unit (CPU), a field programmable gate array (FPGA), and an application specific integrated circuit (ASIC), or any combination thereof. Software 810, which can be stored in the storage 808 and executed by the processor 802, can include, for example, a program to implement the functionality of the present disclosure. The software 810 can be stored and / or transmitted in any non-transitory computer-readable storage medium, including a storage device in the storage 808, for use by or in connection with an instruction execution system, apparatus, or device, that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of the present application, a computer-readable storage medium can be any medium, for example, the storage 808, that can contain or store the program for use by or in connection with the instruction execution system, apparatus, or device.

[0061] The software 810 can also be propagated within any transport medium, for example, those illustrated in the above, to facilitate the transfer of the software 810 from one place to another, for example, from one device to another, or from one device to a network. In the context of the present disclosure, a transport medium can be any medium that can communicate, propagate or transport programming for use by or in connection with an instruction execution system, apparatus, or device. The transport readable medium can include, but is not limited to, electrical, magnetic, optical, electromagnetic, or infrared wired or wireless propagation mediums.

[0062] The computer system 800 can be connected to a network, which can be any suitable type of interconnected communication system. The network can implement any appropriate communication protocol and can be secured by any appropriate security protocol. The network can include any appropriate arrangement of network links capable of effecting transmission and reception of network signals, such as wireless network connections, T1 or T3 lines, cable networks, DSL, or telephone lines.

[0063] The computer system 800 can implement any operating system suitable for operating on the network. The software 810 can be written in any suitable programming language, such as C, C++, Java, or Python. In various embodiments, application software embodying the functionality of the present disclosure can be deployed in different configurations, such as in a client / server arrangement or through a web browser as a web-based application or web service.

[0064] The foregoing description has been presented for the purpose of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. The selected and described embodiments are chosen and described for the best explanation of the principles of the technology and its practical applications. Other embodiments will be apparent to those of ordinary skill in the art from consideration of the principles described herein, with various modifications and variations being possible in light of this teaching. It is intended that the scope of the technology and various embodiments thereof disclosed herein should include all changes and modifications that fall within the scope of the claims.

[0065] As used herein, the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. References to "about" a value or parameter or "approximately" a value or parameter include (and describe) variations that are directed to that value or parameter per se. For example, description of "about X" includes description of "X". It is understood that aspects and variations of the present application described herein include "consisting" and / or "consisting essentially of aspects and variations.

[0066] When a range of values is provided, it is understood that each intervening value, to the upper and lower limits of that range is also encompassed within the scope of the present disclosure. When the stated range includes the upper or lower limit, ranges excluding either of those included limits are also included in the present disclosure.

[0067] While the present disclosure and examples have been fully described with reference to the accompanying drawings, it is to be noted that various changes and modifications will become apparent to those skilled in the art. Such changes and modifications are to be understood as being included within the scope of the present disclosure and examples as defined by the claims. Finally, all publications and patent documents cited herein are incorporated-by-reference in their entirety.

[0068] Any of the systems, methods, techniques, and / or features disclosed herein can be combined in whole or in part with any one or more of the other systems, methods, techniques, and / or features disclosed herein.

Claims

1. A system for imaging biological tissues, the system comprising: One or more light sources are configured to provide short-wave infrared (SWIR) illumination and visible light illumination; A sensor is configured to sense the reflective portion of the SWIR illumination and the reflective portion of the visible light illumination; as well as The controller communicates with the sensor and is configured to: Receive first information from the sensor corresponding to the reflective portion of the SWIR illumination; Receive second information from the sensor corresponding to the reflective portion of the visible light illumination; At least one image of the biological tissue is generated using the first information and the second information; as well as The at least one image is output to at least one of a display and / or a memory.

2. The system of claim 1, wherein the one or more light sources are configured to alternately provide the SWIR illumination and the visible light illumination according to a time pulse scheme.

3. The system of claim 1 or 2, wherein the one or more light sources include a first light source and a second light source, the first light source being configured to provide the SWIR illumination and the second light source being configured to provide the visible light illumination.

4. The system according to any one of claims 1 to 3, wherein the one or more light sources include a first light source, a second light source, a third light source and a fourth light source, the first light source being configured to provide the SWIR illumination, the second light source being configured to provide red light illumination, the third light source being configured to provide green light illumination, and the fourth light source being configured to provide blue light illumination.

5. The system according to any one of claims 1 to 4, further comprising: A first filter is configured to block at least some of the SWIR illumination; as well as The second filter is configured to block at least some of the visible light illumination.

6. The system of claim 5, further comprising a filter support device configured to move at least one of the first filter and the second filter between a first position in the optical path of the system and a second position outside the optical path of the system.

7. The system of claim 1 or 2, wherein the one or more light sources are configured to simultaneously provide the SWIR illumination and the visible light illumination.

8. The system according to any one of claims 1 to 7 further includes a filter array configured to spatially selectively block and transmit the reflective portion of the SWIR illumination and the reflective portion of the visible light illumination.

9. The system of claim 8, wherein the filter array comprises a first portion and a second portion, the first portion transmitting some or all of the reflective portions of the visible light illumination and blocking the reflective portions of the SWIR illumination, and the second portion transmitting some or all of the reflective portions of the SWIR illumination and blocking the reflective portions of the visible light illumination.

10. The system of claim 8, wherein the filter array comprises: The first part transmits the green portion of the visible light illumination that is reflected, while blocking the blue and red portions of the reflected portion of the visible light illumination and blocking the reflected portion of the SWIR illumination; The second part transmits the blue portion of the visible light illumination that is reflected, while blocking the green and red portions of the reflected portion of the visible light illumination, and also blocking the reflected portion of the SWIR illumination; The third part transmits the red portion of the visible light illumination that is reflected, while blocking the blue and green portions of the reflected portion of the visible light illumination, and also blocks the reflected portion of the SWIR illumination; as well as The fourth part is the green, blue, and red portions of the reflective portion that block visible light illumination and transmit SWIR illumination.

11. The system of claim 10, wherein the first portion, the second portion, the third portion, and the fourth portion are arranged in a repeating pattern in the filter array. Block layout.

12. The system of claim 10, wherein the first portion, the second portion, the third portion, and the fourth portion are arranged in a repeating pattern in the filter array. Block layout.

13. The system of claim 12, wherein the repetitive Each repeating element in the block layout The green transmission space included in the block is twice the size of the blue transmission space, red transmission space, or SWIR transmission space.

14. The system of claim 12, wherein... The block portion does not include any horizontally or vertically adjacent blue transmissive portion, any horizontally or vertically adjacent red transmissive portion, any horizontally or vertically adjacent green transmissive portion, or any horizontally or vertically adjacent SWIR transmissive portion.

15. The system of claim 12, wherein... The block layout includes four Block, each The block comprises four spatial portions that transmit a single corresponding wavelength range.

16. The system according to claim 12, wherein: The The block layout includes four piece; First The block comprises three red transmission space sub-parts and one SWIR transmission space sub-part; second The block comprises three green transmission space sub-parts and one SWIR transmission space sub-part; third The block comprises three blue transmission space sub-parts and one SWIR transmission space sub-part; as well as fourth The block includes a red transmission space sub-part, a blue transmission space sub-part, a green transmission space sub-part, and a SWIR transmission space sub-part.

17. The system of claim 8, wherein the filter array comprises: The first part transmits some or all of the reflective portions of the visible light illumination and blocks the reflective portions of the SWIR illumination. The second part transmits a first wavelength range of the reflective portion of the SWIR illumination, blocks a second wavelength range of the reflective portion of the SWIR illumination, and blocks the reflective portion of the visible light illumination; as well as The third part transmits the second wavelength range of the reflective portion of the SWIR illumination, blocks the first wavelength range of the reflective portion of the SWIR illumination, and blocks the reflective portion of the visible light illumination.

18. The system according to any one of claims 1 to 17, wherein the biological tissue comprises a first region having a first water content and a second region having a second water content lower than the first water content.

19. The system according to any one of claims 1 to 18, wherein the one or more light sources comprise a laser.

20. The system according to any one of claims 1 to 19, wherein the one or more light sources comprise light-emitting diodes.

21. The system according to any one of claims 1 to 20, wherein the sensor comprises one or more cameras selected from the group consisting of silicon cameras, InGaAs cameras, black silicon cameras, germanium cameras, germanium-tin on silicon cameras, quantum dot shortwave infrared cameras, and mercury cadmium telluride cameras.

22. The system according to any one of claims 1 to 21, wherein the SWIR illumination has a first polarization.

23. The system of claim 22, wherein the reflective portion of the SWIR illumination sensed by the sensor has a second polarization opposite to the first polarization.

24. The system according to any one of claims 1 to 23, comprising a polarizer disposed between the biological tissue and the sensor.

25. The system according to any one of claims 1 to 24, wherein the biological tissue is free of contrast agent.

26. The system according to any one of claims 1 to 25, wherein the controller is configured to generate the at least one image without reference light.

27. The system according to any one of claims 1 to 26, wherein the controller is configured to generate the at least one image in the absence of information from ambient light around the sensor.

28. The system according to any one of claims 1 to 27, wherein the sensor comprises a cluster of photoelectric sensors.

29. The system of claim 28, wherein the stack of photoelectric sensors comprises: A first photoelectric sensor is configured to detect the blue portion of the visible light illumination reflected from the light. A second photoelectric sensor is configured to detect the green portion of the visible light illumination reflected from the light. A third photoelectric sensor is configured to detect the red portion of the visible light illumination reflected from the light. as well as A fourth photoelectric sensor is configured to detect the reflective portion of the SWIR illumination.

30. A method for imaging biological tissue, the method comprising: Short-wave infrared (SWIR) illumination and visible light illumination are provided by one or more light sources; The sensor detects the reflected portion of the SWIR illumination and the reflected portion of the visible light illumination; A controller that communicates with the sensor receives first information from the sensor corresponding to the reflective portion of the SWIR illumination; The controller receives second information from the sensor corresponding to the reflective portion of the visible light illumination; The controller uses the first information and the second information to generate at least one image of the biological tissue; as well as The controller outputs the at least one image to at least one of the display and / or memory.

31. A non-transitory computer-readable storage medium storing instructions for imaging biological tissue, the instructions being configured to be executed by one or more processors of a system to cause the system to: Short-wave infrared (SWIR) illumination and visible light illumination are provided by one or more light sources; The sensor detects the reflected portion of the SWIR illumination and the reflected portion of the visible light illumination; A controller that communicates with the sensor receives first information from the sensor corresponding to the reflective portion of the SWIR illumination; The controller receives second information from the sensor corresponding to the reflective portion of the visible light illumination; The controller uses the first information and the second information to generate at least one image of the biological tissue; as well as The controller outputs the at least one image to at least one of the display and / or memory.

Images