Double-magnification fluorescent imaging camera

Abstract

The invention discloses a dual magnification fluorescent imaging camera. Video camera devices and systems for white light and fluorescent imaging perform imaging of both fluorescent light and non-fluorescent light collected from an illuminated scene. A beam splitter downstream of the optical subsystem separates light for a non-fluorescent image into a first imaging channel and light for a fluorescent image into a second imaging channel. A meniscus lens is positioned downstream of the beam splitter in one of the first imaging channel and the second imaging channel and is used to vary magnification such that magnification of the first imaging channel differs from magnification of the second imaging channel. One or more image sensors downstream of the beam splitter capture the fluorescent image light and the non-fluorescent image light as a fluorescent image and a non-fluorescent image. The fluorescent image has an image size that is smaller in area on the one or more sensors than an image size of the non-fluorescent image.

Description

Technical Field

[0001] The present disclosure relates to systems for collecting reflected and / or scattered light from a scene in addition to fluorescent emitted light. The present disclosure more particularly relates to an imaging camera head connected to an endoscope for performing both visible light and fluorescence imaging (FI). Background Art

[0002] Scopes for use in medical and / or industrial applications, such as endoscopes, exoscopes, and borescopes, can utilize both white light imaging and fluorescence imaging (FI) simultaneously. Endoscopic instruments intended for industrial use rather than medical use are often referred to as borescopes. Since the present invention relates to both medical endoscopes and exoscopes, as well as borescopes, the term "endoscope" is used to include all of these instruments as a whole. Conventional endoscopes are capable of capturing both white light and fluorescence images with a single optical path and only a single image sensor, by alternating collection of frames under staggered white light and excitation illumination. Such system arrangements result in frame rates significantly lower than those of systems using only visible white light, typically reaching (at most) half the frame rate of white light-only systems. The low frame rate results in unwanted motion blur. In addition to the necessary shutter between white light and fluorescence frames, the sensitivity of the fluorescence frame is also limited by the shared optical path, which typically results in one signal not being able to attenuate relative to the other. Image processing then analyzes the images from the alternating frames and merges them into a fluorescence / white light overlay at a low overall frame rate. This technique results in lower sensitivity in the fluorescence range, as well as reduced brightness and resolution of the superimposed image compared to white light images only.

[0003] Fluorescence images (produced by fluorescent light emitted from the scene - usually the result of fluorescent molecules being exposed to excitation light) are typically much weaker than the corresponding white light (visible spectrum) images, which makes simultaneous imaging on a single camera chip difficult. In addition, in a single chip system, it is difficult to focus the fluorescence image independently of the white light image. Chromatic aberrations in some endoscopes make it impossible to focus the white light image and the infrared fluorescent image simultaneously. While this can be corrected by a fixed focus offset in the camera head for a single endoscope, it is often the case that different models of endoscopes have different offsets. In addition, the fluorescence image may be emitted from a different object plane in object space than the white light image, such as when the fluorescence is generated in blood vessels below the surface of the skin and the white light image is from light reflected and scattered from the surface of the skin.

[0004] US2020 / 0088579A1 discloses a hybrid spectral imager, which includes a multi-band filtering optical device having a beam splitter device for generating at least two replica images of a target image, and an adjustable multi-band filtering device inserted into the imaging path and performing adjustable multi-band pass filtering in the image replica. The beam splitter can be a pentaprism, wherein one of the five surfaces of the pentaprism is coated with a polychromatic mirror substrate, and this coated surface is bonded to a triangular prism so that the rear surface of the prism is parallel to the front surface of the pentaprism. In addition, the pentaprism also includes a tilt actuator, which is used to adjust the tilt angle of the prism assembly and thus adjust the direction of the imaging beam. The image of the target object is focused by the objective optical device and split by the tilted prism assembly onto two image sensors, wherein, in each tilt step, a set of new images is simultaneously obtained in snapshot mode, so that, in scanning mode, the entire spectrum can be scanned to create a complete spectral cube for the target object. Therefore, in order to accommodate sensor arrays of various sizes, the path lengths of the two separated beams must be substantially equal. Therefore, the placement of this hybrid spectral imager in the narrow axis of a COTT endoscope is limited.

[0005] In US2014 / 0225992A1, a minimally invasive surgical system is described, which has an image capture unit including a prism assembly and a sensor assembly. The image capture unit includes a shared lens assembly followed by a sensor assembly, wherein the sensor assembly includes a prism assembly, a reflection unit, and a coplanar image capture sensor. Thus, two image capture sensors with similar image sizes are symmetrical across a plane intersecting the longitudinal axis of the stereo endoscope, and only a small gap is arranged between the corresponding surface of the prism assembly and each coplanar image capture sensor.

[0006] In US2021 / 0307613A1, a fluorescence imaging system for imaging an object includes a white light emitter, an excitation light emitter having multiple excitation bands of light for causing the object to emit fluorescence, a component for guiding the white light and the excitation light to the object and collecting the reflected white light and the emitted fluorescence light from the object, a filter that blocks the light of the excitation band and transmits at least a portion of the reflected white light and the fluorescence light, and an image sensor assembly that receives the transmitted reflected white light and the fluorescence light. Various prisms and reflection arrangements are described to guide the fluorescence light and the white light along different optical paths. Summary of the invention

[0007] It is an object of the present invention to provide improved endoscopic dual image acquisition and to allow dual imaging applications using a variety of existing endoscopes. It is another object of the present invention to provide improved compatibility of a video camera device for dual image acquisition with a variety of endoscopic devices, wherein the dual images have video streams with different characteristics, such as different collected light spectra, different image focal planes or focal depths, or different light intensities. It is another object of the present invention to improve such compatibility while providing improved signal-to-noise characteristics.

[0008] According to a first aspect of the present invention, a video camera device for white light and fluorescence imaging is provided, and the device includes a first optical subsystem, a beam splitter, a meniscus lens, and one or more image sensors. The first optical subsystem transmits reflected light and scattered light for non-fluorescent images and transmits emitted fluorescent light for fluorescent images, both of which are collected from an illuminated scene. The beam splitter is positioned downstream of the first optical subsystem and separates the light for the non-fluorescent image into a first imaging channel and the light for the fluorescent image into a second imaging channel. The meniscus lens is positioned downstream of the beam splitter in one of the first imaging channel and the second imaging channel and is used to change the magnification so that the magnification of the first image channel is different from the magnification of the second imaging channel. One or more image sensors are positioned downstream of the beam splitter to receive the fluorescent light and the non-fluorescent light. The one or more image sensors are configured to capture the fluorescent image light and the non-fluorescent image light as fluorescent images and non-fluorescent images, and wherein the fluorescent image has an image size with a smaller area on the one or more sensors than the image size of the non-fluorescent image.

[0009] According to some embodiments of the first aspect, light from a first imaging channel is imaged onto a first image position of one or more image sensors, and light from a second image channel is imaged onto a second image position of the one or more image sensors that is different from the first image position. A processor may be included that communicates with the one or more image sensors to receive captured fluorescent images and non-fluorescent images from the first imaging channel and the second imaging channel, the processor being configured to digitally rescale the fluorescent image to match the size of the non-fluorescent image and overlay the fluorescent image onto the non-fluorescent image for display. The first image position and the second image position may be different and separate areas of a single focal plane array image sensor, or may be on two separate focal plane array image sensors.

[0010] According to some embodiments of the first aspect, the meniscus lens is a singlet.

[0011] According to some embodiments of the first aspect, the meniscus lens is a doublet having an overall meniscus shape suitable for correcting chromatic aberration in white light.

[0012] According to some embodiments of the first aspect, the magnification of the second imaging channel is 0.5 to 0.9 times the magnification of the first imaging channel. A meniscus lens can be used to reduce the magnification of the second imaging channel.

[0013] According to some embodiments of the first aspect, the meniscus lens has a first radius r1 and a second radius r2, wherein a magnification m equal to r2 / r1 is between about 0.56 and 0.88.

[0014] According to some embodiments of the first aspect, the beam splitter reflects light of a wavelength corresponding to the fluorescent channel and transmits light of a wavelength corresponding to the non-fluorescent channel.

[0015] According to some embodiments of the first aspect, the beam splitter transmits light of a wavelength corresponding to a fluorescent channel and reflects light of a wavelength corresponding to a non-fluorescent channel.

[0016] According to some embodiments of the first aspect, the beam splitter reflects light of ICG / OTL fluorescence having a wavelength of approximately 800-850 nm and transmits light of shorter wavelengths.

[0017] According to some embodiments of the first aspect, the beam splitter transmits light of ICG / OTL fluorescence having a wavelength of approximately 800-850 nm and reflects light of shorter wavelengths.

[0018] According to some embodiments of the first aspect, the beam splitter reflects light of Cy5 or Cy5.5 fluorescence having a wavelength of approximately 650-725 nm and transmits light of shorter wavelengths.

[0019] According to some embodiments of the first aspect, the beam splitter transmits light of Cy5 or Cy5.5 fluorescence having a wavelength of approximately 650-725 nm and reflects light of shorter wavelengths.

[0020] According to some embodiments of the first aspect, the beam splitter reflects light of fluorescein fluorescence having a wavelength of approximately 500-550 nm and transmits light of other wavelengths.

[0021] According to some embodiments of the first aspect, the beam splitter transmits light of fluorescein fluorescence having a wavelength of approximately 500-550 nm and reflects light of other wavelengths.

[0022] According to some embodiments of the first aspect, the video camera device also includes a device for focusing one of the first imaging channel and the second imaging channel differently than the other. The meniscus lens can be afocal. The focusing device can include a mechanism for moving the meniscus lens. The movement of the meniscus lens can be performed in such a way that it does not change the magnification of the image light passing therethrough. The focusing device can include a mechanism for moving the image sensor corresponding to one of the imaging channels. The focusing device can include a liquid lens with adjustable deformation. A processor can be included, which is operable to control the focusing device and the processor executes an autofocus routine to automatically adjust the focus of one of the imaging channels.

[0023] According to a second aspect of the present invention, there is provided an optical imaging system comprising the video camera device of the first aspect and a system controller, the system controller being configured to provide image processing and display control functions and to perform functions such as rescaling the downscaled FI image or the upscaled white light image provided by the video camera device so that the two images are the same or nearly the same size. The system controller may also control focus adjustment by operating an autofocus algorithm to perform focus adjustment according to white light imaging or fluorescent imaging and send appropriate control signals to the video camera device.

[0024] These and other features of the present invention will be apparent from the following description of the preferred embodiments (taken in conjunction with the accompanying drawings). BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present invention is further explained by the following exemplary description of specific embodiments.

[0026] Figure 1 is a block diagram of a medical imaging device 100 according to an example embodiment of the present invention;

[0027] Figure 2 is a partial cross-sectional view of a camera head 101 showing an optical assembly configuration according to an example embodiment;

[0028] Figure 3 is a partial cross-sectional view of a camera head 101 showing an optical assembly configuration according to another example embodiment using a single image sensor;

[0029] Figure 4 is a partial cross-sectional view of a camera head 101 showing an optical assembly configuration according to another example embodiment using a single image sensor oriented at right angles to the primary optical axis;

[0030] Figure 5is a partial cross-sectional view of a camera head 101 showing an optical assembly configuration according to another example embodiment providing magnification in an imaging channel for white light using a single image sensor;

[0031] Figure 6 is a partial cross-sectional view of a camera head 101 showing an optical assembly configuration according to an example embodiment employing a liquid lens 602 in one channel for focus adjustment;

[0032] Figure 7 shows a cross-sectional view of a meniscus lens 700 according to an example embodiment;

[0033] Figure 8 shows a cross-sectional view of a meniscus lens 800 according to another example embodiment; and

[0034] Fig. 9 is a hardware block diagram of a system including an example image capture device according to an example embodiment of the present invention. DETAILED DESCRIPTION

[0035] Figure 1 1 is a block diagram of a medical imaging device 100 according to an exemplary embodiment of the present invention. The medical imaging device 100 ("device 100") includes a camera head 101, which can enable an endoscope 102 to be attached via connectors 108 and 109. In some embodiments, the endoscope 102 and the camera head 101 can be integrated into a single housing without the need for connectors. In this embodiment, the connectors 108 and 109 constitute what is commonly known as a "pawl coupling" or dock-clutch coupling, including a clutch that couples two components, at least one or both of which are rotatable. Preferably, the claw (109) of the claw coupling is designed so that the eyepiece cup (108) is pushed toward the interface portion to engage the connection. The connectors 108 and 109 can be any suitable connector that allows light to pass from the endoscope 102 to the camera head 101. Various structural components that support the depicted elements, as well as other components known in the art, such as illumination sources and controllers, are omitted from the figures herein and are not shown to avoid obscuring relevant details of the exemplary embodiments of the present invention.

[0036] The camera head 101 includes an optical subsystem 110 that transmits light for white light images or other non-fluorescent images, and transmits light for images created by fluorescence in the illuminated scene. The optical subsystem 110 is positioned at or behind the central window of the connector 212 to receive and condition optical image light from the endoscope 102. As further described below, the optical subsystem 110 typically includes a plurality of lenses for focusing both white light and fluorescent light. Many suitable lenses and combinations of lenses can be used for the optical subsystem 110.

[0037] Beam splitter 114 is positioned downstream of the first optical subsystem and separates the light for the non-fluorescent image into a first imaging channel and the light for the fluorescent image into a second imaging channel. In this embodiment, the first imaging channel is along an optical path directed to image sensor 214 and the second imaging channel is along an optical path directed to image sensor 216. In other embodiments, the two imaging channels may be directed to separate areas of a single image sensor. The sensor signal containing the two images is shown in FIG. Figure 6 and Figure 8 The images are processed as further described to provide a combined image.

[0038] Meniscus lens 115 is positioned downstream of the beam splitter in the second imaging channel and is used to change the magnification of the second imaging channel relative to the first imaging channel. In this embodiment, a moving mechanism 116 is mechanically attached to the meniscus lens 115 for moving the lens along the second imaging channel to adjust the focus of the second imaging channel. Figure 7 and Figure 8 As further discussed, focus adjustment does not modify the magnification changes performed by the lens, making focus independent of magnification level.

[0039] In some embodiments, as depicted on the left side of the block diagram, the system 100 includes an endoscope 102. The depicted endoscope is merely an example, and many endoscope designs are suitable, including rigid endoscopes and flexible endoscopes. The endoscope 102 includes a cover glass 202 at its distal tip, which in this version faces directly along the longitudinal axis of the endoscope 102, but may also be positioned at an angle relative to the longitudinal axis as is known in the art. Behind or proximal to the cover glass 202, a preferred location for an objective lens 204 is shown, which is disposed against or very close to the cover glass 202 and is preferably assembled with the cover glass in construction. Although a wide-angle lens is preferred for the objective lens 204, this is not limiting and any suitable lens may be used in various embodiments. The objective lens 204 may be part of an objective lens group 104 that may include one or more additional lenses 103. The specific number and arrangement of lenses in the endoscope 102 will vary greatly depending on the application. A flexible fiber bundle or a series of one or more rod lens pairs 107 are optically arranged or attached proximal to the objective lens 204 or the objective lens group 104, which are used to pass light to the endoscope 102 in the proximal direction. For embodiments with a flexible shaft, a flexible fiber bundle is used. For a rigid or semi-rigid shaft, one or more rod lens pairs 107 are used, which can be separated by spacers or other lenses in any suitable manner known in the art. For embodiments with a semi-rigid shaft, there are flexible joints between the rod lenses at one or more places along the shaft, while the shaft is rigid along the portion containing the rod lenses.

[0040] Figure 2 1 is a partial cross-sectional view of a camera head 101 showing an optical assembly configuration according to an example embodiment. The cross-sectional view includes a ray diagram showing image light 302, which passes through the assembly along a central optical axis 201 and is then directed by a beam splitter 114 toward two separate image sensors 214 and 216. The depicted optical elements are diagrammatic only and are not drawn to scale. The depicted optical assembly can be employed with endoscopic devices and systems having an integrated camera or an external, detachable camera head.

[0041] As shown, in this embodiment, image light 302 enters the optical assembly at cover glass 212. Image light 302 then passes downstream to optical subsystem 110, which in this embodiment includes lens pair 111, lens pair 112, and lens pair 113. Lens pair 111 includes a biconcave lens followed by a biconvex lens, which together have a slight positive or negative refractive power. Lens pair 112 includes a biconcave lens followed by a biconvex lens, which have a total positive refractive power. Lens pair 113 includes a biconvex lens and a concave-convex lens, which together have a positive refractive power and are used to focus image light 302. These three lens pairs are designed to focus and align image light 302 along a central optical axis toward beam splitter 114. In various embodiments, various other lens combinations may also be used to achieve the same effect.

[0042] In this embodiment, the beam splitter 114 is composed of two prisms 114-1 and 114-2, wherein along their adjacent surfaces, a suitable dichroic beam splitting layer is formed by a partially reflective coating, through which the image light is separated, wherein the first portion passes along a first optical path shown by the image light 303, and the second portion is reflected downward along a second optical path shown by the imaging light 304. In this embodiment, the partially reflective coating reflects fluorescent light of a desired wavelength band. As depicted, an internal reflective surface is provided at the edge depicted on the left side of the prism 114-2 to reflect the image light. As discussed above, the first and second portions of light include different spectral contents, wherein white light passes along the optical path 303, and fluorescent light passes along the optical path 304. A dichroic beam splitting layer is arranged at the interface between the two prisms 114-1 and 114-2. Other suitable layers, such as filter layers, may also be included in the prisms. For example, a white light filter layer may be provided at the proximal surface of prism 114-2 (where image light 304 exits prism 114-2) and a fluorescent light filter surface may be included at the proximal surface of prism 114-1 (where image light 303 exits prism 114-1).

[0043] In this embodiment, the meniscus lens 115 is positioned downstream of the beam splitter in the second imaging channel along the path of the image light 304 and centered along the 60 degree angle of the prism. The meniscus lens 115 is used to reduce the magnification of the second imaging channel relative to the first imaging channel. Figure 7 and Figure 8 The meniscus lens is constructed as described.

[0044] A first image sensor 214 is positioned downstream of the cover glass or protective layer 213, wherein the image sensor is positioned perpendicular to the longitudinal axis of the camera head 101, and a second image sensor 216 is positioned downstream of the cover glass or protective layer 215, wherein the image sensor is oriented at a 60 degree angle relative to the longitudinal axis.

[0045] As described above, a movement mechanism may be included for moving the meniscus lens 115 along the second imaging channel to adjust the focus of the second imaging channel. Such focus adjustment does not modify the magnification change performed by the lens 115, thereby making the focus independent of the magnification level. By using a mechanical adjustment mechanism, the focus adjustment can be performed on either imaging channel by also moving the corresponding image sensor along the corresponding optical path. The meniscus lens arranged as depicted allows the imaging device to improve the SNR (signal-to-noise ratio) of the fluorescent image by effectively concentrating the image intensity to a smaller area of ​​the image sensor, and provides independent focusing of the white light channel and the fluorescent channel. Due to the reduction effect of the meniscus lens 115, the image size of the image captured by the second image sensor 216 can be reduced, thereby increasing the intensity of the image relative to the image size on the second image sensor 216 for fluorescent light, thereby improving the sensitivity of the image sensor 216 to the generated fluorescent image by increasing the signal-to-noise ratio within the image area. At the same time, the reduction effect provided by the meniscus lens 115 can be used to suppress aberrations in the image captured by the second image sensor 216.

[0046] Figure 3 is a partial cross-sectional view of a camera head 101 showing an optical assembly configuration according to another example embodiment, wherein a single image sensor is used to capture both white light and FI images. The cross-sectional view includes a ray diagram showing image light 302, which passes through the assembly along a central optical axis 201 and is then directed by a beam splitter 114 toward a single image sensor 214. The depicted optical elements are diagrammatic only and are not drawn to scale. The depicted optical assembly may be employed with endoscopic devices and systems having an integrated camera or an external, detachable camera head.

[0047] As shown, in this embodiment, image light 302 enters the optical assembly at cover glass 212. Image light 302 then passes downstream to optical subsystem 110, which in this embodiment includes lens pair 111, lens pair 112, and lens pair 113. As described above with respect to Figure 2 The optical subsystem 110 is configured as described.

[0048] In this embodiment, beam splitter 114 is composed of three right angle prisms 114-1, 114-2 and 114-3, wherein along the adjacent interfaces of prisms 114-1 and 114-2, a suitable dichroic beam splitting layer is formed by a partially reflective coating. The image light is separated, wherein a first portion is reflected along a first optical path shown by image light 303, and a second portion passes along a second optical path shown by imaging light 304. In this embodiment, the partially reflective coating reflects a white light spectral band. As depicted, an internal reflective surface is provided at the edge depicted on the left side of prism 114-3 to reflect the image light within prism 114-3. As discussed above, the first and second portions of light include different spectral contents, wherein white light passes along optical path 303, and fluorescent light passes along optical path 304. Other suitable layers, such as filter layers, may also be included in the prism. For example, a fluorescent light filtering layer may be provided at the proximal surface of prism 114-2, or at the proximal surface of prism 114-3, and a white light filtering surface may be included at the proximal surface of prism 114-1.

[0049] In this embodiment, the meniscus lens 115 is positioned downstream of the beam splitter in the second imaging channel along the path of the image light 304 and is centered along the primary optical axis 201. The meniscus lens 115 is used to reduce the magnification of the second imaging channel relative to the first imaging channel. Figure 7 and Figure 8 The meniscus lens is constructed as described.

[0050] Image sensor 214 is positioned downstream of cover glass or protective layer 213, wherein the image sensor is placed perpendicular to the longitudinal axis of camera head 101 and is positioned offset from axis 201 to allow detection of image light 303 and image light 304 at two separate regions of image sensor 214, wherein image light 304 is detected in a region centered along primary optical axis 201, and image light 303 is detected in a region offset from the primary axis along the first optical path. It should be noted that while image sensor 214 is a single image sensor in a preferred embodiment, in other embodiments, two sensors may be employed in order to utilize two sensors located in a common plane and thereby simplify manufacturing geometry, including the ability to mount both sensors onto a single circuit board.

[0051] As described above, a moving mechanism may be included for moving the meniscus lens 115 along the second imaging channel to adjust the focus of the second imaging channel. Such focus adjustment does not change the magnification change performed by the lens, thereby making the focus independent of the magnification level. The meniscus lens arranged as depicted allows the imaging device to improve the SNR (signal-to-noise ratio) of the fluorescence image and provide independent focusing of the white light channel and the fluorescence channel. The reduction effect produced by the meniscus lens 115 provides a reduced image size of the image captured by the image sensor 214, thereby increasing the intensity of the image relative to the image size for the FI channel on the image sensor 214. The signal-to-noise ratio of the area of ​​the image sensor 214 dedicated to the fluorescence image is also improved. At the same time, the reduction effect provided by the meniscus lens 115 can be used to suppress aberrations in the image captured by the image sensor 214.

[0052] Figure 4 1 is a partial cross-sectional view of a camera head 101 showing an optical assembly configuration according to another example embodiment using a single image sensor oriented at right angles to the main optical axis. The cross-sectional view includes a ray diagram showing image light 302, which passes through the assembly along the central optical axis 201 and is then directed by the beam splitter 114 toward the single image sensor 214. The depicted optical elements are diagrammatic only and are not drawn to scale. The depicted optical assembly can be employed with endoscopic devices and systems having an integrated camera or an external, detachable camera head.

[0053] In this embodiment, image light 302 enters the optical assembly at cover glass 212. Image light 302 then passes downstream to optical subsystem 110, which in this embodiment includes lens pair 111, lens pair 112, and lens pair 113. Figure 2 The optical subsystem 110 is configured as described.

[0054] In this embodiment, beam splitter 114 is composed of three right angle prisms 114-1, 114-2 and 114-3, wherein along the adjacent interfaces of prisms 114-1 and 114-2, a suitable dichroic beam splitting layer is formed by a partially reflective coating. The image light is separated, wherein a first portion passes along a first optical path shown by image light 303, and a second portion is reflected along a second optical path shown by imaging light 304. As depicted, an internal reflective surface is provided at the edge depicted on the right side of prism 114-3 to reflect the first image light. As discussed above, the first and second portions of light include different spectral contents, wherein white light passes along optical path 303, and fluorescent light passes along optical path 304. Other suitable layers, such as filter layers, may also be included in the prism. For example, a white light filter layer may be provided at the proximal surface of prism 114-2 (where image light 304 leaves prism 114-2) and a fluorescent light filter surface may be included at the proximal surface of prism 114-1 (where image light 303 leaves prism 114-1) or at the proximal surface of prism 114-3.

[0055] In this embodiment, a meniscus lens 115 is positioned downstream of the beam splitter in the second imaging channel along the path of the image light 304. The meniscus lens 115 is used to reduce the magnification of the second imaging channel relative to the first imaging channel. Figure 7 and Figure 8 The meniscus lens is constructed as described.

[0056] Image sensor 214 is positioned downstream of cover glass or protective layer 213 , wherein the image sensor is positioned parallel to the longitudinal axis of camera head 101 .

[0057] As described above, a movement mechanism may be included for moving the meniscus lens 115 along the second imaging channel to adjust the focus of the second imaging channel. Such focus adjustment does not alter the magnification changes performed by the lens, thereby making the focus independent of the magnification level. As described above, the demagnification effect of the meniscus lens 115 improves the sensitivity of the image on the sensor 214 for the resulting fluorescence image by improving the signal-to-noise ratio and allows for correction of chromatic aberration in the second imaging channel.

[0058] Figure 5 1 is a partial cross-sectional view of a camera head 101 showing an optical assembly configuration according to another example embodiment providing magnification in an imaging channel for white light using a single image sensor. The cross-sectional view includes a ray diagram showing image light 302, which passes through the assembly along a central optical axis 201 and is then directed by a beam splitter 114 toward a single image sensor 214. The depicted optical elements are diagrammatic only and are not drawn to scale. The depicted optical assembly may be employed with endoscopic devices and systems having an integrated camera or an external, detachable camera head.

[0059] As shown, in this embodiment, image light 302 enters the optical assembly at cover glass 212. Image light 302 then passes downstream to optical subsystem 110, which in this embodiment includes lens pair 111, lens pair 112, and lens pair 113. As described above with respect to Figure 2 The optical subsystem 110 is configured as described.

[0060] In this embodiment, beam splitter 114 is composed of three right angle prisms 114-1, 114-2 and 114-3, wherein along the adjacent interfaces of prisms 114-1 and 114-2, a suitable dichroic beam splitting layer is formed by a partially reflective coating. The image light is separated, wherein a first portion is reflected along a first optical path shown by image light 303, and a second portion passes along a second optical path shown by imaging light 304. As depicted, an internal reflective surface is provided at the edge depicted on the left side of prism 114-3 to reflect the image light. As discussed above, the first and second portions of light include different spectral contents, wherein white light passes along optical path 303, and fluorescent light passes along optical path 304. Other suitable layers, such as filter layers, may also be included in the prism. For example, a white light filter layer may be provided at the proximal surface of prism 114-1 (where image light 304 leaves prism 114-1), and a fluorescent light filter surface may be included at the proximal surface of prism 114-2 (where image light 303 leaves prism 114-2 and enters prism 114-3), or a fluorescent light filter surface may be included at the proximal surface of prism 114-3 (where light 303 leaves prism 114-3).

[0061] In this embodiment, meniscus lens 115 is positioned downstream of the beam splitter in the first imaging channel along the path of image light 303. Meniscus lens 115 is used to increase the magnification of the first imaging channel relative to the second imaging channel. Therefore, the orientation of the meniscus lens is opposite to the above embodiment.

[0062] Image sensor 214 is positioned downstream of cover glass or protective layer 213, wherein the image sensor is positioned perpendicular to the longitudinal axis of camera head 101 and is positioned offset from axis 201 to allow image light 303 and image light 304 to be detected at two separate areas of image sensor 214, wherein image light 304 is detected in an area centered along primary optical axis 201, and image light 303 is detected in an area offset from the primary axis along a first optical path.

[0063] As described above, a moving mechanism may be included for moving the meniscus lens 115 along the second imaging channel to adjust the focus of the second imaging channel. Such focus adjustment does not change the magnification change performed by the lens, so that the focus is independent of the magnification level. The meniscus lens arranged as depicted allows the imaging device to amplify the white light image relative to the image of the FI channel, thereby improving the resolution of the captured white light image by dispersing the image to a larger area (and therefore a larger number of pixels) of the image sensor 214. This configuration is particularly useful in endoscopes with very narrow diameters. Since the amount of light captured from the scene is limited, and the FI image carries a much lower intensity than the white light image, the FI channel image can remain unchanged, and the much brighter white light image can be expanded to fill a larger portion of the image sensor, thereby improving the resolution of the white light image, while advantageously reducing the brightness per unit area of ​​the image sensor to more closely match the brightness of the FI image. At the same time, the magnification provided by the meniscus lens 115 can be used to suppress aberrations in the image captured by the image sensor 214. In some embodiments where the meniscus lens 115 is placed in the white light channel, the meniscus lens is a double lens in order to avoid chromatic aberrations that may be introduced in the captured white light image. In contrast, when the meniscus lens 115 is placed in the FI channel, such as Figure 4 As presented, the narrow spectral band of fluorescent light is less likely to be negatively affected by chromatic aberration.

[0064] Figure 6 6 is a partial cross-sectional view of a camera head 101 showing an optical assembly configuration according to an example embodiment employing a liquid lens 602 in one channel for focus adjustment. The depicted optical assembly is configured to Figure 2 The optical assembly is similar to that of , wherein a deformable liquid lens 602 is added for adjusting the focus of the first optical channel.

[0065] The deformable liquid lens 602 includes a liquid lens layer 604, which is composed of an optical liquid material that can change its shape. Liquid lenses are known in the art and include a focal length that can be changed by changing the radius of curvature of the lens. This change in radius is controlled electronically and can be changed quickly on the order of milliseconds. Various embodiments may use electrowetting, shape-changing polymers, or acousto-optic tuning methods to control the radius of curvature or refractive index of the liquid lens. In this embodiment, the deformable liquid lens 602 is present in the first optical channel and is used to adjust the focus of the image light 303. However, in other embodiments, the deformable liquid lens 602 may be present in the second optical channel and is used to adjust the focus of the second image light 304. When the deformable liquid lens 602 is used in either case, the mechanical position adjustment mechanism for the meniscus lens 115 can be omitted.

[0066] The various embodiments described above can be used to perform FI using various fluorescent wavelengths of light. For example, a fluorescent molecule such as indocyanine green (ICG) or a fluorescent imaging ligand ("OTL38" or "OTL") can be present in the scene and exposed to a corresponding excitation wavelength, thereby producing emitted light of a specific wavelength band. These fluorescent molecules can all be detected in a wavelength band of approximately 800-850nm. The so-called "approximately" means that for specific applications, the boundaries of the bands are usually not clear, but often have longer tails, which makes the precise wavelength bands not fully defined, and therefore the boundaries of the bands can vary by as much as 10nm. For example, OTL fluoresces (i.e., emits fluorescent radiation) at approximately 790 to 800nm, while ICG fluoresces at 800 to 810nm. For these applications, a beam splitter is provided that separates the band of approximately 800-850nm from the image light by selectively reflecting the band or allowing the band to pass. In another example, Cy5 (Cyanine 5) and Cy5.5 (Cyanine 5.5) fluorescent dies typically emit fluorescence in the range of approximately 650-670 nm and 675-695 nm, respectively. Suitable embodiments of beam splitter 114 can be designed to separate the wavelengths again by selectively reflecting wavelengths of approximately 650-725 nm or selectively passing these wavelengths to select this band. As another example, fluorescein typically emits fluorescence in the range of approximately 515-545 nm. Suitable embodiments of beam splitter 114 separate the band by selectively passing or reflecting the band having a wavelength of approximately 500-550 nm.

[0067] Figure 7 FIG. 7 shows a cross-sectional view of a meniscus lens 700. Figure 2-Figure 6 As shown in the meniscus lens 115 in other contemplated embodiments, the meniscus lens 700 is an afocal meniscus lens suitable for all embodiments described herein. In this embodiment, the meniscus lens 700 is constructed as a single lens having a thickness "t", a refractive index "n", a larger radius of curvature "r1", and a smaller radius of curvature "r2". In general, the afocal characteristics of a meniscus lens are derived from a lens having the relationship described by Equation 1:

[0068]

[0069] Thus, the magnification "m" of the afocal meniscus lens 700 is equal to r2 / r1. As described above, the orientation of the lens is flipped with the radius on the far side being larger to provide a reduction effect rather than a magnification effect (see Figure 8). As described herein, the properties of an afocal meniscus lens provide several advantages when used to converge light near the image plane (i.e., the sensor plane). If the input beam is telecentric, the output beam will also be telecentric. If used at moderate magnifications, the meniscus lens introduces a fairly small amount of (monochromatic) aberrations. The magnification is constant with focus, so that there is no need to require the camera control unit to determine what the actual magnification is each time the unit focuses, as might be required with other focus adjustment schemes. This focusing advantage is particularly advantageous in embodiments that employ only a single image sensor, since it is not possible to change the position of the white light sensor independently of the fluorescent sensor. Lens 700 can also provide aberration correction, particularly for chromatic aberrations in white light.

[0070] Figure 8 A cross-sectional view of a meniscus lens 800 is shown according to another example embodiment and may be used as Figure 2-Figure 6 and meniscus lens 115 in other contemplated embodiments. Meniscus lens 800 is an afocal meniscus lens that is constructed as a double lens, including a double convex lens 804 and a double concave lens 802. Lens 804 provides an outer surface having a larger radius r1 as described above, while lens 802 provides a smaller radius r2 at its outer surface. Lens 800 has an overall magnifying effect. This double lens construction helps reduce chromatic aberration, especially in the case of, for example, Figure 5 When used in the white light imaging channel.

[0071] Table 1 below provides several example parameter values ​​that may be employed to construct an afocal meniscus lens suitable for use in various embodiments. Dimensions are in millimeters.

[0072]

[0073]

[0074] Table 1: Example values

[0075] Fig. 9 is a block diagram of an optical instrument system according to an example embodiment of the present invention. The depicted design is suitable for implementing the various embodiments described above as well as other embodiments. Although this example circuit is shown for an endoscope, the present invention is also applicable to other instruments such as, for example, a borescope or an exoscope.

[0076] Light source 8 illuminates subject scene 9, and light 10 reflected or scattered from the subject scene (or, alternatively, in the case of FI, light emitted by the subject scene) forms an optical image via optical channel assembly 11 (wherein a relay system including rod lenses is typically used to pass the light through to the camera head). At the camera head, the light is focused, aligned with the mirror axis or desired optical axis, and passes to the far side of optical channel assembly 11, where a light directing element 12 (such as beam splitter 114 described above) directs different spectral portions of the light to form different portions of the image on first solid-state image sensor 214 and second solid-state image sensor 216. A focus adjustment mechanism may be included to allow control of the focus mechanism (such as the above-described position adjustment mechanism for meniscus lens 115 or adjustable liquid lens 602) as well as a focus element (such as a lens within optical group 110) upstream of beam splitter 114 for adjusting the focus of the entire system (e.g., both channels together). Although two image sensors are shown in this embodiment, a single sensor design, such as those described herein, may be implemented by removing second image sensor 216 when light directing element 12 directs two portions of light to different areas of a single image sensor.

[0077] In this embodiment, the optical channel assembly 11 includes an imaging system and can be constructed according to various known methods. The image sensors 214 and 216 convert incident light into electrical signals by, for example, integrating the charge for each picture element (pixel). The image sensors 214 and 216 can be active pixel complementary metal oxide semiconductor sensors (CMOS APS) or charge coupled devices (CCDs), to name just two possible examples. The output analog signal from the image sensor is processed by an analog signal processor 22 and applied to an analog-to-digital (A / D) converter 24 to digitize the analog sensor signal. In some versions (typically CMOS designs), the analog signal processing and A / D converters can be integrated into individual sensor models attached to each sensor 214 and 216.

[0078] The camera 28 of the system typically includes a timing generator 26 that generates various timing signals to select rows and pixels and synchronize the operation of the image sensors 214 and 216, the analog signal processor 22, and the A / D converter 24. The camera head electronics typically house the image sensors 214 and 216, while the location of each of the analog signal processor 22, the A / D converter 24, and the timing generator 26 may vary, such as in the mirror handle 102. The non-optical functional elements of the camera 28 may be fabricated as a single integrated circuit, as is typically done with CMOS image sensors, or they may be separately fabricated integrated circuits.

[0079] System controller 50 controls the overall operation of the image capture device based on a software program stored in program memory 54. This memory may also be used to store user setting selections and other data that is retained when camera 28 is turned off. Data connections 27 and 29 carry digital image data from image sensors 214 and 216, respectively, to image processing circuitry 30, which may be integrated with system controller 50 in some versions, or may be a separate programmable logic device or data processor. Data bus 52 provides a path for address, data, and control signals. Control signals are sent to focus / magnification adjustment mechanism 13 and timing generator 26. In some variations, data bus 52 may also carry data connectors 27 and 29.

[0080] Image processing circuitry 30 performs image processing operations including operations to rescale the downscaled FI image (or upscaled white light image) so that the two images are the same or nearly the same size, to combine the two images from image sensors 214 and 216 as desired, including in some embodiments, for embodiments with a single image sensor, processing sub-images based on the first portion and the second portion of light. Image processing circuitry 30 and system controller 50 may execute an autofocus algorithm to make focus adjustments based on white light imaging or fluorescence imaging.

[0081] The processed image data is continuously sent to the video encoder 80 to produce a video signal. This signal is processed by the display controller 82 and presented on the image display 88. This display is typically a liquid crystal display (LCD) in HD, UHD or 4K format with a light emitting diode (LED) backlight, but other types of displays may also be used. The processed image data may also be stored in the system memory 56 or other internal or external memory devices.

[0082] The user interface 60 (including all or any combination of the image display 88, the user inputs 64, and the status display 62) is controlled by a combination of software programs executed on the system controller 50. The user inputs typically include some combination of a typing keyboard, a computer pointing device, buttons, rocker switches, a joystick, a rotary dial, or a touch screen. The system controller 50 may manage a graphical user interface (GUI) presented on one or more of the displays (e.g., on the image display 88). The GUI typically includes menus for making various option selections.

[0083] The image processing circuit system 30, the system controller 50, the system and program memories 56 and 54, the video encoder 80, and the display controller 82 can be housed in a camera control unit (CCU) 70. The CCU 70 can be responsible for powering and controlling the light source 8 and / or the camera 28. As used herein, a "CCU" refers to a unit or module that powers, receives data from, manipulates data from, transmits data to, and / or forwards data from an optical instrument camera. The functionality of the CCU can be distributed over multiple units, such as "connection modules," "link modules," or "head modules."

[0084] As used herein, the terms "include," "comprising," "carrying," "having," "containing," "involving," and the like are to be understood as open-ended, i.e., meaning including, but not limited to. Any use of ordinal numbers such as "first," "second," "third," and the like in a claim to modify claim elements does not, by itself, imply any priority, importance, or order of one claim element over another claim element, or the temporal order in which the acts of a method are performed. Rather, unless otherwise specifically stated, such ordinal numbers are used merely as labels to distinguish one claim element having a particular name from another element having the same name (but for the use of the ordinal number).

[0085] The foregoing is a fairly broad overview of the features and technical advantages of the present invention so that the detailed description of the present invention that follows can be better understood. It will be appreciated by those skilled in the art that the disclosed concepts and specific embodiments can be easily used as a basis for modifying or designing other structures for carrying out the same objectives of the present invention. It will also be appreciated by those skilled in the art that such equivalent constructions do not depart from the scope of the present invention as set forth in the appended claims.

[0086] Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and modifications may be made herein without departing from the scope of the invention as defined by the appended claims. The combination of features described herein should not be interpreted as limiting, and the features herein may be used in any working combination or sub-combination according to the present invention. Therefore, according to U.S. patent law and any relevant foreign patent law, this specification should be interpreted as providing written support for any working combination or certain sub-combination of features herein.

[0087] Furthermore, the scope of the present application is not limited to the specific embodiments of the processes, machines, manufactures, compositions of matter, devices, methods, and steps described in the specification. As will be readily appreciated by those skilled in the art from the disclosure of the present invention, processes, machines, manufactures, compositions of matter, devices, methods, or steps currently available or to be developed in the future may be utilized in accordance with the present invention, which processes, machines, manufactures, compositions of matter, devices, methods, or steps perform substantially the same functions or achieve substantially the same results as the embodiments described herein. Therefore, the appended claims are intended to include within their scope such processes, machines, manufactures, compositions of matter, devices, methods, or steps.

Claims

1. A video camera apparatus for white light and fluorescence imaging, comprising: a first optical subsystem that transmits reflected and scattered light for a non-fluorescent image and transmits emitted fluorescent light for a fluorescent image, both fluorescent and non-fluorescent light being collected from an illuminated scene; a beam splitter downstream of the first optical subsystem, the beam splitter splitting the light for the non-fluorescent image into a first imaging channel and splitting the light for the fluorescent image into a second imaging channel; a meniscus lens positioned downstream of the beam splitter in one of the first imaging channel and the second imaging channel, the meniscus lens being configured to change the magnification such that the magnification of the first imaging channel is different from the magnification of the second imaging channel; as well as One or more image sensors positioned downstream of the beam splitter to receive the fluorescent light and the non-fluorescent light; wherein the one or more image sensors are configured to capture the fluorescent image light and the non-fluorescent image light as the fluorescent image and the non-fluorescent image, and wherein the fluorescent image has an image size on the one or more sensors that is smaller in area than the image size of the non-fluorescent image.

2. The video camera device according to claim 1, wherein: The light from the first imaging channel is imaged onto a first image location of the one or more image sensors, and the light from the second imaging channel is imaged onto a second image location of the one or more image sensors that is different from the first image location.

3. The video camera device of claim 2 , further comprising a processor coupled to receive captured fluorescent images and non-fluorescent images from the first imaging channel and the second imaging channel, the processor configured to digitally rescale the fluorescent image to match the size of the non-fluorescent image and overlay the fluorescent image onto the non-fluorescent image for display.

4. The video camera device according to claim 2, wherein: The first image location and the second image location are different and separate areas of a single focal plane array image sensor.

5. The video camera device according to claim 2, wherein: The first image position and the second image position are on two separate focal plane array image sensors.

6. The video camera device according to claim 1, wherein: The meniscus lens is a single lens.

7. The video camera device according to claim 1, wherein: The meniscus lens is a doublet lens having an overall meniscus shape suitable for correcting chromatic aberration in white light.

8. The video camera device according to claim 1, wherein: The magnification of the second imaging channel is 0.5 to 0.9 times the magnification of the first imaging channel.

9. The video camera device according to claim 8, wherein: The meniscus lens is used to reduce the magnification of the second imaging channel.

10. The video camera device of claim 1, wherein: The meniscus lens has a first radius r1 and a second radius r2, wherein a magnification m equal to r2 / r1 is between approximately 0.56 and 0.

88.

11. The video camera device according to claim 1, wherein: The beam splitter reflects light of a wavelength corresponding to the fluorescent channel and transmits light of a wavelength corresponding to the non-fluorescent channel.

12. The video camera device of claim 1, wherein: The beam splitter transmits light of a wavelength corresponding to the fluorescent channel and reflects light of a wavelength corresponding to the non-fluorescent channel.

13. The video camera device of claim 1, wherein: The beam splitter reflects light of ICG / OTL fluorescence having a wavelength of approximately 800-850 nm and transmits light of shorter wavelengths.

14. The video camera apparatus of claim 1, wherein: The beam splitter transmits light of ICG / OTL fluorescence having a wavelength of approximately 800-850 nm and reflects light of shorter wavelengths.

15. The video camera apparatus of claim 1, wherein: The beam splitter reflects light of Cy5 or Cy5.5 fluorescence having a wavelength of approximately 650-725 nm and transmits light of a shorter wavelength.

16. The video camera apparatus of claim 1, wherein: The beam splitter transmits light of Cy5 or Cy5.5 fluorescence having a wavelength of approximately 650-725 nm and reflects light of a shorter wavelength.

17. The video camera apparatus of claim 1, wherein: The beam splitter reflects light of fluorescein fluorescence having a wavelength of approximately 500-550 nm and transmits light of other wavelengths.

18. The video camera apparatus of claim 1, wherein: The beam splitter transmits light of fluorescein fluorescence having a wavelength of approximately 500-550 nm and reflects light of other wavelengths.

19. The video camera apparatus of claim 1, wherein: The meniscus lens is afocal.

20. The video camera apparatus of claim 1, comprising means for focusing one of the first imaging channel and the second imaging channel differently than the other.

21. The video camera apparatus of claim 20, wherein: The focusing arrangement comprises a mechanism for moving the meniscus lens.

22. The video camera apparatus of claim 21, wherein: The movement of the meniscus lens does not change the magnification of the image light passing therethrough.

23. The video camera apparatus of claim 20, wherein: The focusing apparatus includes a mechanism for moving an image sensor corresponding to one of the imaging channels.

24. The video camera apparatus of claim 20, wherein: The focusing device comprises a liquid lens with adjustable deformation.

25. The video camera apparatus of claim 20, further comprising a processor coupled to control the focusing device, and the processor executes an autofocus routine to automatically adjust the focus of one of the imaging channels.

Images