Method and apparatus for depixization and resolution enhancement in microendoscope

Abstract

A method and apparatus for de-pixelating and enhancing resolution of an image obtained by a fiber optic bundle microendoscope. The method and apparatus include using a beam offset element to offset a beam forming an image relative to a fiber bundle, thereby filling gaps in an ineffective area and acquiring image information therein. The beam offset elements may be disposed at the input end of the fiber bundle, or at the input end and the output end of the fiber bundle.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 590,663, filed October 16, 2023, the entire contents of which are incorporated herein by reference. Background Technology

[0003] The ability to acquire images from within the tissues of living organisms is crucial for diagnostic purposes in medicine and for medical, biological, and pharmaceutical research utilizing living organisms. One of the most commonly used and versatile imaging methods is microendoscopy, in which fiber optic systems are inserted into the body to acquire images. The mechanical flexibility, extension length, and biocompatibility of optical fibers make microendoscopes an indispensable tool in many research and application fields.

[0004] Traditional optical fibers are typically a cylindrical, symmetrical glass structure. The optical refractive index of the fiber core is higher than that of its outer cladding material. This difference in refractive index between the core and cladding ensures that light propagating through the core is retained inside the core through total internal reflection. Such fibers can transmit light over long distances, as is used in modern telecommunications infrastructure. However, fibers with a single core are unsuitable for transmitting images because light rays exiting from the object's plane will randomly combine as they propagate through the fiber.

[0005] To transmit images over optical fibers, a large number of fibers are needed in an ordered manner. When an optical image is projected onto the input surface (near end) of a bundle of fibers with an ordered arrangement, the individual fiber elements behave like camera pixels, preserving local intensity information up to the output surface (far end) of the bundle, thus transmitting the correct image. This structure is called an "imaging fiber bundle" (or simply "imaging fiber") or "coherent fiber." Note that the term "coherent" in this usage refers only to the orderliness of the fiber and does not imply optical coherence of the relative phases of different light sources. Combined with traditional optical elements and detectors, coherent fibers form the basis of a large number of practical imaging schemes and applications.

[0006] Imaging fiber bundles can be manufactured in various forms. The total diameter of the fiber bundle determines the field of view, while the diameter of the individual fibers and the spacing between the cores determine the sensitivity and optical resolution. The total number of fibers in the bundle determines the total diameter for a given core spacing. While using a very large number of fibers (e.g., tens of thousands) in a bundle provides sensitive imaging, it also compromises the mechanical flexibility of the microendoscope.

[0007] Another limitation of imaging fiber optic microendoscopes lies in the pixelation characteristics of this method. For example... Figure 1As shown, optical guidance requires the surrounding medium to have a lower refractive index or a "dead region" that does not contain image information. The resulting image is essentially composed of a single "pixel" (i.e., optical fiber) surrounded by a large dead region. The pixelated effect limits the resolution that a fibered microendoscope can achieve. The optical resolution of a fiber bundle can be approximated using the Nyquist sampling criterion, i.e., twice the spacing between cores. (Or, more accurately, if the sampling frequency is greater than twice the highest frequency to be sampled, then a repeated waveform can be correctly reconstructed.) This pixelation is considered the biggest drawback of microendoscopes.

[0008] Most de-pixelation methods applied to fiber bundles rely primarily on physical translation of the entire fiber bundle relative to a stationary image. See Vyas et al., "Fiber Bundle Shifting Endomicroscopy for High-Resolution Imaging." Biomed. Opt. Express. 2018, 9(10): 4649-4664. Other methods can utilize oscillation of a condenser lens (see, e.g., Osanai U.S. Patent 10,959,608), which achieves de-pixelation by translating the image, but at the expense of optical quality of the image projection.

[0009] Accordingly, it is an object of the present application to provide an apparatus and method for eliminating pixelation and improving resolution in microendoscopy. SUMMARY

[0010] Disclosed herein is a method and apparatus for de-pixelating and enhancing resolution of images obtained via a fibered imaging device, including but not limited to a microendoscope. Fibered imaging bundles inherently produce pixelated images with large dead spaces between pixels, limiting the optical resolution, sensitivity, and utility of the device. The method and apparatus provided herein are based on shifting light entering and exiting the fiber bundle, thereby filling in the gaps in the dead regions and producing images with high resolution.

[0011] In particular, disclosed and claimed herein is a fibered imaging device.

[0012] In one embodiment, the device comprises an imaging fiber bundle sized and configured to transmit a light beam from an object to a detector, and a first light beam shifting element disposed at an input end of the imaging fiber bundle sized and configured to shift the light beam entering the imaging fiber bundle.

[0013] The first beam-shifting element can include a translational beam shifter oscillating with an amplitude or a rotational beam shifter rotating with a rate. The translational beam shifter can be a plate made of optically transparent material tilted with a predetermined angle. The rotational beam shifter can be a wedge prism.

[0014] In another embodiment, the fiber-optic imaging device further includes a second beam-shifting element disposed at the output end of the imaging fiber bundle sized and configured to shift the light beam exiting the imaging fiber bundle. The first and second beam-shifting elements shift the light beam in a synchronized manner, the shifts being equal in magnitude but opposite in direction.

[0015] Each of the first and second beam-shifting elements includes a translational beam shifter oscillating with a synchronized amplitude or a rotational beam shifter rotating with a synchronized rate. The translational beam shifter can be a plate made of optically transparent material tilted with a predetermined angle. The rotational beam shifter can be a wedge prism.

[0016] Also disclosed and claimed herein is a method for de-pixelization and resolution enhancement in a fiber-optic imaging device.

[0017] In one embodiment, the method includes shifting a light beam from an object using a first beam-shifting element disposed at the input end of the imaging fiber bundle, transmitting the shifted light beam through the imaging fiber bundle to a detector, and recording an image using the detector. The method further includes reconstructing the image to generate a higher resolution image.

[0018] In another embodiment, the method further includes shifting the light beam exiting the imaging fiber bundle using a second beam-shifting element disposed at the output end of the imaging fiber bundle. The first and second beam-shifting elements shift the light beam in a synchronized manner, the shifts being equal in magnitude but opposite in direction. Preferably, the scanning rate of the first and second beam-shifting elements is faster than the frame rate of the detector. In this configuration, no further image reconstruction is needed because the resulting image on the detector is properly positioned by the second beam-shifting element disposed at the output end of the imaging fiber bundle.

[0019] Objects and advantages of the present disclosure will be apparent to one of ordinary skill in the art upon reading the following detailed description of the preferred embodiments of the present disclosure in conjunction with the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a schematic representation of the cross-section of a coherent (ordered) fiber bundle. Light propagates only within the core. The surrounding area is needed for optical confinement but constitutes an inactive area.

[0021] Figure 2is a schematic of a beam displacer in the form of a tilted sheet or plate of optically transparent material. The beam displacer causes a shift in the light ray without changing its angle of propagation.

[0022] Figure 3 is a schematic of one embodiment of the method / apparatus including a translating beam displacer (shown as a tilted sheet or plate of optically transparent material) disposed at the proximal end of the imaging fiber bundle.

[0023] Figure 4 is a schematic of another embodiment of the method / apparatus including a rotating beam displacer (shown as a rotating wedge prism) disposed at the proximal end of the imaging fiber bundle.

[0024] Figure 5 is a schematic of another embodiment of the method / apparatus including two cooperating beam displacers (shown as tilted sheets or plates of optically transparent material) disposed at the proximal and distal ends of the imaging fiber bundle.

[0025] Figure 6 is a schematic of another embodiment of the method / apparatus including two cooperating beam displacers (shown as rotating wedge prisms) disposed at the proximal and distal ends of the imaging fiber bundle. DETAILED DESCRIPTION

[0026] In this specification, certain terms have been used for brevity, clarity and understanding. No unnecessary limitations are to be implied therefrom beyond the require of the prior art to the inventor's intent. The various systems and methods described herein can be used alone or in combination with other systems and methods. Dimensions and materials determined in the accompanying drawings and application are merely exemplary and are not intended to limit the scope of the disclosure. Any other dimensions and materials can be used as long as they do not deviate from the purpose of the present application. Various equivalent substitutions, alternatives and modifications can be made within the scope of the appended claims. Each of the appended claims defines a separate embodiment of the invention, and is intended to be interpreted independently as such, only to the extent that the corresponding limitation is expressly recited in the claim itself.

[0027] It should be understood that the written description uses examples to disclose the application, including the best mode, and also to enable any person skilled in the art to practice the application, including making and using any devices or systems and performing any incorporated methods. The various versions of the disclosure can be combined in any arrangement that enables endoscopic imaging. Any description of dimensions or other sizes is provided for illustrative purposes and is not intended to limit the scope of the claims in any way. Additional embodiments can include minor variations as well as larger size changes as needed for use in the industry. The patent scope of the devices disclosed and claimed herein can include other embodiments as would occur to one skilled in the art. The disclosure provided herein can be practiced without including any element not expressly disclosed herein.

[0028] As used herein, the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise.

[0029] As used herein, the term "or" is the inclusive "or" operator, and is equivalent to the term "and / or", unless the context clearly dictates otherwise.

[0030] The present method relies on the transmission of images from within a living organism through a fiber optic system and optical relays and probes. Imaging fibers inherently produce a pixelated image with large dead spaces between pixels, limiting the optical resolution, sensitivity, and utility of conventional methods. A method and apparatus are disclosed herein for eliminating this pixelation produced by imaging fibers, resulting in a continuous image with high resolution. The present method is based on shifting the light entering and exiting the fiber, filling in the gaps in the dead space and capturing the image information therein.

[0031] A method and apparatus are disclosed herein for de-pixelating images obtained by a coherent fiber bundle microendoscope. A beam shifting element is used to shift the entire beam forming the image with respect to the coherent fiber bundle. The beam shifting element (or simply "beam shifter") can be any size and structure of optical or electro-optical device configured to shift the propagation axis of an incident beam. One example of a beam shifting element is a thin sheet or plate of optically transparent material (e.g., glass) having parallel front and back surfaces and oscillating with a small amplitude. The beam shifting element shifts the beam in one or two axes by tilting the thin glass sheet or plate at a predetermined angle, such that the beam is shifted in one or two axes (as shown in Figure 2 、 Figure 3 and Figure 5 ). Another example of a beam shifting element is a "wedge prism" that is rotated about its central axis to impart a rotational shift to the beam passing through the element (as shown in Figure 4 and Figure 6The wedge prism has non-parallel front and back surfaces. It is within the scope of the present disclosure to use other types of beam shifters known today or developed in the future to implement this method.

[0032] The beam shifter can be placed at the input end (proximal end) of the imaging fiber bundle, or at both the input end (proximal end) and the output end (distal end). Shifting the projected image at the proximal end of the fiber bundle fills the dead space between individual fiber elements (or pixels), thus removing the inherent lattice pattern in the fiber bundle structure. In this way, the device and the corresponding method improve the overall optical resolution of the imaged sample. The lattice structure of the fiber bundle no longer sets the final optical resolution achievable by the bundle fibers. Performing the beam shifting operation synchronously at both ends of the fiber ensures that no further post-processing of the image signal is required.

[0033] In one embodiment of the method and device, the beam shifter is placed at the proximal end of the fiber bundle. As shown in Figure 3 and Figure 4 , the beam shifter is located between the endoscope objective and the input end of the imaging fiber bundle. The beam shifter can be a translational (single-axis or dual-axis) beam shifter (e.g., a tilted glass sheet or plate as shown in Figure 3 ), or it can be a rotational beam shifter (e.g., a wedge prism as shown in Figure 4 ). The beam shifter shifts the image by a predetermined discrete distance. A detector (such as but not limited to a CMOS camera) at the distal end of the fiber bundle records the image at each position. A reconstruction algorithm combines these images and generates a new image with higher resolution. In this embodiment, there is only one beam shifter at the input end of the fiber bundle. As described above, the images exiting the fiber bundle are reconstructed by the algorithm.

[0034] In another embodiment of the method and device, the beam shifter is placed at both the proximal end and the distal end of the fiber bundle. As shown in Figure 5 and Figure 6 , a first beam shifter is located between the endoscope objective and the input end of the imaging fiber bundle, and a second beam shifter is located between the output end of the imaging fiber bundle and the endoscope projection lens. The beam shifter can be a translational (single-axis or dual-axis) beam shifter (e.g., a tilted glass sheet or plate as shown in Figure 5 ) or it can be a rotational beam shifter (e.g., a wedge prism as shown in Figure 6 ). The beam shifts caused by the two beam shifters are synchronized and equal in magnitude but opposite in direction. The first beam shifter causes the incident image to be shifted at the input end of the imaging fiber. The second beam shifter, oscillating or rotating in synchronization with the first beam shifter, causes the image to be shifted at the detector (at the distal end of the fiber bundle). Figure 5 Figure 6 ​The output image is shifted on the surface of the camera). In this way, the output signal matches the scanned image of the sample being detected. Preferably, the scanning rate of the beam shifter is faster than the frame rate of the detector. The detector continuously reads the image. No further image reconstruction is needed as the image is scanned on the input side and then back-scanned on the output side. The resulting image on the detector has no pixelated fiber lattice structure and is properly positioned by the beam shifter placed at the distal end of the fiber bundle.

[0035] The methods and devices disclosed herein provide a number of advantages. As Figures 3 to 6 As shown, the beam shifting element is mechanically separated from the rest of the microendoscope, simplifying the mechanical structure. Because the coherent fiber bundle itself does not need to be translated or rotated, bulky fiber bundle motion components (e.g., piezoelectric transducers and closed loop servo mechanisms) are eliminated. The mechanical element of the beam shifter has a simple, direct structure, such as a small electromagnetic element or rotor, and requires only low voltage input. The use of beam shifters at both the input and output ends of the coherent fiber bundle, and oscillating (or rotating) the beam shifters in synchronization and at the appropriate speed, allows real-time image acquisition without any additional computation. The optical portion of the beam shifter needs only to have a similar diameter as the fiber bundle itself, making the entire device very compact.

[0036] The methods and devices disclosed herein have important practical value. The success and applicability of any microendoscope system is directly affected by its optical resolution. The ability to view finer biological structures within living tissue opens the door to unprecedented new discoveries. For example, at the current state of the art, imaging fibers can barely resolve individual brain neurons spaced apart from each other. Using the methods and devices described herein allows resolution of subcellular structures, as well as more densely packed brain regions. In clinical imaging applications, fiber endoscopes are often used for the detection and treatment of cancer. The enhanced imaging resolution enabled by the present disclosure greatly improves the diagnostic accuracy and early detection capabilities of such instruments.

Claims

1. A fiber optic imaging device, comprising: Imaging fiber bundles, whose size and structural configuration are designed to transmit a beam of light from an object to a detector; and A first beam deflection element is disposed at the input end of the imaging fiber bundle, and the size and structure of the first beam deflection element are configured to deflect the beam entering the imaging fiber bundle.

2. The apparatus of claim 1, wherein the first beam deflection element comprises a translational beam deflector oscillating with an amplitude or a rotational beam deflector rotating at a rate.

3. The apparatus of claim 2, wherein the translational beam deflector is a plate made of optically transparent material tilted at a predetermined angle.

4. The apparatus according to claim 2, wherein the rotating beam deflector is a wedge prism.

5. The apparatus of claim 1 further comprises a second beam deflection element located at the output end of the imaging fiber bundle, the element being sized and configured to deflect the beam exiting the imaging fiber bundle.

6. The apparatus of claim 5, wherein the first beam deflection element and the second beam deflection element deflect the beam synchronously, the deflections being equal in magnitude but opposite in direction.

7. The apparatus according to any one of claims 5-6, wherein each of the first beam shifting element and the second beam shifting element comprises a translational beam shifter oscillating with synchronous amplitude or a rotational beam shifter rotating with synchronous rate.

8. The apparatus of claim 7, wherein the translational beam deflector is a plate made of optically transparent material tilted at a predetermined angle.

9. The apparatus of claim 7, wherein the rotating beam deflector is a wedge prism.

10. The device according to any one of claims 1-9, wherein the device is a microendoscope.

11. A method for depixelation and resolution enhancement in a fiber optic imaging apparatus, the method comprising: The beam from the object is deflected using a first beam deflection element located at the input end of the imaging fiber bundle; The deflected beam is transmitted to the detector via an imaging fiber optic bundle; as well as The detector is used to record images.

12. The method of claim 11, further comprising reconstructing the image to generate a higher resolution image.

13. The method of claim 11, further comprising using a second beam deflection element disposed at the output end of the imaging fiber bundle to deflect the beam leaving the imaging fiber bundle.

14. The method of claim 13, wherein the first beam deflection element and the second beam deflection element deflect the beam in a synchronous manner, the deflections being equal in magnitude but opposite in direction.

15. The method according to any one of claims 13-14, wherein the scanning rate of the first beam deflection element and the second beam deflection element is faster than the frame rate of the detector.

Images