Multicore fiber imaging

Abstract

To provide a method of generating an image from light received via multicore fiber imaging.SOLUTION: The method includes receiving a digital image containing a plurality of pixels, the digital image including a plurality of regions therein each corresponding to a waveguide core. Each region includes a plurality of pixels, and a first subset of pixels within each region is defined which at least partly correlates with light having been received at a corresponding core in a first spatial arrangement, the subset including less than all of the pixels within the region. A first image is generated from the first subset of pixels from the regions, combined to form an image over the whole waveguide array. The first spatial arrangement may correspond to a measure of angular dimension of the incident light for that region. In addition to increased depth of field, the modified images provided by the invention allow 3D visualization of objects, e.g., using stereographs or depth mapping techniques.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001]

[0001] The present invention relates mainly to multi-core fiber imaging such as used in endoscopic examinations. In a preferred form, a method is described for processing an image captured in such a system to obtain an improved depth-of-field image or extract 3D information about the image without the addition of additional optical components.

Background Art

[0002]

[0002] Multi-core optical fibers (MOF) are widely used as microendoscope probes for imaging the inside of the body. Such MOFs each contain thousands of waveguide cores with a diameter of 2 - 3 μm that relay light from inside the body to an imaging device operating as an external detector. The imaging device generates an image of the distal end of the MOF. The generated image contains a plurality of pixels, each containing a plurality of regions corresponding to the waveguide cores. Imaging of each core is performed over a plurality of pixels in the image. The digital image will also contain pixels corresponding to the interstitial space between the waveguide cores. Figure 1 shows a portion of such an image. The continuous black image portion 2 in Figure 1 corresponds to the pixels that image the interstitial space between the waveguide cores. Bright regions of various gray levels (e.g., 4) correspond to individual waveguide cores, and their illumination levels correspond to the light received at the distal facet of each fiber.

[0003]

[0003] The development of MOF technology includes processing the received data to improve image quality, including filtering techniques for processing image artifacts generated by the interstitial regions between fiber cores, which may be represented as distracting patterns such as visible grids or moiré effects. For example, U.S. Patent No. 5,751,340 discloses reducing the contrast of grid pattern components by appropriate filtering, such as applying an extension process that interpolates from the brighter center of each fiber to the adjacent interstitial regions.

[0004]

[0004] Multicore optical fibers enable a wide range of microendoscopic imaging modes, including optical coherence tomography, reflection, and fluorescence (confocal, wide-field, and multiphoton).

[0005]

[0005] When a microlens is attached to the distal facet, the MOF typically operates with a short working distance of about 50 - 100 μm. Imaging is also possible in a contact mode where the bare fiber facet is in direct contact with the sample. In this case, the Nyquist-limited resolution is twice the core-to-core distance, while in a lens-based system, the resolution scales with the inverse magnification.

[0006]

[0006] Both lensless and lens-based systems experience a rapid degradation of resolution as the distance from the focal plane increases. This is particularly challenging in microendoscopes, where size constraints often result in the absence of a delicate focus control device at the distal end. To avoid out-of-focus haze, confocal sectioning can be employed, which restricts the collected signal to a thin slice near the MOF facet, i.e., the distal surface of the MOF (lensless system) or the focal plane (lens-based system). However, this can make focusing even more difficult as signals outside the optical slice are rejected while sharing the focus. In many instances, it is desirable for a microendoscopic system to collect "all-focus" images that appear sharp over an extended depth of field, even if the object is not precisely positioned at the focal plane or the MOF facet.

[0007]

[0007] In other types of imaging systems, the depth of field can be extended by gradually restricting the collection aperture to paraxial light rays. This reduces the size of the blur circle of out-of-focus objects, thereby sharpening the image over the depth range. However, since the MOF does not have an adjustable aperture, this mechanism cannot be used to control the depth of field.

[0008]

[0008] Previous attempts to increase the depth of field of an endoscope have relied on mechanical engineering, which results in a larger and more complex probe at the distal end.

[0009]

[0009] Other imaging techniques using single-core multimode fibers with an extended depth of field require the use of coherent light. As a result, these techniques are very sensitive to fiber bending and difficult to use in real-world situations. Furthermore, imaging techniques using multicore fibers that use coherent light with an extended depth of field and are insensitive to fiber bending require very advanced image reconstruction algorithms that often fail with complex objects.

Summary of the Invention

Problems to be Solved by the Invention

[0010]

[0010] Accordingly, an object of the present invention is to provide a method for generating an image from light received through a MOF that at least partially addresses one or more of the above drawbacks.

[0011]

[0011] Any reference to prior art in the specification does not admit or suggest that this prior art constitutes a part of the common general knowledge in any jurisdiction or that this prior art can be combined with other prior art by a person skilled in the art.

Means for Solving the Problems

[0012]

[0012] In a first aspect, the present disclosure is a method for generating an image from light received by an imaging device through a variety of waveguides, the method comprising: Receiving a digital image including a plurality of pixels, the digital image including a plurality of regions each corresponding to a waveguide core and including a plurality of pixels, and further including pixels corresponding to the interstitial spaces between the waveguide cores. Defining a first subset of pixels within each region that at least partially correlates with the light received by the corresponding core in the first spatial arrangement, the first subset of pixels including fewer pixels than all the pixels within the region, and generating a first image from the first subset of pixels of the region. A method is provided that includes these steps.

[0013]

[0013] Generating the first image preferably includes, for each region, determining an average pixel value of the pixels of the first subset and assigning the average pixel value as the pixel value of at least one pixel within the first subset of pixels. The method preferably includes generating pixel values for pixels other than the at least one pixel. In one embodiment, the average pixel value is assigned to one pixel within the first subset of pixels. Most preferably, the average pixel value is assigned to a pixel at a predetermined position representing the center of the waveguide core within the image. The average pixel value can also be assigned to all pixels of a group of pixels at or around the predetermined position representing the waveguide core center.

[0014]

[0014] Generating the first image includes generating pixel values for pixels other than the at least one pixel (e.g., a central pixel or a group of pixels centered). Preferably, generating pixel values for pixels other than the at least one pixel includes assigning pixel values according to a pixel value distribution function centered on the at least one pixel or, alternatively includes assigning pixel values by interpolating between the at least one pixel of adjacent regions. The method includes any one of these steps.

[0015]

[0015] Thus, in one modification example, this method includes obtaining an average pixel value of the pixels of the first pixel subset within the region, assigning the average value to a specified pixel (or a specified group of pixels) located at a position representing the center of the waveguide core of the region, and assigning values to the remaining pixels by interpolating (e.g., by linear interpolation) between the specified pixels (or their respective specified groups of pixels) of adjacent regions.

[0016]

[0016] The first pixel subset can include all pixels within a predetermined radius from the center of the region. The radius may be substantially smaller than the radius of the waveguide core. Alternatively, the subset may be any other shape or arrangement region as needed.

[0017]

[0017] The method further includes generating a second image from the received digital image, and generating a final image by compositing the second image with the first image.

[0018]

[0018] The second image can be generated in a manner similar to the first image. Here, this can be performed by defining a second pixel subset within each region that includes fewer pixels than all the pixels within the region and is different from the first pixel subset, and generating the second image from the second pixel subset of the region.

[0019]

[0019] In one embodiment, one of the first and second subsets can include a subset of all pixels within the region excluding pixels that partially or fully correspond to the gap space between the waveguide cores (i.e., the fiber cladding and the void). In this case, the other of the first and second subsets represents a smaller area of the region, preferably a significantly smaller area.

[0020]

[0020] Generating the second image For each region, the average pixel value of the pixels in the second subset can be determined, and the average pixel value can be assigned as the pixel value of at least one pixel in the second pixel subset. Generating the second image can include generating the pixel values of pixels that are not at least one pixel within the second pixel subset.

[0021]

[0021] Generating the pixel values of the pixels that are not at least one pixel within the second pixel subset may further include assigning pixel values according to a pixel value distribution function centered on the at least one pixel within each region, or include assigning pixel values by interpolating between the pixel values of the second subset of adjacent regions, and can include any one of these.

[0022]

[0022] Thus, in one modification example, this method includes determining the average pixel value of the pixels in the second pixel subset within the region, assigning the average value to a specified pixel (or a specified group of pixels) located at a position representing the center of the waveguide core of the region, and assigning values to the remaining pixels (for example, by linear interpolation) by interpolating between the specified pixels (or their respective specified groups of pixels) of adjacent regions.

[0023]

[0023] The second pixel subset includes all pixels within a second predetermined radius from the center of the region. The radius may be significantly smaller than the radius of the waveguide core. Similar to the first pixel subset, the second pixel subset can include pixel subsets of any shape.

[0024]

[0024] Compositing the second image with the first image can include, for each region, modulating, weighting, or modifying one of the first and second images using the other of the first and second images. Next, the modified images for all regions are composited to generate a modified digital image across various waveguides.

[0025] Combining the second image with the first image includes scaling the luminance of one or both images as necessary and subtracting the second image from the first image. The luminance scaling is preferably performed such that the total intensity of both images is equal. It will be understood that other suitable approaches for combining the second image with the first image can also be used.

[0026]

[0026] By appropriate selection of regions, this approach selectively removes light near the periphery of each core and has the effect of limiting the numerical aperture of each waveguide by calculating the difference between the light level of this (first) image and the light level of the unfiltered (second) image.

[0027]

[0027] The first image preferably has a greater effective depth of field than the second image.

[0028]

[0028] Preferably, the generation of the first image is biased towards the selection of light rays received by the waveguide within the first angular range, and the generation of the second image is biased towards the selection of light rays received by the waveguide within the second angular range.

[0029]

[0029] Preferably, the second angular range is wider than the first angular range.

[0030]

[0030] In some embodiments of any of the methods defined above, the method may further include defining at least one other pixel subset within each region that is at least partially correlated with the light received by the corresponding core in a corresponding different spatial arrangement, the other pixel subset including fewer pixels than all the pixels within the region, generating at least one corresponding other image from the received digital image, and combining the first image and the at least one other image according to the weighting to generate a final image. The at least one other image can include the second image.

[0031]

[0031] The predefined weighting can be determined by a calibration process.

[0032]

[0032] In another aspect, the present specification further provides a method for improving the apparent depth of field of an image captured via a multi-core optical fiber (MOF), where the digital image includes a plurality of pixels, the digital image includes a plurality of regions each containing a plurality of pixels corresponding to a core of the MOF, and further includes pixels corresponding to the interstitial space between the waveguide cores, and the method includes defining a first pixel subset within each region that is at least partially correlated with the light received by the corresponding core in the first spatial arrangement, the first pixel subset including fewer pixels than all the pixels within the region; for each region, determining the average pixel value of the pixels in the first subset and assigning the average pixel value as the pixel value of at least one pixel in the first pixel subset; and generating pixel values for pixels that are not the at least one pixel. A method is disclosed that includes generating a first image having an improved depth of field.

[0033]

[0033] Generating pixel values for pixels that are not the at least one pixel in the first pixel subset may include assigning pixel values according to a pixel value distribution function centered on the at least one pixel of each first region, or assigning pixel values by interpolating between pixel values of the first subsets of adjacent regions.

[0034]

[0034] The method may further include generating a second image from the received digital image, and combining the second image with the first image to generate a final image having an improved depth of field, where the second image Defining, in each region, a second pixel subset that includes fewer pixels than all the pixels in the region and is different from the first pixel subset, and Generating a second image from the second pixel subset of the region.

[0035]

[0035] In a preferred embodiment, the first image has a greater effective depth of field than the second image.

[0036]

[0036] The generation of the first image is preferably biased towards the selection of light rays received by the waveguide within a first angular range, and the generation of the second image is biased towards the selection of light rays received by the waveguide within a second angular range.

[0037]

[0037] The second angular range is preferably wider than the first angular range.

[0038]

[0038] Systems configured to perform these methods (e.g., imaging systems and image processing systems) also constitute further aspects of the present disclosure.

[0039]

[0039] Further aspects of the present disclosure relate to light illumination field imaging.

[0040]

[0040] Specifically, in a further aspect, the present disclosure is a method for determining a light illumination field approximation used in image processing corresponding to a pair of images generated from light received by an imaging device via various waveguides, the method comprising: Obtaining a pair of images in which a first member image has a first depth of field and a second member image has a second depth of field, and the first member image and the second member image have the same focal position; Generating a difference image from the pair of images; Calculating a light illumination field approximation from the difference image.

[0041]

[0041] The process of generating a differential image can initially include at least one intensity scaling of the image. This intensity scaling preferably includes dividing each pixel value by the average pixel value of the image so that the total intensities of the image pair are equal.

[0042]

[0042] The process of calculating the optical irradiation field approximation can include using a hypothetical angular distribution of light propagation with respect to the average ray orientation.

[0043]

[0043] The hypothetical angular distribution may be a Gaussian distribution.

[0044]

[0044] The second member image can be obtained using the method of any one of the above-described embodiments of the present disclosure.

[0045]

[0045] The first member image can be obtained using the embodiment of the first aspect of the present disclosure, and the first member image and the second member image use different first pixel subsets within each region.

[0046]

[0046] Preferably, the first member image is obtained from the same digital image as the second member image and is generated from substantially all the pixels within the region of the digital image corresponding to the waveguide core.

[0047]

[0047] In a further aspect, a method of generating an image, obtaining a pair of images in which the first member image has a first depth of field of view and the second member image has a second depth of field of view, and the first member image and the second member image have the same focal position, determining the optical irradiation field approximation using a method embodying the above-described aspect of the present disclosure, processing the image according to the optical irradiation field approximation to generate a final image, is disclosed.

[0048]

[0048] Processing the image according to the optical irradiation field approximation Reconstructing images having different focal positions, Reconstructing images having a different viewpoint from the received image, Reconstructing the 3D viewpoint of an image, can include any one or more of.

[0049]

[0049] The processing of an image of any of this or other aspects of the invention disclosed herein can include assuming a non-linear stereo parallax of a light source as it moves away from the light source.

[0050]

[0050] In a further embodiment, a method of generating one or more images from light received by an imaging device via a variety of waveguides, wherein the light is generated from a light irradiation field in which the light is incident on the variety of waveguides, the method comprising: Receiving a digital image including a plurality of pixels, each corresponding to a waveguide core and including a plurality of regions each including a plurality of pixels, Processing an image intensity pattern across each of the regions to determine a light irradiation field angular dimension measure for the region, Applying the angular dimension measure to one or more pixels included in each region to generate one or more sets of modified image data, Generating one or more images using one or more sets of the modified image data, is provided.

[0051]

[0051] The step of processing an image intensity pattern across each of the regions can include analyzing each region by a simulated aperture involving a computational comparison of the image intensity under a first computational aperture and the image intensity under a second computational aperture for each region. One set of pixels of one of the first and second computational apertures can include a subset of pixels of the other of the first and second computational apertures, or the sets of pixels of each computational aperture can differ by a specific light irradiation field angular dimension measure extracted from the processing step.

[0052]

[0052] Alternatively or additionally, the processing of the image intensity pattern across each of the above regions can include a pattern matching algorithm that compares the image intensity pattern with a stored pattern. The stored pattern can be generated for each of the various waveguides by a pattern calibration process.

[0053]

[0053] As described above, in a further aspect, the present disclosure also provides an imaging system. The imaging system includes a multi-core optical fiber (MOF) extending from a proximal end to a distal end, a light source for illuminating a scene at the distal end of the MOF, an imaging device disposed with respect to the proximal end of the MOF to capture an image of the light propagated along the MOF, and a data processing system configured to receive the image captured by the imaging device, the data processing system being configured to execute instructions to cause a method embodying any of the aspects disclosed herein to be performed.

[0054]

[0054] Preferably, the MOF comprises an endoscope.

[0055]

[0055] The present disclosure describes the use of various illumination geometries for the object or scene being imaged. In one or more embodiments, the methods and systems described utilize reflection, transmission, fluorescence, or combinations thereof.

[0056]

[0056] As described above, in a further aspect, the present disclosure also provides an image processing system comprising at least one processing unit and at least one memory for storing instructions to be executed by the at least one processing unit, the instructions being executed to cause a method embodying any of the aspects disclosed herein to be performed.

[0057]

[0057] Unless the context requires otherwise, as used herein, the term "comprise" and variations of this term such as "comprising", "comprises" and "comprised" are not intended to exclude further additions, components, integers or steps.

[0058]

[0058] Further aspects of the present invention and further embodiments of the aspects described in the preceding paragraphs are given by way of example and will become apparent from the following description with reference to the accompanying drawings.

Brief Description of the Drawings

[0059]

Figure 1

[0059] Figure 1 represents a portion of the original image obtained from the MOF by the imaging device, showing the MOF core and the interstitial area.

Figure 2

[0060] Figure 2 is a schematic MOF imaging system used when capturing an image that can be processed according to an embodiment of the present invention.

Figure 3

[0061] Figure 3 is a flowchart of a first image processing method according to an embodiment of the present invention.

Figure 4

[0062] Figure 4 shows a series of images showing the output from two embodiments of the present invention compared to the original image series. In this example, the images show a portion of Group 5 of the USAF1951 target.

Figure 5A

[0063] Figure 5A is a flowchart of a second image processing method according to an embodiment of the present invention.

Figure 5B

[0063] Figure 5B is a flowchart of another embodiment of the method of Figure 5A.

Figure 6

[0064] Figure 6 shows a series of images (labeled eDOF, meaning "extended depth of field" images) generated using an embodiment of the process of Figure 5A. A series of plots comparing the image series of Figure 6 with the image series of Figure 4 are also shown.

Figure 7

[0065] Figure 7a shows a ray impinging on the input facet of the fiber core in an orientation described by the angle of incidence θ and the azimuth angle φ. Figure 7b shows the simulated input core intensity distribution resulting from a plane wave oriented at the angle (θ, φ) (scale bar: 5 μm). Figure 7c plots the full aperture image (R = 7 px) for the fiber core input specifically enclosed by the circle in Figure 7b and the simulated normalized intensities within two simulated apertures of R = 3 px and 1 px. Figure 7d shows the experimentally recorded output intensity from the output facet of a single core varying the plane wave input angle (θ, φ) using the apparatus of Figure 2.

Figure 7E

[0065] Figure 7E shows the simulated angular PSF normalized by the aperture area.

Figure 8

[0066] Figure 8 shows a series of images of a portion of groups 6 and 7 of the USAF 1951 target used in the experimental testing of an exemplary implementation of the present invention.

Figure 9

[0067] Figure 9a shows cloth fibers imaged using a standard full aperture approach compared to an embodiment of the present invention (right column) using both 10x and 20x objective lenses. Figure 9b shows the intensity profile along the line shown in the middle row of Figure 9a.

Figure 10

[0068] Figure 10 is a flowchart of a method for estimating the light irradiation field of an image using an embodiment of one aspect of the present invention.

Figure 11

[0069] Figure 11 shows the first and second images of an image pair used in the method of Figure 10, and a visualization of the difference in the intensity values of these images. The bottom row shows a composite image generated from the light irradiation field approximation generated from the images of Figure 11.

Figure 12

[0070] Figure 12a represents the depth map of a series of images of a USAF target (Group 5) generated using an embodiment of the present invention. Figure 12b is a plot of the depth metric as a function of the known ground truth depth (distance from the MOF fiber facet to the USAF target) for the target used to generate this embodiment.

Figure 13

[0071] Figures 13a and 13b show the results of computational refocusing of USAF target images at various distances.

Figure 14

[0072] Figure 14 is a block diagram showing a computer processing system suitable for use as an image processing system for processing the images captured by the system of Figure 2.

Figure 15

[0073] Figure 15 shows the application of optical moment imaging (LMI) to the MOF images processed according to an embodiment of the present invention.

Figure 16

[0074] Figure 16 shows the relationship of the stereo parallax of a moving viewpoint (increasing the axial distance from the object) when the method of an embodiment of the present invention is applied.

DETAILED DESCRIPTION OF THE INVENTION

[0060]

[0075] Figure 2 is a schematic diagram of an exemplary optical device of MOF within an imaging system 8 arranged to image a prepared sample showing an embodiment of the present invention. The imaging system may be an endoscope system. Preferably, the imaging system is coupled to, or includes, a data processing system 400 for processing the image data from the imaging system 8.

[0061]

[0076] The proximal facet of MOF 10 (e.g., Fujikura FIGH-30-600S or FIGH-10-350S) is incoherent from the LED 12 (e.g., Thorlabs M565L3 LED, 565 nm center wavelength) - It is illuminated by rental light. The total illuminance at the distal end of the MOF is up to 10 μW in this example. The light from LED12 is collimated by a collimating lens (CL) and passes through a mirror (M), a 200 mm lens (L), a polarizer (P1), a beam splitter (BS), and a 20x objective lens (OBJ). The illumination source 12 is linearly polarized to facilitate the removal of the light reflected from the proximal facet of the MOF. Both ends of the MOF10 and the sample 14 are attached to an independent three-axis translation stage (xyz). It is preferable that there is no lens between the distal MOF facet and the sample, but some embodiments of the present invention may use such a lens device. It is preferable that there is no lens between the distal end of the MOF and the sample, but some embodiments of the present invention may use such a lens device.

[0062]

[0077] The light that reaches the distal end of the MOF illuminates the sample 14, and then the reflected light recombines with the MOF10. The retroreflected light couples into various modes depending on the incident angle to the distal fiber facet. The output intensity pattern at the proximal end in a plurality of cores is imaged through a microscope objective system (e.g., Olympus Plan Achromat 20x 0.4NA), a beam splitter (BS), a 200 mm tube lens (TL), and a second polarizer (P2). The polarization axes of P1 and P2 are orthogonal to filter out the reflected light at the proximal facet. The image is captured by a camera (CAM) (e.g., a black and white camera with an integration time of 10 ms, Thorlabs DCC3240M). In this example, the core refractive index and the cladding refractive index of the MOF are ncore = 1.5 and nclad = 1.446, respectively, and as a result, an NA of 0.398 is obtained, which almost matches that of a 20x, 0.4NA objective lens (OBJ). The light that reaches the distal end of the MOF illuminates the sample 14, and then the reflected light recombines with the MOF10. The retroreflected light couples into various modes depending on the incident angle to the distal fiber facet. The output intensity pattern at the proximal end in a plurality of cores is imaged through a microscope objective system (e.g., Olympus Plan Achromat 20x 0.4NA), a beam splitter (BS), a 200 mm tube lens (TL), and a second polarizer (P2). The polarization axes of P1 and P2 are orthogonal to filter out the reflected light at the proximal facet. The image is captured by a camera (CAM) (e.g., a black and white camera with an integration time of 10 ms, Thorlabs DCC3240M). In this example, the core refractive index and the cladding refractive index of the MOF are ncore = 1.5 and nclad = 1.446, respectively, and as a result, an NA of 0.398 is obtained, which almost matches that of a 20x, 0.4NA objective lens (OBJ).

[0063]

[0078] The inventor has noticed that light reaching the distal (light-receiving) end of a multi-core fiber from different directions is propagated to the proximal end and received with different spatial intensity patterns in an imaging device, and thus light received from a specific direction can be emphasized or not emphasized in the processed image. Therefore, the present invention results from the recognition that the MOF transmits 3D information in the form of light irradiation field information (spatial angular distribution of light rays incident on the distal end of the MOF), and the angular dimension of the light irradiation field is modulated into the intensity pattern within the core that has hitherto been ignored in the fiber bundle. As will be further considered below, these intensity patterns are caused by angle-dependent mode coupling, and the present invention includes associating these patterns with the angular dimension of the light irradiation field.

[0064]

[0079] An important finding is that light incident on the fiber core at various angles will produce various intensity distributions at the output of the fiber core. Specifically, light rays that impinge directly on the fiber core (paraxial rays) tend to mainly excite the fundamental mode of the fiber, resulting in an output pattern in which most of the light is concentrated in the middle of the core. On the other hand, as the incident angle increases, the output light density at the output of the fiber core tends to move towards the periphery of the core. Furthermore, the inventor has noticed that an image with a large depth of field can be generated by emphasizing the light that reaches almost parallel to the axis of the distal end of the fiber.

[0065]

[0080] According to the present invention, these intensity patterns caused by angle-dependent mode coupling are quantitatively related to the angular structure of the light irradiation field.

[0066]

[0081] Embodiments of the present invention generate an image using a “simulated aperture” applied to each core of an optical fiber. The simulated aperture is applied by selectively emphasizing a subset of pixels corresponding to one or more spatial regions of each core from the image, with emphasis placed on the image generation process. In one form, the simulated aperture is applied by selecting a pixel subset that includes only pixels within a certain radius of the center of each core. In some embodiments, the pixel subset corresponding to each core may not be concentrated at the center of each core. To avoid misunderstanding, the pixel subset that constitutes the “simulated aperture” need not be a single spatially continuous subset and may be composed of sub-subsets. Further, the pixel subset can assume any suitable shape.

[0067]

[0082] Embodiments can be applied to multi-core optical fibers used in contact mode (lensless) or lens mode.

[0068]

[0083] FIG. 3 shows an exemplary method of the present invention that can be used to generate an image with a greater depth of field (DOF) than the captured original image 202. This method removes the contribution from rays reaching at large oblique angles by applying a simulated aperture. As described above, paraxial rays mainly contribute to the intensity in the middle of the fiber core, while oblique rays tend to generate a pattern with the intensity pushed towards the periphery of the core. Therefore, by reconstructing the image using only the central pixels of each core, it is possible to remove some of the oblique rays or at least tilt the image generation towards a direction that includes a larger proportion of paraxial rays. This has been found to have the effect of increasing the DOF compared to using all the pixels from each core or downsampling the raw fiber facet image.

[0069]

[0084] Method 200 begins by receiving an original image from a MOF using, for example, an apparatus as shown in FIG. 2. Next, in step 206, a subset of pixels is selected from the region within the image corresponding to the waveguide core (i.e., the portion of the image related to the interstitial space is ignored) and used to generate an image in step 208. These regions are conceptually considered as simulated apertures that have the effect of being more or less selective for light received at different angles of arrival at the distal end of the MOF, which is applied computationally.

[0070]

[0085] In some embodiments, this may be a prerequisite for selecting a subset of pixels to identify the region within the image corresponding to the waveguide core. This can be done using automated image analysis techniques, or alternatively, the waveguide core is spatially arranged in a known pattern, such as a regular grid, and thus it can be recognized that the position of the region within the image is known. In some embodiments, the identification of the region within the image containing the core can be determined by a process of taking a reference image using a mirror instead of the sample. This image may have a high contrast between the core and the cladding within the image, and this image can be used to more easily identify the position of the core.

[0071]

[0086] As will be understood by those skilled in the art, in embodiments of the present invention, other image processing techniques can also be employed to improve the image quality. For example, after obtaining a background image without a sample, it can be subtracted from each raw camera image before further processing.

[0072]

[0087] Next, in step 208, an image is generated based on the pixels within the simulated aperture. This can include averaging the pixel values across the simulated aperture (210) and assigning this value to the pixel at the center of the core. Next, the method includes generating pixel values between the centers of the cores (step 212). Step 212 can include assigning pixel values concentrated at the center of each core by applying a pixel value distribution function, or interpolating between the pixel values of the centers of adjacent cores.

[0073]

[0088] In a preferred embodiment, after averaging the intensity within each simulated aperture, the average value for each region is assigned to the grid position representing the center of the core in the image, and the image is resampled. In the resampled image, the value corresponding to the position of each region (i.e., core) on the grid corresponds to the position on its fiber facet. The image is resampled using a Gaussian intensity profile having a full width at half maximum (FWHM) equal to twice the grid spacing. The FWHM of the Gaussian can be adjusted to provide a balance between signal-to-noise ratio and resolution. The inventors have noticed that a FWHM that is twice the grid sampling slightly low-pass filters the image, but improves the image resolution by averaging the high spatial frequency noise from the non-uniform core spacing. The peak value of the Gaussian distribution representing a given core is equal to the average intensity within a selected pixel subset (i.e., simulated aperture) within the core region.

[0074]

[0089] FIG. 4 shows a series of images showing the output from two embodiments of the present invention compared to the original image series. In this example, the example images show a portion of Group 5 of the USAF 1951 target. Each column of images displays the corresponding image acquired at a given depth. Different columns represent different depths, progressing in 10 μm increments from 10 to 100 μm from left to right.

[0075]

[0090] The top row shows the original image series. As will be appreciated, the original image series has been processed to filter out the pixels in the interstitial space between the fiber cores. This is done using a method similar to that described above. The original image series is constructed by integrating all of the signal within each core (assuming a core radius R = 7 pixels) and then resampling the cores onto a grid. These are referred to as "full aperture" images.

[0076]

[0091] The images in the second and third rows of FIG. 4 are constructed using simulated apertures that are smaller than the overall width of the fiber. The second row uses a "medium" aperture and averages the intensity of the pixels within R = 3 of the center of the core. The bottom row uses a "small" aperture and averages the intensity of the pixels within R = 1 pixel of the center of the core.

[0077]

[0092] As can be seen from the figure, simulated apertures of smaller size have higher contrast at greater depths. For example, the third element lattice (the top of each image) is resolvable using a small simulated aperture at 70 μm but not resolvable using the full aperture. The imaged lattice cannot be resolved in a full aperture image at depths greater than 60 μm. In fact, higher order modes provide less light to the central pixel, and the diffraction imparted by the microscope objective also tends to mix light from the edge and center of the core into the camera image. As a result, the increase in contrast between the full aperture image and the small aperture image is modest in these examples.

[0078]

[0093] FIG. 5A shows a further embodiment of the present invention that can further improve the DOF of the generated image. In FIG. 5A, using the method of FIG. 3, a first image 302 is generated using a first simulated aperture. Also, a second image 304, for example, an original image or an image generated using the method of FIG. 3, can be obtained using a second simulated aperture different from that used to generate the first image 302. The first image 302 is combined with the second image 304 to generate a final image 308. The combining step 306 is a simple defocus correction step that further improves the image quality. In this step, an image created using all the pixels within the fiber core (or a simulated aperture that effectively provides an image that is less sharp and has a shallower DOF compared to that of the first image 302) is subtracted from the first image 302 created using only the central pixel. By doing so, out-of-focus light is subtracted from the first image 302.

[0079]

[0094] Figure 6 shows an image series (labeled eDOF, representing "extended depth of field" images) generated using the embodiment of the process of FIG. 5A. More specifically, it is obtained by subtracting the image series with a radius R = 3 pixels from the image series with a radius R = 1 pixel of FIG. 4 (scale bar: 100 μm).

[0080]

[0095] Returning temporarily to the basic principle behind the present invention, consider a MOF illuminated with a ray (or plane wave) traveling towards an input facet as shown in FIG. 2. The output light intensity profile within each core of the MOF depends on the orientation of the excitation plane wave.

[0081]

[0096] FIG. 7a shows a ray impinging on the input facet of the fiber core in an orientation described by the angle of incidence θ and the azimuth angle φ. These angles are related to the orientation of the ray and not to the shape of the core.

[0082]

[0097] FIG. 7b shows the simulated input core intensity distribution resulting from a plane wave oriented at the angle (θ, φ) (scale bar: 5 μm). The intensity distribution is calculated by forming an incoherent superposition of the linearly polarized (LP) modes of the fiber. The amplitude of each LP mode in the superposition is given by the coupling efficiency of the plane wave oriented at the angle (θ, φ). And the intensity distribution is integrated over the emission spectrum of the light-emitting diode (LED) (Thorlabs M565L3 LED, central wavelength 565 nm, maximum FWHM 100 nm) used in the experiment. The central image is the intensity pattern in the input resulting from a normally incident plane wave, i.e., a paraxial ray. The corner image is an intensity pattern close to θc = sin−1(NAc). rate. And the intensity distribution is integrated over the emission spectrum of the light-emitting diode (LED) (Thorlabs M565L3 LED, central wavelength 565 nm, maximum FWHM 100 nm) used in the experiment. The central image is the intensity pattern in the input resulting from a normally incident plane wave, i.e., a paraxial ray. The corner image is an intensity pattern close to θc = sin−1(NAc). tor. The central image is the intensity pattern in the input resulting from a normally incident plane wave, i.e., a paraxial ray. The corner image is an intensity pattern close to θc = sin−1(NAc).

[0083]

[0098] The intensity pattern is assumed to be invariant from the input facet to the output facet due to the temporally incoherent nature of the illumination. That is, the intensity distribution simulated in Fig. 7b is located at the input facet. Since the field within the core is temporally incoherent, it is expected that the output intensity pattern does not change significantly from the input. This has been experimentally verified by imaging the output of a fiber core that has received plane-wave excitation input at various angles. Fig. 7d shows the experimentally recorded output intensity from the output facet of a single core while varying the plane-wave input angle (θ, φ) using the apparatus of Fig. 2 (scale bar: 5 μm). The plane-wave input was achieved by digitally raster scanning a small circular aperture with a digital micromirror device (DMD) placed conjugate to the rear pupil plane of the microscope objective system. The measured output intensity distribution in Fig. 7d shows a good qualitative agreement with the distribution generated by the simulation for the input facet (Fig. 7b), supporting the view that the intensity profile is relatively unchanged from the input facet to the output facet. Considering this finding, the intensity pattern at the core output can be used as a surrogate for angular filtering of the rays at the core input.

[0084]

[0099] The relationship between the input plane wave and the output intensity pattern within the core can be represented by the matrix equation Ax = b. In the equation, the column A is the intensity pattern created by a specific plane-wave input orientation (i.e., the pattern of Fig. 7d rearranged as a vector), the vector x is the ratio of the input intensity at a given plane-wave orientation, and the vector b is the recorded core output intensity pattern. That is, the output intensity pattern within the core is a linear combination of the intensity patterns created by individual plane waves (i.e., rays). By solving for x, it is possible to separate the contributions coupled to the input facet of the core at low angles, and thus, the collection aperture is reduced and the depth of field is increased. However, the coupling matrix A varies for each core of the fiber due to the non-uniform shape.

[0085]

[0100] Therefore, in order to solve for the contribution (x) for each ray direction within each core, it is necessary to measure the angular coupling matrix A for each core and then perform matrix inversion separately for each core. This can be achieved by careful calibration or simulation.

[0086]

[0101] A preferred embodiment of the present invention uses an approach that does not seek calibration but rather does not require calibration. The preferred technique begins with the finding that light rays traveling perpendicular to the facet interface (the central image in FIG. 7(b), θ = 0) tend to excite the fundamental mode with an intensity that peaks at the core center. Oblique rays (large θ) excite modes with an intensity that is more localized towards the core / clad boundary side. As a result, in the fundamental mode (I = 0), the maximum excitation efficiency occurs at θ = 0. All incident power is coupled to the fundamental mode at normal incidence, but as θ increases, it is distributed between the fundamental mode and increasingly higher-order modes. Thus, by weighting the image towards the boundary (core / clad boundary) between the core and the edge of the fiber, it is possible to extract information regarding the higher-order modes of the received light.

[0087]

[0102] In a preferred embodiment, the present invention relates to the coupling efficiency as a function of the input angle and how the coupling efficiency varies depending on sub-regions of the core output. FIG. 7c plots the normalized total intensity within different sub-regions of the simulated input-output core image (FIG. 7b) as a function of the incident angle θ. The inset in FIG. 7c shows the size of each simulated aperture relative to the full size of the core, overlaid on the intensity pattern enclosed by the circle in FIG. 7b. In FIG. 7, 1px = 2 38.5 nm, which is consistent with the experiments in FIGS. 4 and 6. The shaded background indicates the angular acceptance range within the NA of the MOF.

[0088]

[0103] Specifically, FIG. 7c plots the full aperture image (R = 7px) for the fiber core input circled in FIG. 7b and the simulated normalized intensities within two simulated apertures of R = 3px and R = 1px. The curve labeled "eDOF" is the difference between the R = 1px curve and the R = 3px curve normalized by the aperture area. All curves in FIG. 7c are normalized to have a maximum value of 1. Area normalization is performed as shown in FIG. 7E. FIG. 7E shows the simulated angular PSF normalized by the aperture area. Here, the total intensity within each selected pixel subset that images the core is divided by the area (πR 2 ), i.e., divided by the simulated aperture. The small aperture angle PSF (R = 1px) has the largest size because the average pixel value is the maximum in this sub-region. In contrast, the full aperture angle PSF (R = 7px) has the lowest average value because it contains many blurred pixels that lower the average value. The eDOF curve is calculated as Ismall - Imedium using the curves in this plot directly. This helps to remove the remaining stray light from the R = 1px image. An offset is added so that the curve is positive everywhere. This curve is labeled "eDOF" in FIG. 7E.

[0089]

[0104] In FIGS. 7c and 7E, the intensity within a small "simulated aperture" with a radius R = 1 pixel (1 pixel = 238.5 nm) from the core center has a sharper angular distribution than the "medium" R = 3 image and the "full aperture" R = 7 image. This is qualitatively clear from the example in FIG. 4. As seen in the eDOF plots of FIGS. 7c and 7e, the angular PSF can be made even narrower by subtracting the R = 3px curve from the R = 1px curve after normalizing the total intensity of each curve by the respective aperture area shown in FIG. 7E.

[0090]

[0105] The image formed by this subtraction process will have a large resolution compared to the small aperture image, sacrificing the fact that the signal decreases and the noise increases (noise from the small aperture image and the medium aperture image is added). It should also be noted that the eDOF PSF has a high background level due to the added offset.

[0091]

[0106] Generally, other linear combinations of the simulated aperture-passed images can be used to generate images having various characteristics, such as different depths of field, a trade-off between signal-to-noise ratio (SNR) and angular PSF width. More generally, it is possible to selectively target the imaging of a plane wave oriented at any given angle (θ, φ). FIG. 5B shows such a process. In this example, n images (302, 303, 304) each of which can be obtained from a process similar to that of FIG. 3 but having different simulated apertures are combined in step 306B. That is, each of the n images is generated from a different pixel subset from each image core, and as a result, each image correlates with the light received in the selected spatial arrangement. The images used in such a process may, as in the previous example, result from the average value within an arbitrary pixel subset of the core that is not necessarily concentric with the core center.

[0092]

[0107] The combination in step 306B can be performed according to a given weighting to generate the final image 308B. The weighting used for the combination can be predetermined by a calibration process or derived in an optimization process. For example, the linear combination used in a given situation may be derived by optimizing the combination of a set of images for any image metric such as contrast or signal-to-noise ratio. This optimization can be performed in a greedy manner. Before the combination, the n images can be normalized as shown in FIG. 7e, or the appropriate image normalization factor required to scale the relative amplitudes of the images can be incorporated into the weighting applied during the combination of the images.

[0093]

[0108] As shown, the specific process of FIG. 5A is a special case of the process of FIG. 5B where the image generated by the circular simulated aperture located at the center is combined and the weighting applied to the image with low angular discrimination is -1.

[0094]

[0109] As can be seen from FIG. 6, the resulting eDOF image exhibits a very high fidelity that is resolvable over the entire depth of 100 μm where the top two gratings are continuous. However, the additional contrast in the eDOF image comes at the expense of additional noise that affects the resolution.

[0095]

[0110] FIG. 8 shows an image series of a portion of groups 6 and 7 of the USAF 1951 target. Each column displays an image acquired when the target was placed at a depth of 10 - 100 μm. The top row shows the original image series constructed by integrating all the signals within each core region (R = 7 pixels) and resampling onto the grid. The second and third rows are the same as the top row, but integrate only over pixel subsets centered on each core and having radii of R = 1 and 3 pixels respectively. The bottom row is the eDOF image obtained by subtracting the image series with radius R = 3 pixels from the image series with radius R = 1 pixel. These images are used in addition to the images of FIGS. 4 and 6 to create the SNR curves and resolution curves of FIGS. 6b, 6c, and 6d (scale bar: 100 μm).

[0096]

[0111] To quantify the true gain in image quality as a function of depth, the modulation of the grid lines of group 5 elements 3 - 6 and group 6 elements 1 - 6 is extracted as shown in FIG. 8 and normalized to the noise in the image. The noise (N) is calculated by taking the standard deviation of one group of pixels in the blank region of the image, and the modulation (M) is the average intensity difference between the grid lines and the space between the grid lines. The MATLAB Findpeaks function is used to detect the positions of the grid lines with the condition that the prominence of each peak must be ≧ N / 2. If less than 3 grid lines are resolvable by this criterion, the grid is said to be unresolved and M is set to 0. This noise-normalized modulation is called the signal-to-noise ratio (SNR = M / N). This is conceptually similar to the modulation transfer function (MTF) of an imaging system normalized by noise-equivalent modulation.

[0097]

[0112] Plot 6b shows the SNR as a function of grid spatial frequency at depths of 10, 50, and 100 μm for the image series of FIGS. 4 and 6. The SNR is the degree of modulation of a grid having a given spatial frequency normalized by the noise in the image. The dotted and dashed curves at the same focal position show the original full-aperture image and the final image (of the corresponding eDOF series in FIG. 6), respectively. At a depth of 10 μm, the SNR of the full-aperture image exceeds that of the eDOF image up to 81 lp / mm before becoming irresolvable. Despite the low SNR at low spatial frequencies, the eDOF image has excellent resolution at 91 lp / mm. At larger depths, the eDOF process becomes very advantageous over all spatial frequencies. At 50 μm and 100 μm, the SNR of the eDOF image is the same as or exceeds that of the full-aperture image at all spatial frequencies. At 50 μm, only grids up to 40 lp / mm can be resolved using the full aperture, while in the eDOF image, all grids up to 72 lp / mm can be resolved. At 100 μm, no grid can be resolved using the full aperture, but in the eDOF image, grids of 40 and 45 lp / mm are both resolved.

[0098]

[0113] These data can also be plotted as a function of depth for each spatial frequency as shown in FIG. 6c. In the full-aperture image, the spatial frequency of 81 lp / mm becomes irresolvable before 30 μm, while in the eDOF image, it remains resolvable even beyond 40 μm. The same trend is seen at 57 lp / mm, which is resolvable at depths of up to 30 μm and 70 μm in the full-aperture and eDOF images, respectively. At this spatial frequency, the depth of field is 2.3 times larger. The largest grid period imaged (40 lp / mm) is resolvable at all depths considered in the eDOF image, but only up to 60 μm in the full-aperture image. Similarly, the depth of field of the highest-frequency grid (81 lp / mm) is improved by a factor of 2 from 20 μm in the full aperture compared to 40 μm in eDOF.

[0099]

[0114] The gain in resolution for distant objects becomes even more apparent when plotting the minimum resolvable grid pitch as a function of object depth, as shown in Fig. 6d. The slope of this curve indicates the effective aperture size because the radius of the blur circle predicted by geometric optics grows as Rblur = depth × tanθNA. The line corresponding to the predicted resolution of the full aperture of the MOF (NA = 0.40, dashed line in Fig. 3(d)) has approximately the same slope as the full aperture dataset (dashed line in Fig. 6d). The best fit line for the eDOF data (second dashed line in Fig. 3(d)) has a slope corresponding to NA = 0.15. Thus, the eDOF processing employed effectively stops down the aperture by a factor of 0.40 / 0.15 = 2.67 for resolution purposes.

[0100]

[0115] Next, embodiments of the present invention were tested on 3D objects, namely the fabric fibers of a protective lens pouch. FIG. 9 shows the "full aperture" image and eDOF image of the fabric fibers seen through the MOF. In traditional "full aperture" imaging, only the fabric fibers in contact with the MOF facets are imaged with high fidelity. The remaining fibers further away from the facets appear blurred, contributing to a diffuse background and resulting in low contrast and resolution. FIG. 9a shows fabric fibers imaged using a standard full aperture approach (left column) using both 10x and 20x objective lenses, compared to the eDOF technology according to the present invention (right column). The 10x objective lens sacrifices poor spatial resolution within the core and thus inaccurate aperture filtering to provide a larger field of view than 20x. However, the eDOF technology employed provides significantly improved contrast even in the core region that measures only up to 7 pixels in diameter. A video of fabric fibers moving near the distal MOF facets imaged through the MOF using a 20x objective system was created. The bottom two rows show still images from this video 1 obtained using a 20x objective system. (Scale bar: 100 μm.) The center line (frame 69) indicates the position and direction of the intensity profiles shown in FIG. 9b respectively. FIG. 9b shows the intensity profile along the line shown in the middle row of FIG. 9a. The solid curve is the intensity profile of the eDOF image, and the dotted curve is the intensity profile of the full aperture image. The dots in FIG. 9a indicate the positions of three fabric fibers that are irresolvable in the full aperture image.

[0101]

[0116] Many of the fabric fibers not in contact with the MOF facets are still within the depth of field of the eDOF image and are still resolvable. In particular, the three widely separated peaks (also indicated by dots) of the eDOF curve are brought about by the three fibers in the middle of the line profile, which are not distinguishable in the full aperture curve. This shows that embodiments of this method not only improve contrast but also fundamentally improve the resolution limit at large depths of the 3D structure.

[0102]

[0117] Suitable embodiments preferentially image light coupled to the core at selected angles by selecting a pixel subset from an image region that includes each fiber core for reconstructing an image. Embodiments preferentially select more paraxial rays by selecting a central pixel. Similar to standard imaging devices, this reduction in the converging angle is accompanied by a corresponding increase in the depth of field of view and the noise level. At high spatial frequencies at large depths that are completely suppressed in a low-noise wide-open image, the high resolution of the eDOF image exceeds the additional noise and results in an excellent image in a suitable embodiment. Particularly suitable embodiments may double the depth of field of view at most spatial frequencies and increase the SNR for higher spatial frequencies in the case of distant objects. It should be noted that embodiments of the present invention are fundamentally different from image sharpening techniques such as unsharp masking that can only rescale spatial frequencies and cannot preferentially filter light based on its input angle.

[0103]

[0118] Also, some embodiments employ a more advanced approach to synthesize an image using different simulated apertures, such as a high-low process used in, for example, structured illumination, to further improve the depth of field of view and contrast beyond the exemplary embodiments described in detail herein.

[0104]

[0119] Embodiments of this aspect of the invention provide advantages for MOF imaging, particularly for lensless microendoscopic probes, that enable non-contact imaging without a lens or large scanning device at the distal facet. This means that the MOF probe can be kept slim to reach narrow constrictions within the body. Eliminating the need for a lens assembly at the distal end also reduces the manufacturing cost of the microendoscope. Embodiments of the invention are applicable even when a distal facet lens is required (e.g., to increase magnification). In a MOF microendoscopic system with a lens, an extension of the depth of field may occur not only in front of the MOF facet but also on both sides of the focal plane. Further, embodiments of this technology are completely incoherent and can thus be used in conjunction with wide-field fluorescence imaging. The incoherent nature of this technology dulls the sensitivity to fiber bending, thereby eliminating the need for transmission matrix correction after each fiber perturbation.

[0105]

[0120] As discussed in other parts of this specification, images generated using the methods described herein can advantageously be employed in the context of light field imaging or plenoptic imaging. In other words, an application of the invention is the use of MOF as a light field sensor. The image relayed by the MOF is originally 2D, but the invention provides the insight that a slim MOF-based imaging device can record at least some aspects of the 3D structure of a sample. This is important in a real-world clinical environment where the sample is not a thin, flat tissue section mounted on a microscope slide but a complex, undulating structure.

[0106]

[0121] The invention shows that the MOF transmits 3D image information through the mode structure within each core and uses this information to estimate the average light distribution of the light rays impinging on each core within the MOF. This ray angle information, together with the raw transmission image, describes what is known as the light field. Assuming the light field of the scene, a 3D viewpoint can be reconstructed and the object depth calculated, and the scene can be partially refocused after acquisition.

[0107]

[0122] The light irradiation field dataset includes a complete (θ,φ) parameterization of the incident light ray orientation, enabling 3D viewpoint movement, depth mapping, and refocusing. In conventional light irradiation field imaging, it is generally necessary to collect both the light intensity data and the direction (angle) data of the entire image. In fact, this typically requires taking multiple images of the scene, for example, using the microlens array of the imaging device to capture images at multiple fields of view or focal lengths simultaneously to capture the images at the same time. Alternatively, two images can be acquired at different focal positions, or light irradiation field estimation can be obtained by measuring phase shift information.

[0108]

[0123] As will be appreciated, the inventors have surprisingly determined that it is not necessary to capture images at different focal positions or additionally measure phase shift information to realize at least some of the advantages of light irradiation field photography. And in one aspect, the present invention provides a method capable of estimating the light irradiation field of a scene captured in an image using a single image.

[0109]

[0124] The inventors have determined that multiple images having different effective depths of field (but the same focal position) can be created from a single captured image using the above-described virtual aperture. Next, these images can be used to estimate the light irradiation field of the single captured image. Since only the average direction of light ray propagation within the light irradiation field can be determined, it is necessary to apply an assumption of the angular distribution of light ray propagation at each point. Despite these limitations, the resulting estimated light irradiation field can be used in processes such as generating images at different focal lengths, generating images from different viewpoints, generating a stereoscopic image by synthesizing two images with spatially separated viewpoints, measuring the distance to an object within the image in the same manner as other light irradiation field images.

[0110]

[0125] The inventor further noticed that these techniques are also applicable to multiple images of a scene captured at the same focal position and different depths of field, regardless of how the images are created (i.e., the two images do not have to be generated from a single image described herein using the simulated aperture technique, but can be separately captured in a more traditional way using an optical system to achieve different depths of field). It should be noted that the "focal position" includes the concept of "focal length" when applied to an optical system having a lens.

[0111]

[0126] FIG. 10 shows a method for determining an optical irradiation field approximation according to an embodiment of the present disclosure.

[0112]

[0127] Method 100 begins at step 1002 by acquiring a pair of images of a scene, each having a different depth of field and the same focal position. In a preferred form, the images can be derived using a method according to an aspect of the invention as described, for example, in connection with FIG. 3. Next, at step 1004, a difference image is generated from the pair of images. This difference image is used at step 1006 to calculate the optical irradiation field approximation.

[0113]

[0128] As is known to those skilled in the art, raw MOF images are often downsampled to remove pixelation artifacts due to discrete sampling of the core. This process assumes that there is no useful information contained within the core itself. However, as discussed above in the context of the image generation aspects of the present invention, the core of the MOF is large enough to support a dozen or so modes in the visible spectrum. Such an incoherent superposition of modes is readily observable at the output facet of the MOF. As the angle of incidence of the input light increases, higher-order modes are preferentially excited. As a result, as the input angle of incidence increases, the sub-core output is converted from a central Gaussian spot (fundamental mode) to an expanding ring. In other words, light incident at an oblique input angle tends to produce output light that is localized around the core. Conversely, light incident at a small angle tends to remain preferentially at the center of the core (see Fig. 7d). The depth of field (DOF) of the image can be improved by generating the image using the method described above. This does not teach the full orientation of the light rays required for 3D light field imaging. To extract this information, the technique of "light field moment imaging" (LMI) is employed, which associates the axial intensity derivative of the image with the average (first moment) ray orientation at each pixel. LMI is described in Orth.A. and Crozier.K.B., 2013, Light field moment imaging. Optics letters, 38(15), pp. 2666 - 2668, the content of which is incorporated herein for all purposes for all purposes.

[0114]

[0129] The LMI described therein requires a pair of images with slightly different focal positions as inputs. However, as described above, the bare MOF imaging probe does not have a delicate focus control device. Instead, embodiments of the present invention use images with different depths of field using, for example, the simulated apertures described herein. The inventor has noticed that an image with a large depth of field and a small simulated aperture size is similar to a focused image of an object located at a position away from the fiber facet. Similarly, a largely defocused image with a small depth of field created by a large simulated aperture size is intuitively similar to a defocused image of an object located at a position away from the fiber facet. Images of large simulated apertures can include full aperture images. Hereinafter, this approximation is referred to as the "aperture-focus approximation". Next, the angular illumination field moment can be extracted using the LMI algorithm, and the light illumination field estimation can be constructed by the following equation.

[0115]

Equation

[0116]

[0130] wherein the two images I1 and I2 forming the image pair are a small and a large simulated aperture image, respectively, and M x and M y are the average tilt angles of the light rays from the z-axis to the x-axis and y-axis, respectively (light illumination field moment). Here, Δz is not clearly defined because the two images are used with different effective apertures instead of different focal positions. As a result, Δz is set to an unknown magnification that is adjusted later. The value of the constant Δz does not affect the resulting video, but simply sets the absolute scale of the resulting parallax.

[0117]

[0131] An experimental implementation of this approach is shown in FIG. 11. First, target images of a large simulated aperture and a small simulated aperture (I1 and I2 respectively) are obtained. I2 is obtained by the method described above. However, for the large aperture image I1 (upper left in FIG. 11), the entire core region R (= 5px) is used and the image is generated in the same way as the full aperture image in FIG. 4. The small aperture image I2 (middle upper in FIG. 11) is generated from a pixel subset of the small central region (R = 1px).

[0118]

[0132] Since a lensless MOF is used, the entire scene appears more focused in I2 than in I1 due to the restricted aperture, typically emulating the defocusing process associated with LMI. The subtle difference ΔI between these two images is directly visualized in FIG. 11 (upper right).

[0119]

[0133] Using I1 and I2, ∇U = [M x , M y is used to solve for M x and M y in the Fourier space by the scalar potential U associated with the optical irradiation field moment. Next, the Gaussian optical irradiation field estimate L is constructed as follows using M x and M y .

[0120]

Equation

[0121]

[0134] where u and v are the tilt angles from the z-axis to the x-axis and y-axis respectively. The parameter σ is empirically set to tanθ’, and the optical irradiation field moment is rescaled by a constant coefficient such that max{M x 2 + M y 2} = σ 2 . This ensures that the average optical irradiation field moment is inside the collection aperture.

[0122]

[0135] The Gaussian form L in the (u,v) space is a proof of the fact that when the light irradiation field is densely sampled in the spatial dimensions (x,y), it is necessarily low-pass filtered in the angular (u,v) dimensions due to diffraction limitations. In the most extreme case, this can result in a light irradiation field that includes a single broad spot that effectively reports the inclination of the light rays (i.e., the wavefront) at each spatial position, similar to a Shack-Hartmann wavefront sensor in the angular dimensions.

[0123]

[0136] When the light irradiation field L is estimated according to an embodiment of the present invention, further image processing can be performed as needed.

[0124]

[0137] In one example, the virtual viewpoint of a 3D scene can be changed by selecting 2D slices (fixed angular (u,v) coordinates) of the 4D light irradiation field L. For example, the images of the scene when viewed from horizontally opposed viewpoints are I L = L(x,y,u0,0) and I R = L(x,y,-u0,0). These images then form a stereo pair that can be combined into a red-cyan stereo anaglyph as shown in Fig. 11 (lower right). This image can be viewed using red-cyan glasses to obtain a 3D effect. A parallax scanning animation of the scene viewed from a moving virtual viewpoint can also be constructed. This method is particularly useful for transmitting 3D information via motion parallax.

[0125]

[0138] Parallax is the result of depth changes in a 3D scene (depth = distance from the fiber facet to the object). Assuming a light irradiation field L that includes parallax information in all angular directions, a depth map can be calculated. This can be done using the method described in Adelson E.H. and Wang, J.Y., 1992. Single lens stereo with a plenoptic camera. IEEE transactions on pattern analysis and machine intelligence, 14(2), pp. 99-106.

[0126]

Number

[0127] In the formula, d is the distance from the fiber facet at the position (x, y) to the object.

[0128]

[0139] Hereinafter, d is called the "depth metric" due to the aperture - focus approximation. L x and L y are the (discrete) partial derivative functions of L in the x - direction and y - direction, respectively (similarly, L u and L v are those in the u - direction and v - direction). The summation is performed over the image patch P centered at (x, y) and moving over all (u, v) coordinates. The size of the image patch can be adjusted according to the desired smoothness of the result. A typical size is 9×9 pixels or more. The depth map resulting from a series of images of a USAF target (Group 5) transmitted and illuminated with white light is shown in Fig. 12a. Fig. 12b is a plot of the depth metric as a function of the known ground - truth depth for the target (the distance from the MOF fiber facet to the USAF target). The depth metric value is the average value of all pixels for each ground - truth depth. The error bars indicate one standard deviation of the measured depth metric for each ground - truth depth.

[0129]

[0140] The dataset of FIG. 12a is processed all at once such that the constant Δz is the same for each image. As expected, the color-coded depth metric d indicates that the USAF target moves further away from the fiber facet as the ground truth (applied via a manual micrometer stage) increases from 0 to 90 μm. When a vertical tilt is applied to the USAF target, the depth change can be clearly seen in the color-coded image closer to the fiber facet than the bottom. As a result of the aperture / focus approximation, the relationship with the depth metric is slightly non-linear, especially at particularly short distances, as shown in FIG. 12(b). Nevertheless, useful 3D information such as the relative depth levels of objects within the scene can still be obtained. If necessary, the depth values can be re-scaled to their true values using a calibration look-up table from measurements of the test object at known distances. and

[0130]

[0141] Another well-known use of light field imaging is synthetic refocusing. The data contained in the light field enables the rearrangement of the spatial-angular structure of light in order to digitally process the focus of the captured image. This is most easily understood by first imaging the 3D scene at all viewpoints in the (u, v) space. To create a synthetically refocused image at a given depth, it is first necessary to correct the parallax that can occur for the object at each viewpoint at the above depth. This means a translational shift of the image in the (x, y) space that is proportional to the (u, v) vector describing the viewpoint coordinates. Once this parallax is taken into account, the shifted images are summed to create a synthetically refocused image (this is sometimes called the "shift and add" method). Even with the aperture-focus approximation, synthetic refocusing using the light field estimation obtained from the MOF image using embodiments of the present invention is possible.

[0131]

[0142] Figure 13a shows the USAF target images at 150 μm refocused at various distances increasing from left to right. Next, the optical irradiation field estimation L is computationally refocused at distances continuously increasing from the fiber facet (the order of the images is near → far from left to right). The initially defocused image (leftmost panel (13a)) converges to the focus (middle panel (13a)) and then becomes overfocused and blurred (rightmost panel (13a)). Next, synthetic refocusing was tested for the distance range from the fiber facet to the target. Figure 13b shows the original images of the USAF target at distances of 0 to 250 μm in the upper row (scale bar: 100 μm). The lower row shows the synthetically refocused sharpest images judged by eye for each distance. Note that since refocusing is performed after image capture, the target has not been moved to obtain this effect. This result is purely computational. Also, the difference between the leftmost image in Figure 13a and the "original" 150 μm image in Figure 13b is due to the Gaussian optical irradiation field model used to construct the optical irradiation field estimation. Figure 13a results from this optical irradiation field estimation, while the "original" image in Figure 13b is acquired before or without performing the optical irradiation field estimation since there is no need for optical irradiation field data. When synthetic refocusing is performed, clear sharpening can be confirmed in the images at distances of 50 μm and 100 μm. There may be a discussion that these can be obtained by appropriate deconvolution or unsharp masking. However, the latter cannot be said for the refocused images at 150 - 250 μm where the grid lines that are completely blurred in the original image are resolved in the optimally refocused image.

[0132]

[0143] As described above, various approaches for 3D visualization of an object using the image provided by the present invention. For example, the 3D structure of a scene can be directly observed by stereoscopic images such as stereoscopic photographs and stereoscopic anaglyphs (e.g., using red-cyan stereoscopic glasses or VR goggle devices) and viewpoint movement (parallax) animations. Alternatively, depth mapping techniques can be applied using, for example, a depth map constructed by maximum intensity projection of a deconvolved light illumination field focus stack.

[0133]

[0144] As can be seen from the above, the image processing method described in this specification enables the use of MOF as a light illumination field imaging element. Using MOF for light illumination field imaging enables the use of an endoscope that is significantly slimmer than existing rigid stereomicroendoscopes that rely on a pair of separated optical imaging paths to record stereoscopic data.

[0134]

[0145] More advantageously, a preferred form of the technology disclosed in this specification is that since all the data required for light illumination field estimation is contained within individual cores, no hardware modification to the MOF-based system is required. No hardware modification to the MOF-based system is required.

[0135]

[0146] A test including imaging of scattered animal tissues using the present invention on a cell structure (specifically, a 5-mm thin slice of a mouse brain stained with proflavine imaged through a fiber bundle) showed a very good quantitative agreement between the proflavine depth distribution measured by the light illumination field approach according to the present invention and the proflavine depth distribution obtained using a desktop confocal microscope.

[0136]

[0147] FIG. 14 is a block diagram showing a typical computer processing system 400 suitable for use / configuration as an image processing system for processing an image captured by the camera CAM of the system of FIG. 2 according to any of the various aspects and embodiments described herein. The image processing system may be a separate computer system (which may be located far from the imaging component) or may form part of the control system of the MOF imaging system.

[0137]

[0148] The computer processing system 400 includes a processing unit 402. The processing unit 402 can include a single computer processing device (e.g., a central processing unit, a graphics processing unit, or other computing device) or can include a plurality of computer processing devices. In some examples, the processing is performed only by the processing unit 402, but in other examples, the processing can also be alternatively performed by a remote processing device that is accessible and usable (either shared or dedicated) in the computer processing system 400.

[0138]

[0149] Via a communication bus 404, the processing unit 402 communicates data with one or more machine-readable storage (memory) devices that store instructions and / or data for controlling the operation of the computer processing system 400. In this example, the computer processing system 400 includes a system memory 406 (e.g., BIOS or flash memory), volatile memory 408 (e.g., random access memory such as one or more DRAM modules), and non-volatile / non-transitory memory 410 (e.g., one or more hard disks or semiconductor drives).

[0139]

[0150] Computer processing system 400 also includes one or more interfaces, generally indicated at 412, through which computer processing system 400 interfaces with various components, other devices and / or networks. The other components / devices may be physically integrated with computer processing system 400 or may be physically separate. If such devices are physically separate, their connections to computer processing system 400 may be via wired or wireless hardware and communication protocols, and may be direct or indirect (e.g., networked) connections.

[0140]

[0151] Wired connections to other devices / networks may be via any standard or proprietary hardware and connection protocol. For example, computer processing system 400 may be configured to connect to other devices / communications networks via one or more of the following: USB, FireWire, eSATA, Thunderbolt, Ethernet, parallel, serial, HDMI, DVI, VGA, AudioPort. Other wired connections are possible.

[0141]

[0152] Wireless connections to other devices / networks may similarly be by any standard or proprietary hardware and communication protocol. For example, the computer processing system 400 may communicate with a variety of wireless devices including infrared, Bluetooth (an early version of Bluetooth, Bluetooth 5.0, The device may be configured to wirelessly connect to other devices / communications networks using one or more of the following technologies: Bluetooth 4.0 / 4.1 / 4.2 (also known as Bluetooth Low Energy and future versions of Bluetooth), WiFi, Near Field Communication (NFC), Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Long Term Evolution (LTE), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA). Other wireless connections are possible.

[0142]

[0153] Generally speaking, the devices connected to the computer processing system 400, whether by wired means or wireless means, enable data input to the computer processing system 400 for processing by the processing unit 402 / data reception by the computer processing system 400, and data output by the computer processing system 400. Exemplary devices are described below, but it will be understood that not all computer processing systems will include all of the devices described, and that additional and alternative devices to those described will also be used.

[0143]

[0154] For example, computer processing system 400 may include or be connected to one or more input devices through which information / data is input (received by computer processing system 400). Such input devices can include physical buttons, alphanumeric input devices (such as keyboards), pointing devices (such as mice, trackpads, etc.), touchscreens, touchscreen displays, microphones, accelerometers, proximity sensors, GPS devices, and the like. Computer processing system 400 may also include or be connected to one or more output devices 414 that are controlled by computer processing system 400 to output information. Such output devices can include indicators (such as LEDs, LCDs, or other lights), displays (such as LCD displays, LED displays, plasma displays, touchscreen displays), audio output devices such as speakers, vibration modules, and other output devices. Computer processing system 400 may include or be connected to devices that can both input and output, such as memory devices (hard drives, semiconductor drives, disk drives, compact flash cards, SD cards, etc.) through which computer processing system 400 can read and / or write data, and touchscreen displays that can both display (output) data and receive touch signals (input).

[0144]

[0155] Computer processing system 400 may also be connected to a communication network (such as the Internet, local area network, wide area network, personal hotspot, etc.) to communicate data with networked devices that can be other computer processing systems.

[0145]

[0156] The architecture shown in FIG. 14 can be implemented in various computer processing systems, such as laptop computers, netbook computers, tablet computers, smartphones, desktop computers, and server computers. It will also be understood that FIG. 14 does not show all the functional or physical components of the computer processing system. For example, a power source or power interface is not shown, but the computer processing system 400 can hold a power source (e.g., a battery) and / or be connectable to a power source. A particular type of computer processing system determines appropriate hardware and architecture, and alternative computer processing systems may have additional components, alternative components, or a smaller number of components than shown, combine two or more components, and / or will further be understood to have different configurations or arrangements of components.

[0146]

[0157] The operation of the computer processing system 400 is also provided by one or more computer program modules that configure the computer processing system 400 to receive, process, and output data.

[0147]

[0158] As used herein, the term "module" refers to computer program instructions and other logic for providing a particular function. A module can be implemented in hardware, firmware, and / or software. A module is typically stored in the storage device 408, loaded into the memory 406, and executed by the processor 402.

[0148]

[0159] A module can include one or more processes and / or can provide only a part of a process. Embodiments described herein can include modules that are different and / or distinct from those described herein. Also, functions attributed to a module can be performed by different or distinct modules in other embodiments. Further, this description may sometimes omit the term "module" for convenience and clarity.

[0149]

[0160] The type of computer processing system 400 used can vary depending on the embodiment and the processing capabilities used by the entity. For example, a server system can include a plurality of blade servers that cooperate to provide the functions described herein.

[0150]

[0161] As will be appreciated, the approach of the present invention has a limited camera frame rate, does not require calibration, does not move in response to moderate fiber bending, and is suitable for potential clinical applications.

[0151]

[0162] Other incoherent imaging methods such as brightfield imaging can also be modified to this approach and can also be used with a fiber bundle employing a digital lens.

[0152]

[0163] As discussed above, embodiments of the present invention are related to the relationship between the core intensity pattern and the angular dimension of the light irradiation field incident on the distal end of the fiber bundle. The analysis included in Appendix A provides a quantification of this relationship.

[0153]

[0164] The gist of this relationship is the fact that the regular LMI equation (Equation 2 above) is modified for application to image pairs with the same focal position but different collection apertures. This occurs because the centroid movement of the point source (stereo parallax, or lateral movement) is not linear with respect to z as it is in the case of a standard light irradiation field.

[0154]

[0165] The above disclosure relates to embodiments of the present invention for generating or modifying an image using "simulated aperture" technology applied to a fiber core, but it will be understood that other methods of processing or analyzing an image intensity pattern across each core can be used to extract the light irradiation field angle information of that core. For example, a pattern matching algorithm can be applied that compares the image intensity pattern to a stored pattern generated for the MOF by a pattern calibration process. The calibration process includes obtaining a reference image for a point source at each of a plurality of angles. These reference images are then used to generate a pattern that is stored for each core that can be compared to the received image using a standard computational pattern matching algorithm.

[0155]

[0166] It is understood that the invention disclosed and defined in this specification extends to all alternative combinations of two or more of the individual features described or apparent in the text or drawings. All of those different combinations constitute various alternative aspects of the invention.

[0156] Appendix A Principle of operation Consider the distance z from a point source imaged using an optical fiber bundle to the fiber facet. The light ray makes an angle θ with the fiber facet at positions x, y from the centerline of the fiber bundle, where θ x and θ y are the angles of inclination of the light ray from the yz plane and the xz plane, respectively. To show this, the raw output image of fluorescent beads at an axial distance z = 26 μm received at the proximal end of the fiber bundle is shown in Fig. 15a (scale bar 5 μm). The radially symmetric pattern of the fiber mode is visible from the relationship between the mode coupling efficiency and the input light ray angle θ. The fiber bundle used in this study included up to 30,000 substantially circular cores with an outer diameter of 750 μm, an average center-to-center spacing of 3.2 μm, an average core radius a = 1 μm, and a numerical aperture (NA) of 0.39. On average, each core of this fiber bundle supports approximately (2πaNA / λ) / 2 ≈ 10 modes at λ = 550 nm (24). 2 As discussed in other parts of this specification, post - processing of image data enables digital manipulation of the numerical aperture (NA) of the fiber. This relies on the fact that higher - order modes, which are preferentially excited at larger incident angles, carry more energy near the core / clad boundary than lower - order modes. Light is effectively directed towards the edge of each core as the ray angle increases. A synthetically limited NA is achieved by the digital aperture filtering approach of embodiments of the present invention (removing light near the periphery of each core). This is shown in FIGS. 15, particularly FIGS. 15c and 15d. The full orientation of the input light cannot be confirmed from this observation alone due to the azimuthal degeneracy of the core modes. To address this, LMI is applied. In LMI, the average ray direction (represented by the optical irradiation field moment vector

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

Claims

1. 1. A method for generating an image from light received by an imaging device through multiple waveguides, the method comprising: receiving a digital image including a plurality of pixels, the digital image including a plurality of regions therein each corresponding to a waveguide core and including a plurality of pixels, and further including pixels corresponding to interstitial spaces between the waveguide cores; determining a first subset of pixels in each region that at least partially correlates with light received at a corresponding core in the first spatial arrangement, the first subset of pixels including fewer than all pixels in the region; and generating a first image from the first subset of pixels of the region.

2. Producing the first image comprises: The method of claim 1 , comprising: for each region, determining an average pixel value of the pixels of the first subset; and assigning the average pixel value as the pixel value of at least one pixel in the first pixel subset.

3. Producing the first image comprises: The method of claim 1 or claim 2, comprising generating a pixel value for the at least one non-pixel.

4. 4. A method according to claim 2 or claim 3, wherein the average pixel value is assigned to a pixel at a predetermined position within the image which represents the centre of the waveguide core.

5. generating a pixel value for the at least one non-pixel pixel; assigning pixel values ​​according to a pixel value distribution function centered on said at least one pixel; or 5. The method of claim 2, further comprising: assigning pixel values ​​by interpolating between said at least one pixel of adjacent regions.

6. The method of claim 1 , wherein the first subset of pixels comprises all pixels within a predetermined radius from a centre of the region.

7. generating a second image from the received digital image; and The method of claim 1 , further comprising combining the second image with the first image to generate a final image.

8. The second image is defining a second subset of pixels in each region, the second subset of pixels including fewer than all of the pixels in the region and different from the first subset of pixels; and The method of claim 7 , wherein the second image is generated by generating the second image from the second pixel subset of the region.

9. 9. The method of claim 7 or claim 8, wherein combining the second image with the first image comprises scaling the luminance of one or both images if necessary, and subtracting the second image from the first image.

10. The method of claim 7 , wherein the first image has a greater effective depth of field than the second image.

11. 11. The method of claim 7, wherein the generation of the first image is biased towards a selection of light rays received at the waveguide within a first range of angles, and the generation of the second image is biased towards a selection of light rays received at the waveguide within a second range of angles.

12. The method of claim 11 , wherein the second range of angles is greater than the first range of angles.

13. 1. A method for determining a light field approximation for use in image processing corresponding to a pair of images generated from light received by an imaging device through various waveguides, the method comprising: acquiring a pair of images, a first member image having a first depth of field and a second member image having a second depth of field, the first member image and the second member image having the same focus position; generating a difference image from the pair of images; calculating a light field approximation from the difference image.

14. The method of claim 13 , wherein the process of calculating the light field approximation includes using an assumed angular distribution of light propagation about a mean ray orientation.

15. 15. A method according to claim 13 or claim 14, wherein the second member image is obtained using a method according to any one of claims 1 to 6.

16. 16. The method of claim 15, wherein the first member image is acquired using a method according to any one of claims 1 to 6, and the first member image and the second member image use different first pixel subsets within each region.

17. 16. The method of claim 15, wherein the first member image is obtained from the same digital image as the second member image and is generated from substantially all pixels in an area of ​​the digital image corresponding to the waveguide core.

18. 1. A method for improving the apparent depth of field of a digital image captured through a multi-core optical fiber (MOF), the digital image comprising a plurality of pixels, the digital image including a plurality of regions therein each corresponding to a core of the MOF and including a plurality of pixels, and further including pixels corresponding to interstitial spaces between the waveguide cores, the method comprising: determining a first subset of pixels in each region that at least partially correlates with light received at a corresponding core in the first spatial arrangement, the first subset of pixels including fewer than all pixels in the region; determining, for each region, an average pixel value of the pixels of the first subset and assigning the average pixel value as the pixel value of at least one pixel of the first pixel subset; and generating a pixel value for the at least one non-pixel; generating a first image having an improved depth of field by:

19. generating pixel values ​​for the at least one non-pixel pixel in the first pixel subset; assigning pixel values ​​according to a pixel value distribution function centered on said at least one pixel of each first region; or assigning pixel values ​​by interpolating between pixel values ​​of the first subset of adjacent regions.

20. generating a second image from the received digital image; and Combining the second image with the first image to produce a final image having improved depth of field. and generating The second image is defining a second subset of pixels in each region, the second subset of pixels including fewer than all of the pixels in the region and different from the first subset of pixels; and 20. A method according to claim 18 or claim 19, wherein the second image is generated by generating the second image from the second subset of pixels of the region.

21. 1. A method for generating an image, comprising: acquiring a pair of images, a first member image having a first depth of field and a second member image having a second depth of field, the first member image and the second member image having the same focus position; Determining a light field approximation according to any one of claims 13 to 17, processing the image according to said light field approximation to generate a final image.

22. 1. A method of generating one or more images from light received by an imaging device through multiple waveguides, the light being generated from a light field incident on the multiple waveguides, the method comprising: receiving a digital image including a plurality of regions each corresponding to a waveguide core and including a plurality of pixels; processing the image intensity pattern over each of said regions to determine a light field angle dimension measure for that region; applying the angular dimension measure to one or more pixels contained in each region to generate one or more sets of modified image data; generating one or more images using the one or more sets of modified image data.

23. a multi-core optical fiber (MOF) extending from a proximal end to a distal end; a light source for illuminating a scene at the distal end of the MOF; and an imaging device positioned relative to the proximal end of the MOF to capture an image of light propagated along the MOF; and 23. An imaging system comprising: a data processing system configured to receive an image captured by the imager, the data processing system configured to execute instructions to cause the data processing system to carry out the method of any one of claims 1 to 22.

24. The imaging system of claim 23 , wherein the MOF comprises an endoscope.

25. 22. An image processing system comprising at least one processing unit and at least one memory for storing instructions executed by said at least one processing unit, said instructions being executed to perform the method of any one of claims 1 to 21.

Images