A multimode fiber optic endoscope system based on incoherent light source

Abstract

A multimode fiber optic endoscope system based on an incoherent light source includes a light source, a coupling system, a multimode optical fiber, a collimation system, and a CCD camera. The light source is an incoherent light source, and the CCD camera is a color CCD camera or a black and white CCD camera with different filters. The endoscope system needs to be trained before use to establish a training set of the correspondence between object images and speckle patterns. During use, the outgoing light of the incoherent light source passes through the coupling system, the optical fiber, and the collimation system in sequence and is incident on the detection object. The light reflected by the detection object returns and is incident on the CCD camera. The CCD camera extracts the speckle information corresponding to light of different wavelengths to obtain three speckle patterns of red light, green light, and blue light. The three speckle patterns are then grayscaled and input into a Unet neural network using channel splicing. The correspondence between the object image and the speckle established by the Unet neural network is used to reconstruct the image of the detection object after optical fiber transmission.

Description

Technical Field

[0001] The invention relates to a multimode optical fiber endoscope system based on an incoherent light source, belonging to the technical field of endoscope imaging. Background Art

[0002] An endoscope is a long-range inspection device that can penetrate deep into the human body and into areas inaccessible to traditional optical imaging devices for detection and imaging. Currently, there are two types of endoscopes in use: fiber-optic bundle endoscopes based on multiple single-mode fiber bundles, and scanning endoscopes that use a single single-mode fiber for scanning and sampling. In fiber-optic bundle endoscopes based on multiple single-mode fiber bundles, each single-mode fiber corresponds to a single pixel. To improve imaging resolution, the number of fibers must be increased and the spacing between fibers must be reduced. However, a large number of single-mode fibers results in a larger probe head and reduced flexibility. Furthermore, the spacing between single-mode fibers is affected by the fiber manufacturing process, making it difficult to achieve less than 1μm. For single-mode fiber scanning endoscopes, the probe requires integrating a scanning galvanometer and microlens device, resulting in a larger probe and less flexibility. The large size of endoscopes not only causes pain to the patient during the inspection process, but also limits their use within narrow cavities.

[0003] Compared to single-mode fiber, multimode fiber has a thicker core diameter and can simultaneously transmit multiple modes of light, enabling high-resolution imaging over a wide area. Because the speckle patterns corresponding to different images under incoherent light illumination are much less distinct than those under coherent light illumination, existing multimode fiber endoscopy systems generally use coherent light sources for illumination. However, since coherent light undergoes mode coupling within a multimode fiber, its internal coupling state changes when the fiber morphology changes. Existing multimode fiber endoscopy imaging technologies are based on a fixed coupling state within the fiber. This makes image reconstruction difficult when the fiber morphology and the environment change. To address this issue, scientists have proposed using machine learning to reconstruct images. Machine learning involves training a neural network on the input image and output speckle pattern of a multimode fiber to fit the image's transmission characteristics within the fiber. While previous studies have shown that machine learning methods for multimode fiber imaging can be somewhat robust to environmental disturbances, image reconstruction using trained neural networks remains problematic when large disturbances (such as significant changes in the fiber's bend radius) are encountered. Furthermore, scientists have proposed that incorporating the corresponding input image and speckle pattern of the fiber's shape change as a dataset during neural network training could, to some extent, enhance the fiber's ability to resist disturbances. However, this method requires a significant amount of time for data acquisition, and the increased computing power required due to the larger dataset. Furthermore, the enhanced disturbance resistance this method can provide is relatively limited. Furthermore, since coherent light is used as the light source, only a single-wavelength speckle pattern can be used as input, resulting in less information available for neural network reconstruction. Furthermore, this method still cannot achieve real-time, high-fidelity detection of unknown disturbances, such as changes in ambient temperature and humidity. Summary of the Invention

[0004] The present invention aims to provide a multimode fiber optic endoscope system based on an incoherent light source. This system uses an incoherent light source instead of a traditional laser as the illumination source for the multimode fiber optic endoscope detection system. This addresses the effects of changes in the fiber or the environment on the image transmission characteristics within the fiber, enabling real-time, dynamic, and rapid endoscope detection imaging. Furthermore, the system utilizes the rich wavelength resources of the broadband light source to obtain more object information. Through channel splicing, speckle patterns of different wavelengths are simultaneously input into a neural network to improve its reconstruction performance.

[0005] The technical solution adopted by the present invention to achieve its inventive objectives is: a multimode fiber optic endoscope system based on an incoherent light source, including a light source, a coupling system, a multimode optical fiber, a collimation system, and a CCD camera. Its structural characteristics are: the light source is an incoherent light source, and the CCD camera is a color CCD camera or a black-and-white CCD camera with different filters. When the black-and-white CCD camera with different filters is used, different filters are placed in front of the black-and-white CCD camera to obtain speckle information of different wavelengths;

[0006] Before using the endoscope system, it needs to be trained to establish a training set of correspondences between object images and speckle patterns. The process of establishing each correspondence in the training set is as follows: the outgoing light of the incoherent light source is coupled into the multimode optical fiber through the coupling system, and after being transmitted through the multimode optical fiber, it is incident on the target object used to establish the training set through the collimation system. The light reflected by the target object is again incident on the CCD camera through the collimation system, multimode optical fiber and coupling system. The CCD camera extracts the speckle information corresponding to the light of different wavelengths reflected by the target object, and obtains three speckle patterns of red light, green light and blue light. The three speckle patterns corresponding to the same target object are then grayscaled and input into the Unet neural network using channel splicing. The object image of the target object is then input into the Unet neural network, and the Unet neural network is used to establish the correspondence between the object image and the speckle pattern.

[0007] The process of using the endoscope system is as follows: the output light of the incoherent light source is coupled into the multimode optical fiber through the coupling system, and after being transmitted through the multimode optical fiber, it is incident on the detection object through the collimation system. The light reflected by the detection object is again incident on the CCD camera through the collimation system, multimode optical fiber and coupling system. The CCD camera extracts the speckle information corresponding to the light of different wavelengths reflected by the detection object, and obtains three speckle patterns of red light, green light and blue light. Then, the three speckle patterns corresponding to the same detection object are grayscaled and input into the Unet neural network through channel splicing. The correspondence between the object image and the speckle established by the Unet neural network is used to reconstruct the image of the detection object after transmission through the multimode optical fiber.

[0008] Compared with the prior art, the present invention has the following beneficial effects:

[0009] Compared with single-mode optical fiber, multimode optical fiber can accommodate more modes. Multimode optical fiber of the same size can achieve higher imaging resolution than single-mode optical fiber bundles, and can also enter narrower channels for endoscopic detection. The present invention proposes a multimode optical fiber endoscope system that uses incoherent light illumination detection. Since there is no interference phenomenon in incoherent light, there is no mode coupling inside the multimode optical fiber, which makes the present invention have very strong anti-interference ability. When the multimode optical fiber morphology changes over a large range, or even when it is constantly shaking, a clear image can be reconstructed. However, incoherent light sources have rich wavelength resources. The grayscale speckle pattern captured by the black and white CCD camera in the prior art is a superposition of speckle patterns of all wavelengths. For a multimode optical fiber system, the transmission paths of light of different wavelengths reflected by the same object are inconsistent within the multimode optical fiber, and it is believed that the mapping relationship between the input and output of different wavelengths is inconsistent. Therefore, it is difficult to use grayscale patterns to directly restore object information through neural networks. This invention leverages the channel-splitting capabilities of color CCD cameras or black-and-white CCD cameras with different filters to directly obtain speckle patterns corresponding to multiple wavelengths when a broad-spectrum incoherent light source is used to illuminate the same object. This multi-wavelength channel splicing of speckle patterns is then fed into a neural network to enrich object information and enhance the neural network's reconstruction capabilities. Ultimately, this system achieves a multimode fiber endoscopic imaging system using incoherent broad-spectrum light illumination.

[0010] Furthermore, the incoherent light source of the present invention includes an LED light source.

[0011] Furthermore, the incoherent light source of the present invention is a white light LED light source.

[0012] Compared with other monochromatic LEDs, white light LEDs have richer wavelength resources and can obtain speckle patterns corresponding to multiple wavelengths. These patterns can be spliced ​​and sent to the neural network, providing the neural network with richer information about the object to achieve better reconstruction effects.

[0013] Furthermore, the training process of the present invention selects N target objects to establish a training set, where N≥30,000.

[0014] To ensure better training results, a large dataset is required to train the neural network. Considering both training results and training time, more than 30,000 target objects are more suitable for the technical solution of the present invention.

[0015] Furthermore, in the training process of the present invention, inputting the object image of the target object into the Unet neural network specifically involves loading the grayscale pattern of the target object through the DMD and inputting the grayscale pattern into the Unet neural network.

[0016] A DMD, or digital micromirror device, is composed of numerous tiny mirrors that can be loaded with grayscale images to simulate objects. Neural network training requires sufficient data. Using a DMD instead of actual objects can help obtain more objects and their corresponding speckle data, improving the neural network's reconstruction.

[0017] The present invention will be further described in detail below through specific implementation methods and drawings, but this does not mean to limit the scope of protection of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the overall structure of an endoscope system according to an embodiment of the present invention.

[0019] Figure 2 It is a schematic diagram of a process of extracting speckle information corresponding to light of different wavelengths reflected by a detection object by a CCD camera according to a first embodiment of the present invention.

[0020] Figure 3 This is a diagram showing the reconstruction effect of the image of the detected object by the endoscope system according to the first embodiment of the present invention.

[0021] Figure 4 This is a comparison diagram of the reconstruction effect of the detected object image before and after the optical fiber position of the endoscope system is disturbed in Example 1 of the present invention. DETAILED DESCRIPTION

[0022] Example 1

[0023] A multimode fiber optic endoscope system based on an incoherent light source includes a light source, a coupling system, a multimode fiber, a collimation system, and a CCD camera. In this example, the light source is an incoherent light source, and the CCD camera is a color CCD camera. Figure 1 Schematic diagram of the overall structure of the endoscope system of this embodiment, in which the CCD is a color CCD camera;

[0024] Before using the endoscope system, it needs to be trained to establish a training set of correspondences between object images and speckle patterns. The process of establishing each correspondence in the training set is as follows: the outgoing light of the incoherent light source is coupled into the multimode optical fiber through the coupling system, and after being transmitted through the multimode optical fiber, it is incident on the target object used to establish the training set through the collimation system. The light reflected by the target object is again incident on the CCD camera through the collimation system, multimode optical fiber and coupling system. The CCD camera extracts the speckle information corresponding to the light of different wavelengths reflected by the target object, and obtains three speckle patterns of red light, green light and blue light. The three speckle patterns corresponding to the same target object are then grayscaled and input into the Unet neural network using channel splicing. The object image of the target object is then input into the Unet neural network, and the Unet neural network is used to establish the correspondence between the object image and the speckle pattern.

[0025] The process of using the endoscope system is as follows: the output light of the incoherent light source is coupled into the multimode optical fiber through the coupling system, and after being transmitted through the multimode optical fiber, it is incident on the detection object through the collimation system. The light reflected by the detection object is again incident on the CCD camera through the collimation system, multimode optical fiber and coupling system. The CCD camera extracts the speckle information corresponding to the light of different wavelengths reflected by the detection object, and obtains three speckle patterns of red light, green light and blue light. Then, the three speckle patterns corresponding to the same detection object are grayscaled and input into the Unet neural network through channel splicing. The correspondence between the object image and the speckle established by the Unet neural network is used to reconstruct the image of the detection object after optical fiber transmission. Figure 2 FIG. 1 is a flow chart of the process of extracting speckle information corresponding to light of different wavelengths reflected by a detection object by the CCD camera of this embodiment. Figure 3 This is a diagram showing the reconstruction effect of the image of the detected object by the endoscope system of this embodiment.

[0026] In this embodiment, the anti-interference capability of the endoscope system of this embodiment is tested by disturbing the optical fiber. Figure 4 This is a comparison diagram of the reconstruction effect of the detected object image before and after the optical fiber position of the endoscope system of this embodiment is disturbed. Figure 4 The left side of the middle image shows the reconstruction effect when the optical fiber position does not change and no disturbance occurs, and the right side shows the reconstruction effect when the optical fiber position changes and disturbance occurs. Figure 4 It can be seen that when the position of the optical fiber changes and disturbance occurs, the endoscope system of this embodiment can still obtain a better reconstructed image. Figure 3 and Figure 4 In [1], SSIM is the abbreviation of Structural Similarity, which stands for structural similarity.

[0027] The incoherent light source in this example is a white light LED light source.

[0028] In the training process of this example, N target objects are selected to establish a training set, where N=32,000.

[0029] In the training process described in this example, inputting the object image of the target object into the Unet neural network specifically involves loading the grayscale pattern of the target object through the DMD and inputting the grayscale pattern into the Unet neural network.

[0030] Example 2

[0031] A multimode fiber optic endoscope system based on an incoherent light source includes a light source, a coupling system, a multimode optical fiber, a collimation system, and a CCD camera. In this example, the light source is an incoherent light source, and the CCD camera is a black and white CCD camera with different filters. When the black and white CCD camera with different filters is used, different filters are placed in front of the black and white CCD camera to obtain speckle information of different wavelengths.

[0032] Before using the endoscope system, it needs to be trained to establish a training set of correspondences between object images and speckle patterns. The process of establishing each correspondence in the training set is as follows: the outgoing light of the incoherent light source is coupled into the multimode optical fiber through the coupling system, and after being transmitted through the multimode optical fiber, it is incident on the target object used to establish the training set through the collimation system. The light reflected by the target object is again incident on the CCD camera through the collimation system, multimode optical fiber and coupling system. The CCD camera extracts the speckle information corresponding to the light of different wavelengths reflected by the target object, and obtains three speckle patterns of red light, green light and blue light. The three speckle patterns corresponding to the same target object are then grayscaled and input into the Unet neural network using channel splicing. The object image of the target object is then input into the Unet neural network, and the Unet neural network is used to establish the correspondence between the object image and the speckle pattern.

[0033] The process of using the endoscope system is as follows: the output light of the incoherent light source is coupled into the multimode optical fiber through the coupling system, and after being transmitted through the multimode optical fiber, it is incident on the detection object through the collimation system. The light reflected by the detection object is again incident on the CCD camera through the collimation system, multimode optical fiber and coupling system. The CCD camera extracts the speckle information corresponding to the light of different wavelengths reflected by the detection object, and obtains three speckle patterns of red light, green light and blue light. Then, the three speckle patterns corresponding to the same detection object are grayscaled and input into the Unet neural network through channel splicing. The correspondence between the object image and the speckle established by the Unet neural network is used to reconstruct the image of the detection object after optical fiber transmission.

[0034] The incoherent light source in this example is a white light LED light source.

[0035] In the training process of this example, N target objects are selected to establish a training set, where N=33,000.

[0036] In the training process described in this example, inputting the object image of the target object into the Unet neural network specifically involves loading the grayscale pattern of the target object through the DMD and inputting the grayscale pattern into the Unet neural network.

Claims

1. A multimode fiber optic endoscope system based on an incoherent light source, comprising a light source, a coupling system, a multimode fiber, a collimation system, and a CCD camera, characterized in that: The light source is an incoherent light source, a white light LED light source; the CCD camera is a color CCD camera or a black and white CCD camera with different filters. When the black and white CCD camera with different filters is used, different filters are placed in front of the black and white CCD camera to obtain speckle information of different wavelengths; Before using the endoscope system, it needs to be trained to establish a training set of correspondences between object images and speckle patterns. The process of establishing each correspondence in the training set is as follows: the outgoing light of the incoherent light source is coupled into the multimode optical fiber through the coupling system, and after being transmitted through the multimode optical fiber, it is incident on the target object used to establish the training set through the collimation system. The light reflected by the target object is again incident on the CCD camera through the collimation system, multimode optical fiber and coupling system. The CCD camera extracts the speckle information corresponding to the light of different wavelengths reflected by the target object, and obtains three speckle patterns of red light, green light and blue light. The three speckle images corresponding to the same target object are then grayscaled and input into the Unet neural network through channel splicing. The object image of the target object is then input into the Unet neural network, and the Unet neural network is used to establish the correspondence between the object image and the speckle pattern. The process of using the endoscope system is as follows: the output light of the incoherent light source is coupled into the multimode optical fiber through the coupling system, and after being transmitted through the multimode optical fiber, it is incident on the detection object through the collimation system. The light reflected by the detection object is again incident on the CCD camera through the collimation system, multimode optical fiber and coupling system. The CCD camera extracts the speckle information corresponding to the light of different wavelengths reflected by the detection object, and obtains three speckle patterns of red light, green light and blue light. Then, the three speckle patterns corresponding to the same detection object are grayscaled and input into the Unet neural network through channel splicing. The correspondence between the object image and the speckle established by the Unet neural network is used to reconstruct the image of the detection object after transmission through the multimode optical fiber.

2. The multimode fiber optic endoscope system based on an incoherent light source according to claim 1, characterized in that: The training process selects N target objects to establish a training set, where N is greater than or equal to 30,000.

3. The multimode fiber optic endoscope system based on an incoherent light source according to claim 1, characterized in that: During the training process, inputting the object image of the target object into the Unet neural network specifically involves loading the grayscale pattern of the target object through the DMD and inputting the grayscale pattern into the Unet neural network.

Images