Deep neural network training method and system for multi-core fiber optic endoscope imaging

Abstract

The present invention relates to the field of fiber optic endoscope imaging technology, and more particularly to a deep neural network training method and system for multi-core fiber optic endoscope imaging. The method comprises the following steps: first, using a data set generation system, projecting a phase image onto the fiber end face and acquiring a speckle image from the fiber output end; then, inputting the speckle image into a deep neural network to reconstruct the phase information of the fiber measurement end from the speckle image; wherein the data set generation system uses an optical system based on holographic display to automatically generate a massive data set for fiber optic phase imaging; and the deep neural network uses a network architecture based on a convolutional neural network (Unet) and a residual neural network (ResNet) to simplify the data processing process. The present invention provides a deep neural network training method for multi-core fiber optic endoscope imaging to address the problems of low real-time performance and accuracy in traditional multi-core fiber quantitative phase imaging technology.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of fiber optic endoscope imaging technology, and in particular to a deep neural network training method and system for multi-core fiber optic endoscope imaging. Background Art

[0002] Flexible optical endoscopes are important diagnostic tools in biomedical and clinical research. Thin optical fibers can be fully utilized as imaging media for endoscopes. In order to obtain effective image information, single-mode optical fibers need to be equipped with a scanning lens or spectral disperser at the end, which limits the miniaturization of the endoscope, the imaging frame rate, and the system field of view. For multimode optical fibers, when the complex phase randomization and mode mixing during transmission are measured or compensated, multimode optical fibers can use their characteristics of transmitting multiple different modes to provide two-dimensional image information. Multi-core fiber (MCF) is different from multimode fiber. Multi-core fiber is a multi-core fiber bundle composed of tens of thousands of single-mode / multimode fiber cores. It is mostly used in flexible fiber endoscopes to improve the spatial resolution of imaging. However, existing multi-core fiber (MCF) quantitative phase imaging technology is often limited by traditional calculation methods, resulting in slow imaging speed and the need for pre-measured reference phase distortion. Summary of the Invention

[0003] The embodiments of the present application provide a deep neural network training method and system for multi-core fiber endoscopic imaging, which solves the problems of low real-time performance and accuracy in traditional multi-core fiber quantitative phase imaging technology.

[0004] To solve the above technical problems, in a first aspect, an embodiment of the present application provides a deep neural network training method for multi-core fiber endoscopic imaging, comprising the following steps: first, using a data set generation system, a phase image is projected onto the fiber end face, and a speckle image is collected from the fiber output end; then, the speckle image is input into a deep neural network, and the phase information of the fiber measurement end is reconstructed from the speckle image; wherein, the data set generation system adopts an optical system based on holographic display to automatically generate massive data sets for fiber phase imaging; the deep neural network adopts a network architecture based on a convolutional neural network (Unet) and a residual neural network (ResNet) to simplify the data processing process.

[0005] In some exemplary embodiments, a dataset generation system is used to generate a large number of specialized training images for multi-core fiber phase imaging; the training images are converted into single-precision phase images, and the adjusted phase images have a resolution of 980×980 pixels, which are then zero-padded to 1920×1080 pixels to match the display specifications of the spatial light modulator.

[0006] In some exemplary embodiments, an optical system based on holographic display includes: a single longitudinal mode laser, a beam expansion unit, a spatial light modulator, a frequency domain filtering unit, a first microscope unit, a calibration camera, a second microscope unit and a detection camera; wherein, in the optical system based on holographic display, the laser beam emitted by the laser passes through the beam expansion unit to uniformly illuminate the spatial light modulator; the phase-modulated laser beam passes through the frequency domain filtering unit to remove unnecessary high-order diffraction orders; then, the phase image of the holographic display is projected onto the measurement end face of the multi-core optical fiber through the first microscope system, and part of the incident light beam is reflected by the end face of the multi-core optical fiber and projected onto the calibration camera to achieve precise alignment of the holographic display plane and the end face of the optical fiber.

[0007] In some exemplary embodiments, in an optical system based on holographic display, far-field speckle output by a multi-core optical fiber is imaged on a detection camera through a second microscope system; a spatial light modulator and the detection camera are triggered synchronously; and when a phase image is displayed on the spatial light modulator, the detection camera is simultaneously activated to capture a corresponding speckle image, thereby ensuring a one-to-one correspondence between the displayed phase image and the speckle image captured by the detection camera.

[0008] In some exemplary embodiments, the beam expansion unit includes a linear polarizer, a reflective mirror, and several achromatic lenses; and the frequency domain filtering system includes an aperture, a reflective mirror, and several achromatic lenses.

[0009] In some exemplary embodiments, the spatial light modulator includes a phase-type spatial light modulator, a reflective or transmissive spatial light modulator, or a digital micromirror.

[0010] In some exemplary embodiments, the architecture of the deep neural network includes three downsampling blocks and three upsampling blocks; wherein the downsampling block and the upsampling block each include a rectified linear unit layer and two convolutional layers.

[0011] In some exemplary embodiments, the architecture of the deep neural network further includes a random dropout layer located before the output convolutional layer to perform a regularization operation on the deep neural network.

[0012] In some exemplary embodiments, after inputting the speckle image into a deep neural network and reconstructing the phase information of the optical fiber measurement end from the speckle image, the deep neural network training method further includes: using a 2D correlation coefficient to quantitatively evaluate the phase reconstruction fidelity of the trained deep neural network to compare the similarity between the image reconstructed by the deep neural network and the real image.

[0013] In a second aspect, an embodiment of the present application further provides a deep neural network training system for multi-core fiber endoscopic imaging, comprising: a connected dataset generation module and a deep neural network module; the dataset generation module is configured to use the dataset generation system to project a phase image onto an optical fiber end face and collect a speckle image from the optical fiber output end; wherein the dataset generation system uses an optical system based on holographic display to automatically generate a massive dataset for optical fiber phase imaging; the deep neural network module is configured to input the speckle image into a deep neural network and reconstruct the phase information of the optical fiber measurement end from the speckle image; wherein the deep neural network uses a network architecture based on a convolutional neural network (Unet) and a residual neural network (ResNet) to simplify the data processing process.

[0014] The technical solution provided by the embodiments of the present application has at least the following advantages:

[0015] To address the real-time and accuracy issues in traditional multi-core fiber quantitative phase imaging technology, an embodiment of the present application provides a deep neural network training method and system for multi-core fiber endoscopic imaging. The method includes the following steps: first, using a dataset generation system, a phase image is projected onto the fiber end face and a speckle image is collected from the fiber output end; then, the speckle image is input into a deep neural network, and the phase information of the fiber measurement end is reconstructed from the speckle image; wherein, the dataset generation system adopts an optical system based on holographic display to automatically generate massive datasets for fiber phase imaging; the deep neural network adopts a network architecture based on convolutional neural network Unet and residual neural network ResNet to simplify the data processing process.

[0016] The present application provides a deep neural network training method for multi-core fiber endoscopic imaging, which directly reconstructs the phase image from the speckle at the far-field output end of the optical fiber, thereby achieving high-fidelity phase reconstruction without pre-calibration. In addition, the present application also proposes an optical system based on holographic display for automatically generating massive data sets for fiber optic phase imaging. This solution not only enables fast and accurate endoscopic imaging, but also simplifies the imaging process, laying a solid foundation for the future clinical application of fiber optic endoscopes. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] One or more embodiments are exemplarily described by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Unless otherwise stated, the pictures in the drawings do not constitute proportional limitations.

[0018] Figure 1 A flowchart of a deep neural network training method for multi-core fiber endoscopic imaging provided by one embodiment of the present application;

[0019] Figure 2This is a schematic structural diagram of a massive data set generation system for multi-core fiber optic endoscope phase imaging provided by an embodiment of the present application;

[0020] Figure 3 This is a schematic diagram of the structure of a deep neural network architecture for reconstructing phase information from a speckle image provided by an embodiment of the present application;

[0021] Figure 4 A schematic diagram of the structure of a deep neural network training system for multi-core fiber optic endoscope imaging provided in one embodiment of the present application;

[0022] Figure 5 A schematic diagram of multi-core fiber phase imaging of handwritten digits using a trained deep neural network according to an embodiment of the present application;

[0023] Figure 6 A schematic diagram of multi-core fiber phase imaging of clothing-like objects using a trained deep neural network provided in one embodiment of the present application. DETAILED DESCRIPTION

[0024] As we can see from the background, existing multi-core fiber phase imaging techniques are often limited by traditional computational methods, resulting in slow imaging speeds and the need for pre-measured reference phase distortion. To achieve high frame rates and high fidelity in fiber endoscopic light field imaging, a new, fast phase reconstruction method is needed.

[0025] The present invention aims to address the real-time and accuracy issues in conventional multi-core fiber (MCF) quantitative phase imaging techniques. Although multi-core fiber endoscopic imaging offers a promising approach for non-invasive, real-time in vivo detection, its computational limitations often make high-speed, high-resolution imaging difficult. In particular, conventional methods require pre-measurement of reference phase deviations and rely on iterative phase recovery algorithms, making image reconstruction complex and time-consuming. Furthermore, the current lack of training datasets dedicated to quantitative phase imaging in multi-core fiber endoscopy makes it difficult to reconstruct and analyze speckle images of multi-core fibers using machine learning and deep learning techniques.

[0026] To address the aforementioned technical issues, embodiments of the present application provide a deep neural network training method and system for multi-core fiber endoscopic imaging. The method comprises the following steps: first, using a dataset generation system to project a phase image onto the fiber end face and collect a speckle image from the fiber output end; then, inputting the speckle image into a deep neural network to reconstruct the phase information at the fiber measurement end from the speckle image; wherein the dataset generation system utilizes an optical system based on holographic display to automatically generate a massive dataset for fiber phase imaging; and the deep neural network utilizes a network architecture based on a convolutional neural network (Unet) and a residual neural network (ResNet) to simplify the data processing process. By providing a deep neural network training method and system for multi-core fiber endoscopic imaging, embodiments of the present application address the issues of low real-time performance and accuracy in traditional multi-core fiber quantitative phase imaging technology.

[0027] The following detailed description of the various embodiments of the present application is provided in conjunction with the accompanying drawings. However, those skilled in the art will appreciate that many technical details are provided in the various embodiments of the present application to facilitate a better understanding of the present application. However, even without these technical details and the various variations and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented.

[0028] See Figure 1 , an embodiment of the present application provides a deep neural network training method for multi-core fiber endoscopic imaging, comprising the following steps:

[0029] Step S1: Using a data set generation system, a phase image is projected onto the end face of an optical fiber, and a speckle image is collected from the output end of the optical fiber. The data set generation system uses an optical system based on holographic display to automatically generate a massive data set of optical fiber phase imaging.

[0030] Step S2: Input the speckle image into a deep neural network to reconstruct the phase information of the optical fiber measurement end from the speckle image; wherein the deep neural network adopts a network architecture based on the convolutional neural network Unet and the residual neural network ResNet to simplify the data processing process.

[0031] The present application is applied to the field of fiber optic endoscope imaging, in particular, the use of deep learning for quantitative phase imaging of multi-core fiber optic endoscopes. Since existing multi-core fiber optic phase imaging technology is often limited by traditional computing methods, resulting in slow imaging speed, the present application proposes the use of deep learning methods to directly reconstruct the phase image from the speckle at the far-field output end of the optical fiber, thereby achieving high-fidelity phase reconstruction without the need for pre-calibration. In addition, due to the lack of fiber optic phase imaging datasets in the prior art, based on holographic display technology, the present application proposes a massive dataset generation system for multi-core fiber optic endoscope phase imaging, which is used to automatically generate massive datasets for fiber optic phase imaging.

[0032] In some embodiments, a dataset generation system is used to generate a large number of dedicated training images for multi-core fiber phase imaging; the training images are converted into single-precision phase images, and the adjusted phase images have a resolution of 980×980 pixels, which are then zero-padded to 1920×1080 pixels to match the display specifications of the spatial light modulator.

[0033] In some embodiments, an optical system based on holographic display includes: a single longitudinal mode laser, a beam expansion unit, a spatial light modulator, a frequency domain filtering unit, a first microscope unit, a calibration camera, a second microscope unit and a detection camera; wherein, in the optical system based on holographic display, the laser beam emitted by the laser passes through the beam expansion unit to uniformly illuminate the spatial light modulator; the phase-modulated laser beam passes through the frequency domain filtering unit to remove unnecessary high-order diffraction orders; then, the phase image of the holographic display is projected onto the measurement end face of the multi-core optical fiber through the first microscope system, and part of the incident light beam is reflected by the end face of the multi-core optical fiber and projected onto the calibration camera to achieve precise alignment of the holographic display plane and the end face of the optical fiber.

[0034] In some embodiments, in an optical system based on holographic display, the far-field speckle output by a multi-core optical fiber is imaged on a detection camera through a second microscope system.

[0035] It should be noted that in the optical system of the present application, the spatial light modulator and the detection camera adopt a synchronous triggering mechanism. When the phase image is displayed on the spatial light modulator (SLM), the detection camera is simultaneously started to capture the corresponding speckle image to ensure a one-to-one correspondence between the displayed phase image and the speckle image captured by the detection camera.

[0036] In some embodiments, the beam expansion unit includes a linear polarizer, a reflector, and several achromatic lenses; the frequency domain filtering system includes an aperture, a reflector, and several achromatic lenses.

[0037] In some embodiments, the spatial light modulator (SLM) includes a phase-only spatial light modulator (Phase-only SLM), a reflective or transmissive spatial light modulator, or a digital micromirror (DMD).

[0038] It should be noted that holographic projection onto the end face of a multi-core optical fiber can be either transmissive or reflective. Multi-core optical fibers include optical fiber bundles with 6 to 30,000 cores, and each core can be single-mode or multi-mode.

[0039] Figure 2The schematic diagram of the structure of the optical system based on holographic display is shown. The massive data set generation system for multi-core fiber endoscope phase imaging provided by this application uses holographic display technology to automatically project the phase image onto the fiber end face and automatically collect the speckle image at the fiber output end on the detection camera (CAM2). Among them, Laser represents the laser; LP represents the linear polarizer; L1 to L6 represent the achromatic lenses; M1 and M2 represent the reflectors; ID represents the aperture; BS represents the beam splitter; CAM1 represents the calibration camera; CAM2 represents the detection camera; MCF represents the multi-core fiber; MO1 and MO2 represent the microscope objective lenses.

[0040] The dataset generation system of the present application uses an optical system based on holographic display to generate a large number of dedicated training images for multi-core fiber (MCF) phase imaging. These images are converted into single-precision phase images, and the adjusted phase image resolution is 980×980 pixels. It is then zero-filled to 1920×1080 pixels to match the display specifications of the spatial light modulator (SLM) for automatic display of phase images. In this system, the laser beam is expanded to uniformly illuminate the spatial light modulator SLM. The phase-modulated laser beam passes through a frequency domain filtering system consisting of lenses (L3, L4) and an aperture (ID) to remove unnecessary high-order diffraction orders. After that, the holographically displayed phase image is projected onto the multi-core fiber measurement end face through a first microscope system (L5, MO1). The incident beam is partially reflected by the fiber end face and projected onto the calibration camera (CAM1), thereby achieving precise alignment between the holographic display plane and the fiber end face.

[0041] The far-field speckle pattern output from the optical fiber is imaged on a detection camera (CAM2) via a second microscope system (L6, MO2). The spatial light modulator (SLM) and the detection camera are triggered synchronously. When the phase image is displayed on the SLM, the detection camera is simultaneously activated to capture the corresponding speckle image. This ensures a one-to-one correspondence between the displayed phase image and the speckle image captured by the detection camera.

[0042] In some embodiments, the architecture of the deep neural network includes three downsampling blocks and three upsampling blocks; wherein the downsampling block and the upsampling block each include a rectified linear unit layer and two convolutional layers.

[0043] In some embodiments, the architecture of the deep neural network further includes a random dropout layer located before the output convolutional layer to perform a regularization operation on the deep neural network.

[0044] Figure 3The following diagram shows the deep neural network architecture for fiber phase image reconstruction. Convolution represents a convolutional layer; ReLU represents a rectified linear unit layer; transposed concatenation represents a deconvolution layer; concatenation represents a concatenation layer; and dropout represents a random dropout layer. Max Pooling is a pooling operation that retains only the largest of the extracted feature values, discarding all other feature values. The largest value represents the retention of the strongest feature, while discarding weaker features.

[0045] This application provides a deep neural network architecture for reconstructing phase information from speckle images, where the network input is the speckle image captured by the detection camera, and the network output is the phase information of the optical fiber measurement end. This architecture is based on Unet and ResNet and contains three downsampling blocks and three upsampling blocks, which effectively simplifies the data processing process. This helps to improve the speed and efficiency of the training process. Specifically, each sampling block consists of two convolution layers and a rectified linear unit (ReLU) layer. Figure 3 The depth of each layer is indicated below the mid-sample block. Additionally, dropout layers are added before the final output convolutional layer as a regularization technique for deep neural networks. During training, these layers randomly omit or deactivate certain neurons, temporarily removing them from the network. This reduces the network's reliance on individual neurons. This approach encourages the network to distribute learned features more evenly across all neurons, preventing overfitting and resulting in broader, more robust representations.

[0046] In some embodiments, after inputting the speckle image into a deep neural network in step S2 and reconstructing the phase information of the optical fiber measurement end from the speckle image, the deep neural network training method further includes: quantitatively evaluating the fidelity of the phase reconstruction of the trained deep neural network using a 2D correlation coefficient to compare the similarity between the image reconstructed by the deep neural network and the real image.

[0047] See Figure 4, an embodiment of the present application also provides a deep neural network training system for multi-core fiber endoscopic imaging, characterized in that it includes: a data set generation module 101 and a deep neural network module 102 connected to each other; the data set generation module 101 is used to use the data set generation system to project the phase image onto the fiber end face and collect the speckle image from the fiber output end; wherein, the data set generation system uses an optical system based on holographic display to automatically generate a massive data set for fiber phase imaging; the deep neural network module 102 is used to input the speckle image into the deep neural network and reconstruct the phase information of the fiber measurement end from the speckle image; wherein, the deep neural network uses a network architecture based on the convolutional neural network Unet and the residual neural network ResNet to simplify the data processing process.

[0048] The deep learning-based multi-core fiber quantitative phase imaging system provided in this application can reconstruct phase images directly from speckle patterns using a deep neural network, achieving high-fidelity phase reconstruction. Furthermore, this application also provides a system for generating massive datasets for multi-core fiber endoscope phase imaging based on holographic display technology, which is used to generate a large number of dedicated training images for multi-core fiber phase imaging. The training images are converted into single-precision phase images, and the adjusted phase images have a resolution of 980×980 pixels, which are then zero-padded to 1920×1080 pixels to match the display specifications of the spatial light modulator.

[0049] Compared with existing technologies, the present invention has the advantage of significantly improving image reconstruction speed. On the same computing platform, the method provided by this application takes more than 8 minutes to reconstruct a phase image (including the time required to calibrate the image reconstruction). However, the deep learning method of the present invention, without the need for pre-calibration of the multi-core fiber, can reconstruct a phase image in 5.5ms, achieving an imaging frame rate of 181fps.

[0050] This application quantitatively evaluates the phase reconstruction fidelity of a trained deep neural network by using the 2D correlation coefficient, which is used to compare the similarity between images reconstructed by the deep neural network and the real image.

[0051] For 96 handwritten digits that were not in the training set, the average fidelity of DNN-based phase reconstruction was 0.994, and for 78 images from the Fashion MNIST dataset (clothing images), the fidelity was 0.998, characterizing general high-fidelity phase reconstruction performance.

[0052] Figure 5A schematic diagram of multi-core fiber phase imaging of handwritten digits using a trained deep neural network is shown. (a) shows the ground truth, (b) shows the speckle image used as the deep neural network input (DNN input), and (c) shows the phase image reconstructed by the deep neural network (DNN reconstruction).

[0053] Figure 6 A schematic diagram of multi-core fiber phase imaging of clothing-like objects using a trained deep neural network is shown. (a) shows the ground truth, (b) shows the speckle image used as the deep neural network input (DNN input), and (c) shows the phase image reconstructed by the deep neural network (DNN reconstruction).

[0054] Based on the above technical solution, the embodiments of the present application provide a deep neural network training method and system for multi-core fiber endoscopic imaging. The method includes the following steps: first, using a data set generation system, a phase image is projected onto the fiber end face, and a speckle image is collected from the fiber output end; then, the speckle image is input into a deep neural network, and the phase information of the fiber measurement end is reconstructed from the speckle image; wherein, the data set generation system adopts an optical system based on holographic display to automatically generate massive data sets for fiber phase imaging; the deep neural network adopts a network architecture based on the convolutional neural network Unet and the residual neural network ResNet to simplify the data processing process.

[0055] The embodiments of the present application provide a deep neural network training method and system for multi-core fiber optic endoscope imaging, which directly reconstructs the phase image from the speckle at the far-field output end of the optical fiber, thereby achieving high-fidelity phase reconstruction without the need for pre-calibration. In addition, the present application also proposes an optical system based on holographic display for automatically generating massive data sets for fiber optic phase imaging. This solution not only enables fast and accurate endoscopic imaging, but also simplifies the imaging process, laying a solid foundation for the future clinical application of multi-core fiber optic endoscopes.

[0056] Those skilled in the art will appreciate that the above-described embodiments are specific examples for implementing the present application, and that in actual applications, various changes in form and detail may be made thereto without departing from the spirit and scope of the present application. Any person skilled in the art may make changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be subject to the scope defined in the claims.

Claims

1. A deep neural network training method for multi-core fiber optic endoscope imaging, characterized in that: This method directly reconstructs a phase image from the speckle at the far-field output end of the optical fiber without pre-calibration. It can reconstruct a phase image in 5.5ms, achieving an imaging frame rate of 181fps. The method includes the following steps: A data set generation system is used to project a phase image onto the end face of an optical fiber and collect a speckle image from the output end of the optical fiber. The data set generation system uses an optical system based on holographic display to automatically generate a massive data set of optical fiber phase imaging. Inputting the speckle image into a deep neural network to reconstruct phase information of the optical fiber measurement end from the speckle image; wherein the deep neural network adopts a network architecture based on a convolutional neural network (Unet) and a residual neural network (ResNet) to simplify the data processing process; The optical system based on holographic display includes: a single longitudinal mode laser, a beam expansion unit, a spatial light modulator, a frequency domain filtering unit, a first microscope unit, a calibration camera, a second microscope unit and a detection camera; In the holographic display-based optical system, a laser beam emitted by a laser passes through a beam expansion unit to uniformly illuminate a spatial light modulator; the phase-modulated laser beam passes through a frequency-domain filtering unit to remove unnecessary high-order diffraction orders; the phase image of the holographic display is then projected onto the measurement end face of a multi-core optical fiber through a first microscope system; the incident beam is partially reflected by the multi-core optical fiber end face and projected onto a calibration camera to achieve precise alignment between the holographic display plane and the optical fiber end face; the multi-core optical fiber includes an optical fiber bundle with a core count ranging from 6 to 30,000. In the optical system based on holographic display, the far-field speckle output by the multi-core optical fiber is imaged on a detection camera through a second microscope system; The spatial light modulator and the detection camera are triggered synchronously; When the phase image is displayed on the spatial light modulator, the detection camera is simultaneously activated to capture the corresponding speckle image to ensure a one-to-one correspondence between the displayed phase image and the speckle image captured by the detection camera; The architecture of the deep neural network includes three downsampling blocks, three upsampling blocks and a random dropout layer; wherein, The downsampling block and the upsampling block both include a rectified linear unit layer and two convolutional layers; the random dropout layer is located before the final output convolutional layer and is used to perform regularization operations on the deep neural network; during the training process, the random dropout layer is used to randomly omit or deactivate certain neurons to reduce the network's dependence on individual neurons.

2. The deep neural network training method for multi-core fiber endoscopic imaging according to claim 1, characterized in that: The data set generation system is used to generate a large number of dedicated training images for multi-core optical fiber phase imaging; The training image is converted into a single-precision phase image, and the adjusted phase image has a resolution of 980×980 pixels, which is then zero-padded to 1920×1080 pixels to match the display specifications of the spatial light modulator.

3. The deep neural network training method for multi-core fiber endoscopic imaging according to claim 1, characterized in that: The beam expansion unit includes a linear polarizer, a reflector and several achromatic lenses; the frequency domain filtering system includes an aperture, a reflector and several achromatic lenses.

4. The deep neural network training method for multi-core fiber endoscopic imaging according to claim 1, characterized in that: The spatial light modulator includes a phase-type spatial light modulator, a reflective or transmissive spatial light modulator, and a digital micro-mirror.

5. The deep neural network training method for multi-core fiber endoscopic imaging according to claim 1, characterized in that: After inputting the speckle image into a deep neural network and reconstructing the phase information of the optical fiber measurement end from the speckle image, the method further includes: The fidelity of phase reconstruction of the trained deep neural network is quantitatively evaluated using the 2D correlation coefficient to compare the similarity between the images reconstructed by the deep neural network and the real images.

6. A deep neural network training system for multi-core fiber endoscopic imaging, characterized in that: The system reconstructs a phase image directly from the speckle at the far-field output end of the optical fiber without pre-calibration, and can reconstruct a phase image in 5.5ms, achieving an imaging frame rate of 181fps. The system includes: a data set generation module and a deep neural network module connected to each other; The data set generation module is used to project the phase image onto the optical fiber end face and collect the speckle image from the optical fiber output end using a data set generation system; wherein the data set generation system uses an optical system based on holographic display to automatically generate a massive data set of optical fiber phase imaging; The deep neural network module is used to input the speckle image into a deep neural network and reconstruct the phase information of the optical fiber measurement end from the speckle image; wherein the deep neural network adopts a network architecture based on a convolutional neural network Unet and a residual neural network ResNet to simplify the data processing process; The optical system based on holographic display includes: a single longitudinal mode laser, a beam expansion unit, a spatial light modulator, a frequency domain filtering unit, a first microscope unit, a calibration camera, a second microscope unit and a detection camera; In the holographic display-based optical system, a laser beam emitted by a laser passes through a beam expansion unit to uniformly illuminate a spatial light modulator; the phase-modulated laser beam passes through a frequency-domain filtering unit to remove unnecessary high-order diffraction orders; the phase image of the holographic display is then projected onto the measurement end face of a multi-core optical fiber through a first microscope system; the incident beam is partially reflected by the multi-core optical fiber end face and projected onto a calibration camera to achieve precise alignment between the holographic display plane and the optical fiber end face; the multi-core optical fiber includes an optical fiber bundle with a core count ranging from 6 to 30,000. In the optical system based on holographic display, the far-field speckle output by the multi-core optical fiber is imaged on a detection camera through a second microscope system; The spatial light modulator and the detection camera are triggered synchronously; When the phase image is displayed on the spatial light modulator, the detection camera is simultaneously activated to capture the corresponding speckle image to ensure a one-to-one correspondence between the displayed phase image and the speckle image captured by the detection camera; The architecture of the deep neural network includes three downsampling blocks, three upsampling blocks and a random dropout layer; wherein, The downsampling block and the upsampling block both include a rectified linear unit layer and two convolutional layers; the random inactivation layer is located before the final output convolutional layer and is used to perform regularization operations on the deep neural network.

Images