Image reconstruction method and imaging system based on multimode fiber holography and polarization coding

Abstract

The invention discloses an image reconstruction method and imaging system based on multimode fiber holography and polarization coding, and the method comprises the following steps: S1, outputting a laser light source, eliminating the reflected light through an isolator, and carrying out the light beam expansion through a first lens and a second lens; s2, dividing the expanded light beam into a reference arm and a signal arm by using an optical splitter; s3, in the signal arm, generating a hologram containing spatial position information through a DMD, dynamically switching polarization states through an optical relay device, and forming holographic light field distribution of different polarization states in a three-dimensional space; s4, the modulated multi-polarized light field is coupled through a microscope objective to enter a multimode optical fiber; s5, the optical fiber output end forms a speckle pattern carrying polarization information through the achromatic lens group; and S6, collecting a speckle pattern through a camera, and realizing holographic image reconstruction by adopting a U-Net neural network. According to the invention, high-fidelity image transmission and reconstruction in the multimode optical fiber are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of intersection of multimode optical fiber imaging and microscopic imaging, and in particular relates to an image reconstruction method and an imaging system based on multimode optical fiber holography and polarization coding. Background Art

[0002] Multimode fiber features a large number of spatial modes and low loss. When light signals propagate through multimode fiber, modal dispersion occurs due to slightly different transmission paths and speeds of different modes of light within the fiber. This dispersion spreads the light signal out in time and space. Through specialized detection techniques, spatial information about an object can be acquired, enabling imaging.

[0003] In the field of multimode fiber imaging, artificial intelligence (AI), an emerging tool, shows great potential for decoding images transmitted through multimode fiber (MMF), particularly when combined with deep learning techniques. By transmitting a large number of images, deep neural networks (DNNs) can interpret the correlation between input and output from seemingly disordered speckle patterns, achieving high-fidelity image reconstruction. While reconstruction is effective for datasets with relatively simple internal structures, this often fails to fully exploit the potential of MMF-DNN systems. In the prior art, "Using Holographic Encoded Variance to Transmit Marked Images through Multimode Fiber," published in Optics Express, Vol. 32, No. 11, May 20, 2024, pp. 18896-18908, discloses a method that introduces holographic modulation to introduce more variation in the output speckle pattern, thereby improving the system's transmission capabilities. Specifically, a holographic label is added to the original dataset, and the resulting phase image is injected into the fiber endface via a Fourier lens. The resulting speckle dataset can be effectively clustered based on the holographic label, maintaining high-quality reconstruction while avoiding information loss. As an application example, a method was demonstrated for decomposing a color image into its three color components, RGB, and assigning a unique holographic tag to each component. The ResUNet architecture was then used to decode each type of speckle pattern and ultimately reconstruct the color image. However, its information capacity is limited by the single degree of freedom of holographic encoding. Multi-channel transmission, such as RGB, requires time-sharing processing, significantly increasing system complexity. Speckle noise and signal aliasing reduce the reconstructed signal-to-noise ratio (SNR). The method also lacks polarization-assisted decoupling, making it unable to suppress common-mode noise, such as global phase drift caused by fiber vibration. Holographic modulation relies on high-resolution DMDs and GPU acceleration, which consumes large hardware resources and has poor real-time performance. This makes it difficult to meet the requirements of dynamic environments, such as mobile scenes or video transmission. Its applicability is limited to ideal static conditions, and it faces the triple dilemma of capacity, noise, and real-time performance in MMF image transmission. Therefore, resolving this triple dilemma in MMF image transmission and providing an image reconstruction method and imaging system based on multimode fiber holography and polarization encoding that utilizes channel multiplexing, noise suppression, and expands its application scenarios are pressing technical challenges facing those skilled in the art.

[0004] Traditional image reconstruction methods based on multimode fiber (MMF) typically ignore the polarization state of light. This is because, in most cases, MMF imaging systems focus on scattering and interference effects through the fiber, particularly imaging based on intensity and phase information. When light propagates through a multimode fiber, the influence of polarization is often neglected due to the fiber's modal distribution and internal scattering properties. However, polarization effects can sometimes play a role in multimode fiber imaging, particularly in applications requiring high resolution or sensitive to anisotropy of the optical field. For example, the fiber's structure and inhomogeneities can cause light modes with different polarization states to propagate differently. This difference can, in certain circumstances, affect image quality or mode coupling during imaging. With the further development of multimode fiber imaging technology, the study of polarization state has become a growing focus, particularly in high-resolution imaging and information extraction. By considering polarization information, researchers can better control and optimize the performance of multimode fiber imaging systems, potentially improving image quality or increasing the amount of decodable information. Summary of the Invention

[0005] The first object of the present invention is to provide an image reconstruction method based on multimode fiber holography and polarization coding to address the problems in the prior art.

[0006] To this end, the above-mentioned purpose of the present invention is achieved through the following technical solutions:

[0007] The image reconstruction method based on multimode fiber holography and polarization coding includes the following steps:

[0008] S1, output laser light source, after the isolator eliminates the reflected light, the beam is expanded by the first lens and the second lens so that the beam diameter matches the digital micromirror device DMD;

[0009] S2, uses a beam splitter to split the expanded beam into a reference arm and a signal arm;

[0010] Reference arm: Light intensity and polarization state are controlled by an adjustable attenuator, collimating lens, and half-wave plate;

[0011] Signal arm: calibrated by a collimating lens and half-wave plate to match the DMD liquid crystal orientation to optimize diffraction efficiency;

[0012] In S3, a hologram containing spatial position information is generated by the DMD in the signal arm, and the polarization state is dynamically switched through the optical relay device to form a holographic light field distribution with different polarization states in three-dimensional space, thereby obtaining a dual-encoded holographic phase image after holographic encoding and polarization encoding.

[0013] S4, the modulated multi-polarized light field is coupled into the multimode optical fiber through the microscope objective;

[0014] S5, the optical fiber output end passes through an achromatic lens group to form a speckle pattern carrying polarization information;

[0015] S6, using a camera to collect speckle patterns, and using a U-Net neural network to reconstruct a holographic image. While adopting the above technical solutions, the present invention may also adopt or combine the following technical solutions:

[0016] As a preferred technical solution of the present invention: in step S2, the reference arm: sequentially adjusts the light intensity through the second variable optical attenuator, collimates through the eighth lens, and adjusts the polarization state through the fourth half-wave plate, and then inputs the light into the beam combiner until it is orthogonal to the signal arm and then inputs the light into the beam combiner;

[0017] Signal arm: collimated by the fourth lens and the second half-wave plate in sequence, the second half-wave plate matches the DMD liquid crystal orientation angle and precisely aligns the 624×624 central area to maximize diffraction efficiency.

[0018] As a preferred technical solution of the present invention: in step S3, a digital micromirror device is used to load a phase hologram to control the wavefront distribution, and a multi-focus holographic structure is generated in three-dimensional space. The horizontal, vertical and circular polarization modes are switched through an optical relay device to achieve polarization state control, and each focal position is associated with the horizontal, vertical and circular polarization states to form a polarization-encoded three-dimensional light field. Through the three-dimensional spatial coherent superposition of light fields with different polarization states, a holographic light field that integrates polarization and spatial dimensions is constructed.

[0019] As a preferred technical solution of the present invention: Step S3 includes the following steps:

[0020] S301, a digital micromirror device is loaded with a preset holographic coding pattern, the coding pattern is compressed and relayed to the input end of a multimode optical fiber through a 4f lens system, and a multi-focus holographic structure is generated in a three-dimensional space;

[0021] S302, the incident light is programmed to switch between horizontal polarization state, vertical polarization state and circular polarization state through the optical relay device;

[0022] S303 , associating and encoding each focal position in the three-dimensional space with each polarization state, coherently superimposing light fields of different polarization states in the three-dimensional space, and outputting a holographic light field that integrates polarization and spatial dimensions.

[0023] As a preferred technical solution of the present invention: Step S4 specifically includes the following steps:

[0024] S401, the digital micromirror device (DMD) relays and reduces the encoded holographic image displayed on the DMD to the proximal end face of the multimode optical fiber through two consecutive 4f imaging systems consisting of a fifth lens, a sixth lens, a seventh lens, and a first microscope objective lens;

[0025] In step S402, the light beam is purified by the first polarizer, adjusted to 45° polarization by the third half-wave plate, split into orthogonal linearly polarized light by the beam shifter, and then converted into left / right circularly polarized light by a quarter-wave plate. Finally, it is coupled to the step-index multimode fiber by the microscope objective.

[0026] As a preferred technical solution of the present invention: in step S402, the first polarizer filters out stray orthogonal polarization components, the third half-wave plate polarizes the light beam to 45°, the beam shifter generates two spatially separated orthogonal linear polarized lights, which are recombined into a common optical path to retain the orthogonal polarization state, and before optical fiber coupling, the quarter-wave plate converts the two orthogonal linear polarizations into right-handed and left-handed circular polarizations, respectively.

[0027] As a preferred technical solution of the present invention: in step S5, the light beam output from the multimode optical fiber passes through the second microscope objective and two achromatic doublet lenses: the ninth lens and the tenth lens, to form a speckle pattern carrying holographic and polarization information.

[0028] As a preferred technical solution of the present invention: in step S6, the U-Net neural network is a 7-layer network architecture, including:

[0029] The encoder consists of three convolutional layers, each of which uses the LeakyReLU activation function for downsampling to extract the multi-scale features of the speckle pattern. Each convolutional layer is followed by batch normalization and skip connections to preserve multi-scale feature information.

[0030] The decoder consists of three levels of deconvolution layers, each of which uses the ReLU activation function for upsampling. The feature maps of the corresponding layers of the encoder are spliced ​​through jump connections to restore the image spatial resolution.

[0031] The network uses a Sigmoid activation function in the output layer to limit the pixel values ​​of the reconstructed image to between 0 and 1.

[0032] As a preferred technical solution of the present invention: in step S6, the unified manifold approximation and projection dimensionality reduction methods (UMAP and t-SNE) are used to perform dimensionality reduction and cluster analysis on the speckle image, respectively, to verify the encoding differences between different polarization states and holographic labels;

[0033] The dimensionality reduction method uses the Barnes-Hut algorithm to accelerate t-SNE calculations and improve the processing efficiency of large-scale speckle image sets.

[0034] A second object of the present invention is to provide an image reconstruction system based on multimode fiber holography and polarization coding to address the problems in the prior art.

[0035] To this end, the above-mentioned purpose of the present invention is achieved through the following technical solutions:

[0036] An imaging system based on multimode fiber holography and polarization coding image reconstruction method, comprising:

[0037] Laser: Provides a coherent light source, which is sequentially connected to the isolator, the first lens, and the second lens to form a collimated beam expansion optical path;

[0038] A beam splitter that splits the light beam into a reference arm and a signal arm;

[0039] Reference arm: a second variable optical attenuator, an eighth lens, and a fourth half-wave plate are sequentially arranged;

[0040] Signal arm: The fourth lens, the second half-wave plate, the digital micromirror device, the polarizer, the third half-wave plate, the fifth lens, the quarter-wave plate and the first microscope objective lens are sequentially arranged;

[0041] Multimode fiber transmission module: step-index multimode fiber, the input end is connected to the first microscope objective lens, and the output end is connected to the second microscope objective lens and the ninth lens;

[0042] Interferometric imaging module: The reference arm is introduced into the beam combiner, interferes with the signal arm to complete the collimation, and then the reference arm is removed; the signal light is focused by the tenth lens, and the pure signal speckle pattern is captured by the camera;

[0043] Image reconstruction module: Uses the U-Net neural network to process speckle images, realizes multimodal information decoding, and outputs the reconstruction results.

[0044] Compared with the existing technology, the image reconstruction method and imaging system based on multimode fiber holography and polarization coding of the present invention have the following beneficial effects: the present invention controls the phase distribution of light by loading a hologram with a DMD, further controls the polarization by combining a wave plate and a polarization-maintaining fiber, and synchronously realizes the joint encoding of holography and polarization in a single optical system through the coordinated dual modulation of the DMD and the wave plate, generating a light field with an arbitrary combination of phase and polarization distribution, and achieving a breakthrough in solving the two major technical problems of polarization degradation and spatial information aliasing caused by mode coupling in multimode fiber imaging, breaking through the limitations of traditional holography, improving information capacity, and realizing flexible and dynamic control of multimodality; at the same time, the jump connection of U-Net is used to retain high-frequency details and improve transmission and reconstruction resolution.

[0045] The image reconstruction method and imaging system based on multimode fiber holography and polarization coding of the present invention retain high-frequency details through U-Net jump connections, achieving high-fidelity image reconstruction of speckle images. Polarization information is combined to assist in decoupling overlapping speckle patterns. Holographic-polarization multiplexing enhances feature extraction, and unsupervised clustering UMAP / t-SNE is used to effectively distinguish the encoded speckle images, confirming the robustness of dual coding.

[0046] The present invention uses holography and polarization for dual-mode encoding, which improves the information capacity of multimode fiber imaging. Through the integration of multi-dimensional control of light fields and deep learning, it provides a new paradigm for high-fidelity optical imaging in complex environments, and realizes high-fidelity image transmission and reconstruction in multimode optical fibers. It has great application prospects in biomedical endoscopy, optical encryption communications and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Schematic diagram of the optical path of a step-index multimode optical fiber microscopic imaging system of the present invention;

[0048] Figure 2 Figure a is the unsupervised clustering result (UMAP) of 10 holographic labels in the horizontal polarization state with a total of 2000*10 images; Figure b is the unsupervised clustering result (UMAP) of 10 holographic labels in the vertical polarization state with a total of 2000*10 images; Figure c is the unsupervised clustering result (UMAP) of 10 holographic labels in the horizontal plus vertical dual polarization state with a total of 2000*10 images; Figure d is the unsupervised clustering result (tSNE) of 10 holographic labels in the horizontal polarization state with a total of 2000*10 images; Figure e is the unsupervised clustering result (tSNE) of 10 holographic labels in the vertical polarization state with a total of 2000*10 images; Figure f is the unsupervised clustering result (tSNE) of 10 holographic labels in the horizontal plus vertical dual polarization state with a total of 2000*10 images; Flow chart of coordinated modulation of holography and polarization coding;

[0049] Figure 3 In the figure, Figure a is the unsupervised clustering result (UMAP) of the same holographic label position in horizontal polarization state, vertical polarization state and horizontal plus vertical dual polarization state for a total of 2000*3 pictures; Figure b is the unsupervised clustering result (tSNE) of the same holographic label position in horizontal polarization state, vertical polarization state and horizontal plus vertical dual polarization state for a total of 2000*3 pictures; Figure c is the unsupervised clustering result (UMAP) of 10 different holographic positions for 1000*20 pictures, 1000 pictures each in horizontal polarization state and vertical polarization state; Figure d is the unsupervised clustering result (tSNE) of 10 different holographic positions for 1000*20 pictures, 1000 pictures each in horizontal polarization state and vertical polarization state;

[0050] Figure 4 This is a U-Net neural network architecture diagram of a step-index multimode fiber microscopy imaging device of the present invention;

[0051] Figure 5In the figure, Figure a shows the transmission of MNIST handwritten digits through a multimode optical fiber. The first row shows the original real digital image, and the 2nd to 4th rows show the corresponding speckle patterns obtained at the optical fiber output end under horizontal polarization state (speckle_1), vertical polarization state (speckle_2) and combined horizontal polarization state and vertical polarization state (speckle_3), respectively. The 5th to 7th rows are the corresponding digits reconstructed using U-Net; Figure b is the structural similarity index (SSIM) distribution histogram of the test dataset reconstructed under three polarization states; Figure c is the average SSIM histogram of the test dataset reconstructed under three polarization states; Figure d is the training and validation loss curves (mean absolute error, MAE) of each polarization encoding within 40 epochs.

[0052] Figure 6 This is a flow chart of the image reconstruction method based on multimode fiber holography and polarization coding of the present invention;

[0053] Figure 7 This is a schematic diagram of the principle of holographic polarization dual encoding of the present invention;

[0054] The devices in the accompanying drawings are: 1-laser; 2-isolator; 3-first lens; 4-first reflector; 5-second lens; 6-second reflector; 7-first half-wave plate; 8-third lens; 9-beam splitter; 10-first variable optical attenuator; 11-first polarization-maintaining fiber; 12-fourth lens; 13-second half-wave plate; 14-digital micromirror device; 15-first polarizer; 16-third half-wave plate; 17-fifth lens; 18-beam shifter; 19-sixth lens; 20-seventh lens; 21-quarter wave plate; 22-first microscope objective; 23-step-index fiber; 24-second variable optical attenuator; 25-second polarization-maintaining fiber; 26-eighth lens; 27-fourth half-wave plate; 28-second microscope objective; 29-ninth lens; 30-beam combiner; 31-tenth lens; 32-camera. DETAILED DESCRIPTION

[0055] The present invention will be described in further detail with reference to the accompanying drawings and specific embodiments.

[0056] An image reconstruction method based on multimode fiber holography and polarization coding comprises the following steps:

[0057] S1: Output laser light source, after the reflected light is eliminated by the isolator, the beam is expanded by the first lens and the second lens so that the beam diameter matches the digital micromirror device (DMD);

[0058] S2: Use a beam splitter to split the expanded beam into a reference arm and a signal arm;

[0059] Reference arm: The light intensity is adjusted by the second variable optical attenuator, the eighth lens is collimated, and the fourth half-wave plate is used to adjust the polarization state before inputting into the beam combiner;

[0060] Signal arm: It is collimated by the fourth lens and polarization-adjusted by the second half-wave plate before entering the digital micromirror device (DMD).

[0061] S3: In the signal arm, the dual-coded holographic phase image after holographic encoding and polarization encoding is loaded through the DMD, which realizes the addition of holographic labels in different positions and the manipulation of different polarization states to the transmitted image.

[0062] S4: The diffracted light beam modulated by the DMD passes through two continuous 4f imaging systems consisting of the fifth lens, the sixth lens, the seventh lens and the first microscope objective lens in sequence, relaying and reducing the encoded holographic image displayed on the SLM to the proximal end face of the multimode optical fiber.

[0063] The first polarizer filters out any unwanted orthogonal polarization components introduced by the diffraction process. A third half-wave plate then orients the transmitted polarization at 45° relative to the beam shifter's principal axis, generating two spatially separated, orthogonal, linearly polarized beams. The beam shifter recombines these beams onto a common optical path while preserving their orthogonal polarization states. Prior to fiber coupling, a quarter-wave plate converts the two orthogonal linear polarizations into right-handed and left-handed circular polarizations, respectively. It is well known that circular polarization states exhibit extremely high stability during propagation in step-index multimode fibers.

[0064] The polarization-encoded light beam is focused by the fifth, sixth, and seventh lenses and coupled into a step-index multimode fiber by the first microscope objective. After the first polarizer filters out stray polarization, the third half-wave plate adjusts the polarization direction of the light beam to 45°. A beam shifter generates two orthogonal linearly polarized light beams, which are then converted into left-handed / right-handed circularly polarized light by a quarter-wave plate. The circularly polarized light is then coupled into a step-index multimode fiber by the first microscope objective.

[0065] S5: The light beam output from the multimode fiber passes through the second microscope objective and two achromatic doublet lenses: the ninth lens and the tenth lens, and interferes with the light beam from the reference arm in the beam combiner to form a speckle pattern carrying holographic and polarization information;

[0066] S6: The speckle pattern is captured by a camera and image reconstruction is performed based on the U-Net neural network.

[0067] In step S3, the holographic encoding specifically comprises the following steps: converting the image to be transmitted into a computer generated hologram CGH, and performing phase modulation on the light beam of the signal arm through the DMD;

[0068] After reflection from the DMD, the third half-wave plate and quarter-wave plate convert the beam into left-hand or right-hand circular polarization;

[0069] The U-Net neural network is a 7-layer network architecture, including:

[0070] The encoder consists of three convolutional layers, each of which uses the ReLU activation function for downsampling to extract the multi-scale features of speckle. Each convolutional layer is followed by batch normalization and skip connections to preserve multi-scale feature information.

[0071] The decoder consists of three levels of deconvolution layers, each of which uses the LeakyReLU activation function for upsampling. The feature maps of the corresponding layers of the encoder are spliced ​​through jump connections to restore the image spatial resolution.

[0072] The network uses a Sigmoid activation function in the output layer to limit the pixel values ​​of the reconstructed image to between 0 and 1.

[0073] In step S7, the speckle data are subjected to dimensionality reduction and cluster analysis using the unified manifold approximation and projection UMAP or t-SNE method to verify the encoding differences of different polarization states and holographic labels;

[0074] The dimensionality reduction method uses the Barnes-Hut algorithm to accelerate t-SNE calculation and improve the processing efficiency of large-scale speckle data sets.

[0075] An imaging system based on multimode fiber holography and polarization coding image reconstruction method, comprising:

[0076] Laser: provides a coherent light source, connected in sequence to the isolator, the first lens, and the second lens;

[0077] A beam splitter that splits the light beam into a reference arm and a signal arm;

[0078] Reference arm: a second variable optical attenuator, an eighth lens, and a fourth half-wave plate are sequentially arranged;

[0079] Signal arm: The fourth lens, the second half-wave plate, the digital micromirror device (DMD), the polarizer, the third half-wave plate, the fifth lens, the quarter-wave plate and the first microscope objective lens are arranged in sequence;

[0080] Multimode fiber transmission module: step-index multimode fiber, the input end is connected to the first microscope objective lens, and the output end is connected to the second microscope objective lens and the ninth lens;

[0081] Image reconstruction module: processes speckle images based on the U-Net neural network and outputs the reconstruction results.

[0082] The DMD is a liquid crystal phase modulator with a resolution of ≥1024×768 and a response time of <10ms.

[0083] Polarization-maintaining optical fibers are used in the optical paths of the reference arm and the signal arm, and the extinction ratio is ≥20dB.

[0084] The camera is a polarization-sensitive CMOS sensor that supports left-handed / right-handed circularly polarized light channel capture.

[0085] As a preferred technical solution of the present invention: the core diameter of the step-index multimode optical fiber is 50-200 μm, and the numerical aperture is 0.2-0.5.

[0086] Compared with the prior art, the present invention provides an image reconstruction method and imaging system based on multimode fiber holography and polarization encoding. In the step-index multimode fiber microscopy imaging method, image reconstruction is achieved by decoding the speckle pattern captured by the camera using a U-Net neural network. The generated phase image is injected into the fiber end face using a Fourier lens, thereby embedding holographic information in the speckle pattern. By adding a holographic label to the original data set and introducing more variation in the output speckle pattern, the system's transmission capacity is improved. The uniqueness of the holographic label is used to cluster different types of speckle patterns. For subsequent decoding and reconstruction, the speckle data set is effectively clustered based on the holographic label to maintain high-quality reconstruction and avoid information loss. The signal light and reference light adopt mutually orthogonal polarization states. The polarization-crossed reference light technology uses optical signals with orthogonal polarization states as a reference, which can effectively suppress common-mode noise caused by environmental factors such as fiber vibration and temperature changes, significantly improving the optical system's anti-interference capability. The ResUNet architecture is used to decode each category of speckle pattern separately. It combines the advantages of residual network ResNet and U-type network UNet, has powerful image processing capabilities and high training efficiency, and achieves high-quality reconstruction by learning the mapping relationship between speckle patterns and original images. The average SSIM of the reconstructed images is ≥93%.

[0087] The present invention provides an image reconstruction method and imaging system based on multimode fiber holography and polarization coding. Holographic coding is achieved through a digital micromirror device (DMD), and polarization coding is performed in combination with a half-wave plate and a quarter-wave plate. Holographic modulation and polarization multiplexing are used to realize the modal coupling characteristics of the multimode fiber to generate a speckle pattern carrying multidimensional information, thereby improving the transmission information dimension of the multimode fiber. Combined with deep learning based on a 7-layer neural network architecture based on U-Net, high-fidelity and high-resolution image reconstruction is achieved.

[0088] The present invention solves the problems of low information capacity and poor anti-interference ability in traditional multimode optical fiber imaging, and is applicable to fields such as biological microscopic imaging and industrial endoscopy.

[0089] Example 1

[0090] like Figure 1As shown, an imaging system based on an image reconstruction method based on multimode fiber holography and polarization coding according to the present invention includes:

[0091] A laser is configured to provide light for the system. The light beam output from the laser is passed to an isolator. The function of this isolator is to prevent reflected light from the system from returning to the laser.

[0092] The first lens and the second lens are mainly used for beam expansion processing and are used for expanding the laser beam.

[0093] The first reflector and the second reflector are used to adjust the coupling position and direction of the light beam.

[0094] A first half-wave plate is used to adjust the splitting ratio of light coupled into the optical splitter.

[0095] The third lens is used to focus the light beam.

[0096] A beam splitter is designed to split the expanded light into two paths: a reference arm and a signal arm. The beam splitter divides the expanded light into two arms, one arm is the reference light and the other arm is the signal light.

[0097] The reference arm comprises: a reference light whose intensity is controlled by a second variable optical attenuator, an output light of a first polarization-maintaining optical fiber is collimated by an eighth lens, and is controlled to enter a beam combiner by a fourth half-wave plate;

[0098] The signal arm portion includes: a portion for collimating the output light beam of the first polarization-maintaining optical fiber, passing the output light beam through the fourth lens, and controlling the polarization state through the second half-wave plate before irradiating the output light beam to the digital micromirror device;

[0099] After passing through the digital micromirror device, the light beam is filtered by a polarizer, the third half-wave plate controls the polarization state, and then passes through the fifth lens, the beam shifter, the sixth lens, the seventh lens, the quarter-wave plate changes the polarization state, and the first microscope objective lens, and finally coupled into the step-index fiber;

[0100] The light beam output from the step-index fiber is expanded by the second microscope objective and the ninth lens, and combined with the light beam from the reference arm in the beam combiner. The combined light beam is finally combined by the tenth lens and captured by the camera. This process completes the alignment of the reference arm with the signal arm. After the alignment is completed, the reference arm is removed. The reference arm is only used for calibration before the experiment begins. Only the signal arm is used in subsequent experimental processes.

[0101] The above is the complete set of equipment used in the experimental process of this patent. Figure 1 The dotted-line part (reference arm) is only used during the collimation phase and will be removed after the collimation is completed.

[0102] Based on the above device, the present invention provides an image reconstruction method based on multimode fiber holography and polarization coding, which specifically includes:

[0103] Using digital micromirror devices, we can adjust the amplitude, phase, and polarization state of a light beam entering a multimode fiber;

[0104] like Figure 7 As shown, the present invention's image reconstruction method and imaging system based on multimode fiber holography and polarization encoding adds a holographic label and polarization state information to an original dataset (the MNIST handwritten digit set). The resulting specific phase image is uploaded to a digital micromirror device (DMD). The diffracted light from the DMD is then coupled into a multimode fiber. Holographic labels at different locations and in different polarization states (horizontally polarized, vertically polarized, and horizontally superimposed vertically polarized) correspond to multiple different incident light modes, which serve as input to the fiber. When these light modes are transmitted through the multimode fiber, they interact within the fiber and produce distinct speckle patterns at the fiber's output.

[0105] The coordinates of 10 different holograms under 3 polarization states are as follows:

[0106] .

[0107] In data processing, in order to verify the encoding differences under different polarization states, and the differences between different holographic coupling methods under each polarization state. Under three polarization states, we coupled 2000 handwritten digits (64x64 pixels) in the MNIST dataset through 10 holographic labels. At a wavelength of 532 nanometers, the number of modes carried by the optical fiber is about 7017, so it can carry 64x64 MNIST digits (4096 pixels). Then the unified manifold approximation and projection dimensionality reduction method (UMAP) is used, based on the principles of manifold learning and topology, and using the nonlinear expression of the data set, it aims to find a low-dimensional manifold to approximate high-dimensional data, which is very suitable for representing fiber speckle patterns. UMAP is as follows: Figure 2 As shown in Figures a, b, and c in the figure, they correspond to three polarization states: horizontal polarization, vertical polarization, and horizontal superimposed vertical polarization, and the data are divided into 10 main clusters based on cluster analysis of holographic labels, corresponding to each hologram. In addition, in order to better represent the speckle pattern data set, a method that can more broadly represent each component is needed. We use distributed random neighbor embedding (t-SNE) based on a probability model. Due to the large amount of overall data, we choose the Barnes-Hut method, which approximates the similarity between points through a tree structure, thereby accelerating the calculation. It still maintains the probability-based similarity calculation principle in the core algorithm. Compared with UMAP, label-based clustering is very powerful. As Figure 2As shown in Figures d, e, and f, subclusters can be observed in each of the 10 main clusters. The speckle patterns excited by the same holographic label have strong correlation. Finally, 2000 speckle patterns without additional holographic encoding were taken in three different polarization states, for a total of 6000 images. Figure 3 As shown in Figures a and b in Figure 1, UMAP and t-SNE can clearly divide the data into three main clusters. At the same time, we took 1,000 images from each of the 10 different holographic codes under horizontal polarization and vertical polarization to form a total of 20,000 speckle patterns. UMAP and t-SNE can still easily divide them into 20 different subclusters, which is in line with our expectations. Figure 3 As shown in Figures c and d.

[0108] In the unsupervised case, although both of the above methods can restore the holographic label input under their corresponding polarization states, in order to compare the differences in reconstructed data under different polarization states, a supervised CNN is required. The deep learning model used here is a convolutional neural network with a U-Net architecture, which is used to reveal the phase pattern based solely on the speckle data. We adopted a deeper and larger U-Net architecture with a 7-layer network, which has stronger feature extraction capabilities and multi-scale information integration capabilities. In addition, a larger receptive field helps to capture global background information. In terms of activation functions, ReLU is used for downsampling and LeakyReLU is used for upsampling. Compared with other activation functions, ReLU and LeakyReLU can speed up training, thereby improving the overall efficiency of the learning process. Each convolutional layer in the U-Net architecture is followed by batch normalization and an activation function. As Figure 4 As shown, a comprehensive overview of the network architecture is provided, including the number of layers and the shape of the output tensor of each stage, providing a clear reference for readers to understand the complex structure of the network.

[0109] To verify this, we compared the reconstruction of a set of handwritten digits transmitted through an optical fiber. Without introducing holographic encoding, the full set of 70,000 MNIST handwritten digits was transmitted for each polarization state, of which 60,000 were used to train the aforementioned U-Net network, 7,000 were used for validation, and 3,000 were used for testing. To quantify the difference between the predicted output and the actual output, we used the mean absolute error (MAE) as our loss function:

[0110]

[0111] Y i,j is the label phase diagram, X i,jis the network output image, k is the index within each mini-batch, and i and j are the image indices. N x N represents the image size (in pixels), and M is the training mini-batch size. At the end of each mini-batch training, the weights and biases in the network are updated using the Adaptive Moment Estimation (Adam) optimizer. Training is performed on a powerful computing platform equipped with two NVIDIA GeForce RTX 4090 GPUs and an Intel® Core™ i9-14900K CPU. Each training session takes approximately 3 minutes. Figure 5 As shown in Figure d, the MAE of training loss and validation loss reaches a convergence point after 40 epochs. Figure 5 In Figure (a), the first row shows a subset of original images from the MNIST dataset, while the second through fourth rows show speckle data collected using three polarization encoding schemes. The fifth through seventh rows show U-Net reconstructions of the original images based on the output speckle patterns. In all three cases, the reconstructions closely resemble the original data, making each digit easily recognizable. We quantify this by measuring the structural similarity index (SSIM) between the reconstructed and original images. For the test data, the average SSIM for the CNN-reconstructed images is approximately 0.93. Figure 5 The histogram in Figure b and Figure 5 The histogram in Figure c illustrates this point.

[0112] The U-Net network used in the experiment can be replaced with other neural network architectures such as GAN generative adversarial networks, which may further improve the SSIM value of the final reconstructed image.

[0113] Use the reference arm optical path to align the signal arm optical path and debug the entire optical experiment system. After debugging is completed, remove the reference arm.

[0114] First, we selected 2,000 images from the MNIST handwritten digits dataset and used an algorithm (CGH) to perform dual-holographic polarization encoding at 10 different holographic positions in three polarization states: horizontal polarization, vertical polarization, and horizontal combined with vertical polarization, generating corresponding phase maps. These phase maps were then uploaded to a digital micromirror device (DMD). Using our experimental system, the diffracted light from the incident beam after passing through the DMD was coupled into a multimode optical fiber, where a CMOS camera was used to capture the far-field speckle pattern at the end of the fiber. These speckle patterns were analyzed using the Unsupervised Mapping (UMAP) and T-SNE algorithms for unsupervised learning of different holographic labels in these three polarization states, as well as unsupervised learning of different polarization states within the same holographic label. The results showed clear clustering, confirming our hypothesis.

[0115] Next, we used the complete MNIST handwritten digit dataset of 70,000 images. Using a CGH algorithm, we generated phase maps corresponding to the initial holographic label position in each of the three polarization states, totaling 3 x 70,000 images. For each polarization state, we selected 60,000 speckle images as the training set, 7,000 as the validation set, and 3,000 as the test set. Using a U-net architecture, we trained the speckle data for each polarization state, reaching convergence in approximately 30 minutes. The trained network can restore and reconstruct speckle images in milliseconds. Subsequent data analysis showed that the U-net training process was nearly identical for all three polarization states (as evidenced by the training and validation loss curves), and the structural similarity index (SSIM) of the recovered data for the test set was consistently around 0.93.

[0116] In terms of the optical path, different polarization states of the light beam can be obtained by changing the device of the optical system, rather than using a digital micromirror device (DMD) to dynamically adjust the polarization state of the light beam as in this experiment.

[0117] Traditional image reconstruction methods based on multimode fiber (MMF) typically ignore the polarization state of light. This is because, in most cases, MMF imaging systems focus on scattering and interference effects through the fiber, particularly imaging based on intensity and phase information. When light propagates through a multimode fiber, the influence of polarization is often neglected due to the fiber's modal distribution and internal scattering properties. However, polarization effects can sometimes play a role in multimode fiber imaging, particularly in applications requiring high resolution or sensitive to anisotropy of the optical field. For example, the fiber's structure and inhomogeneities can cause light modes with different polarization states to propagate differently. This difference can, in certain circumstances, affect image quality or mode coupling during imaging. With the further development of multimode fiber imaging technology, the study of polarization state has become a growing focus, particularly in high-resolution imaging and information extraction. By considering polarization information, researchers can better control and optimize the performance of multimode fiber imaging systems, potentially improving image quality or increasing the amount of decodable information.

[0118] The present invention's image reconstruction method and imaging system based on multimode fiber holography and polarization coding achieves dual modulation of holographic and polarization coding through the synergistic effect of a digital micromirror device (DMD) and polarization optical elements. Image reconstruction is performed using a U-Net neural network. Through the innovative design of holographic modulation and polarization multiplexing, combined with U-Net deep learning, high-fidelity image transmission and reconstruction in multimode fiber are achieved. With its high-fidelity reconstruction, anti-interference capabilities, and multi-physics field encoding advantages, this method has disruptive potential in applications such as biological microscopy and industrial endoscopy, and is particularly suitable for high-quality imaging in complex environments.

[0119] The above-mentioned specific implementation methods are used to illustrate the present invention and are only preferred embodiments of the present invention, rather than limiting the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit of the present invention and the scope of protection of the claims shall fall within the scope of protection of the present invention.

Claims

1. An image reconstruction method based on multimode fiber holography and polarization coding, characterized in that: The following steps are involved: S1, output laser light source, after the isolator eliminates the reflected light, the beam is expanded by the first lens and the second lens so that the beam diameter matches the digital micromirror device DMD; S2, uses a beam splitter to split the expanded beam into a reference arm and a signal arm; Reference arm: Light intensity and polarization state are controlled by an adjustable attenuator, collimating lens, and half-wave plate; Signal arm: calibrated by a collimating lens and half-wave plate to match the DMD liquid crystal orientation to optimize diffraction efficiency; In S3, a hologram containing spatial position information is generated by the DMD in the signal arm, and the polarization state is dynamically switched through the optical relay device to form a holographic light field distribution with different polarization states in three-dimensional space, thereby obtaining a dual-encoded holographic phase image after holographic encoding and polarization encoding. S4, the modulated multi-polarized light field is coupled into the multimode optical fiber through the microscope objective; S5, the optical fiber output end passes through an achromatic lens group to form a speckle pattern carrying polarization information; S6, collects speckle patterns through a camera and uses a U-Net neural network to reconstruct holographic images.

2. The image reconstruction method based on multimode fiber holography and polarization coding according to claim 1, wherein: In step S2, the reference arm: the light intensity is adjusted by the second variable optical attenuator, the eighth lens is collimated, and the fourth half-wave plate is adjusted for polarization state, and then input into the beam combiner until it is orthogonal to the signal arm and then input into the beam combiner; Signal arm: collimated by the fourth lens and the second half-wave plate in sequence, the second half-wave plate matches the DMD liquid crystal orientation angle and precisely aligns the 624×624 central area to maximize diffraction efficiency.

3. The image reconstruction method based on multimode fiber holography and polarization coding according to claim 1, wherein: In step S3, a phase hologram is loaded onto a digital micromirror device to control the wavefront distribution, generating a multi-focus holographic structure in three-dimensional space. Polarization state control is achieved by switching horizontal, vertical, and circular polarization modes through optical relay devices, and each focal position is associated with the horizontal, vertical, and circular polarization states to form a polarization-encoded three-dimensional light field. Through the three-dimensional spatial coherent superposition of light fields with different polarization states, a holographic light field that integrates polarization and spatial dimensions is constructed.

4. The image reconstruction method based on multimode fiber holography and polarization coding according to claim 3, characterized in that: Step S3 includes the following steps: S301, a digital micromirror device is loaded with a preset holographic coding pattern, the coding pattern is compressed and relayed to the input end of a multimode optical fiber through a 4f lens system, and a multi-focus holographic structure is generated in a three-dimensional space; S302, the incident light is programmed to switch between horizontal polarization state, vertical polarization state and circular polarization state through the optical relay device; S303 , associating and encoding each focal position in the three-dimensional space with each polarization state, coherently superimposing light fields of different polarization states in the three-dimensional space, and outputting a holographic light field that integrates polarization and spatial dimensions.

5. The image reconstruction method based on multimode fiber holography and polarization coding according to claim 1, wherein: Step S4 specifically includes the following steps: S401, the digital micromirror device (DMD) relays and reduces the encoded holographic image displayed on the DMD to the proximal end face of the multimode optical fiber through two consecutive 4f imaging systems consisting of a fifth lens, a sixth lens, a seventh lens, and a first microscope objective lens; In step S402, the light beam is purified by the first polarizer, adjusted to 45° polarization by the third half-wave plate, split into orthogonal linearly polarized light by the beam shifter, and then converted into left / right circularly polarized light by a quarter-wave plate. Finally, it is coupled to the step-index multimode fiber by the microscope objective.

6. The image reconstruction method based on multimode fiber holography and polarization coding according to claim 5, characterized in that: In step S402, the first polarizer filters out stray orthogonal polarization components, the third half-wave plate polarizes the light beam to 45°, and the beam shifter generates two spatially separated orthogonal linear polarized light beams, which are recombined into a common optical path to retain the orthogonal polarization state. Before fiber coupling, the quarter-wave plate converts the two orthogonal linear polarization beams into right-handed and left-handed circular polarization, respectively.

7. The image reconstruction method based on multimode fiber holography and polarization coding according to claim 1, wherein: In step S5, the light beam output from the multimode optical fiber passes through the second microscope objective lens and two achromatic doublet lenses: the ninth lens and the tenth lens, to form a speckle pattern carrying holographic and polarization information.

8. The image reconstruction method based on multimode fiber holography and polarization coding according to claim 1, wherein: In step S6, the U-Net neural network is a 7-layer network architecture, including: The encoder consists of three convolutional layers, each of which uses the LeakyReLU activation function for downsampling to extract the multi-scale features of the speckle pattern. Each convolutional layer is followed by batch normalization and skip connections to preserve multi-scale feature information. The decoder consists of three levels of deconvolution layers, each of which uses the ReLU activation function for upsampling. The feature maps of the corresponding layers of the encoder are spliced ​​through jump connections to restore the image spatial resolution. The network uses a Sigmoid activation function in the output layer to limit the pixel values ​​of the reconstructed image to between 0 and 1.

9. The image reconstruction method based on multimode fiber holography and polarization coding according to claim 1, wherein: In step S6, the speckle image is subjected to dimensionality reduction and cluster analysis using the unified manifold approximation and projection dimensionality reduction methods (UMAP and t-SNE), respectively, to verify the encoding differences of different polarization states and holographic labels; The dimensionality reduction method uses the Barnes-Hut algorithm to accelerate t-SNE calculations and improve the processing efficiency of large-scale speckle image sets.

10. An imaging system using the image reconstruction method based on multimode fiber holography and polarization coding according to any one of claims 1 to 5, characterized in that: include: Laser: Provides a coherent light source, which is sequentially connected to the isolator, the first lens, and the second lens to form a collimated beam expansion optical path; A beam splitter that splits the light beam into a reference arm and a signal arm; Reference arm: a second variable optical attenuator, an eighth lens, and a fourth half-wave plate are sequentially arranged; Signal arm: The fourth lens, the second half-wave plate, the digital micromirror device, the polarizer, the third half-wave plate, the fifth lens, the quarter-wave plate and the first microscope objective lens are sequentially arranged; Multimode fiber transmission module: step-index multimode fiber, the input end is connected to the first microscope objective lens, and the output end is connected to the second microscope objective lens and the ninth lens; Interferometric imaging module: The reference arm is introduced into the beam combiner, interferes with the signal arm to complete the collimation, and then the reference arm is removed; the signal light is focused by the tenth lens, and the pure signal speckle pattern is captured by the camera; Image reconstruction module: Uses the U-Net neural network to process speckle images, realizes multimodal information decoding, and outputs the reconstruction results.

Images