A deep learning-based anti-disturbance multimode fiber speckle imaging method

Abstract

This invention discloses a deep learning-based method for perturbation-resistant multimode fiber speckle imaging, comprising the following steps: acquiring an image dataset and preprocessing the images; constructing an image transmission system based on a single multimode fiber; under dynamic perturbation, feeding the preprocessed image into the image transmission system to construct a training set containing the input image and speckle pattern; designing a perturbation-resistant speckle imaging network; feeding the training set obtained from optical experiments into the self-designed perturbation-resistant speckle imaging network for training, thereby obtaining a speckle imaging model with excellent generalization ability to dynamic perturbations; using the trained speckle imaging model, high-fidelity reproduction of the input image can be achieved from the speckle pattern generated at the output end of the multimode fiber under arbitrary perturbation conditions. According to this invention, the requirements of multimode fiber in various practical application scenarios are met, significantly improving the fidelity of image reproduction and exhibiting excellent robustness to dynamic perturbations.

Description

Technical Field

[0001] This invention relates to the field of optical imaging technology, and in particular to a deep learning-based method for perturbation-resistant multimode fiber speckle imaging. Background Technology

[0002] Multimode optical fibers, combining high information capacity, small core diameter, and flexible structure, show great promise for applications in communication, sensing, and endoscopic imaging. However, the combined effects of mode excitation, coupling, dispersion, and interference lead to severe distortion of transmitted image information, resulting in highly chaotic speckle patterns at the fiber's output end. It is worth emphasizing that although the features of the original image are no longer interpretable, the image information is not lost but rather encoded in the global distribution of the speckle pattern. This characteristic lays the physical foundation for image reconstruction.

[0003] Phase conjugation, transfer matrix, and compressed sensing methods have been proven to achieve speckle-based image reconstruction, but the extreme sensitivity of optical fibers to perturbations still severely limits their practical applications. Any micro-perturbation can significantly alter optical transmission characteristics, thereby reducing imaging accuracy. Rigid optical fibers that can effectively suppress deformation perturbations have been successfully used for intracranial neuron imaging, but most applications still rely on flexible media. Some studies have introduced coherent beacons, partial mirrors, and metasurface reflector stacks at the fiber output end to recalibrate the transfer matrix at the incident end, ultimately achieving stable endoscopic imaging under dynamic perturbations. However, such methods not only increase system complexity but also severely limit imaging speed. Furthermore, average speckle illumination schemes and low-rank recovery algorithms have been shown to improve the robustness of imaging systems to micro-deformations, but they struggle to cope with large-scale dynamic deformations.

[0004] Neural networks have been shown to learn signal invariance from perturbed scattering media, thus extending the applicability of scattering imaging techniques to various perturbed environments. This achievement is significant, opening up new research avenues for improving the imaging robustness of multimode fibers under dynamic perturbation conditions. Existing research has shown that neural networks possess a certain degree of adaptability to small-scale environmental changes or fiber deformation, and their generalization performance to dynamic perturbations can be enhanced through large-scale training sets, parallel network structures, or the introduction of additional fiber morphological parameters. However, existing deep learning-based research on multimode fiber speckle imaging largely borrows network structures from general computer vision tasks, lacking explicit modeling of the physical effects of light in pattern transmission, and therefore failing to fully meet the underlying task characteristics of speckle imaging. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a deep learning-based method for anti-disturbance multimode fiber speckle imaging, enabling high-fidelity image transmission through strongly disturbed multimode fibers, thus providing an effective solution for application scenarios where deformation is unavoidable. To achieve the above-mentioned objectives and other advantages of the present invention, a deep learning-based method for anti-disturbance multimode fiber speckle imaging is provided, comprising: Obtain the image dataset and preprocess the images; A transmission system based on a single multimode fiber is constructed, and the preprocessed image is sent into the transmission system to build a training set containing the input image and speckle pattern. An anti-disturbance speckle imaging network is constructed, which includes a global sensing module, a frequency domain enhancement module, a residual encoding / decoding module, and a composite loss function. The training set is fed into the anti-disturbance speckle imaging network for training, thereby obtaining a speckle imaging model with excellent generalization ability to dynamic disturbances. By using a trained speckle imaging model, high-fidelity reproduction of the input image can be achieved from the speckle pattern generated at the output end of a multimode fiber under arbitrary perturbation conditions.

[0006] Preferably, the imaging system includes a laser, and a first objective lens, a pinhole filter, a first lens, a half-wave plate, a beam splitter cube, and a spatial light modulator are arranged sequentially on one side of the laser. A second objective lens, a multimode fiber, a third objective lens, a second lens, and a camera are arranged sequentially on one side of the beam splitter cube. The camera and the spatial light modulator are connected to a computer, and the multimode fiber is fixed to an electric displacement platform by cable ties.

[0007] Preferably, after the image is preprocessed by a computer program, it is sequentially loaded onto the liquid crystal surface of the spatial light modulator. The modulated light carrying the image information is transmitted through a dynamically deformable multimode fiber. A CCD camera is used to sequentially collect the corresponding speckle patterns at the output end of the multimode fiber, thereby constructing a training set.

[0008] Preferably, during speckle pattern acquisition, the multimode fiber fixed on the electric displacement platform is subjected to strong mechanical disturbance, and then undergoes continuous dynamic deformation. The constructed training set contains fiber morphology parameters with a large dynamic range, which meets the practical requirements of multimode fiber in different application scenarios.

[0009] Preferably, the anti-disturbance speckle imaging network utilizes three heterogeneous branches guided by physical priors for front-end feature extraction: the first branch learns the global features of the speckle pattern using a global perception module; the second branch introduces a frequency domain enhancement module to specifically suppress low-frequency noise generated by deformation, and then learns the global features of the high-frequency speckle pattern through the global perception module; the third branch preserves the basic spatial structure of the original speckle pattern through spatial downsampling. These three features are concatenated along the channel dimension, providing rich multi-scale fusion features for the residual coding module. The downsampling path refines high-level semantic features layer by layer through a six-level residual coding module, and the embedded frequency domain enhancement module effectively preserves high-frequency structural information that is easily lost in convolution operations while suppressing low-frequency noise. The upsampling path reconstructs image information through a five-level residual decoding module. The introduction of skip connections and residual modules enhances the ability to remember and extract image information encoded in the speckle feature map, thereby improving image reconstruction fidelity.

[0010] Preferably, the global perception module utilizes dense layers to model the global connection between the speckle pattern and the input image, thereby simulating a nonlinear inverse filtering process to crack the highly mixed linear coding in the forward propagation, providing enhanced global feature representation for the back-end residual encoding and decoding module, which excels at spatial structure reconstruction. Simultaneously, a self-designed global attention mechanism is used to suppress redundant features introduced by modal noise and pattern coupling.

[0011] Preferably, the frequency domain enhancement module enhances key frequency components and suppresses perturbation noise by performing a Fourier transform on the speckle pattern with a high-frequency center and applying a learnable Gaussian mask. This frequency domain enhancement module is constructed as a differentiable network layer, where the scale parameters of the Gaussian kernel in different channels can be adaptively optimized through gradient backpropagation based on the loss function. With the help of the frequency domain enhancement module, the network can strengthen the learning of high-frequency information that is relatively insensitive to deformation noise, thereby suppressing low-frequency noise and improving image reconstruction quality.

[0012] Preferably, the residual encoding / decoding module, while retaining the ability to perceive the two-dimensional neighborhood relationship between pixels, effectively ensures the stable propagation of gradients in deep networks by introducing identity jump connections, alleviates the degradation problem, and promotes the efficient fusion of multi-scale features extracted by the encoder and the reconstruction process of the decoder, thereby significantly improving the fidelity of image reconstruction.

[0013] Preferably, the anti-perturbation speckle imaging network includes a composite loss function that fuses mean square error, peak signal-to-noise ratio (PSNR), and structural similarity. The mean square error, as the fundamental term, has a weight coefficient set to 0.7 to ensure training stability and global grayscale fidelity. Simultaneously, it innovatively integrates PSNR and structural similarity into a perceptual optimization term, where structural similarity is responsible for accurately locating structurally distorted regions, while PSNR provides a relative perceptual assessment of the error magnitude. This composite design allows the model to automatically focus on visually sensitive reconstruction challenges without complex attention calculations, thereby improving reconstruction quality to a certain extent.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are: By leveraging the strong mechanical interference generated by an electrodynamic displacement platform, a fiber deformation over a large dynamic range is provided, meeting the practical requirements of multimode fiber in various application scenarios. In the presented anti-disturbance speckle imaging network, the global sensing module and frequency domain enhancement module are designed based on physical priors of mode transmission and perturbation noise, combined with a classic residual encoding / decoding module, effectively achieving robust imaging of multimode fiber under dynamic perturbations. This network framework exhibits good versatility and can be extended to imaging scattering media under different perturbation environments. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the image transmission system of the deep learning-based anti-disturbance multimode fiber speckle imaging method according to the present invention. Figure 2 This is a structural diagram of the anti-disturbance speckle imaging network of the deep learning-based anti-disturbance multimode fiber speckle imaging method according to the present invention. Figure 3 This is a comparison of the speckle imaging network of the deep learning-based anti-disturbance multimode fiber speckle imaging method according to the present invention and the reconstruction of EMNIST images by U-net. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Reference Figure 1 A deep learning-based method for perturbation-resistant multimode fiber speckle imaging includes: Obtain the image dataset and preprocess the images; A transmission system based on a single multimode fiber is built. Under dynamic disturbance conditions, the preprocessed image is sent into the transmission system to construct a training set containing the input image and speckle pattern. Based on the physical priors of pattern transmission and perturbation noise, an anti-perturbation speckle imaging network integrating a global sensing module, a frequency domain enhancement module, a residual encoding and decoding module, and a composite loss number was designed. The training set constructed from optical experiments is fed into a self-designed anti-disturbance speckle imaging network for training, thereby obtaining a speckle imaging model with excellent generalization ability to dynamic disturbances. By using a trained speckle imaging model, high-fidelity reproduction of the input image can be achieved from the speckle pattern generated at the output end of a multimode fiber under arbitrary perturbation conditions.

[0018] Example 1: Image transmission system based on a single multimode fiber, such as Figure 1 As shown, the beam emitted by the helium-neon laser is focused by the first objective lens, and stray light is filtered out at its focal point by a pinhole filter. The resulting divergent beam is collimated by the first lens. The collimated beam is rotated to a horizontal polarization state by a half-wave plate, and then illuminates the spatial light modulator by a beam splitter. According to the loaded image characteristics, the spatial light modulator performs wavefront phase modulation during light reflection. The reflected light carrying image information is guided by the same beam splitter to the second objective lens, and then efficiently coupled into the multimode fiber. During light transmission, the synergistic effect of mode coupling and intermode dispersion causes severe degradation of image information and excites multimode interference at the fiber output, forming a divergent speckle field. After the light field is collimated by the third objective lens and the second lens, a clear speckle pattern is received by the camera and finally stored in the computer for subsequent processing.

[0019] In the speckle pattern acquisition process, to simulate the morphological changes of optical fibers in practical applications, multimode optical fibers were fixed on an electrically driven displacement stage, and dynamically deformed within a range of 0 to 100 mm via mechanical drive. 12,000 letter images were selected from the EMNIST dataset, upsampled to 512×512 grayscale images, and then sequentially loaded onto a spatial light modulator at 0.3-second intervals. At the fiber optic output end, speckle patterns with a resolution of 760×760 were simultaneously acquired. The first 10,000 sets of speckle patterns and letter images were used for network training, and the last 2,000 sets were used for model testing. Each frame of the speckle pattern not only encodes the input image information but also records the transient deformation characteristics of the optical fiber at that moment. The constructed dataset covers rich image content and morphological perturbation parameters, helping the network understand the influence of image features and deformation perturbations on the speckle pattern, thereby establishing a robust mapping from hybrid encoding to the target image.

[0020] To improve robustness to strong mechanical disturbances and achieve high-fidelity image transmission through continuously deformable multimode fiber, this invention proposes a dedicated speckle imaging network integrating a global sensing module, a frequency domain enhancement module, a residual encoding / decoding module, and a composite loss function, guided by the physical priors of mode transmission and disturbance noise. The network architecture is as follows: Figure 2 As shown, the speckle pattern, as the network input, is first used for front-end feature extraction via three heterogeneous branches guided by physical priors: the first branch learns the global features of the complete speckle pattern using a global perception module; the second branch introduces a frequency domain enhancement module to specifically suppress low-frequency noise caused by deformation, and then learns the global features of the high-frequency speckle pattern through the global perception module; the third branch preserves the basic spatial structure of the original speckle pattern through spatial downsampling. These three features are concatenated along the channel dimension, providing rich multi-scale fusion features for the residual coding module. The downsampling path refines high-level semantic features layer by layer through a six-level residual coding module, and the embedded frequency domain enhancement module effectively preserves high-frequency structural information that is easily lost in convolution operations while suppressing low-frequency noise. The upsampling path reconstructs image information through a five-level residual decoding module. The introduction of skip connections and residual modules enhances the ability to remember and extract image information encoded in the speckle feature map, thereby improving the fidelity of image reconstruction.

[0021] (1), The composite loss function integrating mean square error, peak signal-to-noise ratio, and structural similarity is shown in Equation (1). The mean square error, as the basic term, has a weighting coefficient. The value was set to 0.7 to ensure training stability and global grayscale fidelity. Simultaneously, it innovatively integrates peak signal-to-noise ratio (PSNR) and structural similarity into a perceptual optimization term. Structural similarity is responsible for accurately locating structurally distorted regions, while PSNR provides a relative perceptual assessment of the error magnitude. This composite design allows the model to automatically focus on visually sensitive reconstruction challenges without complex attention calculations, thereby improving reconstruction quality to some extent.

[0022] 10,000 image-speckle pairs are fed into... Figure 2 The network shown was trained using a composite loss function, the Nadam optimizer, 60 iterations, and a batch size of 16. We used 2000 test samples containing unknown and random fiber morphology parameters to validate the trained model's performance. Figure 3 As shown, compared to the classic U-net model, the network designed based on physical priors achieves high-fidelity transmission of EMNIST images through continuously deformed multimode optical fibers subjected to strong mechanical disturbances.

[0023] Furthermore, during speckle pattern acquisition, the multimode fiber fixed on the electric displacement platform is subjected to strong mechanical disturbance, which in turn undergoes continuous dynamic deformation. The constructed training set contains fiber morphology parameters with a large dynamic range, which meets the practical requirements of multimode fiber in different application scenarios.

[0024] Furthermore, the global perception module utilizes dense layers to model the global connectivity between the speckle pattern and the input image, thereby simulating a nonlinear inverse filtering process to crack the highly mixed linear coding in the forward propagation, providing enhanced global feature representation for the backend residual encoding / decoding module, which excels at spatial structure reconstruction. Simultaneously, a self-designed global attention mechanism is used to suppress redundant features introduced by modal noise and pattern coupling.

[0025] Furthermore, the frequency domain enhancement module enhances key frequency components and suppresses perturbation noise by performing a Fourier transform on the speckle pattern with a high-frequency center and applying a learnable Gaussian mask. This module is constructed as a differentiable network layer, where the scale parameters of the Gaussian kernel in different channels can be adaptively optimized through gradient backpropagation based on the loss function. With the help of the frequency domain enhancement module, the network can strengthen the learning of high-frequency information that is relatively insensitive to deformation noise, thereby suppressing low-frequency noise and improving image reconstruction quality.

[0026] Furthermore, the residual encoding / decoding module, while retaining the ability to perceive the two-dimensional neighborhood relationship between pixels, effectively ensures the stable propagation of gradients in deep networks by introducing identity jump connections, alleviates the degradation problem, and promotes the efficient fusion of multi-scale features extracted by the encoder and the reconstruction process of the decoder, thereby significantly improving the fidelity of image reconstruction.

[0027] Furthermore, the anti-perturbation speckle imaging network includes a composite loss function that fuses mean square error, peak signal-to-noise ratio (PSNR), and structural similarity. The mean square error, as a fundamental term, has a weighting coefficient of 0.7 to ensure training stability and global grayscale fidelity. Simultaneously, it innovatively integrates PSNR and structural similarity into a perceptual optimization term, where structural similarity is responsible for accurately locating structurally distorted regions, while PSNR provides a relative perceptual assessment of the error magnitude. This composite design allows the model to automatically focus on visually sensitive reconstruction challenges without complex attention calculations, thereby improving reconstruction quality to a certain extent.

[0028] The number of devices and processing scale described herein are for simplification purposes. Applications, modifications, and variations of this invention will be readily apparent to those skilled in the art. Although embodiments of the invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. It can be applied to various fields suitable for this invention, and further modifications can be readily implemented by those skilled in the art. Therefore, without departing from the general concept defined by the claims and their equivalents, this invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A deep learning based anti-disturbance multimode fiber speckle imaging method, characterized in that, Includes the following steps: Obtain the image dataset and preprocess the images; A transmission system based on a single multimode fiber is constructed, and the preprocessed image is sent into the transmission system to build a training set containing the input image and speckle pattern. An anti-disturbance speckle imaging network is constructed, which includes a global sensing module, a frequency domain enhancement module, a residual encoding / decoding module, and a composite loss function. The training set is fed into the anti-disturbance speckle imaging network for training, thereby obtaining a speckle imaging model with excellent generalization ability to dynamic disturbances. By using a trained speckle imaging model, high-fidelity reproduction of the input image can be achieved from the speckle pattern generated at the output end of a multimode fiber under arbitrary perturbation conditions.

2. The anti-disturbance multi-mode fiber speckle imaging method based on deep learning according to claim 1, wherein, The imaging system includes a laser, and a first objective lens, a pinhole filter, a first lens, a half-wave plate, a beam splitter cube, and a spatial light modulator are arranged sequentially on one side of the laser. A second objective lens, a multimode fiber, a third objective lens, a second lens, and a camera are arranged sequentially on one side of the beam splitter cube. The camera and the spatial light modulator are connected to a computer. The multimode fiber is fixed to an electric displacement platform by cable ties.

3. The deep learning based anti-diffusion multi-mode optical fiber speckle imaging method of claim 2, wherein, The construction of the training set, which includes the input image and the speckle pattern, involves the image being preprocessed by a computer program and then sequentially loaded onto a spatial light modulator. The modulated light carrying the image information is transmitted through a dynamically deformable multimode fiber. A camera is used to sequentially acquire the corresponding speckle patterns at the output end of the multimode fiber, thereby constructing the training set.

4. The deep learning based anti-diffusion multi-mode optical fiber speckle imaging method of claim 2, wherein, During speckle pattern acquisition, the multimode optical fiber fixed on the electric displacement platform is subjected to strong mechanical disturbance, and then undergoes continuous dynamic deformation. The constructed training set contains optical fiber morphological parameters with a large dynamic range.

5. The deep learning based anti-diffusion multi-mode optical fiber speckle imaging method of claim 1, wherein, The anti-disturbance speckle imaging network utilizes three physically prior-guided heterogeneous branches for front-end feature extraction, specifically including: The first branch path learns the global features of the speckle pattern with the help of the global perception module; The second branch path introduces a frequency domain enhancement module to specifically suppress low-frequency noise generated by deformation, and then a global perception module learns the global features of the high-frequency speckle pattern. The third branch path preserves the basic spatial structure of the original speckle pattern through spatial downsampling.

6. The deep learning based anti-diffusion multi-mode optical fiber speckle imaging method of claim 5, wherein, The global perception module utilizes dense layers to model the global connection between the speckle pattern and the input image, thereby simulating a nonlinear inverse filtering process to crack the highly mixed linear coding in the forward propagation. This provides enhanced global feature representation for the back-end residual encoding and decoding module, which excels at spatial structure reconstruction. At the same time, it uses a self-designed global attention mechanism to suppress redundant features introduced by modal noise and mode coupling.

7. The deep learning based anti-diffusion multi-mode optical fiber speckle imaging method of claim 6, wherein, The frequency domain enhancement module performs a Fourier transform on the speckle pattern with a high-frequency center and applies a learnable Gaussian mask. The frequency domain enhancement module is constructed as a differentiable network layer, and the scale parameters of the Gaussian kernel in different channels can be adaptively optimized according to the loss function through gradient backpropagation.

8. The deep learning based anti-diffusion multi-mode optical fiber speckle imaging method of claim 7, wherein, The residual encoding / decoding module retains the ability to perceive the two-dimensional neighborhood relationship between pixels. By introducing identity jump connections, it effectively ensures the stable propagation of gradients in deep networks and promotes the efficient fusion of multi-scale features extracted by the encoder with the reconstruction process of the decoder, thereby significantly improving the fidelity of image reconstruction.

9. The deep learning based anti-diffusion multi-mode optical fiber speckle imaging method of claim 5, wherein, The anti-disturbance speckle imaging network includes a composite loss function that integrates mean square error, peak signal-to-noise ratio, and structural similarity.

Images