A computational imaging method and system based on multimode optical fiber mode modulation

Abstract

The present invention discloses a computational imaging method and system based on multimode fiber mode modulation, comprising: designing and constructing a multimode fiber image transmission system, and collecting natural scene speckle image datasets under different transmission conditions by replacing multimode fibers with different mode modulations; constructing an optimized network architecture based on a UNET network, and constructing a deep learning network with a serial dual-subnet structure for feature extraction and image reconstruction from speckle images to SLM images, and from SLM images to real images, and learning complex speckle image features; training the deep learning network based on natural scene speckle image datasets with different mode modulation states, and introducing a joint loss function to optimize the network's reconstruction performance; and verifying the quality of the model-reconstructed images using speckle images under different transmission conditions. The present invention enhances target information in the speckle image through mode modulation, and achieves high-quality image reconstruction using the deep learning network.

Description

Technical Field

[0001] The present invention relates to the technical field of computational optical imaging and image reconstruction, and more particularly to a computational imaging method and system based on multimode optical fiber mode modulation. Background Art

[0002] Multimode fiber (MMF) has broad application prospects in optical communications, biomedical imaging, optical sensors and other fields due to its high mode capacity and tiny size. For example, in optical communications, MMF can support thousands of guided wave modes, making it a new choice for short-distance, high-capacity data transmission. However, the effects of inter-mode coupling, dispersion and scattering in the MMF transmission process can lead to strong randomness in the output light field, forming a complex speckle pattern. This speckle pattern is extremely sensitive to external interference, making it difficult to accurately demodulate the image information transmitted through the MMF. Therefore, how to recover a high-quality input image from the speckle pattern output by the MMF is a hot and difficult problem in current research.

[0003] Traditional methods such as wavefront modulation, phase conjugation, and transfer matrix technologies can solve the MMF imaging problem to a certain extent, but these methods rely on precise physical model calibration and have poor adaptability to environmental changes and fiber disturbances. For example, although the transfer matrix-based method can accurately demodulate information, its demodulation capability will be significantly reduced in long-distance MMFs that are susceptible to environmental interference.

[0004] In recent years, deep learning technology has provided new ideas for solving this problem. Deep learning can effectively reconstruct distorted images by learning the complex relationship between input and output. However, existing deep learning-based imaging methods mostly focus on reconstructing simple patterns and still have limitations in imaging complex natural scenes. These methods are not adaptable enough to changes in MMF transmission conditions, have low image resolution, and require a large amount of data for training. In addition, although the application of deep learning in fiber optic imaging has made significant progress, it still faces challenges such as data scarcity, insufficient robustness, and insufficient integration of models with physical properties.

[0005] Therefore, how to improve the imaging system's ability to reconstruct complex natural scenes without changing the physical properties of the optical fiber, and adaptively adjust in different optical fiber transmission environments to improve the robustness and accuracy of the system, is an urgent problem that technical personnel in this field need to solve. Summary of the Invention

[0006] In view of this, the present invention provides a computational imaging method and system based on multimode optical fiber mode modulation to solve some of the technical problems mentioned in the background technology.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] A computational imaging method based on multimode optical fiber mode modulation comprises the following steps:

[0009] S1. Design and build a multimode fiber image transmission system, and collect natural scene speckle image datasets under different transmission conditions by replacing multimode fibers with different modulation modes.

[0010] S2. Build an optimized network architecture based on the UNET network and a deep learning network with a serial dual-subnet structure for feature extraction and image reconstruction from speckle images to SLM images and from SLM images to real images, and learn complex speckle image features;

[0011] S3. A deep learning network is trained based on a dataset of natural scene speckle images with different modal modulation states, and a joint loss function is introduced to optimize the network's reconstruction performance.

[0012] S4. Use the trained deep learning network model to reconstruct the speckle images under different transmission conditions and verify the quality of the reconstructed images.

[0013] Preferably, the natural scene speckle image dataset includes a real image at the front end of the multimode optical fiber, an image in the spatial light modulator, and a speckle image actually formed by transmission through the multimode optical fiber, and the speckle image, SLM image, and real image correspond one to one.

[0014] Preferably, the process of collecting natural scene speckle image datasets under different transmission conditions by replacing multimode optical fibers with different mode modulations includes: first, using an unprocessed multimode optical fiber to collect speckle; then, fusing a seven-core optical fiber at the output end of the multimode optical fiber, and then collecting speckle; based on the fusion of the seven-core optical fiber, performing a grating writing operation at the near input end of the multimode optical fiber, and then collecting speckle.

[0015] Preferably, the optimized network architecture based on the UNET network includes an encoder and a decoder; the encoder includes multiple downsampling modules for extracting features of the speckle image; the decoder includes multiple upsampling modules for reconstructing the image.

[0016] Preferably, the optimized network architecture based on the UNET network also includes a Transformer module, which is respectively arranged in the encoder and decoder networks, and the feature maps extracted by the downsampling module and the upsampling module are respectively embedded in the Transformer layer for propagation, and the contextual information between different areas in the speckle image is extracted through the self-attention mechanism.

[0017] Preferably, a skip connection is set between the downsampling and upsampling stages so that the feature information of the lower layer can be directly transmitted to the higher layer; each downsampling layer includes a pooling layer, a convolutional layer Conv, a ReLU activation function, a batch normalization layer BN and a CBAM attention module.

[0018] Preferably, the joint loss function includes the structural similarity index, the Pearson correlation coefficient, and the mean square error, specifically:

[0019] Total Loss=α×MSE(outputs, labels)+β×(1-SSIM Score)+γ×(1-PCCScore)

[0020] Among them, Total Loss is the total loss, α is the weight of MSE loss, MSE is the mean square error, which is an indicator to measure the numerical difference between the predicted value outputs and the true label labels, β is the weight of SSIM Score loss, SSIM is the structural similarity index used to evaluate the structural similarity between two images, γ is the weight of PCC Score loss, PCC is the Pearson correlation coefficient, which measures the linear correlation between the predicted value and the true label.

[0021] A computational imaging system based on multimode fiber mode modulation, based on the aforementioned computational imaging method based on multimode fiber mode modulation, comprising: a multimode fiber image transmission system, a trained deep learning network model, and a model construction and training module;

[0022] A multimode fiber image transmission system is used to collect natural scene speckle image datasets under different transmission conditions by replacing multimode fibers modulated in different modes;

[0023] A trained deep learning network model is used to recover high-quality input images from the speckle pattern output by a multimode optical fiber;

[0024] The model building and training module is used to build an optimized network architecture based on the UNET network and a deep learning network with a serial dual-subnetwork structure to perform feature extraction and image reconstruction from speckle images to SLM images and from SLM images to real images, and learn complex speckle image features; the deep learning network is trained based on a natural scene speckle image dataset with different mode modulation states, and a joint loss function is introduced to optimize the network's reconstruction performance; speckle images under different transmission conditions are used to verify the quality of the reconstructed image by the trained deep learning network model.

[0025] The multimode fiber image transmission system includes a laser, a lens, a transmissive spatial light modulator, a beam splitter, an objective lens, a mode-modulated multimode fiber, a CCD camera, and a PC.

[0026] The laser is used as a light source to emit laser light, which is then irradiated onto a transmissive SLM through a lens. The SLM is loaded with a natural scene image. The laser light carrying the image information passes through a beam splitter and an objective lens and enters the input end of a multimode optical fiber. After being transmitted through the multimode optical fiber, it is emitted from the output end. The speckle formed at the output end is captured by a CCD camera, thereby constructing a speckle image dataset.

[0027] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the computational imaging method based on multimode optical fiber mode modulation.

[0028] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a computational imaging method and system based on multimode optical fiber mode modulation, which effectively controls and modulates the mode inside the optical fiber through mode modulation technology, so that different transmission modes can be adjusted as needed, and can effectively optimize the mode propagation characteristics of the optical fiber without changing the optical fiber size, improve the propagation efficiency of the signal in the optical fiber, thereby improving the performance of the imaging system; through mode modulation, the inter-mode coupling, dispersion and scattering effects in the optical fiber are effectively controlled, reducing the negative impact on image transmission, and providing new possibilities for achieving higher quality image reconstruction; combined with deep learning methods, mode modulation technology can further enhance the imaging system's ability to reconstruct complex natural scenes, and adaptively adjust in different optical fiber transmission environments, thereby improving the robustness and accuracy of the system, which not only solves the limitations of traditional methods, but also improves the adaptability and effect of deep learning-based imaging algorithms in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0030] Figure 1 A schematic diagram of a computational imaging method based on multimode optical fiber mode modulation provided by the present invention;

[0031] Figure 2 Schematic diagram of the multimode optical fiber mode modulation speckle image acquisition device provided by the present invention;

[0032] Figure 3 Schematic diagram of the fusion splicing of a seven-core optical fiber and a common optical fiber provided by the present invention;

[0033] Figure 4 A schematic diagram of the grating writing operation near the input end of a multimode optical fiber provided by the present invention;

[0034] Figure 5 A schematic diagram of the deep learning network provided by the present invention;

[0035] Figure 6 Provide a convolution module schematic diagram for the present invention;

[0036] Among them, 1-laser, 2-lens, 3-transmissive spatial light modulator, 4-beam splitter, 5-objective lens, 6-mode modulated multimode fiber, 7-CCD camera, 8-PC terminal, 9-seven-core fiber, 10-seven-core light cross section, 11-multimode fiber cross section, 12-multimode fiber, 13-carbon dioxide laser, 14-long period fiber grating, 15-fiber coating, 16-fiber core, 17-fiber cladding. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] The embodiment of the present invention discloses a computational imaging method based on multimode fiber mode modulation, such as Figure 1 , including the following steps:

[0039] S1. Design and build a multimode fiber image transmission system, and collect natural scene speckle image datasets under different transmission conditions by replacing multimode fibers with different modulation modes.

[0040] S2. Build an optimized network architecture based on the UNET network and a deep learning network with a serial dual-subnet structure for feature extraction and image reconstruction from speckle images to SLM images and from SLM images to real images, and learn complex speckle image features;

[0041] S3. A deep learning network is trained based on a dataset of natural scene speckle images with different modal modulation states, and a joint loss function is introduced to optimize the network's reconstruction performance.

[0042] S4. Use the trained deep learning network model to reconstruct the speckle images under different transmission conditions and verify the quality of the reconstructed images.

[0043] To further implement the above technical solution, a natural scene speckle image dataset includes the real image at the front end of the multimode fiber, the image inside the spatial light modulator, and the speckle image actually formed by multimode fiber transmission. The speckle image, SLM image, and real image correspond one-to-one.

[0044] In this embodiment, the multimode optical fiber image transmission system designed and constructed in step S1 includes a laser 1, a lens 2, a transmissive spatial light modulator 3, a beam splitter prism 4, an objective lens 5, a mode-modulated multimode optical fiber 6, a CCD camera 7 and a PC terminal 8, and the laser 1 is a 532nm laser.

[0045] In order to further implement the above technical solution, the process of collecting natural scene speckle image datasets under different transmission conditions by replacing multimode optical fibers with different mode modulations includes: first, using unprocessed multimode optical fibers to collect speckle; then, fusing the seven-core optical fiber 9 at the output end of the multimode optical fiber 12, such as Figure 3 , and then perform speckle acquisition. This processing can filter out speckle noise and improve sampling density and signal-to-noise ratio;

[0046] like Figure 4 On the basis of fusion-splicing seven-core optical fibers, a grating writing operation is performed at the near input end of the multimode optical fiber, and then speckle collection is performed; specifically: the optical fiber coating 15 is removed to expose the optical fiber cladding 17, and the output light of the carbon dioxide laser 13 is directly applied to the optical fiber cladding 17. The carbon dioxide laser 13 is used to irradiate the optical fiber cladding 17, and the refractive index of the multimode optical fiber cladding 17 and the optical core 16 is periodically changed by point-by-point exposure, forming a long-period fiber grating 14. Its purpose is to stimulate the coupling of high-order modes in the optical core 16 to enhance the transmission of high-frequency information and provide richer image detail features for subsequent computational reconstruction.

[0047] To further implement the above technical solution, an optimized network architecture based on the UNET network includes an encoder and a decoder; the encoder includes multiple downsampling modules for extracting the features of the speckle image; the decoder includes multiple upsampling modules for reconstructing the image.

[0048] To further implement the above technical solution, the optimized network architecture based on the UNET network also includes a Transformer module. The feature maps extracted by the downsampling module and the upsampling module are embedded in the Transformer layer for propagation, and the contextual information between different regions in the speckle image is extracted through the self-attention mechanism.

[0049] like Figure 5 ,The dual subnetwork series structure composed of the optimized network architecture based on the UNET network imitates the inverse process of reconstructing the speckle image to the SLM image and the SLM image to the real image;

[0050] The architecture takes a three-channel speckle image of size h×w as input①; in the initial feature extraction stage, the input image passes through the first downsampling module②, which increases the number of channels to 16 while keeping the original image size unchanged at h×w; then, the feature map enters the second downsampling module③ in the architecture to complete the first spatial downsampling, reducing the image size to h / 2×w / 2, while the number of channels increases to 32 accordingly; after further downsampling, the feature map enters the third downsampling module④, where the number of channels doubles to 64 and the image size is further reduced to h / 4×w / 4; then, Transf is introduced. The general module ⑤ is used to model global context information and enhance the long-range dependency of features while keeping the number of channels constant. After that, the image features are downsampled for the fourth time by the fourth downsampling module ⑥, increasing the number of channels to 128 and further compressing the image size to h / 8 × w / 8. In the fifth downsampling module ⑦, the number of channels is further expanded to 256, and the image size is reduced to h / 16 × w / 16. Finally, in the center layer ⑧, after the sixth downsampling, the feature map size is compressed to h / 32 × w / 32, and the number of channels reaches a peak of 512, completing the extraction of deep semantic features.

[0051] In the decoding stage, the network first enters the first-layer upsampling module ⑨, upsamples the feature map to h / 16×w / 16, and reduces the number of channels to 256; then the Transformer module ⑤ is introduced again to further enhance the global modeling capability of the features; then it continues to enter the second-layer upsampling module ⑩, and the image size is restored to h / 8×w / 8, and the number of channels is reduced to 128; then the feature map is restored to h / 4×w / 4 through the third-layer upsampling module ⑪, and the number of channels is reduced to 64; it is further restored to h / 2×w / 2 through the fourth-layer upsampling module ⑫, with the number of channels being 32; in the fifth-layer upsampling module ⑬, the image size finally returns to the original input size h×w, and the number of channels is reduced to 16; finally, the network output stage ⑭ generates the target SLM image, completing the full-process mapping from initial input to final reconstructed output.

[0052] Then, the output SLM image is input into the optimized network architecture based on the UNET network in series to realize the reconstruction of the SLM image to the real image⑮.

[0053] Furthermore, a variety of connection methods are designed between various layers of the network to achieve effective transmission and fusion of feature information, thereby improving the overall performance and accuracy of the network.

[0054] To further implement the above technical solution, a skip connection is set between the downsampling and upsampling stages so that the feature information of the lower layer can be directly transferred to the higher layer, which helps to preserve the image details and enhance the network's ability to capture image features.

[0055] like Figure 6 Each downsampling layer includes a pooling layer, a convolutional layer Conv, a ReLU activation function, a batch normalization layer BN, and a CBAM attention module, which further enhances the network's attention to important features and improves the discriminability of feature expression; specifically, they are: pooling layer, convolutional layer Conv, normalization layer BN, ReLU activation function, convolutional layer Conv, normalization layer BN, ReLU activation function, and CBAM attention module.

[0056] The dual-subnet series structure composed of the optimized network architecture based on the UNET network not only reduces the network reconstruction pressure and reduces the network load, but also further fits the speckle acquisition process and improves the reconstruction effect.

[0057] Different pattern-modulated speckles are input as data into the tandem dual-subnetwork network for training and validation. A joint loss function of structural similarity index (SSIM), Pearson correlation coefficient (PCC) and mean square error (MSE) is used. Appropriate learning rate and batch size are set. The learning rate, weight, batch size and network parameter settings are adjusted to ensure that the training loss and validation loss show a downward trend. When the training loss tends to be constant and the validation loss also tends to be constant, it means that the network learning has reached the optimal state and the model is saved.

[0058] To further implement the above technical solution, the joint loss function includes the structural similarity index, Pearson correlation coefficient and mean square error, specifically:

[0059] Total Loss=α×MSE(outputs, labels)+β×(1-SSIM Score)+γ×(1-PCCScore)

[0060] Among them, Total Loss is the total loss, α is the weight of MSE loss, MSE is the mean square error, which is an indicator to measure the numerical difference between the predicted value outputs and the true label labels, β is the weight of SSIM Score loss, SSIM is the structural similarity index used to evaluate the structural similarity between two images, γ is the weight of PCC Score loss, PCC is the Pearson correlation coefficient, which measures the linear correlation between the predicted value and the true label.

[0061] In this embodiment, a reconstruction test is performed on the trained model using a test set. The images in the test set are not repeated in the training set images. The reconstructed images are compared with the original labeled images. The comparison results are displayed together with the evaluation indicators SSIM, PCC and peak signal-to-noise ratio PSNR to measure the imaging performance of the model in demodulating multimode optical fibers for complex images.

[0062] The calculation formulas for SSIM, PCC and PSNR are:

[0063]

[0064]

[0065]

[0066]

[0067] Among them, SSIM measures the similarity of two images based on brightness, contrast and structure, with a value range of -1 to 1; PCC measures the correlation between two images, with a value range of -1 to 1; MSE in PSNR represents the mean square error between the two images; the calculation results of the three indicators SSIM, PCC and PSNR are all calculated. The larger the value, the better the quality of the image reconstructed by the network model; u x is the average brightness value of image x, u y is the average brightness value of image y, u x 2 +u y 2 is the sum of the squared brightness of the two images, σ x 2 and σ y 2 are the variances of image x and image y, respectively. The variance represents the contrast of the image. c1 and c2 are usually small positive constants, which can be determined according to the brightness range of the image. σ xy is the covariance of image x and image y, which reflects the structural similarity of the two images; cov(X,Y) represents the covariance of X and Y, E is the expected value (average) operator; X(i,j) and Y(i,j) are the pixel values ​​of image X and Y at position (i,j), respectively. n -1 represents the maximum pixel value of the image, and n is the number of image bits.

[0068] A computational imaging system based on multimode fiber mode modulation is based on a computational imaging method based on multimode fiber mode modulation, comprising: a multimode fiber image transmission system, a trained deep learning network model, and a model construction and training module;

[0069] A multimode fiber image transmission system is used to collect natural scene speckle image datasets under different transmission conditions by replacing multimode fibers modulated in different modes;

[0070] A trained deep learning network model is used to recover high-quality input images from the speckle pattern output by a multimode optical fiber;

[0071] The model building and training module is used to build an optimized network architecture based on the UNET network and a deep learning network with a serial dual-subnetwork structure to perform feature extraction and image reconstruction from speckle images to SLM images and from SLM images to real images, and learn complex speckle image features; the deep learning network is trained based on a natural scene speckle image dataset with different mode modulation states, and a joint loss function is introduced to optimize the network's reconstruction performance; speckle images under different transmission conditions are used to verify the quality of the reconstructed image by the trained deep learning network model.

[0072] The multimode fiber image transmission system includes a laser, a lens, a transmissive spatial light modulator, a beam splitter, an objective lens, a mode-modulated multimode fiber, a CCD camera, and a PC.

[0073] The laser is used as a light source to emit laser light, which is then irradiated onto a transmissive SLM through a lens. The SLM is loaded with a natural scene image. The laser light carrying the image information passes through a beam splitter and an objective lens and enters the input end of a multimode optical fiber. After being transmitted through the multimode optical fiber, it is emitted from the output end. The speckle formed at the output end is captured by a CCD camera, thereby constructing a speckle image dataset.

[0074] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements a computational imaging method based on multimode optical fiber mode modulation.

[0075] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0076] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A computational imaging method based on multimode fiber mode modulation, characterized in that: The following steps are involved: S1. Design and build a multimode fiber image transmission system, and collect natural scene speckle image datasets under different transmission conditions by replacing multimode fibers with different modulation modes. S2. Build an optimized network architecture based on the UNET network and a deep learning network with a serial dual-subnet structure for feature extraction and image reconstruction from speckle images to SLM images and from SLM images to real images, and learn complex speckle image features; S3. A deep learning network is trained based on a dataset of natural scene speckle images with different modal modulation states, and a joint loss function is introduced to optimize the network's reconstruction performance. S4. Use the trained deep learning network model to reconstruct the speckle images under different transmission conditions and verify the quality of the reconstructed images.

2. The computational imaging method based on multimode fiber mode modulation according to claim 1, characterized in that: The natural scene speckle image dataset includes the real image of the multimode fiber front end, the image inside the spatial light modulator, and the speckle image actually formed by multimode fiber transmission. The speckle image, SLM image, and real image correspond one to one.

3. The computational imaging method based on multimode fiber mode modulation according to claim 1, characterized in that: The process of collecting natural scene speckle image datasets under different transmission conditions by replacing multimode optical fibers with different mode modulations includes: first, speckle collection is performed using unprocessed multimode optical fibers; then, a seven-core optical fiber is fused at the output end of the multimode optical fiber, and speckle collection is performed; based on the fusion of the seven-core optical fibers, a grating writing operation is performed at the near input end of the multimode optical fiber, and then speckle collection is performed.

4. The computational imaging method based on multimode fiber mode modulation according to claim 1, characterized in that: The optimized network architecture based on the UNET network includes an encoder and a decoder; the encoder includes multiple downsampling modules for extracting the features of the speckle image; The decoder includes multiple upsampling modules to reconstruct the image.

5. The computational imaging method based on multimode fiber mode modulation according to claim 4, characterized in that: The optimized network architecture based on the UNET network also includes a Transformer module, which is located in the encoder and decoder networks respectively. The feature maps extracted by the downsampling module and the upsampling module are embedded in the Transformer layer for propagation, and the contextual information between different regions in the speckle image is extracted through the self-attention mechanism.

6. The computational imaging method based on multimode fiber mode modulation according to claim 4, characterized in that: A skip connection is set between the downsampling and upsampling stages so that the feature information of the lower layer can be directly transmitted to the higher layer; each downsampling module includes a pooling layer, a convolutional layer Conv, a ReLU activation function, a batch normalization layer BN and a CBAM attention module.

7. The computational imaging method based on multimode fiber mode modulation according to claim 1, characterized in that: The joint loss function includes the structural similarity index, Pearson correlation coefficient and mean square error, specifically: Total Loss=α×MSE(outputs, labels)+β×(1-SSIM Score)+γ×(1-PCC Score) Among them, Total Loss is the total loss, α is the weight of MSE loss, MSE is the mean square error, which is an indicator to measure the numerical difference between the predicted value outputs and the true label labels, β is the weight of SSIM Score loss, SSIM is the structural similarity index used to evaluate the structural similarity between two images, γ is the weight of PCC Score loss, PCC is the Pearson correlation coefficient, which measures the linear correlation between the predicted value and the true label.

8. A computational imaging system based on multimode fiber mode modulation, characterized in that: A computational imaging method based on multimode fiber mode modulation according to any one of claims 1 to 7, comprising: a multimode fiber image transmission system, a trained deep learning network model, and a model construction and training module; A multimode fiber image transmission system is used to collect natural scene speckle image datasets under different transmission conditions by replacing multimode fibers modulated in different modes; A trained deep learning network model is used to recover high-quality input images from the speckle pattern output by a multimode optical fiber; The model building and training module is used to build an optimized network architecture based on the UNET network and a deep learning network with a serial dual-subnetwork structure to perform feature extraction and image reconstruction from speckle images to SLM images and from SLM images to real images, and learn complex speckle image features; the deep learning network is trained based on a natural scene speckle image dataset with different mode modulation states, and a joint loss function is introduced to optimize the network's reconstruction performance; speckle images under different transmission conditions are used to verify the quality of the reconstructed image by the trained deep learning network model.

9. The computational imaging system based on multimode fiber mode modulation according to claim 8, characterized in that: The multimode fiber image transmission system includes a laser, a lens, a transmissive spatial light modulator, a beam splitter, an objective lens, a mode-modulated multimode fiber, a CCD camera, and a PC. The laser is used as a light source to emit laser light, which is then irradiated onto a transmissive SLM through a lens. The SLM is loaded with a natural scene image. The laser light carrying the image information passes through a beam splitter and an objective lens and enters the input end of a multimode optical fiber. After being transmitted through the multimode optical fiber, it is emitted from the output end. The speckle formed at the output end is captured by a CCD camera, thereby constructing a speckle image dataset.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the computational imaging method based on multimode optical fiber mode modulation according to any one of claims 1 to 7.

Images