Non-coherent digital holographic weak light three-dimensional imaging method based on multi-stage deep learning

Abstract

The application discloses a kind of based on multi-stage deep learning's incoherent digital holography weak light three-dimensional imaging method and system, belong to optical imaging technical field.The method includes: obtaining the multiple phase shift incoherent digital hologram collected under weak light condition;It is input to the first stage deep learning network model trained in advance, and the corresponding enhanced hologram is output, and the model is trained with weak light hologram as input, normal light hologram as label;The preliminary image is obtained by digital reconstruction to enhanced hologram;Preliminary image is input to the second stage deep learning network model trained in advance, and the final high-quality reconstructed image is output, and the model is trained with the image obtained by digital reconstruction to enhanced hologram as input, and the digital reconstructed image of normal light hologram is label;The fidelity and signal-to-noise ratio of three-dimensional reconstruction of FINCH system under weak light condition can be significantly improved without changing the existing optical hardware structure.

Description

Technical Field

[0001] This invention belongs to the field of optical imaging technology, and in particular relates to an incoherent digital holographic low-light three-dimensional imaging method based on multi-stage deep learning. Background Technology

[0002] Currently, Fresnel incoherent correlation holography (FINCH) systems suffer from high light energy loss due to spatial light modulators (SLMs), leading to reduced light intensity and noise interference, resulting in a low signal-to-noise ratio. To overcome these limitations and improve imaging performance, researchers have proposed several alternatives to SLMs. For example, by introducing transmissive liquid crystal graded refractive index (TLCGRIN) elements to replace SLMs, phase modulation can be achieved without complex electrical actuation, effectively reducing light loss. TLCGRIN elements possess high transmittance and good stability, which helps improve the system's signal acquisition capabilities. However, the refractive index distribution of TLCGRIN elements is usually preset and cannot be flexibly adjusted, making it difficult to achieve real-time control of complex wavefronts, thus limiting its application in dynamic scenes. Another approach is to use birefringent crystal lenses (such as calcite or α-BBO crystals) for modulation, utilizing their natural birefringence properties to achieve spatial separation and reconstruction of light fields with different polarization states, completing light field reconstruction without external control. While this approach offers advantages in simplifying system structure and reducing power consumption, its modulation dimensionality is limited, it is sensitive to the incident light angle, and it is prone to optical artifacts, limiting its effectiveness in high-resolution or dynamic scenarios. Although the aforementioned approaches have made some progress in improving light transmission efficiency and simplifying system structure, they still have significant limitations: TLCGRIN elements lack flexibility and tunability, and the birefringent crystal approach is limited by modulation dimensionality and angle sensitivity, both of which are difficult to meet the requirements of high-resolution and dynamic imaging. Summary of the Invention

[0003] The purpose of this invention is to provide an incoherent digital holographic low-light 3D imaging method based on multi-stage deep learning. This low-light imaging method does not require changes to the hardware structure. By optimizing the data processing flow, it significantly improves imaging accuracy and signal-to-noise ratio, providing a new solution for the application of FINCH systems in complex scenarios.

[0004] To solve the above-mentioned technical problems, the specific technical solution of the present invention is as follows:

[0005] In some embodiments of this application, an incoherent digital holographic low-light 3D imaging method based on multi-stage deep learning is provided, including the following steps:

[0006] Step S1: Acquire multiple phase-shifted incoherent digital holograms of the object to be processed under low-light conditions;

[0007] Step S2: Input the multiple phase-shifted incoherent digital holograms into the pre-trained first-stage deep learning network model, and output the corresponding multiple enhanced phase-shifted incoherent digital holograms; wherein, the first-stage deep learning network model uses multiple phase-shifted incoherent digital holograms acquired under low light conditions as input, and the corresponding phase-shifted incoherent digital holograms acquired under normal lighting conditions as supervision labels, and is a multi-input multi-output deep learning model obtained through supervised training;

[0008] Step S3: Perform digital reconstruction processing on the multiple enhanced phase-shifted incoherent digital holograms to obtain a preliminary reconstructed image;

[0009] Step S4: Input the preliminary reconstructed image into the pre-trained second-stage deep learning network model and output the final high-quality reconstructed image; wherein, the second-stage deep learning network model takes the enhanced phase-shifted incoherent digital hologram reconstructed image output by the first-stage deep learning network as input, and the high-quality reconstructed image under normal lighting conditions as supervision label, and is an image enhancement network model obtained through supervised training.

[0010] In some embodiments of this application, in step S1, the multiple phase-shifted incoherent digital holograms include three holograms with different phase shifts, namely 0, 2π / 3 and 4π / 3.

[0011] In some embodiments of this application, the first-stage deep learning network model adopts a multi-input multi-output network structure based on the U-Net architecture, including:

[0012] Multiple shared encoder paths are used to extract common features from multiple phase-shifted incoherent digital holograms of the input;

[0013] Multiple parallel decoder paths, the number of which is the same as the number of input holograms, and each decoder path is used to generate a corresponding enhanced hologram.

[0014] In some embodiments of this application, the loss function used when training the first-stage deep learning network model is the root mean square error loss function.

[0015] In some embodiments of this application, in step S3, the angular spectrum reconstruction algorithm is used to perform digital reconstruction processing on the multiple enhanced phase-shifted incoherent digital holograms.

[0016] In some embodiments of this application, the second-stage deep learning network model adopts a deep learning architecture that integrates the U-Net structure and the residual channel attention mechanism.

[0017] In some embodiments of this application, the implementation is carried out in a Fresnel incoherent correlation holography (FINCH) system without altering the optical hardware composition and optical path structure of the FINCH system.

[0018] Some embodiments of this application disclose an incoherent digital holographic low-light three-dimensional imaging system for performing the above-described method. The system includes:

[0019] The data acquisition module is configured to acquire multiple phase-shifted incoherent digital holograms of the object to be processed under low-light conditions.

[0020] The data processing module includes:

[0021] Memory, which stores computer-executable instructions;

[0022] A processor configured to execute the computer-executable instructions to implement the method as described in any one of the preceding statements;

[0023] The image output module is configured to output the final high-quality reconstructed image.

[0024] In some embodiments of this application, the data acquisition module includes:

[0025] Incoherent light sources are used to provide illumination.

[0026] Spatial light modulators are used to phase modulate object light waves to produce a phase-shifting effect;

[0027] An image sensor is used to record the phase-shifted incoherent digital hologram.

[0028] Some embodiments of this application disclose a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0029] Compared with existing technologies, the beneficial effects of this invention lie in the structural improvements made to the traditional U-Net structure by proposing the UNet-3to3 network. Addressing the problem that existing networks require independent frame-by-frame learning and neglect inter-frame information correlation when processing multiple phase-shifted incoherent digital holograms, this invention achieves joint modeling and collaborative optimization of three holograms through a multi-input multi-output (MIMO) structure. Specifically, the network simultaneously receives three phase-shifted interferograms acquired under low-light conditions as input, extracts common features between the images through a shared coding path, and fuses their respective feature information during the decoding stage, achieving simultaneous prediction output for the three target holograms. Through multi-frame joint input and staged supervised optimization, information sharing and collaborative enhancement among holograms are achieved, significantly improving the structural consistency and image reconstruction quality of holograms under low-light conditions. Attached Figure Description

[0030] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0031] Figure 1 A schematic diagram of an incoherent digital holographic low-light 3D imaging method based on multi-stage deep learning provided in an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram of the FINCH experimental apparatus provided in an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram illustrating the denoising effect of a low-light hologram based on a first-stage network, provided in an embodiment of the present invention. The scale bar in the diagram is 300 μm.

[0034] Figure 4 A schematic diagram of quality comparison analysis of hologram reconstruction based on the first stage enhancement provided for an embodiment of the present invention, wherein the scale bar in the diagram is 300 μm;

[0035] Figure 5 This is a schematic diagram illustrating the comparative analysis of holographic reconstruction effects based on the first-stage network output provided in an embodiment of the present invention, wherein the scale bar in the diagram is 300 μm;

[0036] Figure 6 A schematic diagram illustrating the comparative analysis of holographic reconstruction quality based on multi-stage enhancement, provided for an embodiment of the present invention;

[0037] Figure 7 The following is a schematic diagram of the image enhancement imaging results of two targets at different axial positions provided in an embodiment of the present invention. (a) and (b) are the reconstructions of target 1 and target 2 at different axial positions under low light conditions, and (c) and (d) are the effects of denoising the reconstructed images. The scale bar in the figure is 300 μm.

[0038] Figure 8 A schematic diagram of the FINCH principle provided for an embodiment of the present invention. Detailed Implementation

[0039] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0040] To better understand the purpose, structure, and function of this invention, the invention will be described in further detail below with reference to the accompanying drawings.

[0041] Example 1

[0042] This application discloses a supervised imaging algorithm for denoising and reconstructing incoherent digital holographic low-light holograms based on multi-stage deep learning, including the following steps:

[0043] System Setup: The system consists of two identical light sources, LED1 and LED2 (THORLABS, M625L4-C1, output power 630mW, center wavelength λ=625nm), positioned at the front focal plane of lenses L1 and L2, respectively. Lenses L1 and L2 are used to direct the diverging light emitted by the light sources directly to objects 1 and 2, achieving uniform parallel illumination. The optical information carried by objects 1 and 2 is acquired by lens L3, with objects 1 and 2 located 150mm and 156mm in front of L3, respectively. The polarization direction of polarizer P1 is at a 45° angle to the principal axis of the pure phase spatial light modulator (SLM, Hamamatsu, X15213-16, 1280×1024 pixels, pixel pitch 12.5µm). A 300mm focal length diffraction lens phase mask is applied to the SLM, and through polarization multiplexing, the object diffracted waves acquired by L3 are separated into two beams with different wavefront curvatures. The SLM loads a 200mm lens phase, and the modulated object wave and the unmodulated object wave travel 100mm before reaching the camera (Sony IMX304, TRI120S-MC).

[0044] Dataset Acquisition: Based on the aforementioned optical path system, this invention acquires 2500 sets of holographic data. Each set contains three weak-light phase-shift interferograms and three normal-light phase-shift interferograms, corresponding to phase-shift images at angles of 0, 2π / 3, and 4π / 3, respectively. The dataset is divided into training and testing sets in an 8:2 ratio to ensure the stability of network training and the model's generalization ability.

[0045] Network Construction and Training: To improve the signal-to-noise ratio of low-light phase-shift interferograms, this invention designs a first-stage UNet-3to3 deep neural network. This network, referencing the U-Net design, consists of a symmetrical encoder and decoder, incorporating multiple layers of convolution, pooling, upsampling, and downsampling operations for extracting and reconstructing holographic features. The network input consists of three phase-shift holograms with phase shifts of 0, 2π / 3, and 4π / 3 under low-light conditions, and the output is the predicted hologram with the corresponding phase shift under normal illumination conditions. The root mean square error loss (MSE) is used as the loss function to ensure the accuracy of the predicted image. Based on this, this invention designs a second-stage UNet-RCAN fusion structure image enhancement network for supervised enhancement of details in the reconstructed hologram image. Specifically, the three phase-shift images output from the first-stage UNet-3to3 network are processed by the reconstruction algorithm and used as the input to the UNet-RCAN network; the image obtained by the reconstruction algorithm from three phase-shift interferograms acquired under normal illumination conditions is used as the supervision label. Through this structure, the network learns the mapping relationship between low-light and normal-light reconstructed images, further improving the texture details and overall visual quality of the reconstructed images.

[0046] It's important to note that the UNet-3to3 network model is a modification of the existing UNet technology. Specifically, the UNet-3to3 network structure references the classic U-Net framework, consisting of a symmetrical encoder and decoder. The encoder comprises multiple convolutional layers, batch normalization, and ReLU activation functions, with max pooling used between layers to downsample features. The decoder employs deconvolution followed by convolution, and uses skip connections to concatenate features from corresponding layers in the encoder with features from the decoder layers to preserve detailed information.

[0047] The loss function chosen is the root mean square error (RMSE).

[0048] The formula for the loss function RMSE is as follows:

[0049] ;

[0050] UNet-RCAN combines existing UNet and RCAN technologies; specifically, this neural network can be described as a UNet-RCAN fusion structure.

[0051] Backbone structure: U-Net is used as the backbone, multi-scale features are extracted and spatial resolution is restored layer by layer.

[0052] Enhancement Module: An RCAN module is embedded in the decoder stage to enhance texture details using channel attention.

[0053] Testing the Network Model: After completing network training, the trained model was applied to an independent test set to verify the network's reconstruction accuracy and generalization performance on unknown data. During testing, the input consisted of three phase-shifting interferograms acquired under low-light conditions. The network trained in the first stage was used to generate predicted holograms corresponding to normal lighting conditions. The holograms predicted in the first stage were then used to generate reconstructed images using a reconstruction algorithm, which were then input into the second-stage UNet-RCAN network to obtain the final reconstructed image under normal lighting conditions.

[0054] To verify the effectiveness and practicality of the proposed two-stage deep learning network in low-light holographic image reconstruction, quantitative metrics such as structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) are used to comprehensively evaluate the quality of the reconstructed image. By calculating PSNR and SSIM, the consistency between the final image and the reference image in terms of structural restoration and noise suppression is quantitatively evaluated, further verifying the performance advantages of this invention in holographic image reconstruction under low-light conditions.

[0055] Example 2

[0056] The embodiments of this application adopt the technical features of the above embodiments, wherein, the appendix Figure 8 The optical apparatus on display is used to illustrate the basic mechanism of FINCH.

[0057] When a point light source is placed at a distance of focal length... lens Polarizers were placed before and after the phase mask. and By utilizing the response characteristics of a phase mask to polarization states, beam splitting of the incident wavefront is achieved. The distance between the phase mask and the lens is... After passing through the phase mask, the light is split into two beams, which are then focused towards different axial positions. Finally, at a distance from the phase mask... On the CCD plane at the location, two beams of light interfere with each other, and the resulting point spread hologram has an intensity distribution as shown in formula (1).

[0058] (1)

[0059] Dot diffusion hologram ( This can be viewed as a point spread function in the recording system. One encoding method of ). , and These are spatial coordinates. It is the center wavelength of the point. This represents convolution. When a lens phase is applied to the phase mask, the modulation expression is described as:

[0060] (2)

[0061] It is the focal length of the lens loaded on the phase mask. This is the additional phase shift value introduced by the phase mask. Object holography (OH) can be represented as the convolution of the object and the point source hologram.

[0062] (3)

[0063] Combined with phase-shifting technology, using different Multiple phase-shifted incoherent digital holograms are recorded and superimposed to obtain a complex-valued hologram without background terms and conjugate images. (Object) The complex-valued hologram can be represented as:

[0064] (4)

[0065] The hologram can be reconstructed using a numerical propagation algorithm, and the reconstructed image can be obtained as follows:

[0066] (5)

[0067] and These represent the corresponding spatial coordinates. and The frequency domain coordinates. Unlike intensity images, Theoretically, it contains all object information (amplitude and phase). In the FINCH system, angular spectral propagation is typically used to reconstruct digital holograms. Based on the theory of light diffraction, angular spectral propagation can accurately describe the diffraction and propagation of light in the frequency domain. Angular spectral propagation in free space ( The transfer function of can be expressed as:

[0068] (6)

[0069] It is the reconstruction distance that can be obtained through an autofocus algorithm.

[0070] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0071] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0072] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0073] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0074] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A non-coherent digital holographic low-light 3D imaging method based on multi-stage deep learning, characterized in that, Includes the following steps: Step S1: Acquire multiple phase-shifted incoherent digital holograms of the object to be processed under low-light conditions; Step S2: Input the multiple phase-shifted incoherent digital holograms into the pre-trained first-stage deep learning network model, and output the corresponding multiple enhanced phase-shifted incoherent digital holograms; wherein, the first-stage deep learning network model uses multiple phase-shifted incoherent digital holograms acquired under low light conditions as input, and the corresponding phase-shifted incoherent digital holograms acquired under normal lighting conditions as supervision labels, and is a multi-input multi-output deep learning model obtained through supervised training; The first-stage deep learning network model adopts a multi-input multi-output network structure based on the U-Net architecture, including: Multiple shared encoder paths are used to extract common features from multiple phase-shifted incoherent digital holograms of the input; Multiple parallel decoder paths, the number of which is the same as the number of input holograms, and each decoder path is used to generate a corresponding enhanced hologram; Step S3: Perform digital reconstruction processing on the multiple enhanced phase-shifted incoherent digital holograms to obtain a preliminary reconstructed image; Step S4: Input the preliminary reconstructed image into the pre-trained second-stage deep learning network model and output the final high-quality reconstructed image; wherein, the second-stage deep learning network model takes the preliminary reconstructed image, which is digitally reconstructed from the enhanced phase-shifted incoherent digital hologram output by the first-stage deep learning network, as input, and the high-quality reconstructed image under normal lighting conditions as supervision label, and obtains the image enhancement network model through supervised training.

2. The incoherent digital holographic low-light 3D imaging method based on multi-stage deep learning according to claim 1, characterized in that, In step S1, the multiple phase-shifted incoherent digital holograms include three holograms with different phase shifts, namely 0, 2π / 3 and 4π / 3.

3. The incoherent digital holographic low-light 3D imaging method based on multi-stage deep learning according to claim 1, characterized in that, The loss function used when training the first stage deep learning network model is the root mean square error loss function.

4. The incoherent digital holographic low-light 3D imaging method based on multi-stage deep learning according to claim 1, characterized in that, In step S3, the angular spectrum reconstruction algorithm is used to perform digital reconstruction processing on the multiple enhanced phase-shifted incoherent digital holograms.

5. The incoherent digital holographic low-light 3D imaging method based on multi-stage deep learning according to claim 1, characterized in that, The second-stage deep learning network model adopts a deep learning architecture that integrates the U-Net structure and the residual channel attention mechanism.

6. The incoherent digital holographic low-light 3D imaging method based on multi-stage deep learning according to claim 1, characterized in that, It is implemented in the Fresnel incoherent correlation holography FINCH system without changing the optical hardware composition and optical path structure of the FINCH system.

7. A non-coherent digital holographic low-light three-dimensional imaging system, characterized in that, The system includes: The data acquisition module is configured to acquire multiple phase-shifted incoherent digital holograms of the object to be processed under low-light conditions. The data processing module includes: Memory, which stores computer-executable instructions; A processor configured to execute the computer-executable instructions to implement the method as described in any one of claims 1-6; The image output module is configured to output the final high-quality reconstructed image.

8. The incoherent digital holographic low-light three-dimensional imaging system according to claim 7, characterized in that, The data acquisition module includes: Incoherent light sources are used to provide illumination. Spatial light modulators are used to phase modulate object light waves to produce a phase-shifting effect; An image sensor is used to record the phase-shifted incoherent digital hologram.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images