Real-complex valued cascaded neural network holographic reconstruction method based on multi-constraint optimization

Abstract

The application discloses a real-complex value cascade neural network holographic reconstruction method based on multi-constraint optimization, and is used for solving the problems of twin image artifacts caused by non-well-posed inverse problems in coaxial holographic reconstruction technology, which leads to the decline of reconstruction quality, and the lack of interpretability caused by the black box characteristics of the deep learning method, etc. iφ As the input of the complex value Net, the output is the reconstructed object plane complex amplitude field Be iθ , the amplitude term of Be iθ is constrained, and then the simulation diffraction is propagated to the holographic surface to obtain the physical term constraint composed of the phase term φ' and φ, the complex domain characteristic function is designed to optimize Be iθ , the complex amplitude term constraint is constructed, and then the phase label and θ are obtained through a traditional algorithm to form the fidelity term constraint, and the three constraints jointly guide the training of the real-complex value cascade network. The method can realize high-fidelity and low-noise holographic phase reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of digital holographic reconstruction, and more specifically, relates to a holographic reconstruction method based on real-complex value cascaded neural networks with multi-constraint optimization. Background Technology

[0002] Holographic reconstruction technology, due to its non-invasiveness, high resolution, and three-dimensional imaging capabilities, has significant application value in fields such as biomedicine, micro / nano structure detection, and dynamic process observation. However, the application of traditional coaxial holographic reconstruction methods in complex scenarios still faces many challenges. The holographic recording process can only acquire the intensity information of light waves, losing the crucial phase component, making object plane reconstruction an unsteady inverse problem, leading to the generation of twin images and thus affecting reconstruction quality. Traditional phase retrieval algorithms, such as the Gerchberg-Saxton iterative algorithm and the intensity transfer equation algorithm, can recover phase information to some extent, but they rely on strong prior assumptions and weak scattering approximations, making them prone to getting trapped in local optima under complex samples or low signal-to-noise ratio conditions, resulting in a significant decrease in reconstruction accuracy. Traditional holographic reconstruction algorithms, such as off-axis holographic reconstruction, multi-step phase-shift reconstruction, and multi-distance holographic reconstruction algorithms, can achieve high-precision reconstruction, but their complex optical path structures, high system stability requirements, and extreme sensitivity to vibration and noise in the experimental environment limit the promotion of these methods in dynamic observation and practical engineering applications.

[0003] Deep learning-based methods have offered new solutions for holographic reconstruction due to their powerful nonlinear fitting capabilities. However, most existing neural network architectures only process real-valued data, failing to fully model the complex physical characteristics of light wave propagation, resulting in insufficient fidelity and poor interpretability in phase reconstruction. In recent years, the introduction of complex-valued neural networks has opened new avenues for deep learning-based holographic phase reconstruction. The characteristics computed in the complex domain better reflect the physical nature of light wave propagation, enabling more natural coupling of amplitude and phase information, thereby improving reconstruction accuracy. However, existing complex-valued neural network methods still face certain challenges: the network input is typically only real-valued hologram intensity data, lacking complete complex amplitude information of the initial light field, making it difficult to accurately model the angular spectrum propagation process. This limitation stems from the mismatch between the dual-channel input of the complex-valued network and the single-channel hologram, preventing the network from fully utilizing the physical constraints of the complex domain. The key to solving this problem lies in designing a novel complex amplitude encoding mechanism. This involves transforming the single-channel intensity input into a complex numerical expression that conforms to the laws of light field propagation through physical prior-guided complex initialization or the introduction of a learnable phase estimation module. This allows for a more accurate embedding of the wave optics model into the neural network, achieving high-fidelity end-to-end positional reconstruction. Meanwhile, existing methods often lack explicit constraints on the physical processes of holographic imaging, resulting in poor performance in noise suppression and microstructure recovery. This has prompted researchers to explore a novel reconstruction method that deeply integrates physical prior constraints with data-driven learning, aiming to achieve a more accurate, robust, and physically interpretable deep learning-based digital holographic reconstruction method. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a real-complex value cascaded neural network holographic reconstruction method based on multi-constraint optimization. The aim is to overcome the difficulties of traditional holographic reconstruction methods in terms of phase recovery accuracy, noise robustness, and insufficient deep learning interpretability by cleverly integrating the predictive ability of real-valued networks for the initial phase of the holographic surface with the modeling advantage of complex-valued networks for wavefront propagation characteristics, and combining a joint optimization strategy of multiple physical constraints.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is:

[0006] A holographic reconstruction method based on real-complex value cascaded neural networks with multi-constraint optimization, the method comprising the following steps:

[0007] S1. Construct a coaxial digital holographic optical path system, collect N holograms of samples with structural diversity at a fixed diffraction distance z, and construct a dataset of amplitude A after normalization preprocessing and square root operation.

[0008] S2. Construct a real-complex cascaded neural network. First, input the amplitude A into the real-valued Net to predict the initial phase φ of the holographic surface. Then, combine the initial phase φ of the holographic surface with the amplitude A to form a complex amplitude field Ae.iφ As input to the complex-valued Net, the final output is the reconstructed complex amplitude field Be of the object plane. iθ ;

[0009] S3, the reconstructed complex amplitude field Be of the object plane iθ After constraining the amplitude term B, simulated optical diffraction propagates forward to the holographic surface at a distance z. The phase term in the complex amplitude field at this location is taken as φ'. The physical term constraint is constructed by calculating the consistency with the initial phase φ of the holographic surface. The complex amplitude field Be of the object plane, which is reconstructed by optimizing the complex domain eigenfunction function, is designed. iθ The complex amplitude term constraint is constructed by calculating the consistency of the complex amplitude field before and after optimization; the phase recovered by the traditional holographic reconstruction algorithm is used as the prior of the network label, and the complex amplitude field Be of the label and the reconstructed object plane is calculated. iθ The similarity of the phase term θ is used to construct the fidelity constraint; wherein, the complex domain feature function is an optimization framework composed of a complex-valued total variation difference regularization term and an L2 norm fidelity term, and is solved by a split Bregman iterative algorithm; in this complex domain feature function, firstly, a nonlinear constraint based on a contraction operator is applied to the complex-valued total variation difference regularization term, and adaptive thresholding is performed in the horizontal and vertical gradient directions to effectively filter out high-frequency noise components. At the same time, the fast Fourier transform is used to accurately solve the L2 norm fidelity term, achieving the best balance between preserving the original data features and suppressing noise in the frequency domain; then, an iterative update mechanism for the Bregman distance parameter is introduced, setting the number of iterations to be greater than 10, and finally, a complex amplitude field with smooth characteristics is output, and compared with the complex amplitude field Be before optimization. iθ The difference is calculated to determine the consistency of the complex amplitude field before and after optimization by obtaining the magnitude, real part, or imaginary part of the difference.

[0010] S4. A dual-branch optimization strategy is adopted, with independent parameter update optimizers configured for the real-valued Net and the complex-valued Net respectively. The network parameters of the real-valued Net are optimized by backpropagation through the calculation of the gradient of the physical constraint, while the network parameters of the complex-valued Net are updated by backpropagation through the calculation of the gradient of the complex amplitude constraint and the fidelity constraint. The real-complex cascaded neural network is trained alternately and iteratively, so that the constraint indices of the physical constraint, the complex amplitude constraint and the fidelity constraint gradually converge. When the sum of the constraint indices of the physical constraint, the complex amplitude constraint and the fidelity constraint is lower than the preset threshold α, the network training is terminated. The real-complex cascaded neural network after training can be directly integrated into the holographic imaging system to achieve end-to-end real-time phase reconstruction.

[0011] Preferably, in step S1, the samples with structural diversity are all pure phase-type samples with an amplitude of 1, and the number of images N should be greater than 3000, and the diffraction distance z should be greater than 1 mm.

[0012] Preferably, in step S2, both the real-valued Net and the complex-valued Net adopt an end-to-end neural network framework. The real-valued Net consists of a real-valued convolutional layer, a real-valued pooling layer, and a real-valued activation function; the complex-valued Net consists of a complex-valued convolutional layer, a complex-valued pooling layer, and a complex-valued activation function.

[0013] Preferably, the real-valued Net and complex-valued Net use one of U-Net, ResUNet, Efficient-Net, and Attention-Net as the base model, and the base model must ensure that the size and dimension of the network output are consistent with those of the network input.

[0014] Preferably, in step S3, the constraint method for the amplitude term B is an absorption constraint or a substitution constraint, wherein,

[0015] The absorption constraint is as follows: set the values ​​greater than 1 in amplitude term B to 1, and keep the values ​​less than or equal to 1 unchanged;

[0016] The permutation constraint is: all values ​​of amplitude term B are permuted to 1.

[0017] Preferably, in step S3, the simulated optical diffraction is calculated using angular spectrum diffraction propagation or Fresnel diffraction propagation. Before the simulated diffraction propagation, the original complex amplitude field is filled with edge complex values; after the simulated diffraction propagation, it needs to be trimmed back to the size before filling.

[0018] Preferably, in step S3, the traditional holographic reconstruction algorithm adopts one of the following: off-axis holographic reconstruction algorithm, multi-step phase-shift reconstruction algorithm, and multi-distance holographic reconstruction algorithm. The optical path required for the reconstruction algorithm is extended in the original coaxial digital holographic optical path, and the reconstructed phase label should be matched one-to-one with the input dataset.

[0019] Preferably, in step S3, the physical constraint and the fidelity constraint are calculated using one of the following methods: mean square error, L1 norm, L2 norm, and structural similarity index.

[0020] Preferably, in step S4, the parameter update optimizer employs one of the following algorithms: stochastic gradient descent, adaptive gradient descent, and Adam algorithm.

[0021] Compared with the prior art, the present invention has the following advantages:

[0022] 1. The core advantage of this invention lies in its ingenious design of a real-complex value cascaded neural network. This real-valued network accurately predicts the initial phase of the holographic surface from a single amplitude image, providing crucial prior information for the poorly determined inverse problem of holographic reconstruction. Compared to the limitation of traditional complex valued neural networks requiring two inputs (amplitude and phase), this invention only requires a single amplitude input, significantly improving the network's practicality and applicability. It more effectively fits the distribution characteristics of large-scale holographic data, thus obtaining more accurate phase prediction results. Furthermore, the initial phase predicted by the real-valued network can be directly used as a high-quality initial value for traditional iterative algorithms. This innovative design avoids the slow convergence speed caused by random initialization and achieves the organic integration of data-driven methods and physical models, providing an efficient and reliable new approach for the field of holographic reconstruction.

[0023] 2. This invention employs a split Bregman iterative algorithm based on complex-valued full difference. By designing a complex domain eigenfunction, the complex amplitude optimization process is decomposed into sub-problems with denoising constraints, effectively achieving a balance between noise suppression and original feature preservation. Simultaneously, simulated optical diffraction propagation is introduced into the system, constructing a strict physical constraint mechanism for the initial phase of the holographic surface. In terms of implementation, the above two constraints, along with the fidelity constraint, are embedded as a post-processing module into the complex-valued network training process. This allows the reconstructed object plane complex amplitude field output by the network to be alternately optimized with the numerical solutions of traditional denoising iterative algorithms and wave equations. This unique fusion strategy not only ensures that the reconstruction results conform to the data-driven feature distribution but also strictly follows the physical laws of light field propagation, achieving significant improvements in noise resistance and original data feature preservation, providing a reliable solution for high-quality phase reconstruction in complex environments. Attached Figure Description

[0024] Figure 1 This is a flowchart of the real-complex value cascaded neural network holographic reconstruction method based on multi-constraint optimization of the present invention.

[0025] Figure 2 This is an embodiment diagram of the real-complex value cascaded neural network holographic reconstruction method based on multi-constraint optimization of the present invention. Detailed Implementation

[0026] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0027] See Figure 1 The holographic reconstruction method of real-complex valued cascaded neural networks based on multi-constraint optimization of the present invention includes the following steps:

[0028] S1. Construct a coaxial digital holographic optical path system, collect 5000 holograms of samples with structural diversity at a fixed diffraction distance of 10mm, and construct a dataset of amplitude A after normalization preprocessing and square root operation.

[0029] S2. Construct a real-complex cascaded neural network. First, input the amplitude A into the real-valued Net to predict the initial phase φ of the holographic surface. Then, combine the phase φ and the amplitude A to form a complex amplitude field Ae. iφ As input to the complex-valued Net, the final output is the reconstructed complex amplitude field Be of the object plane. iθ ;

[0030] S3, the reconstructed complex amplitude field Be of the object plane iθ After constraining the amplitude term B, simulated optical diffraction propagates forward to the holographic surface at a distance z. The phase term in the complex amplitude field at this location is taken as φ'. The physical term constraint is constructed by calculating the consistency with the initial phase φ of the holographic surface. The complex amplitude field Be of the object plane, which is reconstructed by optimizing the complex domain eigenfunction function, is designed. iθ The complex amplitude term constraint is constructed by calculating the consistency of the complex amplitude field before and after optimization; the phase recovered by the traditional holographic reconstruction algorithm is used as the prior of the network label, and the complex amplitude field Be of the label and the reconstructed object plane is calculated. iθ The similarity of the phase term θ is used to construct the fidelity constraint;

[0031] S4. A dual-branch optimization strategy is adopted, with independent parameter update optimizers configured for the real-valued Net and the complex-valued Net respectively. The network parameters of the real-valued Net are optimized by backpropagation through the calculation of the gradient of the physical constraint, while the network parameters of the complex-valued Net are updated by backpropagation through the calculation of the gradient of the complex amplitude constraint and the fidelity constraint. The neural network is trained alternately and iteratively to make the three constraint indices gradually converge. When the sum of the three constraint indices is lower than the preset threshold of 0.01, the network training is terminated. The trained neural network can be directly integrated into the holographic imaging system to achieve end-to-end real-time phase reconstruction.

[0032] See Figure 2In step S1, the coaxial digital holographic optical path system is constructed using a 532nm semiconductor laser as the light source, generating uniform plane wave illumination through a collimating lens group. Experimental samples are fabricated on a quartz substrate using micro-nano fabrication technology, all being pure phase-type structures with a fixed amplitude of 1, primarily including various biomimetic biological cell models. Hologram acquisition uses a Basler acA2040-90μm CCD camera with a resolution of 2048×2048 pixels and a pixel size of 5.5μm, recording coaxial holograms at a fixed diffraction distance z=10mm. To ensure data quality, the system is built on an active vibration isolation optical platform and undergoes rigorous dark-field correction and background normalization. Each image is cropped to 256×256 pixels, and the final dataset of amplitude A is obtained through square root calculation and stored as an 8-bit unsigned integer PNG format.

[0033] See Figure 2 In step S2, the real-complex value cascaded neural network is constructed using an improved ResUNet architecture as the basic model. The encoder part of the real-valued Net contains four downsampling stages, each consisting of two real-valued convolutional layers combined with a batch normalization layer and a ReLU activation function. Downsampling is achieved through a max pooling layer. The decoder part performs upsampling through transposed convolution and adds skip connections to fuse low-level features. The complex-valued Net replaces each component with a complex value version while maintaining the same topology, including complex-valued convolutional layers with complex weight matrices and specific complex activation functions. The input and output dimensions of the network must be strictly consistent. The real-valued Net outputs the phase prediction φ, which is concatenated with the amplitude A to form a complex amplitude field as the input of the complex-valued Net. Finally, the output is the complex field of the object plane.

[0034] See Figure 2 In step S3, the amplitude term B adopts a permutation constraint, that is, all values ​​are permuted to 1. The simulated optical diffraction is calculated using angular spectrum diffraction propagation. Before propagation, the original complex amplitude field is filled with edge complex values, and after propagation, it is trimmed back to the size before filling.

[0035] See Figure 1In step S3, the complex domain feature function is mainly an optimization framework composed of a complex-valued total variation difference regularization term and an L2 norm fidelity term, which is efficiently solved using a split Bregman iterative algorithm. This function first applies a nonlinear constraint based on a contraction operator to the complex-valued total variation difference regularization term, performing adaptive thresholding in the horizontal and vertical gradient directions to effectively filter out high-frequency noise components. Simultaneously, a fast Fourier transform is used to accurately solve the L2 norm fidelity term, achieving the optimal balance between preserving original data features and suppressing noise in the frequency domain. Then, an iterative update mechanism for the Bregman distance parameter is introduced, setting the iteration count to be greater than 10. Finally, a complex amplitude field with smooth characteristics is output, and compared with the complex amplitude field Be before optimization. iθ The difference is calculated to determine the consistency of the complex amplitude field before and after optimization by obtaining the magnitude, real part, or imaginary part of the difference.

[0036] See Figure 1 In step S3, the traditional holographic reconstruction algorithm can adopt an off-axis holographic reconstruction algorithm, and the optical path required for off-axis holography is designed by extending the original coaxial digital holographic optical path. The reconstructed phase labels are paired one-to-one with the input dataset. The physical term constraints are calculated using mean square error, and the fidelity term constraints are calculated using structural similarity index.

[0037] See Figure 1 In step S4, the dual-branch optimization strategy is implemented using the PyTorch framework. The real-valued Net and the complex-valued Net are configured with independent Adam optimizers. The training process optimizes the real-valued Net and the complex-valued Net simultaneously. The total loss is calculated in each iteration and training is terminated when it falls below a preset threshold. The hardware platform is configured with an NVIDIA Tesla V100 GPU, the batch size is set to 8, and the learning rate is set to 0.01.

[0038] See Figure 2 In step S4, the trained neural network can be directly integrated into the holographic imaging system. A light beam emitted from the light source illuminates the sample through a collimating lens to generate a coaxial hologram. Then, the hologram is square rooted and its phase is predicted using a real-valued Net. After normalization, the phase term is extracted to reconstruct the initial phase. Finally, the holographic surface information is mapped to the object plane using a complex-valued Net to achieve end-to-end real-time reconstruction. The entire process does not require iterative optimization, effectively eliminating twin artifacts and reconstructing complete object plane phase information.

[0039] The above are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above content. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for holographic reconstruction of a real-complex valued cascaded neural network based on multi-constraint optimization, characterized in that, The method comprises the following steps: S1, a coaxial digital holographic optical system is built, N holograms of samples with structural diversity at a fixed diffraction distance z are collected, and a data set of amplitude A is constructed after normalization preprocessing and square root operation; S2, construct real-complex value cascade neural network, first input amplitude A into real value Net to predict initial phase φ of holographic surface, then input initial phase φ of holographic surface and amplitude A to form complex amplitude field Ae iφ As the input of complex value Net, the final output is the reconstructed object plane complex amplitude field Be iθ ; S3, the reconstructed object plane complex amplitude field Be iθ After the amplitude term B is constrained, the simulated optical diffraction is forward propagated to the holographic surface at a distance z, and the phase term in the complex amplitude field at this position is taken as φ', and the physical term constraint is constructed by calculating the consistency with the initial phase φ of the holographic surface; the complex domain characteristic function is designed to optimize the reconstructed object plane complex amplitude field Be iθ The consistency of the complex amplitude field before and after optimization is calculated to construct the complex amplitude term constraint; the phase recovered by the traditional holographic reconstruction algorithm is used as the network label prior, and the fidelity term constraint is constructed by calculating the similarity between the label and the phase term θ of the reconstructed object plane complex amplitude field Be iθ ; wherein the complex domain characteristic function is an optimization framework composed of a complex total variation regularization term and an L2 norm fidelity term, and is solved by a split Bregman iteration algorithm; in the complex domain characteristic function, first, the nonlinear constraint based on the shrinkage operator is implemented on the complex total variation regularization term, and adaptive threshold processing is performed in the horizontal and vertical gradient directions to effectively filter out high-frequency noise components; meanwhile, the L2 norm fidelity term is accurately solved by using fast Fourier transform, and the best balance between feature preservation and noise suppression of the original data is realized in the frequency domain; then the Bregman distance parameter iterative updating mechanism is introduced, the iteration number is set to be greater than 10, and finally the complex amplitude field with smooth characteristics is output, and the difference between the complex amplitude field before and after optimization Be iθ is calculated by obtaining the size of the modulus, real part or imaginary part of the difference value. S4, a double-branch optimization strategy is adopted, independent parameter update optimizers are configured for the real-valued Net and the complex-valued Net, the network parameters of the real-valued Net are optimized through back propagation of the gradient of the physical item constraint, the network parameters of the complex-valued Net are updated through back propagation of the gradient of the complex amplitude item constraint and the fidelity item constraint, the real-complex value cascade neural network is iteratively trained alternately, the constraint indicators of the physical item constraint, the complex amplitude item constraint and the fidelity item constraint are gradually converged, the network training is terminated when the sum of the constraint indicators of the physical item constraint, the complex amplitude item constraint and the fidelity item constraint is lower than a preset threshold α, and the trained real-complex value cascade neural network can be directly integrated into a holographic imaging system to realize real-time phase reconstruction in an end-to-end manner.

2. The method of claim 1, wherein, In step S1, the samples with structural diversity are all pure phase samples with an amplitude of 1, and the number of shots N should be greater than 3000 and the diffraction distance z should be greater than 1 mm.

3. The method of claim 1, wherein, In step S2, the real-valued Net and the complex-valued Net both adopt an end-to-end neural network framework, wherein the real-valued Net is composed of a real-valued convolution layer, a real-valued pooling layer and a real-valued activation function; and the complex-valued Net is composed of a complex-valued convolution layer, a complex-valued pooling layer and a complex-valued activation function.

4. The method of claim 3, wherein, The real-valued Net and the complex-valued Net adopt one of U-Net, ResUNet, Efficient-Net and Attention-Net as a basic model, and the basic model needs to satisfy that the size and dimension of the network output are consistent with those of the network input.

5. The method of claim 1, wherein, In step S3, the constraint mode of the amplitude item B is absorption constraint or replacement constraint, wherein The absorption constraint is that values greater than 1 in the amplitude item B are set to 1, and values less than or equal to 1 remain unchanged; The replacement constraint is that all values of the amplitude item B are replaced with 1.

6. The method of claim 1, wherein, In step S3, the simulation optical diffraction adopts angular spectrum diffraction propagation or Fresnel diffraction propagation for calculation, and edge complex value filling is performed on the original complex amplitude field before simulation diffraction propagation; and the size before filling needs to be cropped after simulation diffraction propagation.

7. The method of claim 1, wherein, In step S3, the traditional holographic reconstruction algorithm adopts one of off-axis holographic reconstruction algorithm, multi-step phase shift reconstruction algorithm and multi-distance holographic reconstruction algorithm, and a light path required by the related reconstruction algorithm is designed in the original coaxial digital holographic optical system, and the phase labels of reconstruction should be paired with the input data set one by one.

8. The method of claim 1, wherein, In step S3, the physical item constraint and the fidelity item constraint both adopt one of calculation modes of mean square error, L1 norm, L2 norm and structural similarity index.

9. The method of claim 1, wherein, In step S4, the parameter update optimizer adopts one of stochastic gradient descent algorithm, adaptive gradient descent algorithm and Adam algorithm.

Images