A Holographic Reconstruction Method Based on Wavelet Domain Feature Correlation Complex-Valued Neural Network

Abstract

This invention belongs to the field of coaxial holographic reconstruction and discloses a holographic reconstruction method based on a wavelet domain feature correlation complex-valued neural network. Specifically, the method involves: adding a phase term to the coaxial hologram A to construct a complex amplitude B, and performing an n-level discrete wavelet transform on it. The transformed result and the complex amplitude B are used as inputs to the complex-valued neural network, outputting the object field complex amplitude C. Next, noise suppression is applied to the object field complex amplitude C, and the suppressed complex amplitude D is used to simulate the forward propagation process of coaxial holography to generate a hologram E. Finally, the similarity between coaxial hologram A and hologram E is calculated, and a smoothing total variational constraint is applied to the phase of the complex amplitude D. Holographic reconstruction is achieved through a gradient descent optimization algorithm of the complex-valued neural network. This invention aims to utilize the inherent coupling relationship between the wavefront amplitude and phase terms in optical holography to construct a complex-valued feature correlation model to separate the target object from the background information, preserving target features while suppressing noise, thus achieving superior holographic reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of coaxial holographic reconstruction, and more specifically, relates to a holographic reconstruction method based on wavelet domain feature correlation complex value neural network. Background Technology

[0002] Coaxial holography is a holographic imaging technique based on the theory of coherent optical interference. It utilizes the coherent superposition of reference light and object light to generate an interference pattern, which is then recorded by a camera. The recorded interference pattern contains the amplitude and phase information of the target, enabling the reconstruction of the target object's three-dimensional structure. Due to its advantages such as compact structure, cost-effectiveness, and non-contact operation, this technology has been widely applied in optical imaging, material microstructure analysis, biomedical detection, and data storage. With the development of modern science and technology, integrating image processing algorithms and deep learning into coaxial holographic reconstruction can achieve the expression of more complex physical information and superior reconstruction results. It shows greater potential in various fields such as three-dimensional dynamic imaging of cells in biomedicine and high-precision detection in industrial instruments, becoming an important tool for many scientists in their research and applications.

[0003] However, due to unavoidable factors such as noise and turbulence during the imaging process of coaxial holographic interference patterns, background and target information are mixed in the hologram, which makes the ill-conditioned inverse process of holographic reconstruction more difficult. Existing methods directly force denoising on the reconstructed phase, which leads to the loss of target features.

[0004] Complex-valued neural networks, by directly calculating complex numbers, can better express the coupling relationship between the amplitude and phase terms of the wavefront in optical diffraction propagation, providing richer feature representation capabilities than real-valued neural networks in coaxial holographic reconstruction. Since noise or perturbation terms exist simultaneously in both amplitude and phase terms during holographic imaging, constructing denoising or edge detection models based on complex values ​​can lead to more stable noise suppression and more robust inter-layer feature information transfer, thereby improving the information representation capabilities of holographic imaging. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of current technology and provide a holographic reconstruction method based on wavelet domain feature correlation complex-valued neural network. It aims to utilize the inherent coupling relationship between wavefront amplitude and phase terms in optical holography to construct a complex-valued feature correlation model to divide the target object and background information, retain the target features while suppressing noise, and achieve better holographic reconstruction.

[0006] The technical solution of the present invention to solve the above-mentioned technical problems is:

[0007] A holographic reconstruction method based on wavelet domain feature correlation complex-valued neural networks includes the following steps:

[0008] S1. Based on the basic principle of coaxial holography, construct an optical path and use a camera to capture a phase-type coaxial hologram A;

[0009] S2. Take the square root of the pixel value of the acquired phase-type coaxial hologram A as the amplitude term, and randomly generate an initial phase term to construct the complex amplitude B. Perform n-level discrete wavelet transform on the complex amplitude B to generate low-frequency complex amplitude components and high-frequency complex amplitude components at each level. Together with the original complex amplitude B, these are used as the input of the complex-valued network model. The phase term of the suppressed complex amplitude D output in each iteration will replace the phase term of the input complex amplitude B to generate a new complex amplitude B.

[0010] S3. For the input of the complex-valued network model, preliminary feature extraction is first performed through a shallow feature extraction layer. This shallow feature extraction layer consists of multiple complex-valued convolutional blocks with a kernel size of 3×3. Each complex-valued convolutional block contains a complex-valued convolution operation, a complex-valued batch normalization layer, and a complex-valued ReLU activation function. Then, the edge masks of the low-frequency components at each level of the output of the shallow feature extraction layer are constructed using the complex-valued Prewitt operator module to enhance the weight of the edge features of the low-frequency components. Subsequently, the enhanced low-frequency components and the corresponding high-frequency components are used as the input of the cross-wavelet domain feature association module.

[0011] S4. In the cross-wavelet domain feature association module, four complex-valued depthwise separable convolutions are used to calculate the Q and V values ​​for enhancing the low-frequency components. L And the corresponding high-frequency components K and V H The weight W of the low-frequency text information focused in the high-frequency components is calculated by combining multi-head attention and the softmax function. LH And the weight W of the low-frequency component on the semantic representation of the high-frequency component. HL Finally, the weight W LH With V L Weight W HL With V H By combining and applying the enhanced low-frequency component and the corresponding high-frequency component to the input respectively, the enhanced low-frequency feature ENL and the high-frequency feature ENH are obtained.

[0012] S5, ENL of the nth level low-frequency feature n and high-frequency characteristics ENH n Perform a discrete wavelet inverse transform and then pass it through a complex-valued convolutional block. Finally, combine the output of the complex-valued convolutional block with the (n-1)th level low-frequency feature ENL. n-1 The combination generates an optimized low-frequency feature ENL. ' n-1 ;Optimized low-frequency features ENL ' n-1 Then, with the corresponding high-frequency characteristic ENH n-1After performing discrete wavelet inverse transform and a complex convolution block, and executing the same operation n times, a complex amplitude C of the object field with the same size as the complex amplitude B is generated.

[0013] S6. Noise suppression is achieved by using a complex-valued smooth total variational denoising model for the complex amplitude C of the object field. The suppressed complex amplitude D is then used to simulate the forward propagation process of a coaxial hologram to generate a hologram E. Finally, the similarity between the phase-type coaxial hologram A and the hologram E is calculated, and the phase of the complex amplitude D is subject to smooth total variational constraints. A self-supervised iterative optimization process for a single image is then performed using the gradient descent optimization algorithm of a complex-valued neural network, ultimately achieving holographic reconstruction.

[0014] Preferably, in step S1, the phase-type coaxial hologram A is a pure phase hologram, and the amplitude of the original object wave is set to 1.

[0015] Preferably, in step S2, the size of the randomly generated initial phase term is the same as that of the phase-type coaxial hologram A, ranging from 0 to 1.

[0016] Preferably, in step S2, the n-level discrete wavelet transform adopts a two-dimensional Haar discrete wavelet transform, outputting a low-frequency component LL, a high-frequency horizontal component HL, a high-frequency vertical component LH, and a high-frequency diagonal component HH; n in the n-level discrete wavelet transform corresponds to the number of times the two-dimensional Haar discrete wavelet transform is used; the complex amplitude B is used as the first-level low-frequency component for the two-dimensional Haar discrete wavelet transform, and the output low-frequency component LL1, high-frequency horizontal component HL1, high-frequency vertical component LH1, and high-frequency diagonal component HH1 are 1 / 4 the size of the original input size. Each subsequent level of the two-dimensional Haar discrete wavelet transform only operates on the low-frequency component, and the size of each output component is 1 / 4 of the input size of that level; the input of the complex value network model includes the input complex amplitude B and all high-frequency and low-frequency components generated at each level.

[0017] Preferably, in step S3, the complex-valued Prewitt operator module uses the horizontal and vertical 3×3 convolution templates of the Prewitt operator as convolution kernels to perform neighborhood convolution operations on the multi-channel features composed of the real and imaginary parts of the complex amplitude, and calculates the spatial gradient changes of the real and imaginary parts respectively; after normalizing the gradient changes, an edge mask is created (the mask value is between 0 and 1, with the edge region close to 1 and the non-edge region close to 0); the real and imaginary parts are modulated by the edge mask respectively (the mask is multiplied by the real and imaginary parts), and then the modulation result is superimposed on the original real and imaginary parts respectively to achieve edge-enhanced complex-valued signal optimization.

[0018] Preferably, in step S5, the complex amplitude C of the object field needs to pass through an additional complex convolutional layer with a channel number reduced to 1, which contains only one complex convolution operation.

[0019] Preferably, in step S6, the complex-valued smoothing total variational denoising model incorporates the complex-valued total variation into a Huber function, and constructs a Huber-TV regularization layer based on the iterative steps of the proximal gradient method using this Huber function; the proximal operator is approximated using gradient descent, wherein the gradient update step size s is selected as 0.1; the complex amplitude generated iteratively is compared with the input complex amplitude to ensure data fidelity, so that the gradient direction is pulled back towards the direction of data consistency, avoiding over-smoothing; the number of iterations a of the Huber-TV regularization layer is set in the range of 5-10 times.

[0020] Preferably, in step S6, the smoothing total variation constraint is a real-valued constraint on the phase term, which is composed of the real-valued total variation incorporating a differentiable Huber function, and a hyperparameter α is set to prevent the constraint from becoming too smooth. The range of α can be from 0 to 1.

[0021] Compared with the prior art, the present invention has the following advantages:

[0022] 1. The holographic reconstruction method based on wavelet domain feature correlation complex value neural network of the present invention constructs a Prewitt operator module based on complex values, convolves the Prewitt operator on multi-channel complex value features, calculates the gradient magnitudes of the real and imaginary parts of the complex number in the horizontal and vertical directions respectively, enhances the edge feature expression by constructing edge weight masks for the real and imaginary parts, and fuses the Prewitt operator processing results with the original feature map to realize edge determination in the complex domain. The sensitivity to low-frequency features is improved by low-frequency enhancement and complex processing.

[0023] 2. The holographic reconstruction method based on wavelet domain feature association complex-valued neural networks of the present invention achieves the extraction of low-frequency text information focused by high-frequency components and the enhancement of the weights expressing high-frequency detail information in low-frequency components by constructing a cross-wavelet domain feature association module that explores the correlation between low-frequency and high-frequency components. This cross-wavelet domain feature association module uses the low-frequency components and the sum of each high-frequency component as dual inputs, and uses four complex-valued depthwise separable convolutions to calculate the feature representations Q and V of the low-frequency components respectively. L and the characteristic representations of high-frequency components K and V H By combining multi-head attention and the softmax function, the gain of high-frequency components on low-frequency components and low-frequency components on high-frequency components is calculated, and this gain is superimposed on the feature components of the original input to achieve the correlation between cross-component features.

[0024] 3. The holographic reconstruction method based on wavelet domain feature correlation complex value neural network of the present invention achieves the differentiability of the model under phase term regularization constraint by incorporating the total variation into a differentiable Huber function; further, it constructs Huber-TV regularization layer by iterative steps of the fused function using the proximal gradient method, uses gradient descent to approximate the proximal operator, iteratively generates complex amplitudes that suppress noise, and performs data fidelity calculation between the generated complex amplitude and the input complex amplitude, so that the gradient direction is pulled back towards the direction of data consistency, avoiding excessive smoothing of the generated complex amplitude due to noise suppression. Attached Figure Description

[0025] Figure 1 This is a flowchart of the holographic reconstruction method based on wavelet domain feature correlation complex value neural network of the present invention.

[0026] Figure 2 This is an algorithmic framework diagram of the holographic reconstruction method based on wavelet domain feature correlation complex value neural network of the present invention. Detailed Implementation

[0027] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0028] See Figures 1-2 The holographic reconstruction method based on wavelet domain feature correlation complex value neural network of the present invention includes the following steps:

[0029] S1. Based on the basic principle of coaxial holography, construct an optical path and use a camera to capture a phase-type coaxial hologram A;

[0030] S2. Take the square root of the pixel value of the acquired phase-type coaxial hologram A as the amplitude term, and randomly generate an initial phase term to construct the complex amplitude B. Perform n-level discrete wavelet transform on the complex amplitude B to generate low-frequency complex amplitude components and high-frequency complex amplitude components at each level. Together with the original complex amplitude B, these are used as the input of the complex-valued network model. The phase term of the suppressed complex amplitude D output in each iteration will replace the phase term of the input complex amplitude B to generate a new complex amplitude B.

[0031] S3. For the input of the complex-valued network model, preliminary feature extraction is first performed through a shallow feature extraction layer. This shallow feature extraction layer consists of multiple complex-valued convolutional blocks with a kernel size of 3×3. Each complex-valued convolutional block contains a complex-valued convolution operation, a complex-valued batch normalization layer, and a complex-valued ReLU activation function. Then, the edge masks of the low-frequency components at each level of the output of the shallow feature extraction layer are constructed using the complex-valued Prewitt operator module to enhance the weight of the edge features of the low-frequency components. Subsequently, the enhanced low-frequency components and the corresponding high-frequency components are used as the input of the cross-wavelet domain feature association module.

[0032] S4. In the cross-wavelet domain feature association module, four complex-valued depthwise separable convolutions are used to calculate the Q and V values ​​for enhancing the low-frequency components. L And the corresponding high-frequency components K and V H The weight W of the low-frequency text information focused in the high-frequency components is calculated by combining multi-head attention and the softmax function. LH And the weight W of the low-frequency component on the semantic representation of the high-frequency component. HL Finally, the weight W LH With V L Weight W HL With V H By combining and applying the enhanced low-frequency component and the corresponding high-frequency component to the input respectively, the enhanced low-frequency feature ENL and the high-frequency feature ENH are obtained.

[0033] S5, ENL of the nth level low-frequency feature n and high-frequency characteristics ENH n Perform a discrete wavelet inverse transform and then pass it through a complex-valued convolutional block. Finally, combine the output of the complex-valued convolutional block with the (n-1)th level low-frequency feature ENL. n-1 The combination generates an optimized low-frequency feature ENL. ' n-1 ;Optimized low-frequency features ENL ' n-1 Then, with the corresponding high-frequency characteristic ENH n-1 After performing discrete wavelet inverse transform and a complex convolution block, and executing the same operation n times, a complex amplitude C of the object field with the same size as the complex amplitude B is generated.

[0034] S6. Noise suppression is achieved by using a complex-valued smooth total variational denoising model for the complex amplitude C of the object field. The suppressed complex amplitude D is then used to simulate the forward propagation process of a coaxial hologram to generate a hologram E. Finally, the similarity between the phase-type coaxial hologram A and the hologram E is calculated, and the phase of the complex amplitude D is subject to smooth total variational constraints. A self-supervised iterative optimization process for a single image is then performed using the gradient descent optimization algorithm of a complex-valued neural network, ultimately achieving holographic reconstruction.

[0035] See Figure 1 In step S1, the phase-type coaxial hologram A is a pure phase hologram, and the amplitude of the original object wave is set to 1.

[0036] See Figures 1-2 In step S2, the size of the randomly generated initial phase term is the same as that of the hologram A, ranging from 0 to 1.

[0037] See Figures 1-2In step S2, the n-level discrete wavelet transform uses a two-dimensional Haar discrete wavelet transform, outputting a low-frequency component LL, a high-frequency horizontal component HL, a high-frequency vertical component LH, and a high-frequency diagonal component HH. The n in the n-level discrete wavelet transform corresponds to the number of times the two-dimensional Haar discrete wavelet transform is used. The complex amplitude B is used as the first-level low-frequency component for the two-dimensional Haar discrete wavelet transform, outputting a low-frequency component LL1, a high-frequency horizontal component HL1, a high-frequency vertical component LH1, and a high-frequency diagonal component HH1 with a size of 1 / 4 of the original input size. Each subsequent level of the two-dimensional Haar discrete wavelet transform only operates on the low-frequency components, and the size of each output component is 1 / 4 of the input size of that level. The input of the complex value network model includes the input complex amplitude B and all the high-frequency and low-frequency components generated at each level.

[0038] See Figures 1-2 In step S2, n corresponds to the number of times the two-dimensional Haar discrete wavelet transform is used in the n-level discrete wavelet transform. In this embodiment, the input complex amplitude size is 256×256, and n is set to 3. After three discrete wavelet samplings, the complex amplitude sizes are 128×128, 64×64 and 32×32, respectively.

[0039] See Figure 2 All complex convolutional blocks that convolve the initial multi-level complex amplitudes have 1 input channel and 64 output channels; the complex convolutional blocks after channel concatenation have 128 input channels and 64 output channels, and the complex convolutional blocks in the remaining intermediate processes have 64 input and output channels. Setting a larger number of channels can enhance the model's ability to learn complex amplitude features, but it requires more GPU memory and increases the time cost of model learning. The generated complex amplitude C of the object field needs to pass through a complex convolutional layer with the number of channels shrunk to 1, which contains only one complex convolution operation.

[0040] See Figure 1 In step S6, the Huber-TV regularization layer uses gradient descent to approximate the near-end operator and employs iterative optimization to suppress noise. In this embodiment, the number of iterations a is set to 7 and the gradient update step size s is set to 0.1.

[0041] See Figure 2In this embodiment, the loss consists of two parts. The first part is the mean square error calculation of the intensity of the input hologram A and the intensity of the generated hologram E, where p and q represent the length and width of the hologram, respectively. The second part is the Huber-TV smoothing total variation constraint on the phase term in the complex amplitude D, where the smoothing total variation constraint is composed of a real-valued total variation incorporated into a differentiable Huber function. The hyperparameter α is used to prevent over-smoothing of the phase term. A larger weight value of α can be assigned to images with greater noise. In this embodiment, the value of the hyperparameter α is set to 0.1.

[0042] See Figures 1-2 In step S6, the propagation method for generating hologram E by simulating the forward propagation process of coaxial holography after suppression of complex amplitude D is the angular spectrum propagation method in this embodiment.

[0043] The above are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above content. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A holographic reconstruction method based on a complex-valued neural network associated with wavelet domain features, characterized in that, The method comprises the following steps: S1, on the basis of the coaxial holographic basic principle, an optical path is built and a phase type coaxial hologram A is photographed by a camera; S2, the pixel value of the collected phase type coaxial hologram A is squared as an amplitude item, and an initial phase item is randomly generated to construct a complex amplitude B, the complex amplitude B is subjected to n-level discrete wavelet transform to generate low-frequency complex amplitude components and high-frequency complex amplitude components at each level, which are used as inputs of a complex value network model together with the original complex amplitude B; the phase item of the suppressed complex amplitude D output by each iteration is used to replace the phase item of the input complex amplitude B to generate a new complex amplitude B; S3, the inputs of the complex value network model are all subjected to preliminary feature extraction through a shallow feature extraction layer, the shallow feature extraction layer is composed of a plurality of complex value convolution blocks with a convolution kernel size of 3*3, each complex value convolution block comprises a complex value convolution operation, a complex value batch normalization layer and a complex value ReLU activation function; then, an edge mask of each low-frequency component output by the shallow feature extraction layer is constructed by using a complex value Prewitt operator module to enhance the weight of the edge feature of the low-frequency component; subsequently, the enhanced low-frequency component and the corresponding high-frequency component are used as inputs of a cross wavelet domain feature correlation module; S4, in the cross wavelet domain feature correlation module, using four complex depth separable convolution respectively to calculate the enhanced low frequency component Q, V L And the corresponding high frequency component K, V H , combined with multi-head attention and softmax function to calculate the weight W of the focused low frequency text information in the high frequency component LH And the weight W of the low frequency component to the semantic expression of the high frequency component HL ; finally, the weight W LH And V L , the weight W HL And V H Combined and respectively acted on the input enhanced low frequency component and the corresponding high frequency component, to obtain the enhanced low frequency feature ENL and high frequency feature ENH; S5, the n-level low-frequency feature ENL n and high-frequency feature ENH n discrete wavelet inverse transform and a complex convolution block, and then combine the output of the complex convolution block with the n-1-level low-frequency feature ENL n-1 to generate an optimized low-frequency feature ENL ' n-1 ; the optimized low-frequency feature ENL ' n-1 and the corresponding high-frequency feature ENH n-1 discrete wavelet inverse transform and a complex convolution block, and after performing the same operation n times, a complex amplitude C of the object field with the same size as the complex amplitude B is generated. S6, a complex value based smoothing total variation denoising model is used for noise suppression on the object field complex amplitude C, the suppressed complex amplitude D is used to simulate a coaxial holographic forward propagation process to generate a hologram E; finally, the similarity between the hologram phase type coaxial hologram A and the hologram E is calculated, and the phase of the complex amplitude D is subjected to smoothing total variation constraint, and a single image self-supervised iterative optimization process is performed through a gradient descent optimization algorithm of the complex value neural network, so that holographic reconstruction is finally realized.

2. The holographic reconstruction method based on wavelet domain feature correlation complex-valued neural network according to claim 1, characterized in that, In step S1, the phase type coaxial hologram A is a pure phase hologram, and the amplitude of the original object wave is set to 1.

3. The holographic reconstruction method based on wavelet domain feature correlation complex-valued neural network according to claim 1, characterized in that, In step S2, the size of the randomly generated initial phase item is the same as that of the phase type coaxial hologram A, and the range is 0 to 1.

4. The holographic reconstruction method based on wavelet domain feature correlation complex-valued neural network according to claim 1, characterized in that, In step S2, the n-level discrete wavelet transform adopts two-dimensional Haar discrete wavelet transform, and outputs low-frequency components LL, high-frequency horizontal components HL, high-frequency vertical components LH and high-frequency diagonal components HH; n corresponds to the number of times of two-dimensional Haar discrete wavelet transform; The complex amplitude B is subjected to two-dimensional Haar discrete wavelet transform as the first low-frequency component, and outputs a low-frequency component LL1, a high-frequency horizontal component HL1, a high-frequency vertical component LH1 and a high-frequency diagonal component HH1 with a size of 1 / 4 of the original input size; each two-dimensional Haar discrete wavelet transform only operates on the low-frequency component, and the size of each component output is 1 / 4 of the input size at this level; the inputs of the complex value network model include the input complex amplitude B and all high-frequency components and low-frequency components generated at each level.

5. The holographic reconstruction method based on wavelet domain feature correlation complex-valued neural network according to claim 1, characterized in that, In step S3, the complex Prewitt operator module uses the horizontal and vertical 3*3 convolution templates of the Prewitt operator as a convolution kernel to perform neighborhood convolution operation on the multi-channel features composed of the real part and the imaginary part of the complex amplitude, and calculate the spatial gradient changes of the real part and the imaginary part, respectively; create an edge mask after normalizing the gradient changes; modulate the real part and the imaginary part through the edge mask, and then superimpose the modulation results with the original real part and the original imaginary part, respectively, to realize the optimization of the edge-enhanced complex value signal.

6. The holographic reconstruction method based on wavelet domain feature correlation complex-valued neural network according to claim 1, characterized in that, In step S5, the complex amplitude C of the object field needs to pass through an additional complex value convolution layer with one channel contraction to 1, and the complex value convolution layer only contains one complex value convolution operation.

7. The holographic reconstruction method based on wavelet domain feature correlation complex-valued neural network according to claim 1, characterized in that, In step S6, the complex-based smooth total variation denoising model integrates the complex total variation into a Huber function, and constructs a Huber-TV regularization layer by using the iterative steps of the proximal gradient method on the basis of the Huber function; the gradient descent is used to approximate the proximal operator, wherein the gradient update step s is selected as 0.1; the generated complex amplitude in iteration is data-faithful to the input complex amplitude, so that the gradient direction is pulled back to the data consistency direction, and over-smoothing is avoided.

8. The holographic reconstruction method based on wavelet domain feature correlation complex-valued neural network according to claim 7, characterized in that, The setting range of the iteration number a of the Huber-TV regularization layer is 5-10 times.

9. The holographic reconstruction method based on wavelet domain feature correlation complex-valued neural network according to claim 1, characterized in that, In step S6, the smooth total variation constraint is a real value constraint for the phase term, which is composed of a Huber function by integrating the real value total variation.

Images