A method for generating a binarized airspace-constrained x-ray coherent diffraction image recovery

Abstract

The application discloses a kind of generation binary space constraint X-ray coherent diffraction image recovery method.This application is used to realize from the coherent X-ray diffraction image of acquisition recovery out sample image to be measured, using the method of deep learning with good predictive ability to generate corresponding space constraint support to X-ray coherent diffraction image, then it is carried out HIO phase iteration, the application can make up the condition that space constraint is not accurate in traditional HIO iterative method, and then can be more quickly converged, higher quality is recovered;Compared with directly using deep learning image recovery method, the result of the application is more accurate, there is no pixel error situation, can realize through small amount of iteration recovery better result, improve the robustness of X-ray diffraction image recovery.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical imaging, and in particular to a method for generating a binary spatially constrained X-ray coherent diffraction image restoration. BACKGROUND

[0002] Coherent X-ray Diffractive Imaging (CXDI) is a new type of diffraction imaging technology, which has the ability of high resolution and three-dimensional reconstruction of internal structure information of the sample, and does not require the sample to be a crystal. CXDI is also known as X-ray coherent diffraction microscopy or lensless imaging. The basic principle of CXDI is to use the coherence of X-rays for diffraction. By measuring the diffraction pattern on the sample, the phase information of the sample can be reconstructed, and thus the structure information of the sample can be obtained. Unlike traditional optical microscopes, CXDI does not require the use of lenses to focus light, but through the calculation of the diffraction pattern to reconstruct the image of the sample. This lensless imaging method enables CXDI to achieve high-resolution imaging. The application fields of CXDI are wide, including material science, biological science, nanoscience, etc. It can be used to study the crystal structure of materials, surface topography, distribution of nanoparticles, etc. At the same time, CXDI can also be used to observe the internal structure of biological samples such as cells, proteins, etc. Due to its high resolution and the characteristic of not requiring a crystalline sample, CXDI has broad prospects in scientific research and industrial applications.

[0003] The development history of CXDI can be traced back to 1952, when Sayre proposed the idea of CXDI, and in 1980, he began the diffraction imaging experiment of amorphous samples. With the continuous improvement of the phase reconstruction algorithm by Fienup et al., until 1999, Miao et al. completed the first verification experiment, successfully obtaining the reconstructed image of a non-periodic arrangement of gold particles with a diameter of about 100 nm. Since then, CXDI technology has developed rapidly and has achieved good experimental results in the fields of material science and biological science.

[0004] In X-ray coherent diffraction imaging technology, the collected diffraction image contains the amplitude and phase information of the sample. Due to the invisibility of the phase information, the real image of the sample can only be recovered by a phase recovery algorithm. At present, the existing X-ray coherent diffraction image phase recovery methods and technologies at home and abroad mainly include the iterative-based hybrid input-output (Hybrid Input-Output, HIO) algorithm, which optimizes the optimal solution by using the constraints of the X-ray diffraction image in the object space and the spectral space; the phase recovery algorithm technology requiring prior knowledge, such as the Kramers-Kronig relationship, which converts the phase recovery problem into a least squares problem by minimizing the residual sum of squares, and the basis function is selected according to the geometry and material properties of the object. Using the technology of compressed sensing, only a small amount of non-random sampling is performed on the image, and then the sparse representation technology is used in the calculation to recover the image. In recent years, with the continuous development of computer technology and image processing technology, the phase recovery algorithm of coherent diffraction imaging has also made great progress. Deep learning has also been applied in coherent diffraction recovery. By using the tools of deep learning and a large amount of data training, the neural network can directly recover the object from the X-ray diffraction image.

[0005] Although there are many methods for X-ray coherent diffraction image recovery, in order to recover the information of the sample to be tested from a single diffraction intensity image, most of the related mainstream recovery technologies are based on phase iteration of alternating projection. In the process of phase iteration, how to select the spatial domain constraint is a difficulty. It is found through experiments that the iterative recovery result is strongly dependent on the spatial domain constraint condition on the object plane. When the spatial domain constraint is closer to the object shape, the recovered X-ray coherent diffraction image effect is better. However, in the traditional X-ray coherent diffraction image recovery method, due to the unpredictability of the spatial domain constraint, it is difficult to approach the true shape of the object. When the spatial domain constraint is weak, the iterative algorithm is difficult to converge, resulting in a poor final recovery result. In addition, the traditional method may also appear the phenomenon of inverted recovery image. Deep learning as a new type of computational imaging tool has received very extensive attention in recent years. In 2018, Cherukara et al. first recovered the coherent diffraction image by deep learning, proving that deep learning can well recover the X-ray coherent diffraction image, and has the characteristics of fast speed, almost real-time imaging, and good low-frequency information recovery. However, this method is more dependent on the training data set, and the robustness is poor, and it cannot cope with different types of object diffraction images. At the same time, when this method is used to recover the X-ray coherent diffraction image, there may be partial pixel recovery errors and high-frequency information loss.

[0006] In general, although the iterative-based coherent diffraction image restoration method can accurately restore the image of the sample to be measured when the spatial domain constraint is strong, the degree of dependence of the method on the spatial domain constraint is high, and it is difficult to restore a better result without knowing the accurate object boundary. Although the existing black-box deep learning-based method can realize fast image restoration, its robustness and stability are poor. Therefore, it is necessary to invent an efficient and high-quality X-ray coherent diffraction image restoration method. SUMMARY

[0007] In view of the problems existing in the prior art, the present application provides an X-ray coherent diffraction image restoration method, when the constraint of the object plane is closer to the true shape of the object, the effect of the HIO iterative restoration of the X-ray coherent diffraction image is better, and the present application predicts a constraint very close to the true shape of the object before iteration, greatly improving the performance of the iterative algorithm.

[0008] The X-ray coherent diffraction image restoration method for generating a binary spatial domain constraint of the present application comprises the following steps:

[0009] 1) Obtain a training sample image:

[0010] a) Obtain an original sample image :

[0011] Select images from an open-source data set as the original sample image ;

[0012] b) Generate a coherent X-ray diffraction intensity image:

[0013] According to the Fraunhofer diffraction theory, perform numerical simulation on all original sample images using a computer to perform Fourier transform and take modulus square, to obtain the coherent X-ray diffraction intensity image corresponding to the original sample image :

[0014]

[0015] Wherein, the symbol FT represents the Fourier transform operation;

[0016] c) Obtain a diffraction autocorrelation image:

[0017] Perform inverse Fourier transform on the coherent X-ray diffraction intensity image to obtain the autocorrelation image of the original sample image :

[0018]

[0019] Wherein, the symbol represents a correlation operation, and the symbol

[0020] denotes an inverse Fourier transform operation;

[0021] d) obtaining a binary spatial support image S:

[0022] performing binaryzation on the original sample image to obtain a binary spatial support image S;

[0023] taking the autocorrelation image and the corresponding binary spatial support image S as training sample images of the deep neural network;

[0024] 2) constructing and training the deep neural network:

[0025] constructing the deep neural network, training the deep neural network, taking the autocorrelation image as the input of the deep neural network and taking the binary spatial support image S as the output of the deep neural network, saving the network parameters after reaching a preset iteration number, and obtaining the trained deep neural network;

[0026] 3) obtaining a spatial support condition :

[0027] performing inverse Fourier transform on the measured coherent X-ray diffraction intensity image to obtain a measured autocorrelation image ; inputting the measured autocorrelation image to the trained deep neural network, and taking the binary spatial support image S output by the deep neural network as a spatial support condition for a hybrid input-output (HIO) iteration algorithm based on iteration;

[0028] 4) restoring a diffraction imaging result image:

[0029] performing the HIO iteration algorithm on the measured coherent X-ray diffraction intensity image, in the nth iteration, n is a natural number satisfying n≤N, performing Fourier transform on a spatial complex amplitude distribution n

[0030] to obtain a corresponding spectral distribution , performing spectral constraint on the spectral distribution of the X-ray diffraction intensity image, in the spectral constraint, performing square root operation on the measured coherent X-ray diffraction intensity image to obtain a coherent X-ray diffraction amplitude image , and replacing the amplitude part in the complex amplitude of the spectral distribution with the obtained coherent X-ray diffraction amplitude image, while keeping the spectral distribution ​The phase portion of the spectrum remains unchanged, resulting in an updated frequency domain complex amplitude distribution. :

[0031]

[0032] The updated frequency domain complex amplitude distribution is inversely Fourier transformed back to the spatial domain to obtain a new spatial domain complex amplitude distribution. Based on the airspace support conditions obtained in the previous step The spatial domain constraint is performed using the following formula, thereby obtaining the updated spatial complex amplitude distribution. :

[0033]

[0034] in, This is the feedback factor for the HIO iterative algorithm, which controls the convergence performance of the algorithm. It typically takes a value between 0 and 1, and will yield updated spatial complex amplitudes. The distribution is used as input for the next iteration, and a new round of iterations is performed; the above iterative steps are repeated until the preset number of iterations is reached. N Finally, the output image is used as the diffraction imaging reconstruction result image.

[0035] In step 1), the original image The number is over 1000.

[0036] In step 2), the activation function in the deep neural network is set to the Rectified Linear Unit (ReLU) to learn nonlinear mapping relationships; the loss function is set to the Mean Absolute Error (MAE) to adjust and optimize the structural parameters of the deep neural network. The weight parameters are continuously optimized through backpropagation during training. The deep neural network used is one of the following for image restoration: Convolutional Neural Network (CNN), U-Net, or Deep Convolutional Generative Adversarial Network (DCGAN). All tests in this invention use the U-Net network.

[0037] In step 4), the preset number of iterations N It achieves very good results in the range of 300 to 2000. 0.9.

[0038] Advantages of this invention:

[0039] The application is used for realizing the recovery of a sample image to be measured from a collected coherent X-ray diffraction image, and a deep learning method with good prediction ability is used to generate a corresponding space domain constraint support for the X-ray coherent diffraction image, and then the HIO phase iteration is performed on the X-ray coherent diffraction image, so that the application can make up for the inaccuracy of the space domain constraint in the traditional HIO iteration method, and then the convergence can be faster, and a higher quality recovery can be realized; compared with directly using the deep learning image recovery method, the result of the application is more accurate, and the pixel error condition does not occur, the better result can be recovered through a small number of iterations, and the robustness of the X-ray diffraction image recovery is improved. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 It is a comparison chart of different support recovery results of the X-ray diffraction intensity image;

[0041] Figure 2 It is a comparison chart of the result of the X-ray coherent diffraction image recovery method for generating a binary space domain constraint of the application and the traditional HIO method;

[0042] Figure 3 It is a result chart of the X-ray coherent diffraction image recovery method for generating a binary space domain constraint of the application using a small number of iterations. DETAILED DESCRIPTION

[0043] The application will be further described through specific embodiments in combination with the drawings.

[0044] The X-ray coherent diffraction image recovered by the iterative phase recovery method depends on the constraint condition on the object plane, and when the constraint of the object plane is closer to the true shape of the object, the effect of the X-ray coherent diffraction image recovered by the iteration is better. Figure 1 As shown in the first row, a rectangle conforming to the size of the image information is made; then the original image is used to generate a binary processing to generate the third row; the morphological dilation operation is performed on the third row support to generate the second row; and the morphological erosion operation is performed on the third row support to generate the fourth row. As known from the above, the support is gradually reduced, and the third row is just consistent with the shape of the object. Figure 1 As shown in the second row, when the support is closer to the shape of the object, the recovery effect is better.

[0045] The X-ray coherent diffraction image recovery method for generating a binary space domain constraint of the embodiment comprises the following steps:

[0046] 1) Obtain a training sample image:

[0047] a) Obtain an original sample image :

[0048] 6100 images are selected from an open source data set as the original sample image ;

[0049] b) generating a coherent X-ray diffraction intensity image:

[0050] According to the Fraunhofer diffraction theory, all original sample images are numerically simulated by using a computer Fourier transform and modulus square are performed to obtain the coherent X-ray diffraction intensity image corresponding to the original sample image :

[0051]

[0052] wherein the symbol FT represents the Fourier transform operation;

[0053] c) obtaining a diffraction autocorrelation image:

[0054] Inverse Fourier transform is performed on the coherent X-ray diffraction intensity image to obtain the autocorrelation image of the original sample image :

[0055]

[0056] wherein the symbol represents the correlation operation, and the symbol represents the inverse Fourier transform operation;

[0057] d) obtaining a binarized spatial support image:

[0058] The original sample image is binarized to obtain the binarized spatial support image S;

[0059] The autocorrelation image and the corresponding binarized spatial support image S are taken as the training sample images of the deep neural network;

[0060] 2) constructing and training the deep neural network:

[0061] The deep neural network is constructed, and the deep neural network is trained, to obtain the autocorrelation image As the input of the deep neural network, the binarized spatial support image S is taken as the output of the deep neural network; 100 images are taken as the test set, and 6000 images are taken as the training set; the deep neural network adopts the U-Net network, the activation function in the deep neural network is set as the rectified linear unit (Rectified Linear Unit, ReLU), which is used to learn the nonlinear mapping relationship; the loss function is set as the mean absolute error (Mean Absolute Error, MAE), which is used to adjust and optimize the structure parameters of the neural network; the learning rate of the neural network is set to 0.001, and the iteration number is 50 times; in the training process, the weight parameters are continuously optimized through back propagation; after reaching the preset iteration number, the network parameters are saved, and the trained deep neural network is obtained;

[0062] 3) Obtain the spatial support condition :

[0063] The measured coherent X-ray diffraction intensity image is subjected to inverse Fourier transform operation to obtain a measured autocorrelation image ; the measured autocorrelation image is input into the trained deep neural network, and the deep neural network outputs a binarized spatial support image S, which is taken as the spatial support condition of the hybrid input-output HIO iterative algorithm based on iteration ;

[0064] 4) Restore the diffraction imaging result image:

[0065] The HIO iterative algorithm is performed on the measured coherent X-ray diffraction intensity image, and in the nth iteration, n is a natural number less than or equal to N, the spatial complex amplitude distribution of this iteration n is subjected to Fourier transform to obtain the corresponding spectral distribution ; The spectral distribution of the X-ray diffraction intensity image is subjected to spectral constraint, in which the measured coherent X-ray diffraction intensity image is subjected to square root operation to obtain a coherent X-ray diffraction amplitude image , and the obtained coherent X-ray diffraction amplitude image replaces the amplitude part in the complex amplitude of the spectral distribution , while keeping the spectral phase part of the spectral distribution unchanged, to obtain an updated frequency domain complex amplitude distribution :

[0066]

[0067] The updated frequency domain complex amplitude distribution is subjected to inverse Fourier transform to return to the spatial domain to obtain a new spatial domain complex amplitude distribution Based on the airspace support conditions obtained in the previous step The spatial domain constraint is performed using the following formula, thereby obtaining the updated spatial complex amplitude distribution. :

[0068]

[0069] in, This is the feedback factor for the HIO iterative algorithm, which controls the convergence performance of the algorithm. Its value is typically between 0 and 1. In this embodiment... =0.7 or =0.9, the resulting updated spatial complex amplitude distribution Use this as input for the next iteration to start a new round of iterations; repeat the above iterative steps until the preset number of iterations is reached. N Preset number of iterations N The value can be 100, 200, 500, 1000 or 2000, and the final output image is the diffraction imaging reconstruction result image.

[0070] like Figure 2 As shown, this invention can significantly improve image restoration speed, and the image details are well restored without any restoration errors. To further verify the advantages of this invention, observations were made on a small number of iterations, such as... Figure 3 As shown, even with a few iterations, the present invention still achieves good recovery results.

[0071] Finally, it should be noted that the purpose of disclosing the embodiments is to help further understand the present invention. However, those skilled in the art will understand that various substitutions and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the scope of protection of the present invention is defined by the claims.

Claims

1. A method for restoring X-ray coherent diffraction images with binary spatial constraints, characterized in that, The X-ray coherent diffraction image restoration method includes the following steps: 1) Obtain training sample images: a) Obtain the original sample image : Images were selected from an open-source dataset as the original sample images. ; b) Generate coherent X-ray diffraction intensity images: Based on Fraunhofer diffraction theory, numerical simulations were performed using a computer to analyze all original sample images. Perform a Fourier transform and square the modulus to obtain the coherent X-ray diffraction intensity image corresponding to the original sample image. : Wherein, the symbol FT represents the Fourier transform operation; c) Obtain the diffraction autocorrelation image: Performing an inverse Fourier transform on the coherent X-ray diffraction intensity image yields the autocorrelation image of the original sample image. : Among them, symbols Indicates related operations, symbols This represents the inverse Fourier transform operation; d) Obtain the binarized spatial support image: Original sample image Binarization is performed to obtain a binary spatial support image S; With autocorrelation image The corresponding binarized spatial support image S is used as the training sample image for the deep neural network. 2) Construct and train a deep neural network: Construct a deep neural network, train the deep neural network, and use autocorrelation images As input to the deep neural network, the binarized spatial support image S is used as the output of the deep neural network. After reaching a preset number of iterations, the network parameters are saved to obtain the trained deep neural network. 3) Obtain airspace support conditions : The intensity image of the coherent X-ray diffraction of the target obtained by an X-ray diffractometer The autocorrelation image to be measured is obtained by performing an inverse Fourier transform. ; the autocorrelation image to be tested The input is fed into a trained deep neural network, which outputs a binarized spatial support image S. This binarized spatial support image S is used as the spatial support condition for an iterative hybrid input-output (HIO) algorithm. ; 4) Reconstruct the diffraction imaging result image: The HIO iterative algorithm is applied to the coherent X-ray diffraction intensity image under test, and at the first... n In this iteration, n is a natural number ≤ N, representing the spatial complex amplitude distribution of this iteration. Perform a Fourier transform to the spectral domain to obtain the corresponding spectral distribution. Spectral constraints are applied to the spectral distribution of the X-ray diffraction intensity image. Within these constraints, the square root of the measured coherent X-ray diffraction intensity image is calculated to obtain the coherent X-ray diffraction amplitude image. The obtained coherent X-ray diffraction amplitude image is then used to replace the spectral distribution. The amplitude component of the complex amplitude, while maintaining the spectral distribution. The phase portion of the spectrum remains unchanged, resulting in an updated frequency domain complex amplitude distribution. : The updated frequency domain complex amplitude distribution is inversely Fourier transformed back to the spatial domain to obtain a new spatial domain complex amplitude distribution. Based on the airspace support conditions obtained in the previous step The spatial domain constraint is performed using the following formula, thereby obtaining the updated spatial complex amplitude distribution. : in, The feedback factor for the HIO iterative algorithm controls the convergence performance of the algorithm and yields updated spatial complex amplitudes. The distribution is used as input for the next iteration, and a new round of iterations is performed; the above iterative steps are repeated until the preset number of iterations is reached. N Finally, the output image is used as the diffraction imaging reconstruction result image.

2. As described in claim 1, characterized in that, In step 1), the original image The number is over 1000.

3. As described in claim 1, characterized in that, In step 2), the activation function in the deep neural network is set to the modified linear unit; the loss function is set to the mean absolute error.

4. As described in claim 1, characterized in that, In step 2), the deep neural network employs one of the following: convolutional neural network, U-shaped network, and deep convolutional generative adversarial network.

5. As described in claim 1, characterized in that, In step 4), the preset number of iterations N The range is 300 to 2000.

6. As described in claim 1, characterized in that, In step 4), 0.9.

Images