Quantitative and high-anti-interference complex wavefront recovery method based on unsupervised deep learning

Abstract

The invention relates to a quantitative and high-anti-interference complex wavefront recovery method based on unsupervised deep learning. The quantitative and high-anti-interference complex wavefront recovery method comprises the following steps: acquiring an actual measurement value of an optical system; inputting the actual measurement value into the DNN model, and directly outputting the predicted complex wavefront on the target plane by the DNN model in an end-to-end manner; the predicted complex wavefront is further subjected to digital spatial light modulator modulation and digital wave propagation which are the same as those in the actual acquisition process of the optical system, and a predicted measurement value on a digital camera plane is obtained; calculating a loss function based on a difference between the predicted measurement value and the actual measurement value; and repeating the above steps, iteratively optimizing the parameters of the DNN model through the minimum loss function, and outputting a final complex wavefront prediction result after iteration is completed. Compared with the prior art, the method has the advantages that the precision is high, a large amount of training data does not need to be acquired, and a high spatial bandwidth product is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of complex wavefront reconstruction, and in particular to a quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning. Background Art

[0002] The basic feature of a light field can be a complex wavefront (CW), i.e., U(x,y) = A(x,y) exp[jφ(x,y)], where A(x,y) and φ(x,y) represent the amplitude and phase of U(x,y), respectively. In coherent imaging, once a continuous wave on one plane of the light field is obtained, a continuous wave on any plane of the light field can be obtained by numerical propagation. The continuous wave reflected by an object contains the surface information of the object, with its three-dimensional surface shape encoded as φ(x,y) and the surface reflectivity encoded as A(x,y). Therefore, obtaining the continuous wave, especially its phase, is the key to three-dimensional metrology. However, cameras such as charge-coupled devices (CCDs) and complementary metal-oxide semiconductors (CMOSs) can only detect the intensity of the light field but not the phase, resulting in the loss of continuous wave information. To address this problem, various techniques have been developed, which are classified as complex wavefront reconstruction (CWR) techniques. Complex wavefront reconstruction techniques can recover the complex wavefront of the light wave from the intensity information recorded by the camera.

[0003] In general, complex wavefront reconstruction techniques can be divided into two categories: interferometry and non-interferometry. Digital holography (DH) is a typical interferometry technique. Due to the nature of interferometry, digital holography allows quantitative complex wavefront reconstruction with high-precision traceability of wavelength. Therefore, digital holography has become the benchmark for quantitative 3D metrology. However, digital holography and other interferometric complex wavefront reconstruction techniques have a major limitation, namely, the interferometric structure has a high sensitivity to external interferences (such as vibration, fast airflow, etc.). This seriously affects the accuracy of the measurement. Therefore, in industries with severe interference, it is difficult for interferometric techniques to go out of the laboratory for field application. On the other hand, such high-precision, high-interference-resistance 3D quantitative metrology techniques are lacking and urgently needed in industry for quality control. For field purposes, non-interferometric techniques with better anti-interference performance are required.

[0004] The second category is non-interferometric complex wavefront reconstruction technology. Among them, the Shack-Hartmann wavefront sensor (SHWS) has a simple structure based on geometric optical design. SHWS uses a microlens array to divide the wavefront into multiple sub-wavefronts and obtain the average phase slope of each sub-wavefront. Due to this division, the spatial resolution of the retrieved continuous wave is very low. SHWS is limited to continuous waves with smooth changes.

[0005] After SHWS, non-learning computational complex wavefront reconstruction techniques were proposed. These methods directly obtain complex wavefronts from measurements through non-learning computational methods. However, these methods all use or inherit the GS algorithm, so they have low computational power when solving the problem of pathological complex wavefront reconstruction. Therefore, more powerful computational tools are needed.

[0006] In recent years, deep learning (DL) has developed rapidly as a more powerful computing tool. DL has been introduced into the field of complex wavefront reconstruction. The paradigm of using deep neural networks (DNN) to solve the problem of pathological complex wavefront reconstruction is defined as DL-based CWR technology (DL-based CWR). Most of these technologies use supervised methods to train DNNs, which requires a large amount of training data. However, in practice, it is sometimes difficult to obtain a large amount of data and the true value of complex wavefronts. Therefore, the supervised method is difficult to implement. To solve this problem, unsupervised DL-based CWR technology was proposed, which introduces the physical model of measurement formation in the optical system into DNN. Unlike the supervised method, the unsupervised method does not require true values ​​and training data sets. PhysenNet pioneered the work of unsupervised DL-based CWR, but it is limited to pure phase objects and cannot obtain the wavefront of complex value objects. Subsequently, a dual-input Y-Net for obtaining the wavefront of complex value objects was proposed, but Y-Net still has room for improvement in imaging quality.

[0007] For all the above deep learning based complex wavefront reconstruction techniques, there is a major limitation, namely the limited space-bandwidth product (SBP), which is actually the number of pixels measured. Due to limited computational resources, the input measurements must be cropped to a smaller size, such as 512×512 pixels, for DNN processing. This results in a low SBP for the DNN. While the state-of-the-art optical systems can easily reach more than 2000×2000 pixels. Therefore, there is a mismatch between the low SBP of the DNN and the high SBP (HSBP, defined as SBP greater than 2000×2000) of the optical system. The low SBP limits the ability of DL based techniques in acquiring high lateral resolution and large field of view.

[0008] The key advantages and disadvantages of existing complex wavefront reconstruction techniques are listed in Table 1.

[0009] Table 1 Advantages and disadvantages of complex wavefront reconstruction techniques

[0010]

[0011]

[0012] However, all existing deep learning-based non-interference complex wavefront reconstruction techniques do not have these four required advantages at the same time. In addition, most existing complex wavefront reconstruction techniques require precise knowledge of the diffraction distance parameter in the optical system, which is difficult to achieve and very time-consuming. To solve this problem, BlindNet recently automatically calibrated this parameter while DNN optimization. However, this technology is limited to pure phase objects. There is still room for improvement in accuracy and learning range. Summary of the invention

[0013] The purpose of the present invention is to solve the limitations of existing complex wavefront reconstruction technology in terms of practicality, accuracy and space-bandwidth product, and to provide a quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning.

[0014] The purpose of the present invention can be achieved by the following technical solutions:

[0015] A quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning, comprising the following steps:

[0016] S1, collects the actual measurement values ​​of the optical system;

[0017] S2, inputting the actual measurement value into the DNN model, and the DNN model directly outputs the predicted complex wavefront on the target plane in an end-to-end manner, wherein the predicted complex wavefront includes amplitude and phase;

[0018] S3, the predicted complex wave is further subjected to the same digital spatial light modulator modulation and digital wave propagation as in the actual acquisition process of the optical system, and a predicted measurement value on the digital camera plane is obtained;

[0019] S4, calculating the loss function based on the difference between the predicted measurement value and the actual measurement value;

[0020] S5, repeat steps S2-S4, iteratively optimize the parameters of the DNN model by minimizing the loss function, and iteratively output the final complex wavefront prediction result.

[0021] The optical system includes an object to be measured, a 4f system, a beam splitter, a spatial light modulator and a camera. The object to be measured is imaged on the spatial light modulator through the 4f system. The plane of the spatial light modulator is named the target plane. The spatial light modulator modulates the complex wavefront amplitude of the object to be measured on this plane. The modulated complex wavefront reaches the camera after passing through the light wave propagation distance z. Since the spatial light modulator works in a reflection mode, the complex wavefront is guided to the camera through a beam splitter. The camera captures a diffraction intensity pattern as a measurement value. During the operation of the optical system, the spatial light modulator will sequentially generate a series of random phase modulation patterns, and the camera will synchronously capture the corresponding measurement values.

[0022] The mathematical description of a real continuous wave on the target plane is:

[0023]

[0024] Among them, A and are the amplitude and phase of U on the target plane respectively; the random phase pattern M of the spatial light modulator i After phase modulation, the continuous wave propagates the diffraction distance z and reaches the camera, and the measurement value recorded by the camera is expressed as:

[0025]

[0026] in, represents the Hadamard product, and M i Element-wise multiplication of P z represents the wave propagation operator at the diffraction distance z;

[0027] Assuming that the size of the spatial light modulator modulation sequence is N, N measurement values ​​are obtained; the predicted continuous wave on the target plane is directly generated by the DNN model and is represented by the actual measurement values ​​{I1,...,I N} as input, the solution to the underdetermined complex wavefront recovery problem is expressed as:

[0028]

[0029] Among them, G represents the deep neural network DNN, and θ represents the set of neural network layer parameters that need to be learned, including the weights and biases in each neuron.

[0030] In the DNN model optimization process, the diffraction distance parameter z′ is learned synchronously with the model parameter θ from any initial value range. By learning the parameter z′, z′ is made to approach the given and unknown actual diffraction distance z, and the solution to the underdetermined complex wavefront recovery problem is reformulated as:

[0031]

[0032] The DNN model includes an interleaving layer, a U-Net network and a de-interleaving layer connected in sequence, wherein the interleaving layer and the de-interleaving layer are used to achieve a high space-bandwidth product, and the input measurement value is sequentially passed through the interleaving layer, the U-Net network and the de-interleaving layer to obtain a HSBP feature map with an H×W spatial size, wherein H and W represent the length and width of the input measurement value image respectively; the HSBP feature map is further processed through two convolutional layers to generate a predicted phase stack and an amplitude stack of sizes H×W×N respectively, and the N predicted results of the phase stack and the amplitude stack are averaged respectively to obtain the predicted phase of the complex wavefront on the target plane and predicted amplitude The size is H×W×1; the predicted phase and predicted amplitude Combined to obtain the final predicted complex wavefront as output.

[0033] The interleaving layer rearranges the input measurements of size H×W×N into data, where the third dimension N and (s 2 ×N) represents the number of channels, and s represents the scaling factor. By rearranging the interleaved layers, the spatial dimension is reduced from H×W to The number of channels increases from N to s 2 ×N; the de-interlacing layer performs the inverse operation of the interlacing layer, The data is rearranged into data of size H×W×N.

[0034] The U-Net network is a fully convolutional network, which consists of an encoding path and a decoding path, including convolution blocks, convolution downsampling blocks, transposed convolution layers and jump connections; in the encoding path, shallow features are first extracted from the data rearranged by the interleaved layer through the first convolution block, and then deep features are extracted at multiple scales through the second and third convolution blocks, wherein a convolution downsampling block is connected after the second and third convolution blocks to reduce the spatial size; in the decoding path, the encoded result is first decoded through the fourth and fifth convolution blocks, and a transposed convolution layer is respectively connected after the fourth and fifth convolution blocks to increase the spatial size, and then at the end of the decoding path, high-level features are further extracted through the sixth and seventh convolution blocks; between the encoding path and the decoding path, features of the same scale are connected through three jump connections, and the information on both sides is fused by splicing.

[0035] The convolutional block consists of two 3×3 convolutional layers, each followed by a ReLU activation layer and a batch normalization layer.

[0036] The convolutional downsampling block is constructed by a 6×6 convolutional layer followed by a ReLU activation layer and a batch normalization layer.

[0037] The loss function is expressed as:

[0038]

[0039] Among them, the MSE function calculates the predicted amplitude With measured amplitude The mean square error between , N is the number of measurements, TV is the total change regularizer, α is the relative weight of the total change regularizer, is the predicted complex wavefront output by the DNN model;

[0040] During the model optimization process, the gradient of the loss function relative to each neural parameter is calculated through the back-propagation algorithm, and then the gradient descent algorithm is used to update the neural parameters.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) High-precision quantitative complex wavefront reconstruction: The random phase modulation introduced by the spatial light modulator (SLM) in this invention increases the measurement constraint information and effectively alleviates the underdetermination of the CWR problem. Compared with the existing GS algorithm and CDI technology, this complex wavefront modulation based on SLM can provide more efficient constraints, thereby achieving faster CWR convergence and greatly improving the accuracy of complex wavefront reconstruction.

[0043] (2) High anti-interference ability: The present invention is based on a non-interference design and has strong robustness against external interference such as vibration, and can work stably in industrial sites.

[0044] (3) Applicable to complex value objects: The present invention can process complex value objects commonly found in industrial sites and is not limited to pure phase objects, which greatly expands the scope of application.

[0045] (4) High space-bandwidth product (HSBP): The present invention draws on the downsampling-upsampling (DU) module of IPMNet and uses interleaving layers and de-interleaving layers to achieve high space-bandwidth product, which can achieve high lateral resolution and large field of view, meeting the resolution and field of view requirements of three-dimensional measurement in industrial sites.

[0046] (5) Automatic calibration of parameters: The present invention can automatically calibrate the key parameters of the optical system while optimizing through deep learning, without the need for tedious manual settings.

[0047] (6) No need for a large amount of training data: Unlike supervised DL-CWR, which requires a large amount of training data, the present invention utilizes physical model constraints to process wavefront reconstruction of complex value objects without the need for training data. This overcomes the limitation of supervised DL-CWR that it is difficult to obtain a large amount of training data in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the working process of the complex wavefront recovery method of the present invention;

[0049] Figure 2 It is a schematic diagram of the DNN model structure of the present invention;

[0050] Figure 3 A schematic diagram of experimental results of quantitative measurement using USAF as an object in one embodiment;

[0051] Figure 4 A schematic diagram of experimental results of quantitative measurement using a stepped object in an embodiment. DETAILED DESCRIPTION

[0052] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0053] According to the analysis of existing technologies, a prospective complex wavefront reconstruction technology should have the following functions:

[0054] 1) High-precision quantitative complex wavefront reconstruction;

[0055] 2) Strong robustness to environmental interference;

[0056] 3) Applicable to complex value objects, because pure phase objects are very rare in industry;

[0057] 4) High sampling bandwidth and field of view (HSBP), providing sufficient lateral resolution and field of view.

[0058] Based on this, this embodiment provides a quantitative and highly anti-interference complex wavefront recovery method based on unsupervised deep learning, such as Figure 1 As shown, the following steps are included:

[0059] S1, collects the actual measurement values ​​of the optical system;

[0060] S2, inputting the actual measurement value into the DNN model, and the DNN model directly outputs the predicted complex wavefront on the target plane in an end-to-end manner, wherein the predicted complex wavefront includes amplitude and phase;

[0061] S3, the predicted complex wave is further subjected to the same digital spatial light modulator modulation and digital wave propagation as in the actual acquisition process of the optical system, and a predicted measurement value on the digital camera plane is obtained;

[0062] S4, calculating the loss function based on the difference between the predicted measurement value and the actual measurement value;

[0063] S5, repeat steps S2-S4, iteratively optimize the parameters of the DNN model by minimizing the loss function, and iteratively output the final complex wavefront prediction result.

[0064] like Figure 1 As shown in a, the optical system includes an object to be measured, a 4f system, a beam splitter (BS), a spatial light modulator and a camera. The object to be measured is imaged on the spatial light modulator through the 4f system, so the target plane and the spatial light modulator plane are conjugate. The spatial light modulator plane is named the target plane, and the spatial light modulator modulates the complex wavefront amplitude of the object to be measured on this plane. The modulated complex wavefront reaches the camera after the light wave propagates a distance z, where the complex wavefront is guided to the camera through a beam splitter because the spatial light modulator works in reflection mode. The camera captures the diffraction intensity pattern as the measurement value. During the operation of the optical system, the spatial light modulator will sequentially generate a series of random phase modulation patterns, and the camera will synchronously capture the corresponding measurement values.

[0065] Once the measurements are obtained, enter them into Figure 1 The processing flow in b. Figure 1 The measurements in b3 are fed into the DNN as input, as Figure 1 As shown in b4, DNN directly outputs the predicted complex wave on the target plane in an end-to-end manner, including amplitude and phase, as shown in Figure 1 b5 and Figure 1 b6. Then, the predicted complex wave further undergoes the same digital spatial light modulator modulation (b7) and digital wave propagation at distance z as in the actual acquisition process, resulting in a sequence of predicted measurement values ​​(b8) on the digital camera plane. The loss function is calculated based on the difference between the predicted measurement value (b8) and the actual measurement value (b3). By minimizing the loss function, the parameters of the DNN are optimized. The algorithm is iterative, and as the predicted measurement value gradually approaches the true measurement value, the predicted complex wavefront will gradually approach the true complex wavefront on the target plane.

[0066] In this embodiment, a coherent laser with a wavelength of λ = 532nm was used as the light source. A phase-modulated spatial light modulator (HOLOEYE GAEA-2, 3.74μm pixel size) was used to randomly modulate the phase of the complex wave on the target plane. In addition, a CCD camera with a pixel size of 2.4μm was used to capture the measurements. The diffraction distance z between the spatial light modulator and the sensor was about 47mm. During the acquisition process, each time the random pattern loaded on the spatial light modulator was updated, the corresponding measurement value was captured.

[0067] The complex wavefront recovery method proposed in this paper (hereinafter referred to as DeepCWR) can recover the complex wavefront from the measurement value of 2048×2048 pixel size. In the recovery process, the DeepCWR algorithm was implemented on the RTX 3090 graphics card using Python and Pytorch framework. For unsupervised training, the Adam optimization method was used to optimize the network and the propagation distance z, with a learning rate of 3×10 -4 The neural network parameters are initialized randomly.

[0068] The principle of DeepCWR is as follows:

[0069] The mathematical description of a real continuous wave on the target plane is:

[0070]

[0071] Among them, A and are the amplitude and phase of U on the target plane respectively; the random phase pattern M of the spatial light modulator i After phase modulation, the continuous wave propagates the diffraction distance z and reaches the camera, and the measurement value recorded by the camera is expressed as:

[0072]

[0073] in, represents the Hadamard product, and M i Element-wise multiplication of P z represents the wave propagation operator over the diffraction distance z.

[0074] Select Angular Spectrum Method (ASM) as P z The wave propagation model is:

[0075] U′=P z (U(x0,y0)) (3)

[0076] =F -1 (F(U(x0,y0))·H(f x ,f y)) (4)

[0077] Among them, F and F -1 represent Fourier transform and inverse Fourier transform respectively. x0 and y0 are the coordinates of the target plane, and x1 and y1 are the coordinates of the camera plane. x and f y is the coordinate in Fourier domain. The transfer function H(f x ,f y )for:

[0078]

[0079] where k and λ are the wave number and wavelength respectively.

[0080] Assume that the size of the spatial light modulator modulation sequence is N, resulting in N measurement values; the solution to this underdetermined complex wavefront recovery problem is:

[0081]

[0082] This problem is non-convex and nonlinear. In the DeepCWR framework, the predicted continuous waves on the target plane are directly generated by the DNN and are expressed as actual measured values ​​{I1,...,I N} as input, therefore, the solution to the underdetermined complex wavefront recovery problem can be expressed as:

[0083]

[0084] Where G represents the deep neural network DNN, and θ represents the set of neural network layer parameters that need to be learned, including the weights and biases in each neuron. Initially, the parameters of the DNN are randomly initialized. During the optimization process, since the diffraction distance z is unknown, it is also learned synchronously with θ, which realizes the self-calibration function.

[0085] like Figure 2 As shown in Figure 1, the DNN model includes an interleave (Int) layer, a U-Net network, and a de-interleave (De-Int) layer connected in sequence. The core of the DNN is a U-Net, such as Figure 2As shown in a, it is a fully convolutional network. U-Net is able to process data at multiple scales, so low-level and high-level features can be used to achieve reconstruction. An interleaving layer and a de-interleaving layer are added to the front and back ends of U-Net, respectively, to achieve high space-bandwidth product (HSBP). The input measurement values ​​are sequentially passed through the interleaving layer, U-Net network, and de-interleaving layer to obtain an HSBP feature map with a spatial size of H×W, where H and W represent the length and width of the input measurement value image, respectively; the HSBP feature map is further processed by two convolutional layers (CL) to generate predicted phase stacks and amplitude stacks of size H×W×N, respectively, and the N predicted results of the phase stack and amplitude stack are averaged to obtain the predicted phase of the complex wavefront on the target plane. and predicted amplitude The size is H×W×1. This method of averaging N prediction results can effectively reduce the error. Finally, the predicted phase and predicted amplitude Combined to obtain the final predicted complex wavefront as output.

[0086] Figure 2 b shows the structure of U-Net in DeepCWR. The U-Net network consists of an encoding path and a decoding path, including convolution blocks (CB), convolution downsampling blocks (CD), transposed convolution layers (CT) and skip connections (SC). In the encoding path, the first convolution block is used to extract shallow features from the data rearranged by the interleaved layer, and then the second and third convolution blocks are used to extract deep features at multiple scales, where a convolution downsampling block is connected to each of the second and third convolution blocks to reduce the spatial size; in the decoding path, the encoded result is first decoded by the fourth and fifth convolution blocks, each of which is connected to a transposed convolution layer to increase the spatial size, and then at the end of the decoding path, the sixth and seventh convolution blocks are used to further extract high-level features; between the encoding path and the decoding path, three skip connections are used to connect features of the same scale, and the information on both sides is fused by splicing.

[0087] like Figure 2 As shown in (c), the convolutional block (CB) consists of two 3×3 convolutional layers, each followed by a ReLU activation layer and a batch normalization (BN) layer.

[0088] like Figure 2 As shown in Fig. d, the convolutional downsampling block (CD) is constructed by a 6×6 convolutional layer followed by a ReLU activation layer and a batch normalization (BN) layer.

[0089] In order to improve the spatial bandwidth product of DNN, the collaboration of Int and De-Int layers is constructed to rearrange input and output. On the one hand, this collaboration enables DNN to accept measurements of 2000×2000 pixels and output complex waves of the same spatial size, thus matching the HSBP of the optical system. On the other hand, it can also simultaneously reduce the size of data entering U-Net to meet the computational power requirements of U-Net, because the limited computational resources make DNN only accept low SBP input.

[0090] Specifically, if Figure 2 As shown in e, the interleaving layer rearranges the input measurements of size H×W×N into a data, where the third dimension N and (s 2 ×N) represents the number of channels, and s represents the scaling factor. By rearranging the interleaved layers, the spatial dimension is reduced from H×W to The number of channels increases from N to s 2 × N. At the same time, it can be found This means that all the measurement data is retained, just represented in a different form. Each channel consists of a periodic sampling of pixels along the input Figure 2 The spatial dimensions shown in e are separated by a specific distance s. The de-interlacing layer performs the inverse operation of the interlacing layer, converting the dimensions The data is rearranged into data of size H × W × N. Through the above operations, the measured global information is preserved in different data forms to maintain HSBP, while the reduced spatial dimension of the rearranged form is small enough to save computing resources for U-net.

[0091] As an unsupervised DL technique, the proposed DeepCWR optimizes the neural parameters without prior training. Based on the optimization principle of equation (7), the loss function is expressed as:

[0092]

[0093] Among them, the MSE function calculates the predicted amplitude With measured amplitude The mean square error between them; N is the number of measurements; TV is the total variation regularizer, which can suppress noise while maintaining edges and local structures; α is the relative weight of the total variation regularizer, which is recommended to be set in the range of 0.0 to 0.1; is the predicted complex wavefront output by the DNN model.

[0094] During the model optimization process, the gradient of the loss function relative to each neural parameter is calculated through the back-propagation algorithm, and then the gradient descent algorithm is used to update the neural parameters. In this optimization process, the loss is minimized. Approximately This makes the output of the DNN finally produce

[0095] In general, calibration of optical system parameters is crucial for many complex wave reconstruction techniques. In particular, the diffraction distance z between the SLM and the CCD needs to be known accurately to ensure the accuracy of the propagation operator H. However, accurate calibration requires a lot of work, and a slight approximation error may result in the inability to solve the complex wavefront. In the present invention, there is no need to calibrate z. Because the DeepCWR algorithm is fully differentiable, z and the neural parameters of the DNN can be optimized simultaneously to achieve self-calibration. Therefore, in DeepCWR, the diffraction distance parameter z′ can be optimized together with the complex wave reconstruction while starting to learn from an arbitrary range of initial values.

[0096] The self-calibration process is realized by learning the parameter z′ so that z′ approaches the given and unknown actual diffraction distance z. Then the optimization process in equation (7) can be expressed as:

[0097]

[0098] Quantitative complex wavefront reconstruction technology can provide quantitative three-dimensional shape information of an object. Therefore, quantitative complex wavefront reconstruction can not only perform three-dimensional imaging, but also perform quantitative three-dimensional metrology or metrology that is very important for manufacturing. Quantitative measurement means that the measurement of the three-dimensional size of an object is accurate enough to be compared with its true value in a quantitative manner. In order to test the capability of DeepCWR proposed in the present invention, this embodiment was experimented on different targets. In each experiment, the same system parameters were also used to implement off-axis digital holography DH for comparison.

[0099] First, we experimented with the standard 1951 United States Air Force resolution plate (USAF) as the target. N = 16 measurements were captured as input to DeepCWR, and the results are shown in Figure 2. Figure 3 The continuous wave retrieved by DeepCWR is compared with the digital hologram in terms of amplitude and phase.

[0100] The amplitude distributions of DeepCWR and digital holography (DH) are listed in Figure 3 Left side. First row ( Figure 3 a1-a2), the second line ( Figure 3 b1-b2) and the third row ( Figure 3 c1-c2) show the 2048×2048 pixel amplitudes of DeepCWR and DH, respectively, at magnifications of 5 times and 20 times. From these amplitude distributions, it can be seen that both DeepCWR and DH are able to resolve all features of groups 2-5 in the USAF target.

[0101] Figure 3 c1 and c2 show that the best resolved feature of DeepCWR is the 2nd element of group 6 (G6E2), while that of DH is the 1st element of group 6 (G6E1). The lateral resolutions corresponding to G6E2 and G6E1 are 71.8lp / mm and 64.0lp / mm, respectively. This indicates that the proposed DeepCWR has better lateral resolution capability than DH under the same optical system parameters. At the same time, DeepCWR has fewer artifacts and a clearer background with less noise. In order to highlight the details, the Figure 3 Profiles of purple and green lines in b1 and b2 ( Figure 3 e1), and Figure 3 Profiles of purple and green lines in c1 and c2 ( Figure 3 f1). Figure 3 Comparison of the profiles of e1 shows that DeepCWR provides better amplitude contrast than DH. Figure 3 The comparison of f1 profiles shows that DeepCWR is able to resolve G6E2, while DH cannot.

[0102] The phases of DeepCWR and DH are listed in Figure 3 Right. First row ( Figure 3 a3-a4), the second line ( Figure 3 b3-b4) and the third row ( Figure 3 c3-c4) are the magnification stages of ×5 and ×20 for DeepCWR and DH, respectively, with 2048 × 2048 pixels. From these stages, it can be seen that DeepCWR is able to resolve G6E2, while DH is able to resolve G6E1. This indicates that the phase lateral resolution of DeepCWR is 71.8 lp / mm, which is also higher than 64.0 lp / mm for DH. To demonstrate the quantitative capability of height measurement, Figure 3 The phase-derived height profiles of the red and blue lines in a3 and a4 are plotted on Figure 3 d. Considering that the true height of this element is about 100nm, these two profiles show that both DeepCWR and DH are able to quantitatively measure the height with high accuracy. While the height profile retrieved by DH is more noisy, the height profile of DeepCWR is less noisy and has better phase contrast. To highlight the capability of DeepCWR in high-resolution elements, Figure 3 Height profiles of the yellow and black lines in b3 and b4 ( Figure 3 e2), and Figure 3 Height profiles of the yellow and black lines in C3 and C4 ( Figure 3 f2). Figure 3 Comparison of e2 shows that DeepCWR not only provides better phase contrast for high-resolution elements, but also has lower noise. Figure 3 The comparison of f2 profiles shows that the proposed DeepCWR can resolve the phase of G6E2, while DH cannot.

[0103] Next, in order to further study the quantitative measurement capability of DeepCWR, Figure 4 The step is located at the center of the sample and the true height provided by the manufacturer is 104.30nm. Figure 4 a1, a2, and a3 show the amplitude, phase, and 3D morphology obtained from the phase reconstructed by DeepCWR, respectively. Figure 4 b1 to b3 show the amplitude, phase, and 3D morphology of DH. Figure 4 The height profiles of the red and blue lines in a2 and b2 are as follows Figure 4 c. The height measured by DeepCWR is 104.6nm, while the height obtained by DH is 109.5nm. Considering that the true height is 104.30nm, DeepCWR provides a more accurate quantitative measurement result.

[0104] In addition, during the experiment, the air suspension function of the optical experimental platform was turned off in this embodiment to check the anti-interference ability of the proposed DeepCWR compared with DH. Figure 4 There is an undesirable background fluctuation phase in b2, as shown by the dashed black line, and the profile is as follows Figure 4 d. However, Figure 4 There is no such background fluctuation in a2, as shown by the dotted yellow line, and the profile is also Figure 4 As shown in d. Figure 4 The comparison of d shows that DeepCWR has a more stable background than DH and thus has better anti-interference ability. This is because DH is interferometric, while DeepCWR is not.

[0105] In summary, the complex wavefront reconstruction method DeepCWR proposed in the present invention effectively solves the shortcomings of existing complex wavefront reconstruction technology in terms of accuracy, anti-interference, applicability and spatial bandwidth product through innovations such as spatial light modulator assistance, unsupervised deep learning and high spatial bandwidth product design. Therefore, DeepCWR is expected to become an effective solution to the lack of high-precision, high-anti-interference, quantitative three-dimensional measurement technology suitable for complex value objects and high spatial bandwidth product in industrial sites, providing new technical support for quality control and feedback control in the manufacturing industry.

[0106] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning, characterized in that: The following steps are involved: S1, collects the actual measurement values ​​of the optical system; S2, inputting the actual measurement value into the DNN model, and the DNN model directly outputs the predicted complex wavefront on the target plane in an end-to-end manner, wherein the predicted complex wavefront includes amplitude and phase; S3, the predicted complex wave is further subjected to the same digital spatial light modulator modulation and digital wave propagation as in the actual acquisition process of the optical system, and a predicted measurement value on the digital camera plane is obtained; S4, calculating the loss function based on the difference between the predicted measurement value and the actual measurement value; S5, repeating steps S2-S4, iteratively optimizing the parameters of the DNN model by minimizing the loss function, and iteratively outputting the final complex wavefront prediction result.

2. A quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 1, characterized in that: The optical system includes an object to be measured, a 4f system, a beam splitter, a spatial light modulator and a camera. The object to be measured is imaged on the spatial light modulator through the 4f system. The plane of the spatial light modulator is named as the target plane. The spatial light modulator modulates the complex wavefront amplitude of the object to be measured on this plane. The modulated complex wavefront reaches the camera after traveling a light wave propagation distance z. Since the spatial light modulator works in reflection mode, the complex wavefront is guided to the camera through a beam splitter. The camera captures the diffraction intensity pattern as a measurement value. During the operation of the optical system, the spatial light modulator sequentially generates a series of random phase modulation patterns, and the camera synchronously captures the corresponding measurement values.

3. A quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 1, characterized in that: The mathematical description of the real continuous wave on the target plane is: Among them, A and are the amplitude and phase of U on the target plane respectively; the random phase pattern M of the spatial light modulator i After phase modulation, the continuous wave propagates the diffraction distance z and reaches the camera, and the measurement value recorded by the camera is expressed as: in, ○ represents the Hadamard product, which means and M i Element-wise multiplication of P z represents the wave propagation operator at the diffraction distance z; Assuming that the size of the spatial light modulator modulation sequence is N, N measurement values ​​are obtained; the predicted continuous wave on the target plane is directly generated by the DNN model and is represented by the actual measurement values ​​{I1,...,I N } as input, the solution to the underdetermined complex wavefront recovery problem is expressed as: Among them, G represents the deep neural network DNN, and θ represents the set of neural network layer parameters that need to be learned, including the weights and biases in each neuron.

4. A quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 3, characterized in that: In the DNN model optimization process, the diffraction distance parameter z′ is synchronously learned together with the model parameter θ from any initial value range. By learning the parameter z′, z′ is made to approach the given and unknown actual diffraction distance 2, and the solution to the underdetermined complex wavefront recovery problem is reformulated as:

5. The quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 1, characterized in that: The DNN model includes an interleaving layer, a U-Net network and a de-interleaving layer connected in sequence, wherein the interleaving layer and the de-interleaving layer are used to achieve a high space-bandwidth product, and the input measurement value is sequentially passed through the interleaving layer, the U-Net network and the de-interleaving layer to obtain a HSBP feature map with an H×W spatial size, wherein H and W represent the length and width of the input measurement value image respectively; the HSBP feature map is further processed through two convolutional layers to generate a predicted phase stack and an amplitude stack of sizes H×W×N respectively, and the N predicted results of the phase stack and the amplitude stack are averaged respectively to obtain the predicted phase of the complex wavefront on the target plane and predicted amplitude The size is H×W×1; the predicted phase and predicted amplitude Combined to obtain the final predicted complex wavefront as output.

6. A quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 5, characterized in that: The interleaving layer rearranges the input measurements of size H×W×N into a data, where the third dimension N and (s 2 ×N) represents the number of channels, and s represents the scaling factor. By rearranging the interleaved layers, the spatial dimension is reduced from H×W to The number of channels increases from N to s 2 ×N; the de-interlacing layer performs the inverse operation of the interlacing layer, The data is rearranged into data of size H×W×N.

7. The quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 5, characterized in that: The U-Net network is a fully convolutional network, which consists of an encoding path and a decoding path, including convolution blocks, convolution downsampling blocks, transposed convolution layers and jump connections; in the encoding path, shallow features are first extracted from the data rearranged by the interleaved layer through the first convolution block, and then deep features are extracted at multiple scales through the second and third convolution blocks, wherein a convolution downsampling block is connected after the second and third convolution blocks to reduce the spatial size; in the decoding path, the encoded result is first decoded through the fourth and fifth convolution blocks, and a transposed convolution layer is respectively connected after the fourth and fifth convolution blocks to increase the spatial size, and then at the end of the decoding path, high-level features are further extracted through the sixth and seventh convolution blocks; between the encoding path and the decoding path, features of the same scale are connected through three jump connections, and the information on both sides is fused by splicing.

8. The quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 6, characterized in that: The convolutional block consists of two 3×3 convolutional layers, each followed by a ReLU activation layer and a batch normalization layer.

9. The quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 6, characterized in that: The convolutional downsampling block is constructed by a 6×6 convolutional layer followed by a ReLU activation layer and a batch normalization layer.

10. The quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 1, characterized in that: The loss function is expressed as: Among them, the MSE function calculates the predicted amplitude With measured amplitude The mean square error between , N is the number of measurements, TV is the total change regularizer, α is the relative weight of the total change regularizer, is the predicted complex wavefront output by the DNN model; During the model optimization process, the gradient of the loss function relative to each neural parameter is calculated through the back-propagation algorithm, and then the gradient descent algorithm is used to update the neural parameters.

Images