Quantitative and high anti-interference complex wavefront recovery method based on unsupervised deep learning

Abstract

The application relates to a quantitative and high anti-interference complex wavefront recovery method based on unsupervised deep learning, which comprises the following steps: collecting actual measurement values of an optical system; inputting the actual measurement values into a DNN model, and directly outputting a predicted complex wavefront on a target plane in an end-to-end manner; the predicted complex wavefront further undergoes the same digital spatial light modulator modulation and digital wave propagation as in the actual acquisition process of the optical system to obtain predicted measurement values on a digital camera plane; a loss function is calculated based on the difference between the predicted measurement values and the actual measurement values; the above steps are repeated, the parameters of the DNN model are iteratively optimized by minimizing the loss function, and the final complex wavefront prediction result is outputted after iteration. Compared with the prior art, the application has the advantages of high precision, no need to obtain a large amount of training data, high spatial bandwidth product, etc.

Description

Technical Field

[0001] The present invention relates to the technical field of complex wavefront reconstruction, and in particular to a quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning. Background Art

[0002] The fundamental characteristic of a light field can be a complex wavefront (CW), defined as U(x,y) = A(x,y)·exp[jφ(x,y)], where A(x,y) and φ(x,y) represent the amplitude and phase of U(x,y), respectively. In coherent imaging, once a continuous wave is obtained on one plane of the light field, it can be obtained on any plane of the light field through numerical propagation. The continuous wave reflected by an object contains surface information about the object, with its three-dimensional surface shape encoded as φ(x,y) and its surface reflectivity encoded as A(x,y). Therefore, obtaining the continuous wave, particularly its phase, is crucial for three-dimensional metrology. However, cameras such as charge-coupled devices (CCDs) and complementary metal-oxide semiconductors (CMOS) can only detect the intensity of the light field, not its phase, resulting in loss of continuous wave information. To address this issue, various techniques have been developed, categorized as complex wavefront reconstruction (CWR). CWR techniques can recover the complex wavefront of light from the intensity information recorded by the camera.

[0003] Generally speaking, complex wavefront reconstruction techniques can be divided into two categories: interferometry and non-interferometry. Digital holography (DH) is a typical interferometry technique. Due to the nature of interferometry, DH allows for quantitative complex wavefront reconstruction with high-precision traceability of wavelengths. Therefore, DH has become the benchmark for quantitative 3D metrology. However, DH and other interferometry complex wavefront reconstruction techniques share a major limitation: the interferometric structure is highly sensitive to external interferences (such as vibrations, fast airflow, etc.). This seriously affects the accuracy of the measurement. Therefore, in industries with severe interference, interferometric techniques are difficult to apply outside the laboratory for field application. On the other hand, industry lacks and urgently needs such high-precision, highly anti-interference 3D quantitative metrology techniques for quality control. For field purposes, non-interferometric techniques with better anti-interference performance are required.

[0004] The second category involves non-interferometric complex wavefront reconstruction techniques. Among these, the Shack-Hartmann wavefront sensor (SHWS) features a simple structure based on geometric optics. The SHWS utilizes a microlens array to divide the wavefront into multiple sub-wavefronts, deriving the average phase slope of each sub-wavefront. Due to this division, the spatial resolution of the retrieved continuous wave is very low. The SHWS is limited to continuous waves with smooth variations.

[0005] After SHWS, non-learning computational complex wavefront reconstruction techniques were proposed. These methods directly obtain complex wavefronts from measurements by non-learning computational methods. However, these methods all use or inherit the GS algorithm, so they have low computational capacity when solving the ill-posed complex wavefront reconstruction problem. Therefore, more powerful computational tools are needed.

[0006] In recent years, deep learning (DL) has rapidly developed as a more powerful computational tool. DL has been introduced into the field of complex wavefront reconstruction. The paradigm of solving the ill-posed complex wavefront reconstruction problem using deep neural networks (DNNs) is defined as DL-based CWR technology. Most of these technologies train DNNs in a supervised manner, which requires a large amount of training data. However, in practice, it is sometimes difficult to obtain a large amount of data and the true value of the complex wavefront. Therefore, it is difficult to implement the supervised manner. To solve this problem, unsupervised DL-based CWR technology was proposed, which introduced the physical model of measurement in the optical system into the DNN. Unlike the supervised manner, the unsupervised manner does not require true values and training data sets. PhysenNet pioneered the work of unsupervised DL-based CWR, but was limited to pure phase objects and could not obtain the wavefront of complex-valued objects. Subsequently, a dual-input Y-Net was proposed for obtaining the wavefront of complex-valued objects, but the Y-Net still has room for improvement in imaging quality.

[0007] For all the above deep learning-based complex wavefront reconstruction techniques, there is a major limitation, i.e., the limited spatial bandwidth product (SBP), which is actually the number of pixels of the measurement. Due to limited computational resources, the input measurement must be cropped to a smaller size, such as 512x512 pixels, for the DNN to process. This results in a low SBP for the DNN. While the most advanced optical systems can easily reach more than 2000x2000 pixels. Therefore, there is a mismatch between the low SBP of the DNN and the high SBP (HSBP, defined as an SBP greater than 2000x2000) of the optical system. The low SBP limits the ability of DL-based techniques to obtain high lateral resolution and large field of view.

[0008] The key advantages and disadvantages of existing complex wavefront reconstruction techniques are listed in Table 1.

[0009] Advantages and disadvantages of complex wavefront reconstruction techniques

[0010]

[0011]

[0012] However, all existing deep learning-based non-interferometric complex wavefront reconstruction techniques do not simultaneously possess these four desired advantages. Furthermore, most existing complex wavefront reconstruction techniques require precise knowledge of the diffraction distance parameter in the optical system, which is difficult and time-consuming to achieve. To address this issue, BlindNet recently automatically calibrated this parameter during DNN optimization. However, this technique is limited to pure phase objects, and there is still room for improvement in accuracy and learning range. Summary of the Invention

[0013] The purpose of the present invention is to address the limitations of existing complex wavefront reconstruction technology in terms of practicality, accuracy and space-bandwidth product, and to provide a quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning.

[0014] The purpose of the present invention can be achieved by the following technical solutions:

[0015] A quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning, comprising the following steps:

[0016] S1, collects the actual measurement values ​​of the optical system;

[0017] S2, inputting the actual measurement value into the DNN model, and the DNN model directly outputs the predicted complex wavefront on the target plane in an end-to-end manner, wherein the predicted complex wavefront includes amplitude and phase;

[0018] S3, the predicted complex wavefront further undergoes the same digital spatial light modulator modulation and digital wave propagation as in the actual acquisition process of the optical system, and obtains a predicted measurement value on the digital camera plane;

[0019] S4, calculates the loss function based on the difference between the predicted measurement value and the actual measurement value;

[0020] S5, repeat steps S2-S4, iteratively optimize the parameters of the DNN model by minimizing the loss function, and iteratively output the final complex wavefront prediction result.

[0021] The optical system includes a measured object, a 4f system, a beam splitter, a spatial light modulator, and a camera. The measured object is imaged on the spatial light modulator through the 4f system. The plane of the spatial light modulator is named the target plane. The spatial light modulator modulates the complex wavefront amplitude of the measured object on this plane. The modulated complex wavefront reaches the camera after traveling a light wave propagation distance z. Since the spatial light modulator operates in reflection mode, the complex wavefront is guided to the camera through a beam splitter. The camera captures the diffraction intensity pattern as a measurement value. During the operation of the optical system, the spatial light modulator sequentially generates a series of random phase modulation patterns, and the camera synchronously captures the corresponding measurement values.

[0022] The mathematical description of a true continuous wave on the target plane is:

[0023]

[0024] Among them, A and are the amplitude and phase of U on the target plane respectively; the random phase pattern M of the spatial light modulator i After phase modulation, the continuous wave propagates the diffraction distance z and reaches the camera. The measurement value recorded by the camera is expressed as:

[0025]

[0026] in, represents the Hadamard product, and M i Element-wise multiplication of P z represents the wave propagation operator on the diffraction distance z;

[0027] Assuming that the size of the spatial light modulator modulation sequence is N, N measurement values ​​are obtained; the predicted continuous wave on the target plane is directly generated by the DNN model and the actual measurement values ​​{I1,...,I N} as input, the solution to the underdetermined complex wavefront recovery problem is expressed as:

[0028]

[0029] Here, G represents the deep neural network DNN, and θ represents the set of neural network layer parameters that need to be learned, including the weights and biases in each neuron.

[0030] During the DNN model optimization process, the diffraction distance parameter z′ is learned synchronously with the model parameter θ from an arbitrary initial value range. By learning the parameter z′, z′ is made to approach the given and unknown actual diffraction distance z, and the solution to the underdetermined complex wavefront recovery problem is reformulated as:

[0031]

[0032] The DNN model includes an interleaving layer, a U-Net network and a de-interleaving layer connected in sequence, wherein the interleaving layer and the de-interleaving layer are used to achieve a high space-bandwidth product. The input measurement value passes through the interleaving layer, the U-Net network and the de-interleaving layer in sequence to obtain an HSBP feature map with an H×W spatial size, wherein H and W represent the length and width of the input measurement value image respectively; the HSBP feature map is further processed by two convolutional layers to generate a predicted phase stack and amplitude stack of size H×W×N respectively, and the N prediction results of the phase stack and the amplitude stack are averaged to obtain the predicted phase of the complex wavefront on the target plane. and predicted amplitude The size is H×W×1; the predicted phase and predicted amplitude Combined to obtain the final predicted complex wavefront as output.

[0033] The interleaving layer rearranges the input measurements of size H×W×N into Data, where the third dimension N and (s 2 ×N) represents the number of channels, and s represents the scaling factor; by rearranging the interleaved layers, the spatial dimension is reduced from H×W to The number of channels increases from N to s 2 ×N; the de-interleaving layer performs the inverse operation of the interleaving layer, The data is rearranged into data of size H×W×N.

[0034] The U-Net network is a fully convolutional network, consisting of an encoding path and a decoding path, including convolution blocks, convolution downsampling blocks, transposed convolution layers and jump connections. In the encoding path, shallow features are first extracted from the data rearranged by the interleaved layer through the first convolution block, and then deep features are extracted at multiple scales through the second and third convolution blocks, wherein the second and third convolution blocks are each connected to a convolution downsampling block to reduce the spatial size. In the decoding path, the encoded result is first decoded through the fourth and fifth convolution blocks, and the fourth and fifth convolution blocks are each connected to a transposed convolution layer to increase the spatial size. Then, at the end of the decoding path, high-level features are further extracted through the sixth and seventh convolution blocks. Between the encoding path and the decoding path, features of the same scale are connected through three jump connections, and the information on both sides is fused by splicing.

[0035] The convolutional block consists of two 3×3 convolutional layers, each followed by a ReLU activation layer and a batch normalization layer.

[0036] The convolution downsampling block is constructed by a 6x6 convolution layer, followed by a ReLU activation layer and a batch normalization layer.

[0037] The loss function is represented as:

[0038]

[0039] Where the MSE function calculates the mean square error between the predicted amplitude and the measured amplitude , N is the number of measurements, TV is the total variation regularizer, and alpha is the relative weight of the total variation regularizer. is the predicted complex wavefront output by the DNN model.

[0040] During the model optimization process, the gradient of the loss function with respect to each neural parameter is calculated by the backpropagation algorithm, and then the neural parameters are updated using the gradient descent algorithm.

[0041] Compared with the prior art, the present application has the following beneficial effects:

[0042] (1) High-precision quantitative complex wavefront reconstruction: The present application increases the measurement constraint information by introducing random phase modulation through a spatial light modulator (SLM), effectively alleviating the underdetermination of the CWR problem. Compared with existing GS algorithms and CDI technology, this complex wavefront modulation based on SLM can provide more efficient constraints, thus achieving faster CWR convergence and significantly improving the precision of complex wavefront reconstruction.

[0043] (2) High interference resistance: The present application is based on a non-interferometric design, which has strong robustness to external vibrations and other disturbances, and can work stably in industrial sites.

[0044] (3) Suitable for complex-valued objects: The present application can handle complex-valued objects commonly found in industrial sites, not limited to pure phase objects, which greatly expands the application range.

[0045] (4) High spatial bandwidth product (HSBP): The present application learns from the IPMNet's downsampling-upsampling (D-U) module, uses interleaving layers and deinterleaving layers to achieve high spatial bandwidth product, and can achieve high lateral resolution and large field of view, meeting the resolution and field of view requirements of three-dimensional measurement in industrial sites.

[0046] (5) Automatic calibration of parameters: The present application can automatically calibrate the key parameters of the optical system while optimizing the deep learning, without the need for tedious manual settings.

[0047] (6) No need for a large amount of training data: Unlike supervised DL-CWR, which requires a large amount of training data, the present invention utilizes physical model constraints to process wavefront reconstruction of complex value objects without the need for training data. This overcomes the limitation of supervised DL-CWR in practical applications that it is difficult to obtain a large amount of training data. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 Schematic diagram of the working process of the complex wavefront recovery method of the present invention;

[0049] Figure 2 Schematic diagram of the DNN model structure of the present invention;

[0050] Figure 3 FIG1 is a schematic diagram of experimental results of quantitative measurement using USAF as the object in one embodiment;

[0051] Figure 4 Schematic diagram of experimental results of quantitative measurement using a stepped object in one embodiment. DETAILED DESCRIPTION

[0052] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0053] Based on the analysis of existing technologies, a prospective complex wavefront reconstruction technology should have the following functions:

[0054] 1) High-precision quantitative complex wavefront reconstruction;

[0055] 2) Strong robustness to environmental interference;

[0056] 3) Applicable to complex value objects, because pure phase objects are very rare in industry;

[0057] 4) High sampling bandwidth and field of view (HSBP), providing sufficient lateral resolution and field of view.

[0058] Based on this, this embodiment provides a quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning, such as Figure 1 As shown, the following steps are included:

[0059] S1, collects the actual measurement values ​​of the optical system;

[0060] S2, inputting the actual measurement value into the DNN model, and the DNN model directly outputs the predicted complex wavefront on the target plane in an end-to-end manner, wherein the predicted complex wavefront includes amplitude and phase;

[0061] S3, the predicted complex wavefront further undergoes the same digital spatial light modulator modulation and digital wave propagation as in the actual acquisition process of the optical system, and obtains a predicted measurement value on the digital camera plane;

[0062] S4, calculates the loss function based on the difference between the predicted measurement value and the actual measurement value;

[0063] S5, repeat steps S2-S4, iteratively optimize the parameters of the DNN model by minimizing the loss function, and iteratively output the final complex wavefront prediction result.

[0064] like Figure 1 As shown in Figure a, the optical system consists of an object under test, a 4f system, a beam splitter (BS), a spatial light modulator (SLM), and a camera. The object under test is imaged on the SLM via the 4f system; therefore, the target plane and the SLM plane are conjugate. The SLM plane is called the target plane, where the SLM modulates the amplitude of the complex wavefront of the object under test. The modulated complex wavefront reaches the camera after traveling a light wave propagation distance z. Since the SLM operates in reflection mode, a beam splitter directs the complex wavefront to the camera. The camera captures the diffraction intensity pattern as the measurement value. During the operation of the optical system, the SLM sequentially generates a series of random phase modulation patterns, and the camera synchronously captures the corresponding measurement values.

[0065] After obtaining the measurement, enter it into Figure 1 The processing flow in b. Figure 1 The measurements in b3 are fed into the DNN as input, as Figure 1 As shown in b4, DNN directly outputs the predicted complex wave on the target plane in an end-to-end manner, including amplitude and phase, as shown in Figure 1 b5 and Figure 1 As shown in Figure 6, the predicted complex wavefront then undergoes further digital spatial light modulator modulation (b7) and digital wave propagation over distance z, identical to the actual acquisition process, resulting in a sequence of predicted measurement values ​​(b8) on the digital camera plane. A loss function is calculated based on the difference between the predicted measurement values ​​(b8) and the actual measurement values ​​(b3). By minimizing the loss function, the parameters of the DNN are optimized. This algorithm iterates, and as the predicted measurement values ​​gradually approach the actual measurement values, the predicted complex wavefront also gradually approaches the actual complex wavefront on the target plane.

[0066] In this embodiment, the experiment uses a coherent laser with a wavelength of λ = 532 nm as the light source. A phase-modulated spatial light modulator (HOLOEYE GAEA-2, 3.74 μm pixel size) is used to randomly modulate the phase of the complex wave on the target plane. In addition, a CCD camera with a pixel size of 2.4 μm is used to capture the measurement values. The diffraction distance z between the spatial light modulator and the sensor is about 47 mm. During the acquisition process, the corresponding measurement values are captured every time the random pattern loaded on the spatial light modulator is updated.

[0067] The complex wavefront recovery method proposed in the present application (hereinafter referred to as DeepCWR) can recover the complex wavefront from the measurement values with a size of 2048 × 2048 pixels. In the recovery process, the DeepCWR algorithm is implemented on an RTX 3090 graphics card using Python and Pytorch framework. For unsupervised training, the Adam optimization method is used to optimize the network and the propagation distance z, and the learning rate is 3 × 10 -4 The neural network parameters are randomly initialized.

[0068] The principle of DeepCWR is as follows:

[0069] The mathematical description of the real continuous wave on the target plane is:

[0070]

[0071] Where A and are the amplitude and phase of U on the target plane, respectively; the continuous wave is phase-modulated by the random phase pattern M i of the spatial light modulator, propagates and diffracts a distance z, and reaches the camera, and the measurement value recorded by the camera is represented as:

[0072]

[0073] Where, represents the Hadamard product, represents the element-level multiplication of i and M z , and P z represents the wave propagation operator with a diffraction distance of z.

[0074] The angular spectrum method (ASM) is selected as the wave propagation model of P z , and its formula is:

[0075] U' = P z (U(x0,y0)) (3)

[0076] = F -1 (F(U(x0,y0)) · H(f x ,f y)) (4)

[0077] Among them, F and F -1 Represents Fourier transform and inverse Fourier transform respectively. x0 and y0 are the coordinates of the target plane, and x1 and y1 are the coordinates of the camera plane. x and f y is the coordinate in Fourier domain. The transfer function H(f x ,f y )for:

[0078]

[0079] where k and λ are the wave number and wavelength respectively.

[0080] Assume that the size of the spatial light modulator modulation sequence is N, resulting in N measurement values; the solution to this underdetermined complex wavefront recovery problem is:

[0081]

[0082] This problem is non-convex and nonlinear. In the DeepCWR framework, the predicted continuous wave on the target plane is directly generated by the DNN and is expressed as the actual measured values ​​{I1,...,I N} as input, so the solution to the underdetermined complex wavefront recovery problem can be expressed as:

[0083]

[0084] Here, G represents the deep neural network (DNN), and θ represents the set of neural network layer parameters to be learned, including the weights and biases in each neuron. Initially, the DNN parameters are randomly initialized. During the optimization process, since the diffraction distance z is unknown, it is also learned synchronously with θ, achieving self-calibration.

[0085] like Figure 2 As shown in , the DNN model includes interleaving (Interleave, Int) layers, U-Net networks and de-interleaving (De-Interleave, De-Int) layers connected in sequence. The core of DNN is a U-Net, as shown in Figure 2As shown in a, it is a fully convolutional network. U-Net is able to process data at multiple scales, so it can use low-level and high-level features to achieve reconstruction. An interleaving layer and a de-interleaving layer are added to the front end and back end of U-Net, respectively, to achieve high space-bandwidth product (HSBP). The input measurement value passes through the interleaving layer, U-Net network and de-interleaving layer in sequence to obtain an HSBP feature map with H×W spatial size, where H and W represent the length and width of the input measurement value image, respectively; the HSBP feature map is further processed by two convolutional layers (CL) to generate predicted phase stacks and amplitude stacks of size H×W×N, respectively. The N predicted results of the phase stack and amplitude stack are averaged to obtain the predicted phase of the complex wavefront on the target plane. and predicted amplitude The size is H×W×1. This method of averaging N prediction results can effectively reduce the error. Finally, the predicted phase and predicted amplitude Combined to obtain the final predicted complex wavefront as output.

[0086] Figure 2 Figure b shows the structure of the U-Net in DeepCWR. The U-Net network consists of an encoding path and a decoding path, including convolutional blocks (CB), convolutional downsampling blocks (CD), transposed convolutional layers (CT), and skip connections (SC). In the encoding path, the first convolutional block first extracts shallow features from the permutated data of the interleaved layer. Then, the second and third convolutional blocks extract deep features at multiple scales. Each of the second and third convolutional blocks is followed by a convolutional downsampling block to reduce the spatial size. In the decoding path, the encoded result is first decoded by the fourth and fifth convolutional blocks, each of which is followed by a transposed convolutional layer to increase the spatial size. At the end of the decoding path, the sixth and seventh convolutional blocks further extract high-level features. Between the encoding and decoding paths, three skip connections are used to connect features of the same scale, fusing the information on both sides through splicing.

[0087] like Figure 2 As shown in Figure c, the convolutional block (CB) consists of two 3×3 convolutional layers, each followed by a ReLU activation layer and a batch normalization (BN) layer.

[0088] like Figure 2 As shown in Figure d, the convolutional downsampling block (CD) is constructed by a 6×6 convolutional layer followed by a ReLU activation layer and a batch normalization (BN) layer.

[0089] To improve the space-bandwidth product of the DNN, a collaboration between the Int and De-Int layers was constructed to rearrange the input and output. This collaboration enables the DNN to accept measurements of 2000×2000 pixels and output complex waves of the same spatial size, thus matching the high-speed bandwidth (HSBP) of the optical system. Furthermore, it also reduces the size of the data entering the U-Net, meeting its computational requirements. Limited computational resources limit the DNN to accepting inputs with low SBP.

[0090] Specifically, if Figure 2 As shown in e, the interleaving layer rearranges the input measurements of size H×W×N into a Data, where the third dimension N and (s 2 ×N) represents the number of channels, and s represents the scaling factor. By rearranging the interleaved layers, the spatial dimension is reduced from H×W to The number of channels increases from N to s 2 × N. At the same time, it can be found This means that all the measurement data are retained, just represented in a different form. Each channel consists of a periodic sampling of pixels along the input. Figure 2 The spatial dimensions shown in e are separated by a specific distance s. The deinterlacing layer performs the inverse operation of the interlacing layer, The data is rearranged into data of size H × W × N. Through the above operations, the measured global information is preserved in a different data form to maintain HSBP, while the reduced spatial dimension of the rearranged form is small enough to save computing resources for U-net.

[0091] As an unsupervised DL technique, the proposed DeepCWR optimizes the neural parameters without pre-training. Based on the optimization principle of equation (7), the loss function is expressed as:

[0092]

[0093] Among them, the MSE function calculates the predicted amplitude With measured amplitude The mean square error between them; N is the number of measurements; TV is the total variation regularizer, which can suppress noise while maintaining edges and local structures; α is the relative weight of the total variation regularizer, which is recommended to be set in the range of 0.0 to 0.1; is the predicted complex wavefront output by the DNN model.

[0094] During the model optimization process, the gradient of the loss function relative to each neural parameter is calculated through the back-propagation algorithm, and then the gradient descent algorithm is used to update the neural parameters. In this optimization process, the loss is minimized. Approximate This makes the output of DNN finally produce

[0095] In general, calibration of optical system parameters is crucial for many complex wave reconstruction techniques. In particular, the diffraction distance z between the SLM and the CCD needs to be known accurately to ensure the accuracy of the propagation operator H. However, accurate calibration requires a lot of work, and a slight approximation error may result in the complex wavefront being unable to be solved. In the present invention, there is no need to calibrate z. Because the DeepCWR algorithm is fully differentiable, z and the neural parameters of the DNN can be optimized simultaneously to achieve self-calibration. Therefore, in DeepCWR, the diffraction distance parameter z′ can be optimized together with the complex wave reconstruction while starting from an arbitrary range of initial values.

[0096] The self-calibration process is achieved by learning the parameter z′ so that z′ approaches the given and unknown actual diffraction distance z. The optimization process in equation (7) can be expressed as:

[0097]

[0098] Quantitative complex wavefront reconstruction technology can provide quantitative three-dimensional shape information of an object. Therefore, quantitative complex wavefront reconstruction can not only perform three-dimensional imaging, but also perform quantitative three-dimensional metrology or metrology that is very important for manufacturing. Quantitative measurement means that the three-dimensional size of an object is measured accurately enough to be compared with its true value in a quantitative manner. In order to test the capability of DeepCWR proposed in the present invention, this embodiment conducted experiments on different targets. In each experiment, the same system parameters were also used to implement off-axis digital holography DH for comparison.

[0099] First, we experimented with the standard 1951 USAF resolution plate (USAF) as the target. N = 16 measurements were captured as input to DeepCWR, and the results are shown in Figure 2. Figure 3 The continuous wave retrieved by DeepCWR is compared with the digital hologram in terms of amplitude and phase.

[0100] The amplitude distributions of DeepCWR and digital holography (DH) are listed in Figure 3 Left. First row ( Figure 3 a1-a2), the second line ( Figure 3 b1-b2) and the third row ( Figure 3 Figures c1-c2) show the 2048×2048 pixel amplitudes of DeepCWR and DH, respectively, at magnifications of 5x and 20x. As can be seen from these amplitude distributions, both DeepCWR and DH are able to resolve all features in groups 2-5 of the USAF target.

[0101] Figure 3 c1 and c2 show that the best resolved feature of DeepCWR is the second element of group 6 (G6E2), while that of DH is the first element of group 6 (G6E1). The lateral resolutions corresponding to G6E2 and G6E1 are 71.8lp / mm and 64.0lp / mm, respectively. This shows that the proposed DeepCWR has better lateral resolution capability than DH under the same optical system parameters. At the same time, DeepCWR has fewer artifacts and a clearer background with less noise. In order to highlight the details, the Figure 3 The profiles of the purple and green lines in b1 and b2 ( Figure 3 e1), and Figure 3 The profiles of the purple and green lines in C1 and C2 ( Figure 3 f1). Figure 3 Comparison of the profiles of e1 shows that DeepCWR provides better amplitude contrast than DH. Figure 3 The comparison of f1 profiles shows that DeepCWR is able to resolve G6E2, while DH cannot.

[0102] The phases of DeepCWR and DH are listed in Figure 3 Right. First row ( Figure 3 a3-a4), the second line ( Figure 3 b3-b4) and the third row ( Figure 3 c3-c4) are 2048×2048 pixels, and the magnification stages of ×5 and ×20 for DeepCWR and DH, respectively. From these stages, it can be seen that DeepCWR is able to resolve G6E2, while DH is able to resolve G6E1. This indicates that the phase lateral resolution of DeepCWR is 71.8 lp / mm, which is also higher than 64.0 lp / mm of DH. To demonstrate the quantitative ability of height measurement, Figure 3 The phase-derived height profiles of the red and blue lines in a3 and a4 are plotted on Figure 3 d. Considering that the true height of this element is about 100nm, these two profiles show that both DeepCWR and DH are able to quantitatively measure the height with high accuracy. While the height profile retrieved by DH is more noisy, the height profile of DeepCWR is less noisy and has better phase contrast. To highlight the capability of DeepCWR in high-resolution elements, Figure 3 Height profiles of yellow and black lines in b3 and b4 ( Figure 3 e2), and Figure 3 Height profiles of yellow and black lines in C3 and C4 ( Figure 3 f2). Figure 3 Comparison with e2 shows that DeepCWR not only provides better phase contrast for high-resolution elements, but also has lower noise. Figure 3 The comparison of f2 profiles shows that the proposed DeepCWR can resolve the phase of G6E2, while DH cannot.

[0103] Next, in order to further study the quantitative measurement capability of DeepCWR, Figure 4 The step is located at the center of the sample, and the manufacturer provides a true height of 104.30 nm. Figure 4 a1, a2, and a3 show the amplitude, phase, and 3D morphology reconstructed by DeepCWR, respectively. Figure 4 b1 to b3 show the amplitude, phase and 3D morphology of DH. Figure 4 The height profiles of the red and blue lines in a2 and b2 are as follows Figure 4 As shown in c. The height measured by DeepCWR is 104.6nm, while the height obtained by DH is 109.5nm. Considering that the true height is 104.30nm, DeepCWR provides a more accurate quantitative measurement result.

[0104] In addition, during the experiment, the air suspension function of the optical experimental platform was turned off in this embodiment to check the anti-interference ability of the proposed DeepCWR compared with DH. Figure 4 There is an undesirable background fluctuation phase in b2, as shown by the dashed black line, and the profile is as follows Figure 4 d. But in Figure 4 There is no such background fluctuation in a2, as shown by the dotted yellow line, and the profile is also as Figure 4 As shown in d. Figure 4 The comparison of d shows that DeepCWR has a more stable background than DH and thus has better anti-interference ability. This is because DH is interferometric while DeepCWR is not.

[0105] In summary, the complex wavefront reconstruction method DeepCWR proposed in this paper effectively addresses the shortcomings of existing complex wavefront reconstruction techniques in terms of accuracy, interference immunity, applicability, and spatial-bandwidth product through innovations such as spatial light modulator assistance, unsupervised deep learning, and high spatial-bandwidth product design. Therefore, DeepCWR is expected to become an effective solution to address the lack of high-precision, highly interference-resistant, and high-spatial-bandwidth quantitative 3D measurement technology in industrial sites, providing new technical support for quality control and feedback control in the manufacturing industry.

[0106] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning, characterized in that: The following steps are involved: S1, collects the actual measurement values ​​of the optical system; S2, the actual measurement value is input into the DNN model, and the DNN model directly outputs the predicted complex wavefront on the target plane in an end-to-end manner, wherein the predicted complex wavefront includes amplitude and phase; the DNN model includes an interleaving layer, a U-Net network and a de-interleaving layer connected in sequence, wherein the interleaving layer and the de-interleaving layer are used to achieve a high space-bandwidth product, and the input measurement value is sequentially passed through the interleaving layer, the U-Net network and the de-interleaving layer to obtain a high space-bandwidth product. H × W HSBP feature map of spatial size, where H 、 W Represent the length and width of the input measurement value image respectively; the HSBP feature map is further processed by two convolutional layers to generate a size of The predicted phase stack and amplitude stack of N The predicted phase of the complex wavefront on the target plane is obtained by averaging the predicted results. and predicted amplitude , size is 1; predict the phase and predicted amplitude Combined to obtain the final predicted complex wavefront As output; S3, the predicted complex wavefront further undergoes the same digital spatial light modulator modulation and digital wave propagation as in the actual acquisition process of the optical system, and obtains a predicted measurement value on the digital camera plane; S4, calculates the loss function based on the difference between the predicted measurement value and the actual measurement value; S5, repeat steps S2-S4, iteratively optimize the parameters of the DNN model by minimizing the loss function, and iteratively output the final complex wavefront prediction result.

2. The quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 1, characterized in that: The optical system includes an object to be measured, a 4f system, a beam splitter, a spatial light modulator, and a camera. The object to be measured is imaged on the spatial light modulator through the 4f system. The plane of the spatial light modulator is named the target plane. The spatial light modulator modulates the complex wavefront amplitude of the object to be measured on this plane. The modulated complex wavefront travels a distance z before reaching the camera. Because the spatial light modulator operates in reflection mode, a beam splitter directs the complex wavefront to the camera. The camera captures the diffraction intensity pattern as a measurement. During the operation of the optical system, the spatial light modulator sequentially generates a series of random phase modulation patterns, and the camera synchronously captures the corresponding measurement values.

3. The quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 1, characterized in that: The mathematical description of a true continuous wave on the target plane is: in, and They are Amplitude and phase at the target plane; random phase pattern via a spatial light modulator After phase modulation, the continuous wave propagates diffraction distance and reaches the camera, and the measurement value recorded by the camera is expressed as: in, ,◦ represents the Hadamard product, which means and Element-wise multiplication of , The diffraction distance is Wave propagation operator on ; Assume that the size of the spatial light modulator modulation sequence is N , thus we get N The predicted continuous wave on the target plane is directly generated by the DNN model and is expressed as the actual measured value. As input, the solution to the underdetermined complex wavefront recovery problem is expressed as: in, represents the deep neural network DNN, Represents the set of neural network layer parameters that need to be learned, including the weights and biases in each neuron.

4. The quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 3, characterized in that: During the DNN model optimization process, the diffraction distance parameter Starting from any range of initial values ​​and model parameters are learned together synchronously, by adjusting the parameters Learning makes Approaching the given and unknown actual diffraction distance , the solution to the underdetermined complex wavefront recovery problem can be reformulated as: 。 5. The quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 1, characterized in that: The interwoven layer will be of size The input measurements are rearranged to dimensions of data, where the third dimension N and( ) indicates the number of channels, represents the scale factor; by rearranging the interleaved layers, the spatial dimension is changed from Reduce to , the number of channels from Increase to The de-interlacing layer performs the inverse operation of the interlacing layer, and converts the size The data is rearranged into a size of data.

6. The quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 1, characterized in that: The U-Net network is a fully convolutional network, consisting of an encoding path and a decoding path, including convolution blocks, convolution downsampling blocks, transposed convolution layers and jump connections. In the encoding path, shallow features are first extracted from the data rearranged by the interleaved layer through the first convolution block, and then deep features are extracted at multiple scales through the second and third convolution blocks, wherein the second and third convolution blocks are each connected to a convolution downsampling block to reduce the spatial size. In the decoding path, the encoded result is first decoded through the fourth and fifth convolution blocks, and the fourth and fifth convolution blocks are each connected to a transposed convolution layer to increase the spatial size. Then, at the end of the decoding path, high-level features are further extracted through the sixth and seventh convolution blocks. Between the encoding path and the decoding path, features of the same scale are connected through three jump connections, and the information on both sides is fused by splicing.

7. The quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 6, characterized in that: The convolutional block consists of two 3×3 convolutional layers, each followed by a ReLU activation layer and a batch normalization layer.

8. The quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 6, characterized in that: The convolutional downsampling block is constructed by a 6×6 convolutional layer followed by a ReLU activation layer and a batch normalization layer.

9. The quantitative and highly interference-resistant complex wavefront recovery method based on unsupervised deep learning according to claim 1, characterized in that: The loss function is expressed as: Among them, the MSE function calculates the predicted amplitude With measured amplitude The mean square error between N is the number of measured values, is the total variation regularizer, α is the relative weight of the total variation regularizer, is the predicted complex wavefront output by the DNN model; During the model optimization process, the gradient of the loss function relative to each neural parameter is calculated through the back-propagation algorithm, and then the gradient descent algorithm is used to update the neural parameters.

Images